How can we help researchers get better study data (and improve the participant UX)?

TL;DR

Redesigned the mobile-first experience for recording naturalistic study tasks. I pitched a restructured capture methodology, leading to a major reduction in user friction. I also implemented advanced help options and robust researcher customization. The redesigned UX and IA significantly improved usability, leading to...

Reduced support tickets due to task confusion

Less rejected study data due to task confusion

Reduced backend tech debt

Highlight

The foundational task recording methodology was making key UX features impossible, so I approached the developers to pitch an alternative methodology for capture. It was a challenging redesign for both teams, but it reduced our tech debt and made the product far easier to use and gather better data. An improved UX is always worth the extra effort.

Note

The exact, quantitative effect of our product remains confidential to the clients so my success is evidenced in their continual reliance on the features I designed that help drive million-dollar product decisions.

Role

UI design

Interactive prototyping

Developer handoff

Collaborators

Design Lead

Researchers

Developers

Timeline

Q1 2024

Background

Context

Pulse Labs’ main B2B SaaS product is a platform that helps researchers draw out product and user insights from naturalistic studies by helping researchers run studies and recording participant input.

A major part of naturalistic studies involves participants recording themselves conducting predetermined tasks. These tasks are tracked and recorded through the Pulse Labs app, and carefully designed by researchers to test usability and draw out product/user insights.

Problem

When I joined the team, the flow for recording tasks was highly unfriendly to participants. It required participants to do a host of tedious actions, including:

Copy/paste the tasks to another app (or write them down) for future reference

Don't return to the app for help until recording is over

Inability to restart a single task recording at the end of a session (must upload or restart all)

After uploading, manually set the start/end timestamps of each task

Ultimately, the UX was making it easy for participants to make mistakes and become frustrated, which can affect research data and participant retention.

Approach

Recording methodology

I studied the old flow and realized the main issue is the system records everything as one large media file. I talked to the developers to hear why and pitched an alternative:

Most of the bad UX would be solved if the system recorded every task separately as smaller files. This would have two main advantages:

Engineering said it was possible to rework how recording was handled. This opened up great opportunities for the user experience.

Feature requests

There were some other feature requests that made sense to design and ship together:

Researchers had been asking for a way to display modality (which is how users should complete a task, such as "with tap" or "while parked" or "while outdoors") during a task without typing it into the task itself.

My design lead also requested we add a context prompt feature, which would allow researchers to add detailed context or directions before one or more tasks.

Exploration & Discovery

Empathy mapping

In addition to these requests, I tried putting myself in the position of our users: "If I was a participant or researcher, what would make recording with Pulse Labs a joy?"

I came up with:

  • Time and equipment requirements screen before starting

  • Camera setup assistance before starting

  • Customizable researcher tools

  • Review individual tasks before uploading

All of these concepts respect the participants' time and the effort put forth by the researchers in pursuit of naturalistic data.

Review methodology

At first I thought reviewing tasks immediately after doing them was the best reviewing method. But on-paper, it felt clunky, forcing participants to slow down too much.

I felt a lot better about an optional Review screen after all tasks are over, right before uploading. This allows participants to review and restart a task if there's an issue without needing to interrupt their task flow mid-recording.

Solution

Note: Another designer had recently refreshed the design system assets, so that's why these newer designs look so different from the original.

Here's how all the ideas, requests, and designs came together to help researchers get better data from their participants:

Instructions screen

Updated

  • Lacks the embarrassing request to copy/paste the tasks to another app (from the original version)

  • Reminder to tap "Next Task" after each task to automatically segment their recordings properly

Modality

New

  • Informs participants about how to conduct a task, helping researchers get more consistent, nuanced task data

  • Above and separated from the task itself, improving its hierarchy and likelihood of being seen, thereby priming participants on how to approach the task

  • Why: Conveys researcher intent, reducing bad test data

Context prompts

New

  • Bottom-anchored modals that appear before tasks (when applicable)

  • Can be recalled at any time

  • Communicates directions beyond the task itself (such as switching apps mid-session or spending a certain min/max time on a task)

  • Confirmation button is on a short, 3-second timer to prevent participants from speeding through without reading it, but doesn't feel like a long, tedious barrier either

  • Why: Conveys researcher intent, reducing bad test data

Task help

New

  • Button that simultaneously tracks progress and gives participants help options mid-task

  • Allows participants to restart a task if they make a recording error

  • Allows participants to contact support for more help

  • Allows participants to skip a task if the task is not applicable to them or their product

  • Why: Helps reduce participant frustration and subsequent support tickets (or dropouts)

Review

Updated

  • Automatically detects and surfaces recording issues to the participant, such as if a task is too short, too long, or corrupted

  • Allows participants to restart recording a specific task in the list, instead of restarting all of them

  • Why: Helps reduce participant frustration, bad data, and subsequent support tickets (or dropouts)

Researcher Customization Tools

Beyond the default Context Prompts and Modalities, researchers have rich customization tools to create ones that perfectly fit their research methods.

Next steps...

Here's what I'd do differently if I had the opportunity, time, and resources:

See more work