By Brenda Reginatto, Research Lead, Care Team, ARCH
When conducting research with patients, it is very tempting to try capturing as much data as possible from them. Research participants are extremely valuable sources of information, but are often difficult to reach. Patient recruitment is challenging for many reasons and when a research team manages to enrol a number of participants successfully, the desire to maximise data collection seems perfectly understandable.
There is one caveat, however. While researchers’ desire to learn certainly comes (or at least should come!) from a good place, there is a risk that by asking participants to do ‘too much’ we may effectively scare them off and lose their commitment to the study altogether.
|Note: While this post draws on my experience of applied research, the same principles apply whether you are conducting user research in academia or in a company. If you are developing an app for example, I’m sure you’ve asked yourself questions around how many steps your sign-up process should have, how many clicks should be required to complete a particular task, etc. In one way or another, the same questions hunt us all!|
So we must ask ourselves, where do we draw the line? How much can we ask of research participants, before it becomes too much? How lengthy should questionnaires be? How often should we use them? How many sensors can we ask someone to wear? How many tasks can we ask someone to perform and how often?
In fact, I have asked myself these questions in virtually every project I have worked in. In one of my first projects, I helped a company collect data from older people who were at risk of falls. Participants had to put on a number of sensors every morning and charge them overnight. The sensors were powerful data collection tools, but operating them was far from straightforward. Recruitment proved very challenging, and many patients explained that the amount of things to they had to do and the complexity of the tasks involved made them not want to take part.
During our recent REACHES study, we worked with a group of companies to explore the burden a remote trial placed on participants. In this project, nursing home residents were asked to wear an activity tracker for 8 weeks, while nursing home staff were asked to help them charge the device twice a week and troubleshoot connection issues, should they arise. While residents reported limited burden, staff explained that these tasks (which initially sounded simple) proved considerably burdensome. In one way or another, the burden stemmed from the fact that these activities were not embedded in staff’s routines and significantly disrupted their workflows.
On another study, a company wanted to understand how a group of patients and a group of healthy participants would comply with wearing different activity trackers, answering daily questions on their phones, as well as performing a number of tasks over one month. Interestingly, while the healthy participants had a long list of complaints about how uncomfortable or difficult it was to put on the activity trackers and how repetitive they found the questions, the patient group did not express the same views. For them, these issues were minor compared to the positive feeling of “accomplishment” and the sensation that they were “doing something to help their condition”.
Reflecting back on these experiences, it is clear to me that the question of “how much is too much” is too simplistic. In order to understand how much you can ask of research participants without negatively impacting their experience, you might consider asking the following questions instead:
- Does it make sense to the user?
The sheer number of tasks may not matter so much if the user feels they understand what they are supposed to do. In “The Design of Everyday Things”, Donald Norman explains ‘mental models’ as an individual’s perception of how something works. If the task at hand fits with an existing mental model, users tend to learn faster and require less mental effort to complete it. It’s not rocket science, if users perceive something as easy to do, they are less likely become overwhelmed and abandon the task.
- Does it fit in the user’s routine?
We all know context matters. Answering one question a day may sound simple, however the time of day and where you are when that question pops up on your phone can dictate whether you actually answer it or not. In other words, tasks that are perceived as disruptive to the user’s routine, are less likely to be completed.
- What is ‘in it’ for the user?
Ultimately, people tend to stick to a task if they perceive this is of benefit to them. On the other hand, if a task is perceived as “not worth it”, it is more likely to be abandoned. Balancing effort and reward is not simple as different people are driven by different factors. Some may see benefit in helping people under the same circumstances, while others expect direct benefits. Some people are motivated by financial reward, while others may enjoy the social aspect of engaging in an activity with their peers. There is no one-size-fits-all.
From my experience, there is no magic formula to answer these questions. In order to understand how much we can ask of participants without hindering their experience, we need to get to know them as best as possible. We need to go beyond demographics and understand their familiarity with the tasks involved, whether these fit with their routines and whether there is enough incentive for them to get involved. In ARCH, we are big advocates of patient-centred design and have seen that a small feasibility study to explore these issues with “real patients” prior to a bigger study can be a great help to answering these questions.
Have you ever asked yourself the “how much is too much” question? How did you go about answering it? We would love to hear about any particular challenges you faced or any suggestions you may have on how to overcome them.