User-centred design approach to surveys

Policy details

What we mean by ‘user-centred design’

User-centred design is defined as learning about the needs of those who will use your service and designing it to meet them.

In terms of surveys the user is the respondent and their needs can relate to the following:

• Who they are
• What their circumstances are
• What information they require before, during and after taking part
• How and where they take part
• What they are trying to do and what they want to be able to do
• What their expectations are at each stage of the journey
• Issues that cause them friction and pain points

If we build the ‘right thing’ (i.e. the right sort of survey) then the following outcomes will be observed:

• Reduction in respondent burden
• Increased data quality
• Reduction in costs

The Government Digital Service principles support this approach.

How user-centred design can reduce respondent burden

Respondent burden can occur at any point in the respondent journey, for example from the point of receiving communications through to completing the survey itself.

It is possible to reduce burden at every touchpoint by developing the survey experience to be user-centred. Researchers, L. Wilson and E. Dickinson, at the Office for National Statistics have distilled their user-centred design knowledge and approaches into a ‘Respondent Centred Design Framework’ (RCDF) (L. Wilson and E. Dickinson, 2021). The RCDF aims to combat the ‘errors of measurement’ and ‘errors of representation’.

This framework can be used alongside: the Total Survey Error Framework (Groves and others, 2004) and the four-step cognitive process developed by (Tourangeau and others, 2000).

 

Respondent Centred Design Framework (RCDF)

1. Establish the data user need

Before designing in a respondent centred way, we must understand the data user need.

Gather information from the data users and analysts to learn about how they intend to use the data. Once this is clear, this information can then be used to inform the design of the communications, questions and next steps.

2. Mental model research

Explore through qualitative research how respondents conceptualise topics in the questionnaire.

This means to learn about what they draw upon to reach that understanding, the thought processes that take place and whether the response options meet their expectations.

Mental models can be learned, based on experience, exposure and environment. It is equally important to design for them in interviewer administered and self-completion modes.

Interviewers can also provide insights into the mental models of respondents. This information will then inform the wording and flow of the questionnaire, grouping questions that are conceptually linked.

3. Understand user experience and needs

User stories and journeys are key tools when designing a survey. They document needs and the path that a user will take, respectively. Often assumptions are made about these which can result in the wrong thing being built.

A user story is a three-part statement which documents the user needs. They are commonly written in the following way, “As a [insert]…I need [insert]…so that [insert]”.

For example, a respondent may say, “As a respondent to the survey, I need to know how long it will take, so that I can complete it at a time that is convenient”.

The insights that inform the statements are gathered through qualitative research with the public. A user journey documents every step taken and task completed by the user when using your product or to complete a goal.

Documenting each step in the journey will help to identify the barriers and friction points. Once these are identified and understood then they will inform the research plan.

4. Use data to design

Data can be used to inform the questionnaire order and flow and to improve quality.

It is possible to assess whether the questionnaire is moving respondents through it in the most efficient way by analysing the pathways of different groups. Using this information, it is possible to design the question flow to minimise burden.

High item missingness (i.e. a high amount of missing data related to people not answering a question), or long question timings for certain questions may indicate a question is burdensome for respondents.

Running these analyses and learning from them can improve data quality and direct future qualitative research to explore the problem.

5. Create using appropriate tone, readability and language

Creating content with appropriate tone, reading age and language is key to reducing burden.

All content, from letters to the questionnaire, should be tested for reading age compliance. There is free online software to do this, such as the Hemingway App.

In the UK the average reading age is nine years old therefore content should be developed to meet this level. There will be some instances where this is not possible to apply to all content, so pragmatism is recommended.

Learning about the language used by the public to describe topics or their circumstances allows us to recycle this content in materials and questions to reduce burden by aiding cognition.

6. Design without relying on help

Questionnaires should not rely on respondents reading additional help in order to provide a response. Questions should be designed in a user centred way to reduce burden.

Issues with questions should be addressed through iterating and testing alternative designs rather than by adding help. It is not always possible to remove all help from a questionnaire – however, it should be used sparingly.

7. Take an ‘optimode’ approach to design

‘Optimode’ means to design the respondent communications and the questionnaire optimally for each mode.

For respondent materials this means tailoring content of letters depending on the mode of interview.

For questionnaires this means optimising the design for each mode.

Having a design that works optimally in the mode that it is being administered will reduce respondent burden as it will be tailored to their needs.

8. Use adaptive design

Questionnaires should be built using adaptive design. This is where the interface adapts to the screen size and displays the content accordingly.

This approach will reduce respondent burden as the screen will render suitably to the device they are using to complete the survey.

When designing questions, it is helpful to think ‘smartphone first’ as this will constrain the amount of space available in turn challenging you to produce a leaner question, or set of questions, to address the data user need and reduce burden.

9. Conduct ‘cogability’ testing

Before going live with respondent materials and the questionnaire, it is important that pre-testing takes place with respondents.

The content must be tested to ensure that it is understood, it is usable and meets the respondent need, in turn reducing burden.

We recommend combining cognitive and usability testing to create ‘cogability’ testing in the same session to maximise learning.

10. Design inclusively

All content developed should be designed to be inclusive. Inclusive designs reduce burden for all users not just those with disabilities. Find out more about making your service more inclusive.