Office for National Statistics (ONS) Survey Strategy Research and Development Principles (SSRDP)

When we find something that works, for example, a letter template or a question pattern, we use it widely. We:

• follow harmonised standards to improve the quality and comparability of our data across government
• use consistency to build legitimacy and brand recognition
• use tried and tested products to improve our ways of working and help us all achieve our goals

But, we also allow ourselves to take a different approach when our evidence shows we need to. This prevents us from complicating the respondent journey, which could compromise user needs. We always aim for consistency and not uniformity.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• aligns with other initiatives across the organisation
• products should reduce complexity and burden that exists in the system

We develop each product, for example a letter or questionnaire, in the best way for the mode, or modes, it is administered in. We also develop each product for its mode specific users. This is known as “optimode” design. By designing in this way, we can help respondents give us the data we need by reducing respondent burden. It also creates products that are more user friendly to our internal users, including interviewers and call centre staff.

We tailor each product to the medium it uses and the specific needs of the users in that mode. This helps us get the data we need and improves data quality.

We use adaptive web design during development which allows the layout to adapt to the screen size appropriately. We design for mobile screens first, and then larger ones. That’s because it helps challenge us to think about the minimum content needed. We justify each piece of content being added, and refer to user needs, user stories and user journeys to do so.

ONS Survey Strategy Delivery Principles

This SSRDP links to the ONS Survey Strategy Delivery Principles that “products should reduce complexity and burden that exists in the system”.

The design of our surveys and their products are based on evidence. We:

• do not make assumptions about our users’ needs
• do not make a design decision if there is no evidence to support and inform it
• avoid assumption led design as this will lead us to producing the wrong thing

Evidence and insights can be gathered from many sources. For example, we could complete some research with respondents or explore existing data to inform our next actions.

ONS Survey Strategy Delivery Principles

This SSRDP links to the ONS Survey Strategy Delivery Principles that “decisions are backed up by evidence”.

Our surveys meet the data users’ needs because our design journey starts with them. We invest time with our data users to learn about their data intention. This includes understanding how they intend to use and analyse the data.

We let our data users lead the way by providing the concepts to be investigated, but they do not design the content itself. We avoid getting data users to design the content because the designs will not be respondent centred.

Once we understand our users’ needs we use this information, alongside the respondent needs, to inform the design of the respondent centred survey products.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• clear ownership and accountability
• decisions at right level
• decisions are backed up by evidence

We listen to our respondents and know what they need. This means we design the right thing. We do not make design decisions based on our assumptions, personal views, and biases. Instead, we carry out research to learn about respondent mental models and needs. We explore the cognition and usability of our household and business survey products through testing. We involve interviewers, call centre staff, survey processors and operational staff in the development of products to incorporate their needs and their insights on respondents. This could include insights about issues with an existing questionnaire, for example.

We always ensure we learn about what respondents need, rather than what they want. We use our analysis of respondent needs to develop assets such as respondent journeys and stories which inform the design of survey products.

We follow the Respondent Centred Design Framework to ensure we design based on needs to create respondent centred products.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• decisions are backed up by evidence
• decisions at right level
• products should reduce complexity and burden that exists in the system

Our statistics reflect the experiences of everyone in our society. This means all our surveys are designed to be inclusive and compliant with accessibility legislation.

We think about all types of respondents from the start because we want everyone to be able to take part in our surveys. We increase response and representation in our data by removing barriers to interaction and participation created through exclusionary design. We follow harmonised standards to ensure our survey questions are inclusive and that we collect representative data.

Inclusive and accessible designs reduce burden for all respondents, not just those with additional needs or disabilities. For example, we aim to design each product to meet the average reading age of the UK. This makes our products easier to understand which improves the overall respondent experience. Inclusive and accessible designs improve the quality of our data and build trust in our statistics.

These ways of working also apply to products that are developed for internal ONS users that are part of running a successful survey. This includes interviewers, call centre staff, survey processors, and operational staff. This provides equal opportunity to our workforce by ensuring everyone can use our products.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• products should reduce complexity and burden that exists in the system
• aligns with other initiatives across the organisation
• contribution to survey and ONS strategic outcomes is clear

We trust and involve the right people at the right time at every phase of a project. We define clear roles and responsibilities which helps us run a successful survey and achieve our goals. We are transparent about the design process with stakeholders and involve them in the development journey.

Everyone involved in a project clearly understands their purpose and expected contribution at every point of designing and developing a survey. They know where their role begins and ends, which helps ensure people with the right skills for the job are assigned to the right part of the design and development process. This allows the organisation to fully benefit from the investment made to employ and train these people who are experts in their roles. It also avoids products being influenced and designed by the wrong people at the wrong time. This can lead to the wrong thing being built. For example, data users are responsible for providing their data needs and analytical requirements to the research and design teams. The research and design teams then fulfil their role in the process, which is to conduct the research to develop the appropriate designs to meet user needs. The roles are clear, the data users do not dictate the design of the questions as the research and design teams are trusted and skilled to produce the right product to meet their needs.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• clear ownership and accountability
• decisions at right level

We have a responsibility to make our surveys easy to understand and use. We do the hard work to make our surveys simple, which removes that burden from our staff and respondents.

We prioritise the respondent experience because we know that without doing so, we risk us not achieving our goals. We develop surveys that meet the needs of respondents and data users by investing time and resources into the early research, design, and testing phases of a project. We monitor respondent burden and use the insights to inform decision making.

We develop surveys that do not rely upon staff intervention and lengthy guidance to get the data we need. Instead, they are clear and highly usable on their own, without the need for much additional support or advice. Through good design we empower our respondents to take part in our surveys and provide us with the data we need. We only add additional help where research shows that further support is needed.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• products should reduce complexity and burden that exists in the system
• aligns with other initiatives across the organisation
• contribution to survey and ONS strategic outcomes is clear

We follow best practice and standards in the design and development of our surveys. This ensures our surveys are high quality, modern, and sustainable.

When we have found something that works, we make it reusable and shareable instead of starting from the beginning of the development process every time. Our harmonised standards are good examples of this. This approach ensures others can benefit from the investment made in developing that product. It also avoids duplication of effort and spending of public money on creating the same thing.

Sometimes we may need to take a different approach. It is important to remember that each survey is different and may need bespoke products or solutions. The decision to do something different and stray from best practice and standards is always based on evidence and respondent needs, not assumptions, personal views, or biases. For example, harmonised standards are used as the starting point, but they can be adapted to meet the needs of specific surveys.

We continuously refresh our knowledge and understanding of best practice and standards, which allows us to constantly add to our evidence base. We look to the research of others around the world to inform our work, but we keep in mind the importance of country context.

We refresh our surveys and carry out continuous improvement to our content to ensure they remain relevant.

We use administrative data or other data sources, where available, to reduce survey length, respondent burden and operational demands while improving processing and the quality of statistical outputs.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• aligns with other initiatives across the organisation
• products should reduce complexity and burden that exists in the system
• proposals can be sustainably resourced and financed
• decisions are backed up by evidence

We take an agile approach to developing our surveys, which helps us avoid the risk of building the wrong thing and finding that out too late. We always test our survey products and processes with respondent users before releasing them. We iterate and refine them based on research insights and not assumptions, which ensures we develop something that meets the users’ needs.

We are transparent about what does not work and we abandon these things when our research shows they are not suitable.  We then work to find an alternative solution. We always test our survey products along the full end-to-end respondent journey to ensure we are providing our respondents with highly usable and coherent products.

Sometimes we may find a problem with a questionnaire after the live phase. There are several ways we might find this out, for example through interviewer feedback, respondent feedback, or by looking at the amount of imputation needed. When this happens, we flexibly adapt and improve the questionnaire rather than needing to run big re-development projects.

We involve topic experts from inside and outside of ONS to support with:

• the design of our surveys and their products
• how our surveys are run

We share our insights and learnings widely with others internally and externally. We abandon what does not work in favour of finding something that does, and we remember that discovering something does not work is a valid insight. We share prototypes and progress widely to gather feedback from people with different areas of expertise to create better products for our users.

We recognise the importance of bringing our stakeholders on our development journey to ensure successful survey design. This is why we involve them at all stages of a project.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• decisions are backed up by evidence
• decisions at right level

We see our surveys and the data they generate as an ecosystem. We are aware of the interdependencies of each phase in the data lifecycle and how our decisions at the beginning and throughout affect the quality of the data we collect and produce.

When a survey uses multiple modes for data collection, we do not view or develop each mode separately. Instead, we think about all aspects together and consider them as one so we can create successful surveys that meet respondent needs. When there is a problem, we think about how it affects the whole respondent journey and data lifecycle. We then work to fix the problem accordingly.

We use intelligence from paradata, processing and analysis of data collection to help the future design of our surveys. For example, we learn about where we are doing lots of imputation as this may mean that we need to review the questions and concepts for clarity.

We think holistically about the design and appearance of our products to maximise response and data quality. We ensure all products respondents interact with are consistent in tone and appearance. This helps build trust and build a brand identity.

ONS Survey Strategy Delivery Principles

This SSRDP links to the following ONS Survey Strategy Delivery Principles:

• products should reduce complexity & burden that exists in the system
• contribution to survey and ONS strategic outcomes is clear
• aligns with other initiatives across the organisation