Harmonisation case studies

The task 

Homelessness is a devolved matter in the UK. This means that each nation produces their own homelessness statistics. 

Each country collects data on the two main concepts of homelessness. These are:

• statutory homelessness
• rough sleeping

Data about these two concepts are published as official statistics.

Legislation and administrative data collection systems are different in each country. This means information about comparability of statistics and data is limited. 

The approach 

The Government Statistical Service (GSS) Strategy Delivery team and the harmonisation team created an interactive tool to help users compare official statistics. It also helps users to compare definitions of homelessness and rough sleeping. The tool:

• provides comparison guidance
• give details about the process a person may go through when they’re seeking support for housing in each UK country
• shows how each country collects their statistics

The harmonisation team published a report with recommendations for improving comparability of homelessness in UK statistics. The report gives an in-depth overview of the comparability of homelessness statistics. 

The outcome

Users have accessed the tool more than 2,500 times since it was launched.

More information

For more details, see the Homelessness harmonisation guidance.  

The task 

The National Survey for Wales is the main social survey carried out by Welsh Government and its partners.

Before the pandemic, interviews were carried out with people in their homes. Interviews lasted around 45 minutes. The team used Computer-Assisted Personal Interviews (CAPI) and Computer-Assisted Self-Interviews (CASI).

In March 2020 fieldwork was suspended. The survey was redesigned as a 25 minute recontact phone survey until December 2020. From January 2021 a new approach to sampling was adopted. A random sample of addresses was taken. Phone numbers were collected by using an online portal, phone book or ‘knock to nudge’ approach.

In July 2021 the National Survey began a trial to test whether a follow-up online section would be possible. The online section took 15 minutes to complete. It allowed self-completion questions to be asked again. It helped to increase the survey content without affecting data quality. The quality of the data can be affected if a phone interview is too long.

The approach 

The sudden change in mode gave the survey team the chance to use some of the mode-optimised harmonised standards. The redesign of the survey included:

• re-structuring questions
• shortening and simplifying text
• reviewing introductions and instructions

The team made changes to the interviewers’ instructions, prompts and answer options. They produced interviewer guidance to explain the changes.

The survey team spoke to the interviewers regularly to gather feedback on the approach and questions. The team used mode-optimised harmonised standards. This included the harmonised standard for ethnicity designed to collect data over the phone. There was not much time to develop the survey, so it was useful to have mode-optimised questions that had already been tested.

Internal stakeholders from Welsh Government worked with the Harmonisation team and the Office for National Statistics (ONS) to make changes to the questions. The ONS carries out the survey.

The outcome 

It is still a little too early to tell if the change in question mode has affected the data. Comments from field interviewers and respondents, indicate that the questions are working well in the field. No issues have been reported. The results of cognitive testing were also positive.

The survey team found that asking the harmonised standards over the phone does not take much longer than using showcards. This is clear from the initial analysis of the timing data. The survey team will continue to monitor and analyse timings, data and feedback throughout the 2021 to 22 survey year.

More information

For more details, see the National Survey for Wales. 

The task

The Office for Statistics Regulation (OSR) are conducting systemic reviews of mental health statistics in the four UK nations. So far they have undertaken reviews of mental health statistics in England and Northern Ireland. The OSR also have plans to review mental health statistics in Wales and Scotland.

In the Mental Health Statistics in England review, OSR recommended that consistent definitions should be developed for publications. This would help users make meaningful comparisons between data sources. The Harmonisation team gathered information on the data landscape of mental health statistics to support these recommendations.

The approach

The team gathered relevant information on mental health. They then contacted stakeholders to make sure they had interpreted the information correctly.

The team decided early in the project that they would concentrate on adult mental health only and exclude data on Coronavirus (COVID-19). This was because of the different needs and requirements of the topics.

The guidance compares the use of mental health definitions. It also compares any relevant legislation and strategies. This means that the team can work out why certain terminology has been used.

Guidance on how to find mental health data in survey and administrative sources is available. The guidance includes information about how the data is collected. It also includes survey questions with clinical measures of mental health.

The team worked with stakeholders and users of the new guidance to make sure that it represented the four nations accurately.

The outcome

Users have welcomed the harmonisation guidance. It is a useful way of bringing together all mental health related data to provide a basis for future work on the topic. Producers of statistics are now able to use the guidance to get wider topic knowledge.

OSR has given the guidance their approval on Twitter. They said:

“Great to see this new guidance from @UKGSS. We’re pleased to see that recommendations made in our report on Mental Health Statistics in England have been included.”

More information

For more details, see the Mental health harmonisation guidance.

The task

Collecting qualifications in surveys is challenging for many reasons. For example, some respondents may struggle to:

• recall their own qualifications
• recall the qualifications of other people in their household
• select their qualifications from the options provided

This is why we offer two options for collecting qualification data. These are the educational attainment harmonised standard and the qualifications harmonised standard.

“Educational attainment” refers to the highest level of education a person has achieved. In most cases, this information is enough. So we would recommend using the educational attainment harmonised standard where possible. Sometimes users need to know specific qualifications, or their outputs need more detail. In these cases, we would recommend using the qualifications harmonised standard instead.

The approach

The qualifications harmonised standard is based on the Transformed Labour Force Survey (TLFS). The TLFS is used to produce labour market statistics. These are usually produced by the Labour Force Survey (LFS). The Office for National Statistics (ONS) Social Survey Research and Design team developed the TLFS using data from their research.

The TLFS is suitable for administration both online and over the telephone. It aims to gather information about several aspects of the respondent, including their:

• employment history
• education history
• nationality

Information on the respondent’s qualifications is used to establish their educational attainment. So, even if the respondent is unsure what their highest qualification is, we can work it out. The respondent is then placed in 1 of 7 categories which reflects this.

The impact

The qualification harmonised standard allows for a more detailed collection of qualifications. The questions capture all qualification types that have ever been an option in the UK. This means that everyone can be placed in a category.

Producers of statistics have welcomed the guidance on the collection of qualification data. The guidance allows for the collation of qualifications, providing a basis for future work.

More information

For more information, see the Qualifications harmonised standard.

The task

Faced with uncertainty and a need for data, the GSS Harmonisation team worked quickly to bring together:

• information about the Coronavirus (COVID-19) pandemic
• recommended questions for government surveys on the COVID-19 pandemic

The GSS Harmonisation team would usually undertake an extensive research plan, including qualitative and quantitative testing. But, because of limited resourcing, immediate data requirements, and the changing and diverse data needs, there was no time for the team to undertake such extensive research.

The approach

The GSS Harmonisation team worked quickly to bring together user and data needs and requirements. This involved contacting stakeholders and talking to them to create user need statements, which informed question specifications and design. The team also reviewed government surveys and the questions they had developed about COVID-19, as well as their comparability to one another. This review helped to inform the remit for the harmonised standards and address important data gaps.

The team worked with ONS colleagues and other government departments to identify and amend relevant questions to be consistent with question design best practice and the ONS style guide. They were then able to provide a coherent and consistent question set, which included topics such as diagnosis, worry, key worker status, and impact on life. The team made further edits, such as introducing response options that allowed respondents to give “don’t know” as an answer. This helped to avoid respondents either deciding not to answer a question, or giving an answer that they think will be satisfactory to the interviewer. It was important to use available evidence to assess the effectiveness of the questions, especially as there was limited time available for this work and the team were unable to test the questions with the public.

The question set was sent to the Harmonisation Champions Network for comment, and then published on the Analysis Function website.

The impact

Since the first publication, the webpages underwent two substantial reviews as data needs and user needs continued to change, and the virus and the pandemic continued to evolve. Using the same approach, the Coronavirus (COVID-19) harmonised guidance is now a more relevant set of questions for use in Government surveys. It includes questions on diagnosis, testing, long-COVID, vaccine status, worry, and impact on life.

By reacting to a quickly a substantial, unprecedented situation like the COVID-19 pandemic we were able to support and influence the collection of very sensitive and high-profile data. This meant we could help ensure it was consistent, comparable, and coherent, which had immediate effects on public understanding and development of policy.

More information

You can find more information about this work in the Coronavirus (COVID-19) harmonised guidance. The guidance includes links to tools such as the Coronavirus (COVID-19) question bank, which was also a reactive piece of work that was developed in a short space of time to address similar issues.

If you would like to contact us, please email the GSS Harmonisation team at Harmonisation@statistics.gov.uk.