Tips for communicating the impact of COVID-19

Policy details

These tips suggest some things to consider if your statistical outputs have been affected by the pandemic.

The COVID-19 pandemic is having an impact on the production and publication of many of our statistics.

For example:

• some of our data collection methods have changed – we are pausing face-to-face interviews and re-thinking the way we collect survey data
• we are increasingly faced with incomplete or missing data
• we have had to prioritise resource away from the production of some official statistics to provide urgent COVID-19 analysis
• even where we are collecting the same data as previously, the pandemic is affecting trends and making comparisons over time difficult

All of these things have an impact on the quality of our statistics.

We must also consider that it may be difficult to tell whether COVID-19 has had an impact on the quality of an output, and even more difficult to estimate what the impact may have been. For example, a survey about participation in the labour market may want to measure the effect of working from home. But the fact that people are now more likely to be at home could mean that different types of people are responding.

We need to consider how best to communicate impacts and uncertainty to our users.

Tip 1: Use the Code of Practice as a framework to guide your actions

Remember Trustworthiness, Quality and Value

Use the three pillars of the Code of Practice for Statistics as a framework to think through your choices.

For example:

• how can you show you are trustworthy in the way you release the statistics?
• what are the quality issues to be managed and explained – for example, are trade-offs needed to achieve sufficient accuracy and timeliness?
• what statistics are needed in the public interest?

Use the guidance

Alongside the code, the Office for Statistics Regulation have published a package of measures to help guide producers through the pandemic:

• guidance on factors that producers should consider when making changes to data collection, statistics production and release
• an approach to rapid regulatory reviews for new outputs
• guidance on suspending accredited official statistics status
• exemptions to the Code of Practice for Statistics due to COVID-19

Use this guidance to ensure that your statistics continue to serve the public good.

Tip 2: Talk to your users

Communicate with users regularly and proactively

Make your users aware of potential impacts and changes as soon as possible in advance of the publication. Consider potential impacts at all stages of the data journey and for all users. For example, you may produce statistics which are supplied to another team or department for onward processing or analysis.

Good practice example

The Department for Transport published guidance about the impact of COVID-19 on their outputs. This identifies both known and potential issues and clearly indicates publications affected in advance of their publication.

Be up front about issues and concerns

Make any differences or changes very clear within your publications, especially where they will have an impact on how the statistics should be used. Put clear notes towards the top of your release as well as footnotes and signposting within the publication itself.

Good practice example

The Crime in England and Wales September 2020 publication from the Office for National Statistics uses bold text and an exclamation mark at the start of the long-term trends section to make clear that the data are not directly comparable.

Reach out to your users and data providers as soon as possible

Be proactive in your engagement with data providers to better understand the impact of COVID-19 on your data sources.  If you have alternative methods you could use, engage formally or informally on which might best meet your users’ needs. If you need to scale back outputs due to resourcing, get user input into which outputs to prioritise. Ensure that users’ needs are considered when any decisions are made.

Tip 3: Be helpful!

Make things easy for your users

For example, if COVID-19 has had an impact on your time series and it is not appropriate to compare the latest data with previous time points, don’t show them on the same charts or tables as the previous series. Keep them separate so that users know not to compare them, or clearly label the affected period. If COVID-19 has not yet had an impact on your statistics because of data lags, make this clear at the start of your publication.

Good practice example

The second paragraph of the ONS house building statistics publication highlights that the impact of COVID-19 is not yet apparent in the data.

Communicate uncertainty clearly

Some of the impact of COVID-19 on the statistics might be unknown – be clear about this. Users need to have enough information to be able to judge whether the statistics are fit for their purpose. There is more detailed guidance available on communicating quality, uncertainty and change.

Be specific

Don’t add a generic ‘use with caution’ notice. We recommend not using such wording because it does not explain what the issues are and how they feed through to the use of the statistics. State what your users can and can’t do with the data.

Good practice example

The first section of the COVID-19 infection survey publication clearly states what the data can and cannot be used for.

Explain the problem

Be transparent and explicit about any known issues and the impact that they have had. Be open about your plans and how and when the issue will be rectified. For example, will you publish or revise the data in the future?

Good practice example

The Department for Culture, Media and Sport very clearly set out potential impacts on their estimates of levels of cultural, digital and sporting engagement.

Show alternatives

Consider any alternative information, resources, or data that might be useful for your users. Are there different ways you could present your statistics that focus attention on estimates that remain more robust? If you have temporarily paused your publication, are there alternative data sources you could highlight?

Good practice example

The Northern Ireland Statistics and Research Agency (NISRA) have published a document containing information from a range of alternative sources in the absence of their regular tourism statistics.

Tip 4: Collaborate

Share your experiences

This is a new and evolving area for all of us. We do not yet understand the full impact of COVID-19 on official statistics. It is more important than ever to share problems, knowledge and ideas with each other. You can do this by:

• emailing best practice to the Good Practice Team who can then share this across the Government Statistical Service (GSS) and Analysis Function
• using the GSS Slack workspace
• sharing with your departmental champions
• emailing the cross-government survey group if related to survey data

Discuss with producers of equivalent or similar statistics

Reach out to producers of similar statistics in other parts of the UK (or in other departments) to discuss and learn from their experiences and to align your approaches where possible. This helps to ensure coherence between releases and provides a better experience for your users.

Ask for advice

Don’t be afraid to ask for support. You can do this by:

• asking fellow analysts via the GSS or Data Science Slack workspaces
• speaking to your Head of Profession for Statistics
• emailing the Office for Statistics Regulation
• contacting the Best Practice and Impact Division

Tip 5: Use consistent language

Use the Coronavirus (COVID-19) A to Z style guide on GOV.UK to make sure the language you use is consistent across government.