Using non-official sources for UK reporting on the Sustainable Development Goals
The Sustainable Development Goals (SDGs) team at the Office for National Statistics (ONS) have published a new protocol to assess statistical information from non-official producers. This protocol will help to expand their range of UK SDG reporting and enable them to consider statistics from charities, academia and businesses.
What are the Sustainable Development Goals?
The Goals are the United Nations’s (UN) radical and ambitious agenda for a more inclusive, sustainable and prosperous world that leaves no-one and nowhere behind by 2030. The SDGs bring together 17 economic, environmental and social goals in a holistic agenda covering people, planet, prosperity, peace, and partnership.
Launched in 2015, the SDGs are part of the UN 2030 Agenda, with progress measured using 247 indicators. Beyond these headline indicators, the Goals encourage reporting of a range of breakdowns, by “income, sex, age, race, ethnicity, migratory status, disability and geographic location, or other characteristics” as appropriate (UN General Assembly, 2017). Some indicators also have specific disaggregations, such as education level or type of funding. In fact, there are over 100 required breakdowns, which provide additional information about the indicators.
How are the SDGs reported?
The ONS SDGs team leads on reporting the United Kingdom’s (UK) progress through SDG Data, the National Reporting Platform.
With such a wide agenda, SDG reporting relies on a similarly wide range of sources. Most of our reporting comes from national statistics or other official statistics from across government. Statistics from official sources inherently are good quality and trustworthy, but even for these the team conduct quality and relevance checks before they are used on the Platform.
What is the challenge?
The SDGs team already report 80% of the indicators for the UK. However, as they make progress, it has become increasingly challenging to report using official sources alone. A wide range of disaggregations for these indicators is also needed, to address the principle of Leave No One Behind at the heart of the SDGs.
They are reporting on many of these disaggregations using official statistics, and the Inclusive Data Taskforce is looking at how to increase the inclusivity of such sources. Non-government sources can complement their Platform to tell a more complete story of progress on sustainability and its challenges. But as they are non-official, they are not covered by the Code of Practice for official statistics.
So how can the SDGs team ensure that such information is of sufficient quality for SDG reporting? Or indeed for a wider use within government, for anyone in an official setting wishing to use non-official sources.
What is the solution?
The SDG data team at ONS has designed and published a standardised assessment protocol to help assess the quality of non-official sources. This tool informs decisions on the SDG suitability of the non-official source. It seeks to strike a balance between harnessing benefits while managing the risks and ensuring sufficient trustworthiness, quality, and value. Doing so while considering the UN’s requirements for the indicator. This protocol builds on great work done previously by Statistics Netherlands.
The protocol has two stages. First, the source must pass the Quality Gateway, by meeting three criteria: ethics and privacy, transparency and accountability, and need.
Next is the Quality Matrix scoring stage. This has six criteria: relevance, methods, coverage, timeliness and ongoing availability, data journey awareness, and quality assurance. Each of these is scored between 0 (not acceptable) to 3 (the highest score). A source with an average score above 1.5 and no 0 scores – cross-checked by a second assessor – is deemed suitable for use in SDG reporting. Sources will also be reassessed when they are updated, which is usually done once a year.
So what happens if a source fails? The process allows the SDGs team to understand the weaknesses by looking at which criterion led to a fail or had the lowest score. They can then engage with the provider of the source to see how it can be strengthened and potentially pass a reassessment.
You can find out more about the protocol and the details of the criteria in their publication, UK Sustainable Development Goals: use of non-official sources.
What happens next?
The new protocol for assessing non-official sources is very much a work in progress. The SDGs team will be improving it as it is used and based on feedback. They welcome your thoughts on their approach, and encourage people to adapt it to their needs, even if they are not directly involved in SDG reporting. Please let them know if you do!
They are also looking at extending the protocol to cover non-official qualitative sources. These could be journal articles or other reports. They could use these to temporarily fill gaps as they search for official or non-official statistical sources. They might also offer useful contextual information for those indicators already being reported. This work is underway, so please let them know if you want to know more by emailing SustainableDevelopment@ons.gov.uk.