Zeroing in on the Code: Enhancing statistics to monitor our “BUS” journey towards net zero

The net zero challenge

The UK is committed to reaching net zero by 2050. This means that total greenhouse gas emissions would be equal to the emissions removed from the atmosphere, aiming to limit global warming and the resulting climate change.

In order to meet the UK’s target, the heating of buildings will need to be decarbonised. The Boiler Upgrade Scheme (BUS) aims to incentivise and increase the deployment of low-carbon heating technologies by providing an upfront capital grant towards the costs of installing an air source heat pump, a ground source heat pump and, in limited circumstances, a biomass boiler. The scheme launched in April 2022 and has just completed its third year.

Here we share our journey in developing a new set of statistics to monitor the BUS, labelling them as “in development” and then subsequently going through a process to remove that label.

Monitoring the scheme

As the scheme was rolled out, there was demand from a range of stakeholders for statistics on its uptake. So, we put the (BUS!) wheels into motion to develop a new monthly official statistics series. Throughout the development of the statistics, we kept the Code of Practice for Statistics in mind to help us provide a better service to our users. In the initial design of the statistics, we held workshops with internal policy and analytical colleagues to identify the most useful content. We tested different options and highlighted their trade-offs, e.g. more granular breakdowns potentially leading to loss of information where suppression is then needed on small counts.

When we launched the statistics in August 2022, as they were a brand-new series we categorised them as ‘official statistics in development’. By definition, these are official statistics that are ‘undergoing a development; they may be new or existing statistics, and will be tested with users, in line with the standards of trustworthiness, quality, and value in the Code of Practice for Statistics.’

We marked our statistics as being in development for a few reasons. For one thing, our input data was data from a third party – the administrators of the scheme, Office of Gas and Electricity Markets (Ofgem) – and primarily collected for operational purposes. In the early days of the scheme, the data collection involved lots of manual processes which are prone to error and we didn’t feel that we had a good enough understanding of the data quality, the implications or how it might evolve over time. Secondly, whilst we had invested time in developing what we considered to be a robust production process, we hadn’t had the chance to test it out on real data.

Using the “official statistics in development” label, allowed us to inform users about the uptake of BUS. It also gave us the opportunity to get feedback from a wider range of stakeholders; letting us test if we were providing them with sufficient information to make informed judgements on the scheme. Behind the scenes, it allowed us to undertake further methodological development of our processes; test and validate the input data quality; and incorporate user feedback in developing our content.

Moving on! (from “in development” to “official statistics”)

We’d wanted our statistics series to move to regular official statistics for a while, but struggled to know where and how to start. We’d done lots of work on developing and automating our production processes, improving our quality assurance checks, and taking user feedback on board. There had also been improvements in the collection of the data through increased digitalisation of the application process, which had improved the quality of our input data.

We sought advice from other statisticians, as well as from the Office for Statistics Regulation (OSR). After reviewing the OSR guidance on producing official statistics in development, we agreed an approach with our department’s Head of Profession for Statistics. This entailed the production team carrying out a thorough internal assessment of our statistics. We did this by going through each pillar and principal of the Code of Practice and evidencing how we met it (or didn’t!). This was all documented and included justification around why we had initially chosen to launch our statistics as “in development” and why we now felt the time was right to remove that label.

It was both reassuring to see that the statistics were largely compliant with the Code of Practice, and satisfying to summarise everything we had achieved and how far the statistics had come. Our assessment was then reviewed and signed off by the Head of Profession. We agreed how we would notify users in advance of the change in the status of the statistics so that they had the opportunity to raise any concerns and provide further feedback, as well as agreeing how we would communicate when we were no longer classifying the statistics as “in development”.

What did we take away from the process?

Here are our top tips for statistics producers who are graduating from the “official statistics in development” label:

1. Incorporate best practice from the start

When releasing new official statistics in development, doing what you can to incorporate best practice from the outset will make for a better user experience and also save time in the long run when it comes to removing “in development” labels. This means following the principles of the Code of Practice for Statistics, as well as key Analysis Function guidance on things like accessibility and releasing statistics in spreadsheets.

2. Stakeholder engagement is key

Speak to everyone you can! In preparing to remove the “in development” label from your statistics, gather knowledge from the likes of the Office for Statistics Regulation as well as fellow statisticians who have already been through the process. Even though we did get some really helpful suggestions from external stakeholders for further statistics to include, one takeaway for us was that we could have benefited from even more engagement with that group – so be proactive about reaching out and asking them whether your statistics are meeting their needs.

There isn’t a one-size-fits-all approach to removing the “in development” label, so establishing good relationships with a variety of stakeholders is crucial for knowing how, when and why you might remove the label, and agreeing an appropriate approach.

3. Keep learning!

The environment we work in as statistics producers is constantly changing, so we need to adapt with it. This means keeping abreast of changes in the Code of Practice and Analysis Function guidance. But it is also critical to cultivate a deep understanding of the policy landscape and your users, so that you can provide them with relevant and impactful statistics. For us, doing all of this meant that when it came to removing the “in development” label, it was almost just a formality, and there wasn’t a lot of further development required to enable us to progress to official statistics.

We’re very happy to speak with other government statisticians who are going through the process of removing their “in development label”, so please do get in touch. And check out our Boiler Upgrade Scheme monthly statistics.