Publishing experimental statistics

Policy details

Introduction

Experimental statistics are a subset of newly developed or innovative official statistics that are undergoing evaluation. Experimental statistics are developed under the guidance of the Head of Profession for Statistics. They are published to involve users and stakeholders at an early stage in assessing their suitability and quality.

The regulatory guidance document Experimental statistics – official statistics in development sets out the United Kingdom (UK) Statistics Authority policy on this topic.

Experimental statistics are, by definition, also official statistics. The Statistics and Registration Service Act 2007 refers only to official statistics and makes no mention of experimental statistics.

It is the sole responsibility of the producer body (with the guidance of the Chief Statistician or Head of Profession for statistics) to decide when to label a set of official statistics as experimental and when to remove that label. The Office for Statistics Regulation are happy to discuss specific situations with producers as part of the decision making process, but have no formal role in making the decisions.

In developing experimental statistics, producers should build towards full consideration of the standards set out in the Code of Practice for Statistics.

National Statistics and experimental statistics

Experimental statistics development is applicable to and valuable for National Statistics. When National Statistics are redeveloped, they must be re-labelled as experimental statistics.

Producers must obtain permission to remove the National Statistics label from the Office for Statistics Regulation prior to making the change. National Statistics designation can only be removed with the approval of the regulator.

When to use the experimental statistics label

Producers must use their judgement to decide when to use the experimental statistics label for a new output or a modification of an existing output. Consider the following criteria:

• there is a defined and time-limited work programme, either to develop new statistics or to improve existing outputs, and the output falls within the remit of that programme
• you are developing new statistics that have considerable immediate value to users; users are aware of the statistics’ quality and limitations and can make qualified use of them before operational testing has been completed
• the statistics remain subject to testing of quality, volatility and ability to meet user needs
• new methods are being tested and are still subject to modification or further evaluation
• there is only partial coverage (e.g. of subgroups or regions) at this stage of development

Introducing experimental statistics

When introducing experimental statistics, producers should:

• explain what the experimental statistics are
• describe how they might be useful
• describe why they are experimental – what is new or different about them – and how they complement existing official statistics
• explain how and when feedback will be sought and what feedback the producer would like
• include a statement in releases and metadata explaining what the experimental statistics label means
• set out when the producer expects to be able to remove the experimental statistics label

Whenever possible, producers should also say whether and when they plan to request the UK Statistics Authority to assess whether the statistics comply with the Code of Practice so that they can be designated as National Statistics.

Removal of the experimental statistics label

The experimental statistics label is temporary. It can be removed when a producer considers this to be appropriate, which may be as part of a planned transition to designation as National Statistics.

It is up to the producer to decide when it is appropriate for the experimental statistics label to be removed. Factors to consider before removing the label include:

• whether user feedback indicates that statistics are trustworthy, of high quality, and valuable
• whether the methods employed have proved sufficiently robust to suit the variety of circumstances material to the use of the statistics
• whether coverage has reached a sufficient level
• whether the defined development phase has ended
• whether it is judged that the statistics fully comply with the standards of the Code of Practice

Statistics assessed as fully compliant with the Code of Practice are, by definition, no longer experimental. As with all official statistics, a producer can ask the UK Statistics Authority to assess whether experimental statistics comply with the Code of Practice in order that they can be designated as National Statistics.

The Authority may also decide that it is appropriate to assess a set of experimental statistics and will notify the producer accordingly.

Producers may withdraw a set of experimental statistics if they do not meet user needs or are otherwise not fit for purpose. If experimental statistics are to be withdrawn, the Head of Profession or Lead Official should decide who to notify and whether to escalate changes of scope to the National Statistician.

The guidance on changing or ceasing to publish official statistics provides further details.