Methodology division at the Office for National Statistics

The census includes everyone doesn’t it? Well, it doesn’t quite achieve that. Every effort was made to ensure everyone was counted in the 2011 Census, however no census is perfect and some people were inevitably missed.

The methodology division developed a statistical methodology which measured this ‘undercount’ of the population and made sure that the statistics published from the census represented everyone. This statistical methodology produced the key population estimates from the 2011 Census which are used as the basis for all population statistics in the ten years following the census.

Find out more about the Census coverage assessment.

For the last five decades, the ONS has undertaken a census non-response link study alongside the ONS Census for England and Wales. This study has linked data from social surveys administered around census day, with census data, to allow for a test of potential non-response bias in the social survey data.

The ONS link the addresses of responding and non-responding households in the social survey datasets to the same addresses in census dataset. They are then able to use the rich census data to explore the demographic characteristics that explain the profiles of responders and non-responders.

The ONS is again planning to undertake a social survey and census data linkage project alongside the 2021 Census, with a set of ONS social surveys identified for inclusion in the first phase.

These analyses will allow us to consider our non-response models and inform our future transformation of social surveys. The linked dataset will also be a valuable resource for our colleagues in the census team, as part of the evidence base for understanding census coverage and quality.

Find out more about the 2021 Census non-response link study.

New data sources and methods are being introduced into the production of consumer price statistics from 2023. The new data sources, namely web-scraped and scanner, have the potential to improve the quality of UK consumer price statistics through increased coverage and more timely data.

To be able to calculate a price index we must know which category each product in the data belongs to in line with the ONS product hierarchy e.g. women’s dress, men’s shirt. The volume of this new data means we can no longer manually assess each product. Instead we are researching using machine learning to automatically classify products.

Using a mixture of natural language processing (NLP) and classification algorithms (such as XGBoost) we have developed a prototype classification method for clothing data. Once classified the data are able to move through the other stages of calculating a price index. This research will now be broadened to apply these methods to other price datasets such as groceries.

Find out more about the automated classification of web-scraped clothing data in consumer price statistics.

Methodology were involved from the outset of this fast-paced development. As the Government needed timely data of the precise impact of the pandemic on UK businesses, administrative data sources – neither timely nor specific – were not suitable, so a traditional survey, in which ONS excels, was decided upon.

Instead of taking months to develop and design, BICS was in the field collecting data in a matter of days. This was only possible due to questionnaire design experts helping design, test and route questions – this has continued throughout the survey, as questions are added/deleted regularly, and the more data we collect the more we can refine the electronic questionnaire.

The design of the sample, the data cleaning methods, and the estimator used took a little longer – initially the design was business-area led, and resulted in a purposive sample which could only be used to provide information on the businesses that responded, not the whole economy. Innovative imputation methods developed by methodology were implemented quickly to help with the high non-response rates. The methodology redesign of the survey to make it representative of the whole business population, and allow inference from results about the impact on the UK economy not just the sample, took longer to be implemented – although it was developed at the outset.

Latest developments have involved methodology developing measures of precision around estimates, so that standard errors, coefficients of variation and confidence intervals can be published around key variables collected.

Business insights and impact on the UK economy

Business insights and impact on the UK economy – detailed datasets

Business Impact of Coronavirus (COVID-19) Survey: preliminary weighted results

Business impact of COVID-19 survey results

Methodology have been involved throughout this long running project, from questionnaire design and testing to currently providing the methods and support for the transformation to the new processing pipeline. The new DTrades output will bring together short-term output indicators for Retail, Motor Trades and Wholesale industries (standard industrial classification divisions 45, 46 and 47) as part of a large-scale Business Statistics Transformation programme.

Historically, short-term output from these industries have been measured using the Retail Sales Index (RSI) and Monthly Business (MBS) Surveys. These surveys are being transformed such that data will be collected electronically under the newly designed Monthly Turnover Survey, to facilitate the production of the new DTrades output.

As key aspect of this transformation work programme, the DTrades output will incorporate the use of both survey and administrative His Majesty’s Revenue and Customs (HMRC) Value Added Tax (VAT) data. Methodological research has taken place to determine the ‘partition’ between survey and VAT data.

Currently working closely with IT colleagues, we are delivering on five improved methods: Estimation, Selective Editing, Date Adjustment, Winsorization and Imputation. When completed these generic methods will be available through the Statistical Methods Library (SML) for use throughout ONS and wider government areas.

Find out more about DTrades in survey methodology bulletin no. 79.