Statistical Disclosure Control for tables produced from administrative data

The scope of this guidance

This guidance outlines the main steps to be taken in considering disclosure control, while allowing different solutions to be developed for different datasets, taking into account detailed risk assessment and the latest disclosure control methods.

Risk assessment for data in tables

One major distinction between the risk assessment of tables created from survey data and tables created from administrative data is the consideration of ‘response knowledge’. This refers to the consideration that someone looking at the data may know a particular person, household or business is in the dataset.

In surveys, the sample size is usually a small percentage of the population, and someone looking at the data may have private knowledge that a particular person, household or business has responded. The cell counts are more likely to be small, which introduces a high risk that a person, household or business could be identified in the data.

Administrative data usually include most of the eligible population, and one could assume that an eligible individual is in the dataset. Though the cell counts are usually high, any small counts can lead to a high risk that a person, household or business could be identified.

Government produces a very large amount of administrative data, providing a valuable resource if it can be used correctly. There are legal gateways which can allow accredited and approved researchers to access administrative data for research and statistical purposes. There are certain criteria to meet to ensure this can happen, including the assurance that a person’s identity cannot be identified in the information provided for research and statistics.

Administrative data are generally not collected for the sole purpose of producing statistics.

This guidance provides an overview of disclosure control techniques for all administrative data. Administrative data covers a wide area and can be collected from:

• people or households — such as crime data, or benefit data
• businesses — such as VAT transactions

This guidance concentrates on data collections from individuals and households.

Specific disclosure advice is available for outputs based on births and deaths registrations.

Legislation and governance for outputs published from administrative data

As with survey data, there is legislation that applies to data published from administrative data sources. The legal underpinning to the release of these data is the UK Data Protection Act (DPA) 2018, which is the implementation of the UK General Data Protection Regulation (GDPR).

This Act controls how personal data are used by organisations, businesses, and the government. The government is legally bound to ensure the information collected is used fairly, lawfully and transparently.

The 2018 Act includes the following topics of personal data defined as sensitive:

• racial or ethnic origin
• political opinions
• religious or other similar beliefs
• Trade Union membership
• health
• sexual life
• biometric and genetic data
• commission or alleged commission of an offence
• proceedings for any offence committed, or alleged to have been committed

As can be seen, sensitive personal data as defined by the DPA are wide ranging. Although information on income, wealth and financial affairs are not mentioned specifically in the Act, there may be some situations in which they could also be considered sensitive. There is little detail in official guidance as to how personal data is classified, but the variables previously described in this section are useful attributes to consider when producing tables.

The Freedom of Information (FOI) Act also can be invoked when specific pieces of information (such as a single count) are requested. These can be investigated more thoroughly than standard publications with large numbers of tables, often through use of intruder testing.

For the Office for National Statistics (ONS) and some other government bodies, current policy is for responses and released outputs from all FOI requests to be made public. This means the approaches to be followed when ensuring the confidentiality of outputs should be broadly the same as for general requests. There are several exemptions whereby FOI requests can be refused. For ONS, one of these is where the data are defined as personal information under the Statistics and Registration Service Act (SRSA) 2007 (section 39 of the SRSA, section 40 of the SRSA, and section 44 of the FOI Act). For non-ONS outputs, there may be other legislation that applies (section 44 of the FOI Act). There is also an FOI exemption that relates to personal data (section 40 of the FOI Act).

The Code of Practice for Statistics provides producers of official statistics with the detailed practices they must commit to when producing and releasing official statistics. It states that there is a legal obligation under the SRSA to ensure compliance with the Code of Practice in relation to Accredited Official Statistics (section 10 of the SRSA and section 11 of the SRSA).

Common law allows people to bring legal proceedings if the collection, use and disclosure of personal information breaches the obligation of confidence. Under the common law, anyone who receives confidential information must not disclose it without consent or justification. The common law duty of confidentiality extends to confidential information about people who are deceased.

Ultimately the Information Commissioner is responsible for ensuring good practice for publishing data is followed. They are likely to have a view on any refusals to publish on confidentiality grounds. They will need to consider the practical day-to-day aspects of disclosure control alongside the legal considerations.

The variables defined in the DPA as sensitive would likely be of high impact to a person if disclosed. In general, disclosure risk is a constant problem when personal data are released. Risk can be defined as the likelihood of a person, business, household or other statistical unit (or related attributes) being identified in a published output. The impact of any disclosure is increased if the risk involves particularly sensitive variables that could cause excessive and unnecessary harm or distress to particular individuals or groups. Disclosure of sensitive variables is also likely to be high profile and lead to considerable damage to the reputation of the data producers.

It is important to note that the term ‘impact’ refers to ‘statistical impact’. This is the effect disclosure of information will have on the person, business or household concerned. This is different to ‘reputational impact’, which occurs when a dataset is lost in transit or data are released inadvertently.

Levels of risk can be defined for outputs from administrative data by considering the geographical level of the table (or the population at risk) and the sensitivity of the variables within the table. Whether low cell counts are considered to have a potential for disclosure will depend on the level of risk associated with the table.

The sensitivity of variables is a subjective area, and the relationship between disclosure risk, variable sensitivity and the population at risk can be hard to determine. The main thing to think about is how an intruder would approach a table containing low frequencies. The data supplier has the greatest knowledge of the data, which means they are in a good position to assess how an intruder would operate. If an intruder considers the variables to be of great personal or political interest, they may invest more time in attempting to find an individual or an attribute relating to a person in the table.

Low counts in the margins of a table are a particular issue. This is not just because the row or column contributing to the margin will consist of small counts, but also because the marginal total will highlight the small number of contributions to one defined variable subgroup. This could encourage an intruder to investigate the data further.

For frequency tables, cells with low counts are likely to be the major problem. An uneven spread of frequencies throughout the table could also be concerning. In general, cell counts of 0 and 1 and rows or columns where the frequencies are concentrated into a small number of cells are the most problematic. Case Study 2 contains relevant discussion on issues around low counts.

Low risk

For some administrative based statistics, the likelihood of an identification attempt may be considered low if tables are disseminated at a high level of aggregation (such as national or regional level), with only limited tables produced from the one database. In these cases, there is usually a very small risk associated with linking between current and future releases. Statistics in this category will not usually need any protection beyond appropriate selection of variable categories and combining them where necessary. However, care should be taken where rows or columns are dominated by zeros, especially where a marginal total is 1 or 2.

Medium or high risk

Cells of size 1 or 2 may be considered unsafe for many administrative based statistics disseminated at a lower level of aggregation (for example, at small geographies or for small populations), or where many linked tables are produced from the same dataset. Care should also be taken where a row or column is dominated by zeros.

There is a similar recommendation for high-risk outputs where identification attempts may occur frequently. The impact of any successful identification would be significant for these kinds of outputs, an example being statistics on abortions. To ensure these outputs are protected, all cells of size 1 to 2 are considered unsafe and care should be taken where a row or column is dominated by zeros. High risk tables should also be closely examined if they are especially sparse, for example, if they have a low cell average and contain many zeros. Higher levels of protection may be needed for small geographical levels or for certain variables with an extremely high level of interest and impact.

Zeros

The risk levels previously discussed refer to the importance of zeros. The risk of disclosure will be greater if zeros are distributed in particular ways. If all cells in a row or column are zero apart from one, an intruder would know that all members of the row belong to a single category for the column variable. This is a form of group disclosure.

The distinction between structural and non-structural zeros is also important. Structural zeros are those where the counts cannot be anything other than zero. An example would be an impossible set of characteristics, like a mother who is 8 years old. Non-structural zeros occur because nobody with that combination of characteristics happens to be present in the population, though it would be perfectly possible for someone with those characteristics to exist. An example of a non-structural zero could be if there were no mothers aged 15 in a dataset. It is feasible for a 15-year-old mother to exist, but there just happen to be none in the data in this example. Risk in tables with zeros is generally determined by cells which contain non-structural zeros.

Population at risk

Population at risk refers to the underlying number of people that could be present in a particular cell because they share characteristics with other people in a table. The likelihood of disclosure will increase as the population at risk decreases. This means that for a smaller population at risk there is greater possibility of disclosure. For example, if a cell represented the number of women aged 18 to 34 who had given birth in a Local Authority, the population at risk would be all women aged 18 to 34 in that Local Authority.

A table is ‘unsafe’ if it contains one or more cells with an unacceptable risk of disclosure. Disclosure control methods should be used to reduce the risk by modifying these cells. Careful judgement will be required when applying any method to ensure that the most important messages are still valid.

Uncertainty

The aim for most outputs is to introduce enough uncertainty into the data to give an element of doubt to any possible disclosure. This is an alternative to the case of zero risk, where the data are protected to the extent that disclosure is not just highly unlikely but impossible. This may be a requirement for some outputs, but the changes necessary to achieve this can affect the usefulness of the data considerably.

Uncertainty relates to the extent to which cell values and apparent attribute disclosures in tables may not represent real respondents or real attribute disclosures. Published cell counts may not represent the true counts for a variety of reasons. Some of these reasons cannot be easily quantified, such as respondent error or data capture error, or mis-classification. Some can be more easily quantified, such as non-response and ‘edit and imputation’, or pre-tabulation or post-tabulation adjustment as part of disclosure control.

Factors to consider in setting the level of uncertainty include the amount of sensitivity in the data and the likelihood and impact of a real disclosure claim.

Assessing the disclosure risk

There is no single, uniform approach to disclosure protection. Context is very important when you are making decisions on whether outputs pose a disclosure risk.

Determining the users’ requirements

The government produces a wide range of statistics based on administrative sources. Examples include tables of statistics concerning crime, education, benefits, and health.

Producers of statistics should design publications according to the needs of users, as a first priority. It is vital to identify the main users of the statistics and understand why they need the figures and how they will use them in detail. Read more about understanding user needs.

This is necessary to ensure that the:

• design of the output is relevant
• amount of disclosure protection has as little effect on the usefulness of the statistics as possible, while still being sufficient to protect confidentiality

Considering the characteristics of the data and proposed tables

The source of the data may affect the need to protect confidentiality. Sensitive variables may need special attention.

The age of the data may reduce the risk of disclosure. This is because the population of the statistics will change over time and may become less identifiable.

The quality of data may determine the extent of the need for disclosure protection. Data quality can vary considerably for many reasons, such as:

• poor population coverage
• poor information recording during the data processing stage
• a need for a large proportion of imputed data
• respondents not giving sensitive information accurately

Poor data quality may introduce uncertainty for an intruder.

Typically, statistical units are defined as people, households, or businesses. It is important to assess which units are represented in the data and need to be protected. Disclosure risks may also increase if groups of statistical units (for example, people from the same household) are represented in a table. This is because these people could be able to identify each other.

The disclosure risk for event-based data will be different than residence-based data. For example, to identify a person in a table for patients visiting a health clinic, you would need to know that the person is included in the population base for the table. In this case, this would mean knowing that a particular person has attended the clinic. The risk reduces if the population base or coverage of the table is not easily identifiable.

It is also important to consider the characteristics of the tables. Where tables are very simple and presented at a high level of aggregation (including geography), disclosure issues are less likely. However, even at a high level of aggregation, small cell counts can be a risk, and the underlying population or sub population should be considered. When tables become more detailed and the counts in individual cells are small, the risk of identification may increase and protection may be needed. If the spread of values is skewed across a table, the risk in particular cells may increase above an acceptable level.

Issues may arise with linked tables where the risk of disclosure can increase by differencing or through combining with other data. Differencing occurs when two or more tables can be considered together to deduce the value of a potentially disclosive count. For example, this may occur:

• when tables are produced from the same dataset for two non-coterminous geographies (that is, areas with overlapping boundaries leading to small areas or slivers) — for example, Wards and Super Output Areas (SOAs)
• where classifications differ only slightly — for example, a table that includes children aged 0 to 4 and another that includes children aged 0 to 5

Likely disclosure risks

A risk assessment should be undertaken to develop suitable confidentiality protection. This assessment should include factors such as the:

• nature of the variables — for example, considering the risk and impact if a disclosure were to occur
• structure of the table, meaning the number of observations in the table and their distribution

Disclosure can be quantified in terms of both risk and impact.

Disclosure risk is high when a table is designed so there are cells in the table with low frequencies, or when there are rows or columns where all the counts are in a small number of cells. Tables such as these could lead to identification of a person and maybe disclose further information about them.

The impact of any disclosure is higher if the data are sensitive and where great distress may be caused by releasing the data. For example, if a table of high risk was released giving details of treatment for mental health issues in a small geographical area, it would have greater impact than releasing a table of equal risk detailing use of local shops, as this information is much less sensitive.

Decisions on risk and impact should be made by someone with a detailed understanding of the statistics and the user need. To be explicit about the disclosure risks to be managed, you should consider a range of potentially disclosive scenarios and take action to prevent them. The risk assessment should be reviewed on a regular basis as the tolerable level of risk may change over time. The scenarios should be used to identify the specific cells that could lead to disclosure, known as ‘unsafe’ cells. Appropriate confidentiality rules should be applied to these cells. It is not possible to protect against all risks, so it is important to remember this is a risk management not a risk elimination exercise.

General attribute disclosure

Someone with knowledge relating to a statistical unit could, with the help of data from the table, discover details that were previously not known to them. Attribute disclosure includes inferential disclosure, where information about a statistical unit can be deduced, or inferred, with a high degree of confidence.

The possibility of attribute disclosure can be a concern if:

• there is a count of 1 in a marginal total – this could be a row or column total
• the table is distributed in such a way that one or more rows or columns are dominated by zeros

This allows an intruder to work out attribute X if they already know someone in the data with attribute Y. A zero in population data allows us to say that no-one in the population has that combination of attributes, which may narrow down the possibilities for intruders. Attribute disclosure can also occur if there is a count of 2 in a marginal total. One of the people included in the marginal total could identify the other person and potentially discover something new about them.

Disclosure risks may increase where groups of units who appear in the same table know enough about each other to identify each other and potentially discover something new. This can occur where units share characteristics or are grouped in some way. Examples could be people from the same household, or a table population that consists of a group of people who are likely to be aware of each other.

Overall, general attribute disclosure can occur in tables which are not well populated and do not have evenly distributed counts. The data owner must determine whether the sensitivity of the attribute requires the application of disclosure control.

The motivated intruder

Data in a table can be combined with information from other sources to identify a statistical unit and disclose further details. The level of disclosure risk that is tolerable may depend heavily on the sensitivity of the data. This situation may occur when small values are reported for some cells.

Although other sources can reveal the identity of the individual, it may be the statistics that motivate the intruder to start looking and attempting to reveal what is disclosive. Official statistics should not reveal the identity of any respondents. The risk of disclosure should consider other relevant sources of information which can be combined with the statistics. The analyst does not need to consider all data sources, but should consider data which is likely to be available to third parties.

As the base population is decreased by moving to smaller geographies or sub-populations, it becomes easier to find units and discover information. It will also become more likely that an intruder will have greater confidence in any claim they might make. In a large population (for example, a country or region), the effort and expertise required to discover more details about the statistical unit may be deemed to be disproportionate for the intruder.

To protect against a motivated intruder you should, at a minimum, consider all cell counts of 1 or 2 to be potentially disclosive for geographies:

• below Local Authority District (LAD) level – these typically range from 41,049 to 1,144,919 in England (excluding the Isles of Scilly and City of London), based on Census 2021 data
• below Integrated Care Board (ICB) level – these range from 52,131 to 3,146,943, based on the registered population in 2022 to 2023

You should also consider all cell counts of 1 or 2 to be potentially disclosive for geographies equivalent to the size of LADs and ICBs in England.

The threshold value of 3 is chosen here to ensure that there are no instances in a well-defined geography where there is either:

• an isolated individual
• a member of a cell of frequency 2 that may attempt to identify the other person in the cell

Larger cell values are likely to discourage an intruder, as well as increasing the difficulty of disclosure.

As a general guideline disclosure risk increases with smaller geographies.

Identification and self-identification

Where a cell has a large value, risks arising from identification are not usually significant. More consideration is needed where a cell has a small value, particularly if the count is 1. This is because identification or self-identification can lead to the discovery of rareness, or even uniqueness, in the population being considered. For certain types of information – especially information of a sensitive nature – rareness or uniqueness may encourage others to seek out the person in the data. The threat or reality of this could:

• cause harm or distress to the affected person
• lead the affected person to claim that the statistics are inadequate to protect themselves or others

The same is true of cells with a value of 2 representing two units, where one of the units contributing to the cell may identify the other. This could occur when groups of people or organisations with similar characteristics who know enough to identify each other appear in the same table. An example could be people from the same household.

To protect against unique identification or self-identification, you should consider – at a minimum – all cells of size 1 or 2 to be unsafe at all but the highest geographical levels. Although direct identification or self-identification may not be a significant risk, protection is often required since identification can lead to attribute disclosure when more than one table is disseminated from a data source. The identified person in an internal cell of a table can become a marginal cell in another table and a new attribute could be learned.

There are no formal rules to follow when publishing tables but if these disclosure risks are considered, the resulting outputs should be of both useful and low risk.

There are some occasions where specific outputs will need to be assessed differently to determine the disclosure risk.

Mosaic, or jigsaw effect

A more detailed scenario to consider is that of the mosaic, or jigsaw effect. This is where different data releases – each of which may be deemed safe in isolation – can be pieced together to effect disclosure. Another example is where there are multiple releases from a similar collection over time, perhaps for successive months or quarters. Change between each release may be small, which may make it easier to identify people in the data.

As we see a growth in publicly available sources, including social media, this risk has increased considerably.

Assessing the risk

Start by defining the variables carefully. How sensitive are the variables? What would be the impact of disclosure?

Then, think about the order of release. Are the tables released together or at different times? Are they ad-hoc requests to the same customer? This is where a log of releases would be recommended. A log of releases would allow data analysts to have an easily accessible record of previous releases so they could see which combinations of variables (and categories) have been published previously or made available to the same user. Consider whether a new publication could cause disclosure issues. Remember that releases to the same customer would be likely to be more of a problem although the legal position is the same as releasing to different customers.

Consider whether there are other datasets available in the public domain with some of the same variables. They may be collected at different times, through different modes and have different measurements, but will still add a little to the overall disclosure risk.

Creating and releasing tables

If a defined set of tables has been released previously, users are likely to expect they will be published to the same specification in future releases. Users may complain if the publishing strategy is changed, so you may want to consider this when you are selecting the disclosure control method.

If many tables are to be released from a particular dataset, it may be advisable to protect the underlying microdata before creating your tables. This helps address any disclosure by differencing. Read more about protecting microdata created from social surveys.

Flexible table builders are becoming more of an option for users who want to create their own tables. Users can use them to download many tables from the same microdata.

One example is the create a custom dataset (CACD) tool developed for Census 2021 in England and Wales.

There are multiple disclosure issues with flexible table builders. The main challenges are:

• univariate uniques — this is where there are single counts in a requested table with only one variable
• differencing
• ensuring disclosure risk has been reduced to an acceptable level
• how a flexible dissemination system affects microdata releases

Read Stephanie Blanchard’s paper about the methodological challenges of protecting outputs from a flexible dissemination system, published in the Survey Methodology Bulletin (number 79)

Population at risk assessments

The effect of geography on disclosure risk has already been discussed in this guidance. This can be expanded to consider risk for different population sizes alongside variables of differing sensitivity.

You should also consider the safety of frequent releases of the same measures, especially those that are supplied daily rather than monthly, quarterly, or annually. There are likely to be small fluctuations between each release, which means the disclosure risk could be high – even where the population at risk may also be high.

You should carry out a risk assessment exercise to develop suitable confidentiality rules for different datasets. In practice, it is likely that producers of statistics will find that risk levels for outputs from the administrative data can be defined by the population at risk with a further breakdown by the impact of any identification. This means considering whether the output contains any high, medium, or low impact variables.

Decisions on the likelihood and impact of identification should be made by someone with detailed knowledge of the data. When deciding on the risk level and level of protection required, thought will need to be given both to the:

• likelihood of an identification (based on population at risk)
• impact that an identification would have

Tables with sensitive defining variables will almost certainly have a greater impact if an individual is identified correctly.

Impact through the release of specific combinations of variables (or ‘statistical impact’ as described earlier in this guidance) is a subjective term. However, possible categorisations for a small number of common variables are given in this section. Some of these are the same as the broader selection of variables defined as sensitive by the DPA.

Variables with higher impact are more sensitive, meaning that any disclosure would cause a great deal of distress to the person concerned.

High impact variables

These may include:

• income or wealth — any financial variables are generally considered to be high impact
• ethnic origin
• religious beliefs
• physical or mental health
• some types of crime — these variables are high impact for victims
• sexual orientation
• gender identity
• economic activity — especially responses of ‘unemployed’ or ‘never worked’
• industrial classification — this variable is considered to be high impact at the most detailed level
• occupation
• qualifications — especially level of qualification and details of the subject studied

The lower the level of geography at which a table is released, the greater the level of risk for these variables. At local area level (such as Census Output Area), it is possible that variable combinations, including one or more of the variables in the ‘high impact’ list, will be unique for an individual.

Medium or low impact variables

These may include:

• marital status — where this is not covered by vital registration
• single year of age
• household composition
• geography — at Local Authority, or equivalent
• industrial classification — where this is grouped into a small number of categories
• sex
• age group
• house type

The risk will be lower if the geography is at region level or above.

As previously stated, counts of 1 or 2 for tables at a relatively low level of geography need to be thought of as possibly disclosive. However, as populations can differ widely across administrative geographies, the population at risk for a table produced from a lower level of geography will not be consistent. It is difficult to give exact guidance, but one approach is to consider ‘population at risk’. This is not always easy to define conclusively but this section gives examples of population sizes. Please note these examples are indicative rather than prescriptive. The relationship between risk and impact is also considered for both high impact, and medium to low impact.

The levels of geography at which data are likely to be released (such as LAD) vary considerably in population size. This is why population at risk is used alongside the impact of a variable to define the risk of a table.

The values for population at risk are useful as a guide and suggest levels at which an intruder might be more inclined to try to identify an individual in a published table. Smaller populations at risk require more protection. However, experts in different areas of official statistics may want to choose different values for population at risk while maintaining the structure outlined in this guidance.

The likelihood of an attempt at identification and its impact may be heightened and additional protection required if:

• any other disclosive situations are likely to occur
• statistical units are represented more than once in the table — for example, if the statistical unit is a patient and the table reports annual hospital admissions, then a cell of 4 could represent the 4 times the same patient was admitted
• groups of statistical units are represented in the table — for example, people from a particular household
• tables based on the dataset have already been released — the likelihood of identification may increase due to linking and differencing with these past releases, which means protecting large databases may be important
• other freely available datasets can be linked to the tables

Legal, ethical and trust considerations

The 2018 Data Protection Act and Code of Practice for Statistics have already been discussed in this guidance. There is further discussion of legal, ethical and trust considerations in this section.

When establishing whether confidentiality protection is required for a particular statistic, it is necessary to consider:

• public trust and cooperation
• legal rights and obligations
• national and international standards for statistics

This will help determine acceptable disclosure risks and unacceptable disclosure risks.

The production and use of statistics depends on the co-operation and trust of citizens. Such trust cannot be maintained unless the privacy of individuals’ information is protected – and perceived to be protected. Failure to respect privacy might result in harm or distress to a specific individual. Sensitive personal records, therefore, need to be strictly confidential. On the other hand, there is a legitimate public interest in having ready access to aggregate or summary statistical information.

The legal framework covering the use of personal information as defined by the SRSA and DPA is complex. When such information is transformed into statistics, the legal framework is much simpler. The statistical information can be widely and freely used, provided confidentiality protection has been applied such that it is no longer likely that the information can be related to specific identifiable individuals.

When the information in a statistic does not relate to an identifiable individual (either on its own or in combination with other information likely to be available), there can be no breach of the duty of confidence owed or any conflict with data protection or human rights legislation.

Information available in the public domain is not necessarily risk-free when presented in a table or as another statistic. Statistical disclosure control methods may modify the data or the design of the statistics, or a combination of both. They will be judged sufficient when the guarantee of confidentiality can be maintained.

Statisticians or researchers should have no interest in the individual statistical unit, other than to distinguish one unit from another for statistical purposes. This could include conducting data matching or linking exercises for statistical purposes, where identified data are essential for quality reasons.

In contrast, an intruder is someone who, for whatever reason, wishes to distinguish one statistical unit, so they can treat that unit separately or differently from the other statistical units in the dataset for a non-statistical purpose. Producers of statistics ought to be aware that intruders may use other public and private sources of information in their attempt to identify an individual or other member of the published table.

Phrases related to the word ‘identity’ are used frequently in legislation concerning disclosure control and data confidentiality. The 2018 Data Protection Act defines an “identifiable living individual” as a living person who can be identified, directly or indirectly, in particular by reference to both direct identifiers and factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of the individual.

Data relating to an identifiable living individual are defined as personal data. The Act states that it is unlawful to obtain personal data without consent of the data controller.

Selecting disclosure control methods

Where required, disclosure control methods can be used to reduce the risk by modifying the unsafe cells. The choice of method must balance the uses to be made of the information and simplicity of approach. The method chosen should ensure that the released data is of high utility and disclosure control is applied appropriately. The data should not be over protected or under protected.

Table redesign

This is recommended as a simple method that will minimise the number of unsafe cells and preserve original counts. However, the use of this method should be balanced against consistency in publication plans when initially planning the design of tables.

You can use table design to disguise unsafe cells by:

• grouping categories within a table
• aggregating to a higher level geography or for a larger population sub-group
• aggregating tables across several years, months, or quarters

The advantages of this approach are as follows:

• the original counts in the data are not damaged
• it is easy to implement

The disadvantages of this approach are as follows:

• the detail in the table will be reduced
• you may also be restricted in your redesign if there are policy or practical reasons that require a particular table design

Examples

Examples include:

• teenage conception statistics are published for Local Authority or higher level
• City of London is combined with Hackney, Rutland Unitary Authority (UA) is combined with Leicester UA, and Isles of Scilly UA are combined with Cornwall UA — this is demonstrated in the publication about quarterly conceptions to women aged under 18 years for England and Wales

If unsafe cells remain in the output tables, further protection methods should be considered to disguise them. If table redesign is not a feasible solution, the recommended method for post-tabular protection for most frequency tables is controlled rounding (which preserves the additivity of the table). However, this method requires specialist software and a linear programming solver and therefore will not always be practical. In some cases, if the number of unsafe cells is low then cell suppression can be an alternative method. Controlled rounding and cell suppression can be implemented in specialist software such as Tau-Argus.

Cell modification methods

This involves changing the cell values. Some or all of the values are different in the published data compared to the original. Methods are cell suppression, addition of noise and rounding.

Database modification methods

These methods involve adjusting the underlying microdata prior to tabulation. This is possible where a data provider has access to the individual record level data.

There are many other methods of disclosure control.

For example, an alternative approach would be to create a synthetic dataset which maintains all the properties of and relationships in the true dataset. From these data, non-disclosive tables can be created.

Equally, alternative methods for presenting data can be considered to provide users access to information without disclosing the underlying data. In many cases, this will provide a more robust analysis than reliance on the accuracy of small cell counts. These could include presenting data graphically with limited detail in scale or providing commentaries or analytical outputs.

Implementing your selected method, or methods

This guidance will allow data providers to set disclosure control rules and select appropriate disclosure control methods to protect different types of published tables of statistics based on administrative sources. The most important consideration is maintaining confidentiality, but these decisions will also accommodate the need for clear, consistent and practical solutions that can be implemented within a reasonable time and using available resources. The methods used will balance the loss of information against the likelihood of individuals’ information being disclosed. Data providers should be open and transparent in this process. All decisions should be documented, along with the whole risk assessment process so these can be reviewed.

You should think about the relationship between risk and usefulness when setting disclosure control rules for tables produced from administrative data. It is often impractical to aim for zero risk as the resulting outputs would be of little practical use to prospective users of the data. This means that any released data will have a small level of associated risk, but there will also be sufficient uncertainty that any attempted identification or attribute disclosure would be correct.

There is also a relationship between disclosure risk and disclosure impact. What is considered a sufficient level of uncertainty will differ depending on the sensitivity of the release and the impact of a disclosure. This is difficult to quantify, and decisions ought to be taken by those most familiar with the data. A higher level of uncertainty would be required when the table involves a sensitive variable or variables, typically those of high impact. Disclosure of any sensitive or high impact information could have a significant and longstanding effect upon both the affected person and the organisation.

The effect of a false claim must also be considered. For example, a false claim that relates to a person and their involvement in criminal activity could cause great harm or distress, despite being incorrect.

When looking at published statistics, users should be aware that:

• the dataset has been assessed for disclosure risk
• methods of protection may have been applied

For quality purposes, users of a dataset will be provided with an indication of the nature and extent of any modification due to the application of disclosure control methods.

Any techniques used may be specified, but the level of detail made available should not be enough to allow the user to recover disclosive cell counts.

Data providers may need to make judgements in a wider context than the specific statistics that they are producing at any one time. They need to be aware of decisions made by others within their organisation, either in the past or for similar sectors. This helps ensure decisions are consistent with the policy and strategic position of the organisation. Decisions also need to be made in the context of wider information governance arrangements both in an organisation and more widely.

Data producers should also be prepared to respond to claims of disclosure, whether these claims are correct or incorrect.

When providers share data with a second party to be published (assuming this complies with any legal or policy requirements), they must ensure the second party follows the general guidance and any specific confidentiality rules that have been developed. These should be stated in the data sharing agreements. This will ensure consistency between published statistics derived from the same source.

Any change in disclosure control rules for a published statistic raises the issue of revisions to previous releases. In general, new disclosure control rules will be implemented for future releases and the rules will not apply to past releases. An exception can be made in cases where the disclosure control rules are altered to allow more data to be released. In these cases it may be feasible to re-release older datasets with greater detail.

Statistical confidentiality is a public interest which will normally outweigh other relevant public interest in disclosing the underlying confidential records to the public. There will be rare occasions when the public interest in disclosing the records might outweigh the public interest in confidentiality. An example would be a requirement to publish details relating to a spate of crimes in a local area. Such decisions must only be taken at the highest level and in consultation with the Head of Profession. In many cases it will be found that these records are not statistics but factual information, which is subject to a different set of rules or guidance. The difference between statistics and factual information is discussed in the FOI Act.

Freedom of information requests

Under the Freedom of Information (FOI) Act ‘statistical information’ and ‘factual information’ are treated differently within the section 35 exemption. The equivalent exemption in the Freedom of Information (Scotland) Act 2002 (FOISA) is section 29. You can find guidance on the FOI exemption on the Information Commissioner’s website and guidance on the Scottish equivalent of the FOI exemption on the Scottish Information Commissioner’s website.

Put simply, factual information is the record of events or administrative actions. Statistical information is the outcome of a transformation, aggregation or analysis of such records performed using a repeatable methodology. Records of a disease, for example, are factual information, and the aggregation and analysis of those records is statistical information.

People have a general right of access to information on themselves held by public authorities through the FOI or FOISA, as long as this does not contravene confidentiality constraints. Confidentiality policy developed using this guidance can be used to help decide which exemptions in the Act are relevant, and which should be cited when withholding confidential statistical information. It is good practice to explain a general policy for the withholding of information, but this must be done in addition to – and not in place of – the exemptions in the FOI Act.

FOI requests should always be considered on a case-by-case basis. There may be cases when decisions about a case appear to be inconsistent with the general policy for the publication of statistics. This does not mean that the policy is wrong, since it has been developed for use in a production process rather than assessment of every individual cell count. Whilst confidentiality must always be maintained, a decision made under FOI to provide information in a form different to the published outputs is compatible with this guidance.

Help and support

If any problems arise when applying statistical disclosure control to tabular outputs from administrative data, please contact the SDC Expert Team by emailing SDC.Queries@ons.gov.uk.

See a flowchart of the SDC process from receiving data to output.