Analysis Function Annual Report 2022 to 2023

This year we concentrated on developing the support and guidance we provide in two main areas: ensuring analytical advice is influential and in the field of evaluation.

Influential analytical advice

We carried out research to identify the factors that enable a departmental analysis function to ensure its advice is influential. Our approach was holistic. We included perspectives from different professions, and a range of grades.

We developed a strong evidence base for what works and created a set of principles for analysts to follow to ensure that their advice is influential. We now plan to embed these principles across government, to ensure this research informs working practices.

We worked with departments to identify case studies that show how departmental analysis functions have provided influential advice. We are working with the departments to identify areas of learning and best practice, which we will share across the Analysis Function. This will enable analysts to:

• learn from each other
• share knowledge
• help build a culture of continuous improvement

Evaluation and theory of change

Our evaluation specialists provide support to colleagues across government in the use of the Theory of Change (TOC) methodology. TOC is an important way of integrating evaluation into the work of government and ensuring value for money.

This year the team published a Theory of Change toolkit, which is designed to build capability and support the development of TOC across government, particularly in developing departments’ Outcome Delivery Plans. The toolkit brings analytical rigour to the process of designing an intervention and can be used by those with little or no experience. We have received excellent feedback on the toolkit.

We ran a workshop for departmental leads who are responsible for developing their Outcome Delivery Plans. We worked with the Evaluation Task Force (ETF) and HM Treasury (HMT) to ensure TOC is being used as part of departmental planning, and provided feedback on the use of TOC in Outcome Delivery Plans.

Priorities for 2023 to 2024

We will:

• finalise and share the next batch of case studies – we will identify opportunities to embed learning into working practices across government
• explore other factors which enable analysts to produce influential analytical advice
• conduct a cross-government evaluation capability survey – this is in response to a recommendation from the National Audit Office (NAO) in their 2019 report on evaluation in government
• use evaluation capability survey data to identify gaps and deliver capability-building activity

The workstream focused on building capability in key areas and using innovative approaches to analysis. We worked with the analytical community to ensure they have access to the right tools and capabilities to deliver analysis using the latest methods.

Communicating analysis and accessibility

We made substantial improvements in the field of accessibility this year, building capability and capacity in communicating analysis. We developed tools and e-learning resources, including the AF data visualisation e-learning, which has had over 10,000 page views since publication.

We ran multiple interactive sessions, seminars, and set up communities of practice. We established monthly drop-in clinics to help analysts solve problems in data visualisation and accessibility, which proved to be very popular. Our DataConnect22 online event, focused on making charts accessible, had over 650 attendees, and over 1,000 views on YouTube.

We have also provided expertise to partner organisations. The Government Digital Service (GDS) has adopted our guidance into their products to ensure that standard government charts now use accessible colour palettes.

Reproducible analytical pipelines

We published the Analysis Function Reproducible Analytical Pipelines (RAP) Strategy in June 2022. The RAP strategy outlines how, by implementing RAP into analysis processes, analysts can improve efficiency, while also making better quality, more impactful analysis.

To implement the RAP Strategy, we developed common tools that can be used across government, including new guidance for open sourcing analytical code. We ran the annual Coding in Analysis and Research Survey (CARS) and achieved an excellent response.

We worked with departments to support them in developing their own strategies for implementing RAP processes. As a result, 5 departments have now published RAP implementation plans.

Priorities for 2023 to 2024

We will:

• build communities of practice in communicating analysis and accessibility through drop-in clinics and community forums
• bring together guidance for managers on how to manage teams to implement RAP
• share open-source tools for analysts across government
• develop an AF Innovation Strategy, which references the leading role of the Integrated Data Service (IDS), and deliver against the accompanying action plan — this includes delivering regular Innovation Showcases

This year our work has helped to set the expectations for analytical standards across government and support all analysts and departments to apply analytical standards in practice.

The Analysis Function Standard

We refreshed the Analysis Function Standard to ensure it sets clear expectations for the planning and undertaking of robust analysis. The standard gives us a common vocabulary and consistent understanding of analytical work across government. The standard provides clear direction and guidance for users and producers of government analysis, including non-analysts and external consultants, to ensure quality and consistency of analysis across government organisations.

The functional assessment framework

This year, we developed and launched a framework for organisations to assess whether they are meeting the functional standard and where they need to make changes. The objective is to help encourage continuous improvement within and across government organisations. We worked with departments to develop the framework and support them in its use.

The framework has been well received. The results helped departments to understand their strengths, weaknesses, and areas for improvement. We will use the results centrally to target additional support to help organisations.

The Analysis Function Standards Steering Group

Since its formation in March 2022, the Analysis Function Standards Steering Group has provided oversight of the Analysis Function Standard refresh and the development of the functional assessment framework. The group has also started overseeing wider guidance from across the Analysis Function, working with existing groups to ensure guidance is fit for purpose and being developed with appropriate expertise.

Priorities for 2023 to 2024

We will:

• provide support to help colleagues meet the Analysis Function standard, using the results of the assessment framework
• implement a new process for approving wider analytical guidance
• support work on updating the “colour books”, including the Aqua Book

A thriving, vibrant community with regular sharing opportunities is essential to our continuous improvement. This year, we have broadened and deepened the way analysts work with and learn from each other across government. Our activity focused on building the analytical community of the future, sharing examples of excellent analysis and decision making throughout the Civil Service.

Analysis in Government Awards

Our Analysis in Government (AiG) Awards are designed to recognise and celebrate the inspirational individuals and outstanding teams producing analysis across government. The awards recognised the impact of analysis on wider government and society, and the diversity and significance of analysis projects across government. The awards showcased excellence and offered the opportunity for government analysts to hear the experiences of their colleagues and learn from “the best”.

We received 113 high quality nominations across 6 different award categories. The nominations covered some of the most interesting and important work happening across government and represented the broad diversity of departments and professions within the Analysis Function.

The winners of the 2022 AiG Awards were announced in January 2023. We also launched our new “People’s Choice Award” event which gave members of the Analysis Function the opportunity to vote for the piece of work they were most inspired by. We received over 1,500 votes for the award.

We also hosted a new “Celebration Day” event in February 2023, which gave attendees the chance to learn more about the AiG Award winners and their work. More than 120 people attended the first Celebration Day event.

Analysis in Government (AiG) Month

The second Analysis in Government (AiG) Month took place in May 2022. The aim of the event was to support continuous improvement by:

• inspiring analysts by the best practice from across government
• ensuring analysts feel part of a vibrant community

The events in AiG Month brought together government analysts from different departments and from different professions, encouraging them to work together.

Feedback showed that 86% of attendees found the sessions to be interesting and engaging. 75% of attendees felt that the sessions provided them with new skills or knowledge to use in their work. The biggest event was a session on making spreadsheets accessible, which was attended by 409 analysts. The recording of the session has since been viewed over 2,600 times on YouTube.

As well as hosting 18 live events, we published 12 blog posts and hosted 9 Senior Leader question and answer sessions. During the month:

• 2,000 users visited the new Analysis Function website
• we gained 26,000 Twitter page impressions
• our newsletter subscriptions doubled compared to the previous month

Engaging communication channels

Throughout 2022 we continued to improve our communication channels to inform the analysis community of continuous improvement opportunities and support.

We launched our new Analysis Function website in May 2022. It provides our diverse analytical community with updates on guidance, learning, events, and all things related to the Analysis Function. The website includes different types of learning content to appeal to colleagues with a variety of learning styles.

We continued to produce a monthly AF newsletter with varied content from departments and professions. Our newsletter subscribers have increased month on month. We also continued to grow our communities across a range of social media channels including Twitter, YouTube, and LinkedIn.

Priorities for 2023 to 2024

We will:

• build on the success of AiG Month 2022 for AiG Month 2023 and beyond
• design and improve events to support continuous improvement of government analysis skills across government with the fourth annual AiG Awards
• design and improve other AiG events and communications to promote continuous improvement

The skilled people objective has several workstream sponsors, and each concentrates on a specific area:

• the Capability sponsor is Emma Campbell, Director of Analysis and Chief Economist at Department for Environment, Food and Rural Affairs (DEFRA)
• the Capacity sponsor is Dave Johnston, Actuarial Director at Government Actuary’s Department (GAD)
• the Diversity and Inclusion sponsor is Elinor Godfrey, Deputy Director, Head of Centre for Data and Analysis, Foreign Commonwealth and Development Office (FCDO)
• the Consistency sponsor is Dave Johnston, Actuarial Director at GAD
• the Community sponsor is Alex Jones, Director, Insights and Research at Office for Standards in Education, Children’s Services (Ofsted)

Building capability

This year we provided a robust learning offer that develops core analytical skills for analysts and non-analysts across government and refreshed our career framework.

We raised awareness and use of our Analysis Function Online Skills Tool among badged and unaffiliated analysts throughout government. The platform enables analysts to conduct skills assessments against aspirational roles and to access recommended learning to address skills gaps.

We refreshed the Analysis Function Career Framework to ensure it reflects the wide range of multidisciplinary analytical roles available, supporting career planning and progression in the Civil Service alongside the typical “badged” profession roles.

We began work to understand the skills, aspirations, and career trajectory of our unaffiliated analysts (roughly 5,000 colleagues). Led by HMRC and DWP, we have explored how we can improve our offer for this important population. This includes trialling the affiliation of multidisciplinary analysts through the Data Analyst role profile in the AF Career Framework.

We collaborated with the Government Skills and Curriculum Unit to ensure analysis is a core part of the future curriculum offer. This includes a focus on building analytical capability for non-analysts as part of the government reform agenda. In November 2022 we held an “Analysis for All” session for 350 policy professionals as part of a learning campaign focused on “getting things done in government”.

We developed and launched a learning pathway for non-analysts, including bitesize learning videos on core analytical concepts. We also launched a comprehensive set of learning modules to build analytical literacy.

Building capacity

Our capacity building activity has included the delivery of diversity and inclusion interventions and talent management that supports more fluid movement of diverse talent to the highest priority needs.

We delivered the 2022 to 2023 Director talent moderation process and created robust succession plans for our most senior analytical posts in government. In November 2022 we launched our first Senior Civil Service 1 (SCS1) talent approach. The talent board sat for the first time to discuss trends and possible interventions for our SCS1 community in December 2022.

This year we fulfilled commitments made in the Function’s D&I strategy. We have refreshed and relaunched our Inclusion Resources, a toolkit designed to support analysts to create diverse and inclusive teams.

We hosted events showcasing diverse leadership across the Function, with more than 200 attendees throughout the year. We also provided spaces for 14 analysts from 6 departments to participate in ONS leadership development programmes for underrepresented groups.

Building consistency

This year we have built consistency through the delivery of toolkits, frameworks, and people policies. Our aim is a more consistent employee value proposition for our analytical community.

We shared our 2022 workforce data product which gives a clear view of the current shape and scale of the cross-government analytical workforce.

We partnered with the Fast Stream and Early Talent (FSET) team in the Cabinet Office, to influence the future direction of Fast Stream 2024 and ensure the Analysis Function can make the most of this crucial early talent pipeline. We worked with all analytical professions to consider areas where we could improve consistency in the Fast Streamer experience including learning, induction, and alumni offers.​

Building community

This year we offered Function-wide networking, mentoring and knowledge sharing opportunities, connecting analysts beyond their profession and department boundaries.

We submitted a comprehensive report on the current levels of analytical capability among policy professionals to the Cabinet Functional Reform Board. This milestone represents a key commitment of the Government Reform agenda. To formally launch the analytical capability review of policy officials, we delivered a series of activities as part of AiG month. This included a findings session for 125 officials, and our “coffee connect” series, which connected 288 policy professionals and analysts.

To support delivery of the recommendations in the report, we formed an implementation plan steering group. As of March 2023, 40 out of 44 actions in the implementation plan are in progress or completed. We also set up a Function-wide Mutual Mentoring Programme, connecting 31 government analysts across 15 different departments.

We worked collaboratively to increase the learning and development capability of public sector organisations outside of the Civil Service. We worked with two public entities in Scotland: Scottish Water and Consumer Scotland.

Priorities for 2023 to 2024

We will:

• continue to drive uptake of our online skills tool and introduce a learning and skills plan to refresh our learning offer
• introduce a clear pathway for accreditation for unaffiliated analysts across government
• continue to deliver our diversity and inclusion strategy to connect analysts beyond their departments and professions, and launch “AF Links”, a programme to support under-represented groups across the community
• launch the Analysis Function Resourcing Hub to streamline best practice and guidance for resourcing activity across different professions
• produce the 2023 workforce data product to give a clear view on the shape and scale of the cross-government analytical workforce

To deliver the best analysis for the UK, our analytical leadership is becoming more responsive and collaborative. This year, we have continued to build a strong and inclusive functional culture.

Co-ordinating analytical input

The Analysis Function governance structure provides the means to co-ordinate analytical input on cross-cutting issues. Our board of Departmental Directors of Analysis (DDANs) met six times over the year, bringing together the chief analysts from each government department, supporting a functional approach to cross-cutting issues.

Part of the work of DDAN board is to agree analytical priorities and ensure we provide a timely and coordinated response to these. One example is the work by the National Situation Centre (SitCen) to implement a data strategy to improve Crisis Analysis within government. DDAN board members supported SitCen to deliver phase one of the strategy, including the creation of a cross-government network of Crisis Data Liaison Officers. The board also provided input on proposed data sharing guidance.

Our strategy

In July 2022 we launched our Analysis Function Strategy for 2022 to 2025, which articulated a vision that was relevant and inspiring to the whole Function. To develop the strategy, we spoke to leaders and analysts throughout government to understand their requirements.

To raise awareness of the Function and our strategy, we have attended events across multiple departments, professions, and network groups. This has also helped us build relationships, develop our functional culture and collect feedback on the work of the Analysis Function central team.

Monitoring our progress

We updated our annual plan, showing in detail how we will realise our strategy.

The central team provides regular reporting and progress updates to DDAN and Analysis Function boards on the delivery of our work. These boards provide challenge and accountability, and set the direction of the Analysis Function. We have also collected evidence to evaluate the impact of our function including how we are delivering efficiencies.

Priorities for 2023 to 2024

We will:

• ensure co-ordinated analytical leadership in response to the policy challenges of the day
• ensure co-ordinated analytical response to cross-government initiatives, such as efficiencies and crisis response
• provide planning, monitoring, and evaluation for the Analysis Function

The Integrated Data Service (IDS) is a cross-government service for which the ONS is the lead delivery partner. The programme to develop IDS spans several years, and the service will link data and allow rapid analysis on a wide range of integrated data assets.

The IDS is key to delivering the government’s National Data Strategy and the roadmap for digital and data from 2022 to 2025. The IDS will link data to provide insight and give policymakers robust evidence to inform and support decision making.

In 2022 to 2023, IDS operated in a beta phase of development, meaning users had access to a limited version of the service to provide feedback and enable further development. The IDS website went live in July, providing information on how government analysts and accredited researchers could access the service. The IDS added important data assets such as birth registrations and areas related to levelling up.

Sir Patrick Vallance’s Pro-innovation Regulation of Technologies Review in March flagged the importance of data sharing and encouraged departments to use IDS.

Ofsted’s Data & Insight team have developed automated reports to support inspection, adopting statistical techniques to highlight data patterns. Ofsted’s Inspectors use reports to inform their preparations for inspections of schools and Further Education & Skills (FES) provision.

In 2022 and 2023, we developed automation to maximise efficiency and reduce the time inspectors spend analysing data, undertaking user research with inspectors to identify improvements.

We worked with data suppliers to understand how to display data, leading to several developments in how we present the data, sentence wording, data caveats, and related training and guidance.

On average, two hours’ time per inspection has been saved by having performance data summarised and analysed in one place. The team’s maintenance time has reduced from 30 days per year to 1.5 days.

The Evaluation Task Force (ETF) improves how HM Government programmes are evaluated, to inform decisions on whether programmes should be continued, modified, or stopped.

The ETF has had a major impact since its inception in 2021, by March 2023, the ETF had:

• provided advice on 211 evaluations, covering £115bn of government spending

• provided advice to HM Treasury on 97 Spending Review bids and Business Cases
• delivered 31 presentations, conferences, and teach-ins to raise awareness of evaluation methods

The ETF ran an inaugural 5-day Evaluation Academy, attended in person by future evaluation trainers from 14 different departments. The ETF is also addressing barriers to transparency. The ETF, alongside “i.AI” (the Cabinet Office’s Incubator for Automation and Innovation), has developed an Evaluation Registry, which provides a unified location to amass evaluation evidence and increase the visibility of evaluations across government.

As international travel resumed after the COVID-19 pandemic, government departments needed reliable passenger demand estimates. As modelling techniques based on previously observed trends were no longer reliable, a new and innovative model was needed. Department for Transport (DfT) analysts developed short-term demand scenarios which were crucial to a wide range of operational decisions across government.

DfT analysts from different analytical professions came together to work on developing the new model. Social researchers designed and conducted surveys to identify traveller intent for all international travel modes, while economists created bespoke models. These models applied the survey data to observed 2019 passenger behaviour. The methodology was adapted over time as the policy landscape changed and DfT became more familiar with passenger behaviour at the border.

The analysis was used to plan disruption response activities following the P&O ferry situation, including provisions for haulier welfare and public communications. The scenarios were used across government and were adapted as policy landscapes shifted.

The Synthesis team and Digital Publishing team within ONS produced a cost of living insights tool, to bring together insights in an accessible format for the wider public. The tool brings together relevant ONS and Government Statistical Service (GSS) data into a “one-stop shop”, where the content is organised by theme.

Key indicators are visualised as simple charts, and commentary on the themes is provided in clear language. The tool is updated in real time and all indicators were quality assessed.

The tool achieved around 3500 weekly views, with 80% of users being members of the public, and 20% policy influencers. The team collected feedback on the benefits of tool:

• citizen users like being able to get reliable information about the crisis
• citizen users like the personalisation tools and being able to place themselves in the data
• there is still a need for this tool in March 2023, even though the crisis is changing

The tool provides welcome insight to citizens and demonstrates the ability of our analysts to respond to new challenges.

The analysis and data team in the Resettlement and Humanitarian Directorate at the Department for Levelling Up, Housing and Communities (DLUHC) set up a Ukraine Data Dashboard using a wide range of data from across government. The dashboard allowed teams across the directorate to interact with the data and stay up to date with trends such as arrival fluctuations and employment across different regions. Policy teams also had sight of the emerging data and used it to influence strategic priorities, for example sponsor rematching.

The team worked closely with the ONS to commission surveys of the Ukrainian cohort in the UK. These surveys provided rich insight into the experiences of Ukrainian refugees and their sponsors.

The data has been instrumental in understanding the barriers to employment and housing. Using this evidence, the directorate was able to secure funding for a new English for Speakers of Other Languages (ESOL) programme, to support employment and integration opportunities.

There was a cross-government effort to provide evidence to the Office for Budget Responsibility (OBR) to inform their forecast. Analysts across departments, including the ONS, the Department for Work and Pensions (DWP) and the Department of Health and Social Care (DHSC) worked together at pace. The different teams provided evidence to support policy development and design, sharing their internal analysis and academic literature.

The team’s collaboration also went across functions. Our analysts were working side-by-side with our policy, delivery, and spending colleagues. The programme of work saw evidence-based policy making and policy delivery come together as one. Working with policy colleagues from the outset ensured analysts had the time to build evidence and influence policy options, feeding into ministerial advice.

The work led to a labour market package that is forecast to get an additional 110,000 people into work by 2027 to 2028. This was noted by the OBR as their largest ever adjustment to their supply-side forecast in the medium term as result of Government policy.

The HM Revenue and Customs (HMRC) Benefits and Cross-Government analysis team helped the Department for Levelling Up, Housing and Communities (DLUHC) to understand the employment picture for Ukrainian nationals who had arrived in the UK.

A methodology was developed to get the most from Pay As You Earn (PAYE) real time indicators data and migrant worker scan data. The HMRC team presented the findings to several cross-government groups and the devolved administrations. Agreement was gained to publish the data, demonstrating transparency and making the data accessible to external organisations for the greater good.

This innovative analysis, the first on refugee or resettlement cohorts, provided insight to support policy decisions. This evidence allowed DLUHC to support Ukrainians fleeing the war to live independently in the UK.

Fraud is the most commonly experienced crime in the UK. It affects individuals and organisations and costs billions of pounds every year.

Data scientists, engineers, and policy experts came together for 3 days in March 2023 for the first cross-government data hackathon on fraud. The event was organised by the 10 Downing Street Data Science (10DS) team and the National Crime Agency (NCA), and was supported by partners including Ordnance Survey and the Metropolitan Police Service.

Attendees worked in newly-formed teams to build new AI enabled tools to detect fraudulent companies. These tools linked data from Companies House, money laundering sanctions lists and geospatial data. Attendees tested a range of approaches including random forests and network graph analysis. The teams worked on their tools and then presented their products to a panel of judges. The winning product was a “3 tap” tool for stopping spam text messages.

By bringing together the expertise and resources of different departments and agencies in events such as these, the NCA and 10DS ensure that the UK remains at the forefront of applying AI in the fight against fraud.

A common data quality problem is to have multiple different records that refer to the same entity, but no unique identifier that ties these entities together. The Ministry of Justice, (MoJ) developed “Splink”, an open-source Python library for data deduplication and linkage, or entity resolution, to solve this problem at scale.

Splink has been downloaded 6 million times. It has been used in government, academia and the private sector on datasets containing more than 100 million records. Small linkages of up to 1 million records run 100 times faster than earlier versions of Splink. Splink version 3.0 marked a major restructuring of the application programming interface

Splink is now used widely in the courts and probation services. It helps to develop a better understanding of offenders under electronic monitoring orders. Splink also helps create more accurate criminal histories to produce risk and predictive models.

The Kitchen Life 2 project is an award-winning research project from the Food Standards Agency (FSA). The FSA used a range of innovative methodologies to understand how people prepare food and perform other tasks in a kitchen environment and how to improve these behaviours.

The project team installed motion sensitive cameras in 101 domestic and business kitchens. Behaviours were tagged and coded for analysis, such as handwashing and food preparation. The multi-method project combined the data from the cameras with a range of other data from:

• interviews
• food diaries
• shopping receipts
• passive data on fridge and freezer temperatures

The data was integrated into a dashboard to analyse the frequency and context of the behaviours. Analysis also used a behavioural framework to understand why people behave in the way they do.

Cases of foodborne illness cost the UK approximately £9 billion a year. The insight provided by this innovative methodology has transformed the FSA’s understanding of kitchen life. The project has provided fresh and more accurate insight for risk assessment, policy development, and behavioural intervention design.

The Department for Education (DfE) implemented a new system which automatically collects daily attendance data from schools. This method of data collection is revolutionary for the department and its stakeholders and, because it is automated, it created no additional burden for schools.

This real-time monitoring meant that DfE knew how many pupils were in school by the second day of the new academic year. When the Permanent Secretary presented the work at the Public Accounts Committee, Meg Hillier MP stated she had “never heard of a government IT project that has moved quite so fast.”

Schools, trusts and local authorities are provided with daily reports with powerful visualisations. These allow any issues with individual pupils or groups, such as vulnerable children and those eligible for free school meals, to be easily identified and action taken. A public dashboard is published fortnightly providing up-to-date national, regional and local authority level information on school attendance.

The project involved cross-group working within DfE and with schools, local authorities, unions, and sector bodies. The data was also useful for monitoring other areas. For example, it provided a useful way of assessing the number of schools that closed during the recent teacher strikes.