Artificial Intelligence (AI) case studies
Artificial Intelligence (AI) is already being used by analysts across government to improve efficiency, support evidence synthesis, enhance analytical processes, and explore new ways of working. However, one of the most effective ways to build understanding and confidence is through sharing real-world examples.
This collection of case studies showcases how analysts from across the Analysis Function are applying AI in their day-to-day work. The examples highlight a range of use cases, from drafting and summarisation to qualitative analysis, coding, automation, and more advanced analytical applications.
The purpose of these case studies is not to promote a single approach, tool, or solution. Instead, they provide practical examples of how AI can be used responsibly, the challenges encountered, the lessons learned, and the safeguards applied to ensure quality, transparency, and trustworthiness.
As AI adoption continues to evolve, we encourage analysts to learn from the experiences of others, adapt approaches to their own context, and share their own examples with the wider Analysis Function community.
Included in each case study
Each case study includes:
- details of the business challenge or analytical problem being addressed
- information about the AI tool (or tools) used
- details of how AI was applied within the workflow
- a summary of the benefits and outcomes achieved
- evaluation of the risks, limitations, and lessons learned
- quality assurance and governance considerations
- practical tips for analysts considering a similar approach
Case studies
This project was completed by the Advanced Analytics team, based in the Analysis Directorate at the Department for Energy Security and Net Zero (DESNZ).
About this work
The team have developed the DESNZ Analytical Repository and Tracker (DART) chatbot in partnership with the DESNZ’s Monitoring and Evaluation Hub.
DART is a collection of over two-hundred Evaluations and Impact Assessments published by DESNZ. The chatbot was designed to allow users to easily search through these documents for any information or evidence that they might require. In addition to summarising this information, the chatbot can also highlight sections of documents it is quoting, allowing greater transparency and giving the user a jumping-off point for any additional research.
The outcome of this work
The chatbot has just been deployed to 150 initial users and the team are currently gathering their feedback to improve the chatbot before deploying it to a wider group.
By gathering this information in one place, DART will increase access and use of existing evaluation evidence. Being able to interact with the chatbot using natural language will make it faster and easier for colleagues to access the information they need.
Lessons learned
Prompt engineering is often the most effective way to boost performance! Even very small tweaks to the model’s prompt can have a large impact and just experimenting (in tandem with proper evaluation) can go a long way.
About the AI tools used
The app is built around a “Retrieval Augmented Generation” (RAG) methodology, where sections of relevant documents are retrieved based on the user’s query and then fed to the LLM, which drafts a summary. Microsoft Azure infrastructure was used to create a vector store out of all relevant DART documents and to securely send requests to the OpenAI API. The app itself was developed in Plotly DASH (a Python package) and deployed using departmental infrastructure.
The chatbot has been tested extensively using RAGAS, which is a framework for evaluating performance of RAG models. The DART team provided a series of “golden” answers representing how chatbot should respond to certain questions, and then the app’s performance was compared to these along a number of metrics. Continuous evaluation allowed the development team to iteratively improve the chatbot’s performance over time and ensure that it met an appropriate standard.
How this work supports the AF AI Strategy Objective
This work supports the following Strategy Objective from the AF strategy for Analysis in Government 2025 to 2028:
“We will be innovative, driving efficiency across our function, the Civil Service and society, applying artificial intelligence, automation, and the best analytical tools and methods.”
The DART chatbot is an innovative use of AI, which uses cutting edge analytical methods. This tool gives our social research and policy colleagues quick and easy access to the information they need, driving efficiency both across the Analysis Function and the wider department.
This project was completed by operational research analysts from the Central Analysis, Research and Evidence team at the Department for Science, Innovation and Technology (DSIT).
About this work
The DSIT Impact Map is a department-wide Theory of Change tool. It has been developed through workshops and engagement with teams across DSIT to show how activities contribute to outcomes, impacts and departmental priorities. The Impact Map has been used to support planning, collaboration and decision-making. It was inspired by a presentation given at the 2025 Government Operational Research Service (GORS) conference by operational research colleagues from HM Revenue and Customs (HMRC). The primary insight from the presentation was around the use of a relational database to represent the nodes and edges in a Theory of Change diagram. This creates a more interactive and flexible tool than the conventional static map.
Mapping a whole department in this way (rather than a single programme or intervention as is more common) could have been an unrealistic prospect. But as the project evolved, AI enabled the team to review business plans, strategies, business cases and Theory of Change content at scale. The material covered most of DSIT and represented a volume of information that would have been impractical to review and aggregate consistently through manual analysis alone, given resource constraints. Analyst-designed prompts were used to identify evidence, surface gaps and assess causal logic, categorisation, link rationale and strength against a common framework. This helped apply a consistent review approach across directorates, while recommendations were reviewed by subject matter experts before being incorporated into the Impact Map.
The outcome of this work
By using AI the team were able to reduce the time needed to review, refine and aggregate Theory of Change content from across DSIT. This helped accelerate development of the Impact Map and engagement with stakeholders. It also enabled a more consistent assessment of outcomes, causal pathways and supporting rationale across directorates, improving the quality and comparability of content used for department-level aggregation.
AI was particularly valuable in applying a common review framework across large volumes of material, helping identify evidence gaps, inconsistencies and potential improvements that could then be assessed by analysts and subject matter experts.
The team also developed a Copilot agent based on the Impact Map data. This is a strategy and impact analysis assistant that can be prompted to describe how elements of the impact map connect together. The agent can make constrained inferences and provide evidence-based insights with clear references back to the source data.
Lessons learned
AI worked best as a tool to support analysts rather than as something to replace them. It was particularly useful for reviewing large volumes of information, identifying potential gaps and generating options for consideration, but analyst judgement remained essential when assessing causal logic, policy context and final recommendations.
The team also found that output quality depended heavily on the prompts and frameworks provided. Clear analytical standards and quality assurance requirements enabled AI to apply a consistent approach at scale, while helping analysts focus on challenge, assurance and strategic thinking.
About the AI tools used
Microsoft Copilot was used throughout the project to review source material, develop and test analytical frameworks, support app development through coding suggestions, and draft guidance, communications and Theory of Change content.
Copilot enabled rapid analysis of large volumes of material and supported a consistent assessment approach across directorates, making it easier to identify evidence, gaps, inconsistencies and opportunities for improvement.
The agent the team developed is also based on Microsoft Copilot.
How this work supports the AF AI Strategy Objective
This work supports the following Strategy Objective from the AF strategy for Analysis in Government 2025 to 2028:
“We will be innovative, driving efficiency across our function, the Civil Service and society, applying artificial intelligence, automation, and the best analytical tools and methods.”
This project used AI to reduce the time spent gathering evidence, reviewing content and maintaining the DSIT Impact Map. By automating routine tasks, analysts could spend more time on stakeholder engagement, challenge, assurance and strategic thinking.
The approach demonstrates how AI can be integrated into analytical workflows to improve efficiency while maintaining analytical quality, transparency and professional standards.
This project was completed by the Floods and Water Analysis and Evidence Team and Secretariat to the Independent Water Commission, based at Defra.
About this work
The Independent Water Commission’s (IWC) call-for-evidence was open from 27 February to 23 April 2025 and sought views on the water sector regulatory system in England and Wales. It received over 50,000 responses from individuals, organisations and campaign routes, creating a substantial analytical challenge across 73 questions, including around 40 open-text questions.
To manage this volume, the team used Consult (an AI-assisted tool developed by i.AI) to support thematic analysis of open-text responses. The tool:
- generated key themes
- mapped responses to the themes identified
- assessed sentiment and response position
- supported outputs through dashboards and spreadsheets for policy teams
Before using the tool at scale, the team worked with the Consult AI team to evaluate its performance against human reviewers. The evaluation compared AI-generated outputs with expert human review across a sample of responses to check whether the tool could support robust analysis without replacing human judgement.
The outcome of this work
The AI-supported approach enabled the team to process and synthesise a very high volume of responses within tight timelines, supporting both the interim and final reports of the Independent Water Commission. The final summary of responses states that AI-assisted thematic analysis was used to identify key themes and that evaluation provided assurance of alignment between AI-generated outputs and human analysis.
The evaluation found strong alignment between Consult and human reviewers. At least one reviewer made no changes to Consult’s mapped themes for 83% of responses; overall F1 scores were 0.79 and 0.82 when comparing Consult with two reviewer groups, compared with 0.74 between the reviewer groups themselves; and Consult accurately mapped 89% of response positions.
Consult AI estimated substantial efficiency benefits. 2,418 hours were saved compared with manual analysis, with an estimated manual process of around 2,440 hours compared with around 22 hours for AI-assisted mapping, and more than £80,000 saved in evaluation costs. Use of the tool was also provided to use by the consult AI team free of charge. Based on similar consultation analysis outsourced by Defra teams, this saved around £100,000.
Lessons learned
AI can be highly valuable for large-scale consultation or call-for-evidence analysis, particularly where there are high volumes of open-text responses and short timelines. However, the tool worked best when combined with clear human oversight, expert review and a structured quality assurance process.
Data protection and transparency need to be considered early. The team sought data protection advice, clarified that AI would be used to thematically analyse and label responses rather than make decisions on the data, and received advice that the processing described was not high risk and did not require a full Data Protection Impact Assessment (DPIA).
The work also showed the importance of clear analytical scope. Some long-form stakeholder submissions and campaign responses required separate treatment because they did not always follow the structure of the call-for-evidence questionnaire.
About the AI tools used
The main AI tool used was Consult, which is an AI-powered tool developed by i.AI to analyse public consultation and call-for-evidence responses. It supports theme generation, theme mapping, response position assessment, and presentation of findings through spreadsheets and dashboard outputs.
The tool helped the team move from a largely manual approach to an AI-assisted analytical workflow, substantially reducing the effort required to identify and map themes across large volumes of open-text responses. The tool did not replace human judgement: human reviewers remained involved in checking, validating and interpreting outputs.
The impact was both operational and analytical. It helped provide timely analysis for the Commission’s reporting timetable, reduced the estimated manual burden, and provided a more structured and transparent way of reviewing response themes at scale.
How this work supports the AF AI Strategy Objective
This case study supports the AF AI Strategy objective by showing how AI can be used responsibly to drive efficiency and improve analytical delivery across government. The work applied AI to a real, high-volume evidence challenge, while maintaining human oversight, quality assurance and data protection checks.
It demonstrates innovation in public sector analysis: using AI to help synthesise evidence more quickly, reduce manual burden, support policy development, and provide a repeatable example for other consultation and call-for-evidence exercises across Defra and wider government.
This project was completed by the Floods and Water, Analysis and Evidence, Data and Monitoring Team.
About this work
There are 38 Local Resilience Forums (LRFs) in England, each responsible for planning and preparing for localised incidents and major emergencies within their area.
As part of a flooding preparedness exercise, LRFs were asked to assess their confidence against ten distinct flood response elements and provide supporting narrative explaining their ratings.
The project required thematic analysis of these qualitative responses to identify common themes, strengths, gaps, risks, and regional variations. The findings were needed for an internal briefing and slide pack to support decision-making.
The outcome of this work
Copilot was used to support thematic analysis of the free-text responses. Copilot generated initial summaries, identified recurring themes, and highlighted potential strengths and gaps across submissions.
Outputs were reviewed and quality assured by analysts against the source data before inclusion in briefing materials. Copilot was also used as a secondary reviewer to challenge conclusions and identify potential over-generalisations.
The use of AI significantly reduced the time required for analysis. The work was originally expected to take two analysts approximately two weeks to complete. While full analyst review remained essential, the use of Copilot accelerated drafting, resulting in an estimated 35% reduction in delivery time.
Lessons learned
Several prompting approaches were tested during the project. The team found that analysis quality was significantly better when Copilot was provided with responses for a single response element at a time, rather than being asked to analyse the entire dataset in a single prompt.
A particularly valuable quality assurance step was using a “devil’s advocate” prompt, such as:
“Challenge the preceding analysis. What alternative interpretations could be drawn from the same evidence?”
This helped identify potential bias, overstatement, and unsupported conclusions. As well as checking that any quotes could be traced back to the data source.
Stakeholders provided positive feedback on both the quality of the outputs and the speed at which the analysis was delivered. One stakeholder commented: “I stand on the shoulders of giants!”
About the AI tools used
Microsoft 365 Copilot was used to support qualitative thematic analysis by summarising free-text responses, identifying common themes and patterns, highlighting differences across submissions, and assisting with quality assurance.
The tool enabled analysts to process a large volume of qualitative evidence more efficiently, reducing the time spent on manual analysis while maintaining human oversight throughout.
How this work supports the AF AI Strategy Objective
This work supports the following Strategy Objective from the AF strategy for Analysis in Government 2025 to 2028:
“We will be innovative, driving efficiency across our function, the Civil Service and society, applying artificial intelligence, automation, and the best analytical tools and methods.”
Share your case study
Have you used AI to support your analytical work?
We are keen to showcase examples from across all analytical professions, departments, and regions. Sharing your experience helps build collective knowledge, promotes good practice, and supports the responsible adoption of AI across government.
If you would like your work to be featured, please contact the AI Working Group by emailing Analysis.Function@ons.gov.uk.