Northern Ireland Statistics and Research Agency 2024/25 Reproducible Analytical Pipeline Strategy Implementation Plan

In 2022 the Government Analysis Function published a Reproducible Analytical Pipelines (RAP) Strategy. The strategy acknowledges that RAP improves the value, efficiency and quality of outputs when it is used effectively within statistical and analytical processes.

It outlines an ambitious vision for analysis in government that the Northern Ireland Statistics and Research Agency (NISRA) aspires to adopt. The three key goals of the UK strategy to make this possible relate to tools, capability and culture. As part of the strategy public sector organisations have been asked to develop local plans detailing how they will tackle these three priority areas.

The purpose of this document is to outline the work NISRA has already carried out to implement the Analysis Function RAP Strategy and how we will continue to develop the tools, capability and culture within NISRA and our outposted statistical branches in order to move towards full RAP implementation.

Where is NISRA on RAP?

The NISRA Corporate Plan 2025-2029 has six strategic enablers, one of which is Digital Infrastructure. The plan states that we will:

‘Promote the use of new technologies and techniques which will drive efficiency and coherence. This will include the use of Reproducible Analytical Pipelines (RAP) to produce outputs efficiently and to a consistently high standard and examining the feasibility of using artificial intelligence tools in statistical production.’

For NISRA outputs there is usually an existing step-by-step process in place (i.e. a set of processes that must be followed in a specific order to get from raw data to final output). However, legacy processes are typically dated, not efficient and require manual intervention and use of multiple software packages.

Given the importance of RAP and the associated benefits that can be garnered from its implementation, NISRA has plans to implement the use of Reproducible Analytical Pipelines across the Agency and which also applies to NISRA statisticians outposted in Departments across NI.

The long-term goal is to ‘implement Reproducible Analytical Pipelines for all our key statistical processes and key statistical outputs within 5 years’. The overarching objective is to eliminate manual intervention where possible, increase automation and move away from the use of multiple costly software packages, to wrap the streamlined processes into one open-source software such as “R” where feasible.

NISRA has already made significant progress on the implementation of RAP. Many NISRA branches have started to introduce RAP into their processes to make outputs reproducible, improve efficiency and address accessibility requirements. RAP projects have covered all parts of the statistical production process, and many have included the production of the statistical publication in an accessible HTML format alongside spreadsheet outputs. Benefits include improved process documentation, higher quality outputs, enhanced staff coding skills and significantly faster processing times.

RAP Implementation

NISRA’s goal is to implement RAP for all key statistical processes and statistical outputs. It is understood that this will take time, training and support to embed and is likely to be an iterative process

Teams currently have process flows for their statistical production/outputs, but in many cases, they are disjointed, require the use of several software packages and human intervention. In advance of any RAP development, teams should review the current processes and ensure that process maps are in place. A process map allows the pipeline /process to be broken down into manageable chunks for coding.

Each branch and team will develop their RAP processes and experiences over time, continually improving their outputs, code and automation processes. Support will be provided by a RAP Working Group which has been established to promote the use of RAP and support its implementation. The NISRA Tech Lab will also be a key source of support, helping with technical advice and code development.

Peer learning and support will form a significant element of the approach to implementation. Sharing code and approaches with others across teams, branches and departments will facilitate continual improvements across the Agency.

Each team will start with some minimum RAP requirements and develop towards full RAP implementation.

To achieve this, we have included two specific targets in the NISRA 2024/25 Business Plan as follows.

“Convene a NISRA RAP Working Group, made up of volunteers from those who have implemented RAP. This group will take the RAP strategy forward and support branches in the development of their RAP implementation plans.”

“Work areas within NISRA will start to devise and start to deliver implementation plans which roll out NISRA’s Reproducible Analytical Pipelines (RAP) Strategy for all key statistical processes and key statistical outputs over the next five years.”

The Director of Statistical Co-ordination and Analysis within NISRA will be the NISRA RAP owner and will oversee the implementation of this strategy. Branches will be supported in the development of their implementation plans and will be expected to engage with the RAP Working Group.

The NISRA RAP objectives and Implementation Plan are aligned with the three goals outlined in the Analysis Function RAP Strategy:

1. Tools

Ensure that analysts have the tools they need to implement RAP principles.

2. Capability

Give analysts the guidance, support and learning to be confident implementing the Reproducible Analytical Pipelines (RAP).

3. Culture

Create a culture of robust analysis where the RAP principles are the default for analysis, leaders engage with managing analysis as software, and users of analysis understand why this is important.

Our aim is to embed RAP in all appropriate and relevant outputs across the Agency and to empower statisticians to adopt RAP as default going forward. This applies to NISRA statisticians working across all NI government Departments. We will work with our colleagues to deliver these goals as outlined below.