Towards a comprehensive suite of resources for developing and implementing AI/ML solutions for official statistics
THE PROJECT
New advances in artificial intelligence (AI) and machine learning (ML) present promising opportunities for the production of official statistics, with applications across the production pipeline including improvements in efficiency, unlocking potential of novel data sources, enhancing data quality and delivering more user-friendly outputs. Harnessing this potential is a key strategic aim of the European Statistical System (ESS). Through its Innovation Agenda, the ESS is pursuing a coordinated approach and has established a One-Stop-Shop for Artificial Intelligence and Machine Learning in Official Statistics (AI/ML4OS). This is a four-year project, which includes the participation of National Statistical Institutes (NSIs) from 16 countries. The overarching aim of the project is to produce a single point of entry for ESS staff to facilitate accessing and deploying AI/ ML solutions.
THE MOTIVATION
Expertise on new analytical tools for statistical production is dispersed across the ESS, with each NSI having established its own competencies in specific areas. Yet many challenges and opportunities are common to all Member States. As well as technological solutions, there are advantages to sharing guidelines, standards and quality frameworks among organizations. A collaborative approach to developing solutions is therefore desirable. It is further hoped that this collective effort will lead to thriving communities of ESS staff for ongoing mutual support beyond the timeline of the AI/ML4OS project.
THE METHODOLOGY
The AI/ML4OS project is composed of 13 work packages (WP) that are separate but interlinked. The first six WPs are oriented towards supporting the operation of the project and coordinating the delivery of its outputs. The remaining seven WPs are use cases, each of which explores a specific tool or theme within AI/ML in the context of official statistics. Each WP is led by one or two NSIs, with as many as 12 countries participating on individual WPs.
| The six supporting WPs are | The seven use case WPs are |
WP1 – Project management and coordination: Led by the CSO (Ireland), this WP supports, coordinates and facilitates the consortium activities in the project, ensuring that the project achieves its overall objectives. WP2 – Communication and community engagement: Led by ISTAT (Italy), this WP supports the interdependencies between WPs and shares progress and outputs with the statistical community and other external stakeholders. WP3 – ESS AI/ML lab: Technical infrastructure and organisational setup: Led by INSEE (France), this WP provides a testing environment (sandpit) for the use case WPs, and provides materials to support NSIs and Other National Authorities to create their own infrastructure for AI/ML. WP4 – AI/ML state-of-play and ecosystem monitoring: Led by Statistics Denmark, this WP will develop and maintain a comprehensive picture of AI/ML developments and activities. WP5 – Standards, methodological and implementation frameworks: Led jointly by CBS (Netherlands) and DeStatis (Germany), this WP develops methodological and implementation guidelines and supports standardisation in applying AI/ML-based solutions. WP6 – Knowledge repository and training material: Led by Statistics Poland, this WP will collect the knowledge developed through the project and prepare appropriate materials for knowledge sharing | WP7 – AI/ML on earth observation data, satellite imagery: Led jointly by CBS (Netherlands) and ISTAT (Italy), this WP will develop methodological and implementation guidelines for earth observation data-based solutions for official statistics. WP8 – Statistically valid and efficient editing and imputation in official statistics by AI/ML – with a special focus on editing: Led by DeStatis (Germany), this WP will improve automation, efficiency and quality of the editing process using AI/ML solutions. WP9 – Statistically valid and efficient editing and imputation in official statistics by AI/ML – with a special focus on imputation: Led by INE (Spain), this WP will develop, test and implement AI/ML-based solutions for imputation processes. WP10 – From text to code - Experiences and potential of the use of AI/ML for classifying and coding: Led by DeStatis (Germany), this WP will develop cutting-edge solutions to assign classification schemes to text data sources. WP11 – Applying ML for estimating firm-level supply chain networks: Led by CSB (Netherlands), this WP will use ML models to create population-scale firm-level supply chain network datasets. WP12 – Large language models: Led by Statistics Sweden, this WP will explore the vast opportunities presented by large language models (LLMs) across the statistical production pipeline, from data understanding to dissemination. WP13 – Generation of synthetic data in official statistics: techniques and applications: Led jointly by Statistics Austria and ISTAT (Italy), this WP will explore techniques for generating synthetic data across several statistical domains. |
MORE INFORMATION
Project dashboard on Eurostat CROS
THE TEAM
Participating countries: Austria, Denmark, France, Germany, Ireland, Italy, Luxembourg, The Netherlands, Norway, Poland, Portugal, Slovenia, Spain, Sweden including non-beneficiaries: Cyprus and Switzerland.
Contact: projectAIML4OS@cso.ie
LAST UPDATE: December 2024
PARENT PAGE:
