Data and metadata validation is the process of ensuring that data and metadata are clean, correct and useful. In this context of data validation, the following specific definition summaries this process: Validation is an activity verifying whether or not a combination of values is a member of a set of acceptable combinations (see Definitions).
Data and metadata validation is a key task that is performed in all statistical domains to ensure that both, data and metadata outputs are of high quality. Eurostat validation architecture comprises STRUVAL and CONVAL services that support the automatisation of the data validation process. National competent authorities may use these services prior data transmission. National and European metadata reports are also validated via the ESS MH.
The Validation and Transformation Language (VTL) has been developed to standardise the language for defining validation and transformation rules. VTL is linked to the SDMX standard as it builds on the transformation section of the SDMX information model. Furthermore, validation rules will be exchanged using SDMX messages.
In the process of data and metadata validation the following tools/services may be used:
- STRUVAL service, powered by the SDMX Converter, verifying the structure of the dataset
- CONVAL service, verifying the content of the dataset
- COOL will display the results of the validation process (only available to Eurostat users)
- EDAMIS will display the results of the validation processes ESS MH for the structure and content validation of reference metadata reports
For more information on the ESS standards and tools for validation please visit the corresponding pages available on this platform or contact us at ESTAT-DATA-METADATA-SERVICES@ec.europa.eu