Validation reports user guide

VALIDATION REPORTS USER GUIDE

1. Standardization of the Validation Reports
2. Retrieval of the Validation Report
3. Structure of the Validation Report (HRR)
4. See also

Standardization of the Validation Reports

In collaboration with National Statistical Institutes, Eurostat has introduced an SDMX-compliant XML schema for validation reports. This Machine-Readable Report (MRR) contains the results of the validation event (e.g. errors in data, error messages) and can be processed by Eurostat’s Validation Report Formatter Service (VRF) to produce an additional, user-friendly Human Readable Report (HRR) in HTML format. The HTML report enhances the readability and clarity of the information provided and presents information on validation errors in a transparent and ordered manner.

Eurostat’s validation services (STRUVAL and CONVAL) currently produce the SDMX-compliant MRR report and the associated HRR report. 

Retrieval of the Validation Report

The data provider may access the Validation Report as EDAMIS feedback. The Validation Report is never sent directly (e.g. email) to data providers due to possible data confidentiality and security constraints. The EDAMIS service may be, however, configured to send an email to inform users of the availability of the report.

Data providers receive the Validation Report in a human-readable format aimed at statisticians and other generalist groups. The HTML and Excel formats described in this guide are the most widely used, however the human-readable report can be generated in CSV, as well. The machine-readable XML format is currently only available for Eurostat internal use.

Validation Report - HTML

The Validation Report consists of a Header, an Overview section (optional), the Error Summary and the Error Details. The Header contains metadata on the validation process and a general overview of the results of the validation event. In the Overview section, statistical production domains may opt to display additional information on the validation process and the data itself. The Error Summary lists all rules that the data failed with the error message describing the issue. Finally, the Error Details contain the detailed list of all error occurrences, grouped by unique root cause.

Header section

The Header section provides an overview of the validation event: the aggregated results, and the service(s) and assets involved. The Header consists of an information box on process metadata (left), a counter of validation failures per severity (right). Further, the Header provides messages related to the outcome of the validation process. If a report is generated in the Acceptance environment, this is also indicated in the Header.

The metadata list contains the following entries:

Please note that entries that are not relevant for the individual validation event will not appear in the report, e.g. the use of Constraints is optional and therefore the label may not appear.

Error occurrence counter

The Header displays a counter with the total of issues encountered, grouped by Severity. 

Messaging

The Validation Report includes general purpose (not error specific) messages to inform data providers of specific circumstances of the validation flow instance.

Overview section

Production domains may choose to complement the validation results with additional information about the validation event, e.g. calculate statistics on data quality or compliance metrics. Such calculations are not considered part of the data validation itself and therefore are displayed separately in the report, under the Overview header, above the Error Summary. The metrics are produced by the CONVAL service and use a non-standard severity that has no impact on the validation flow. 

Error Summary section

The Error Summary provides the complete list of Rules that the data failed to fulfill. Further, the Summary includes:

The Error Summary and Full report may be exported to Excel using the button in the top right of the section (highlighted in green below). A search bar helps finding details in the list.

Error details section

The Error Details lists all occurrences of errors for a specific rule detected by the validation services. The section presents:

• the rule name
• severity level
• total number of occurrences
• number of occurrences reported - STRUVAL only, the number of occurrences displayed in the report is limited to the first 5 in the order of detection
• error message
• the complete list of series keys (the cross-products of Concept Name-Concept Value pairs) in a tabular format
• in case a secondary message is defined for a Rule to provide instructions for resolution, an Additional details entry will display it

Note: For structural validation, there are no Rule names defined, the Rule entry will display an error ID that may be helpful for diagnostic purposes.

Similarly to the Error Summary, the table may be exported to Excel or CSV formats, copied to the clipboard or printed using the button on the right side of the section.

In case the validation process detects no errors of any severity, the Error Details section does not appear in the report.

Grouping of results

Errors detected in the data are grouped by identifying the unique root cause that prompted them. The grouping logic is displayed below; if there is a difference detected at either step between errors, they will be presented separately, whereas errors with all four attributes identical are considered multiple occurrences of the same error group.

Note: A single root cause may trigger multiple error types, and these will be listed separately (e.g. a code is unexpected and also violates a length constraint).

Errors generated by technical issues detected in the file (e.g. structurally incorrect dataset) will also appear as an error group, with 1 occurrence and no location defined. 

Filtering reports due to confidentiality constraints

Eurostat policy prohibits the inclusion of confidential data in Validation Reports distributed to external parties, including national statistical institutes. In such cases, the data provider will receive a report where potentially confidential elements are filtered out and marked as {REMOVED - CONFIDENTIAL}. Eurostat domain managers collecting and processing confidential data will receive an unfiltered report where all information is retained.

The filtering only affects CONCEPT_VALUE.

Validation Report - Excel

In case the original input dataset is XLSM/XLSX format, there is an option to generate the Validation Report by directly indicating the errors in the original file. Note that this method does not work with the XLS extension files.

Cells where a rule violation is detected are highlighted in orange color and the Error Message is displayed in a comment box attached to the cell, along with the Severity.

In case multiple errors are associated with a single cell, all Error Messages are listed in the same comment box.

The design of certain validation rules would imply to highlight more than one cell in the report, e.g. pointing to multiple cells utilized for a calculation. To avoid a possibly confusing visualization, in such cases the report currently avoids the highlighting of multiple cells and associates the error message with a single cell. In case the error needs to be referenced across multiple tabs in a file, the highlighting will be applied on the first affected cell in each tab.

Excel reports may be complemented with an HTML report in case an executive summary is also desired.

X

See also

STRUVAL error codes and messages