gms | German Medical Science

Introduction: As a result of digitization, the importance of data quality (DQ) is increasing more and more, especially in the field of medicine and science [1], [2]. In the research project “Collaboration on Rare Diseases” of the Medical Informatics Initiative (CORD-MI, FKZ 01ZZ1911R), we developed a framework for assisting the quality of data recorded in different health information systems (HISs) to improve the quality of rare disease (RD) documentation and to support clinical research.

Methods: Effective management of DQ in CORD-MI requires appropriate metrics and tools to assess the quality of data extracted from different HISs. In the literature, there is currently no consensus or standard framework for assessing DQ [3], [4], [5], [6]. The definitions of indicators reflect the individual requirements of implemented use cases. For CORD-MI, the following dimensions are required: completeness, plausibility, uniqueness and concordance. From these dimensions appropriate metrics were derived such as orphaCoding completeness, orphaCoding plausibility and uniqueness rate of RD cases [7]. Our data quality framework is implemented as an R package that provides methods for calculating DQ metrics and creating reporting scripts [7]. Using this package, we developed tools for local and cross-institutional analysis of DQ in CORD-MI. To enable harmonized DQ assessments a FHIR interface was developed and applied to the MII core data set [8] stored in the FHIR servers of heterogeneous HISs. Hence, the application of our methods is not limited to local data models or HIS architectures. Essentially, a distributed DQ analysis was implemented using Personal Health Train (PHT) [9], [10], [11]. We have also used the Alpha-ID-SE terminology [12] for detecting RD cases with tracer diagnoses [13] and assessing the quality of RD documentation.

Results: Our methodology provides a framework that enables a harmonized DQ assessment and cross-site reporting. Using PHT, the developed tools were successfully applied on synthetic data located at the University Medical Center Göttingen, University Hospital Aachen and University Hospital Cologne. The analyzed data include data items that capture information about patients, cases and diagnoses as specified in [14]. The test results indicate the correctness of calculated indicators and key numbers. The developed tools and generated reports can be downloaded from GitHub [15].

Discussion: Our approach yields promising results, which can be used for local and cross-institutional DQ assessments. The fact that our framework is model-independent enables us an easy update and change of used references. Another advantage of our framework is the modular design that allows the user to select desired key numbers and indicators as well as to generate user-defined DQ reports. Since only calculated DQ metrics leave local HISs, our approach provides useful methods for privacy-preserving assessment of DQ.

Conclusion: The framework developed provides valuable DQ metrics and an interoperable solution for evaluating the DQ in distributed data sources of CORD-MI. This study has demonstrated that our methodology can detect DQ issues such as outliers or ambiguity and implausibility of coded diagnoses and that it can be applied for cross-site reporting on DQ. Our approach can therefore be used for DQ benchmarking to improve the quality of RD documentation.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.