gms | German Medical Science

Introduction: The goal of the Medical Informatics Initiative (MII) is to make healthcare data accessible to research. To this end, data is collected and made available at the Data Integration Centers (DIC). To evaluate the progress of this effort, regular Projectathons are conducted. For such secondary uses of healthcare data, consideration of data quality (DQ) is particularly important. Data that is of good quality for healthcare is not automatically suitable for every research question in terms of quality and granularity. Thus, data quality must be re-evaluated in each case with respect to its specific research question.

Methods: We use the DQ terminology by Kahn et al. ([1], p. 18) for all collaborative efforts in the MII. It divides DQ into three levels: Conformance, Completeness and Plausibility, and two contexts: Verification and Validation. Each level includes further sub-levels as well as concrete definitions.

The MII has defined its own FHIR profiles, the Core Data Set (CDS) for data representation. Running FHIR validation against those profiles in the different Data Integration Centres (DIC) fulfils Kahn Conformance Verification and Validation. Starting from the data requirements of the Projectathon use case (Catalogue of Items), we implemented Data Quality Indicators (DQIs) using dataquieR [2], satisfying Kahn Completeness and Plausibility Verification. As the scripts for data extraction and analysis at the DICs were written in R, attaching the DQ scripts was straightforward.

Additionally, we deposited the DQIs in CEDAR, a metadata repository that allows medical codes to be included (via BioPortal) and the core dataset elements to be referenced via FHIRPath. By linking the core dataset, the medical codes and the DQIs, we enable the reuse of the DQIs beyond a single use case. Subsequent use cases, based on the same codes or CDS elements, can build on these DQIs. Therefore, we refer to the DQIs now available through CEDARs REST API as FAIR DQIs.

Results: We defined a formal representation of DQIs specific to the distributed context of the MII [3]. We also created DQIs for all Kahn definitions based on the elements of the CDS [4]. We created a framework for generating DQ reports when extracting data from the DICs. We implemented the DQIs in R using dataquieR as part of the seventh Projectathon. The developed scripts are available on the MII GitHub page [5] and we additionally provide FAIR DQIs through CEDAR. These allow the reuse of our DQ efforts, as well as the comparison of our results across specific implementations. Figure 1 [Fig. 1] shows the framework for a clinical, decentral use case of the 7^th MII Projectathon dealing with “Atrial Fibrillation”. A similar, central use case had previously been addressed in the 6^th Projectathon [6]. The Preparation Phase allows for re-use of existing FAIR DQIs during script implementation, if another use case provided them during its Feedback Phase. In the Operational Phase scripts are run at each DIC independently. We were able to re-use the FAIR DQIs from the 6^th Projectathon, demonstrating the value of the outlined framework.

Discussion & conclusion: Data requirements not only change with secondary use but can also vary between use cases. For DQ analysis, it is therefore difficult to make general statements about a dataset. Instead, the suitability of a dataset for answering a research question should be assessed by the use case experts. It was necessary to map DQIs as more than just tool-specific metadata. The creation of FAIR DQIs prior to implementation enables the sharing and reuse of DQIs, especially in highly distributed contexts, like the MII, where the same tools are not used in all locations. While the Projectathons were limited in terms of the suitable DQ tools, FAIR DQIs with their tool independent conceptualisation provide an opportunity for reuse in future use cases requiring different tooling. This is especially relevant for the path forward in the MII, as different tools for data quality assessment (DQA) are already being developed in the MIRACUM consortium [7] while use case specific DQA implementations are seeing adaption at the same time [8]. This approach also has the potential to benefit from other already established DQA tools specifically designed in the context of secondary use, like the tools developed as part of OHDSI, which are already seeing some support in Germany [9].

The Kahn framework provided a starting point for possible DQIs, but Completeness and Plausibility Validation remained unresolved. The creation of the necessary reference distributions, so-called “gold standards”, would particularly benefit from the wealth of data in the MII. We hope to address this in a dedicated DQ-focused use case.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.

This contribution has already been published: The outlined Framework was already practically applied in the 6^th Projectathon of the MII. We previously published a (German) abstract about the 6^th Projectathon [6].This abstract builds on our previous work, and we want to use this opportunity to update the community of the ongoing efforts in the MII Projectathons and present some new results as well as the updated framework from the 7^th Projectathon.