gms | German Medical Science

Introduction: Data quality assessments (DQA) are crucial for finding and handling data errors, making data comparable and FAIR. However, DQA require efficient generation and communication of results. Different concepts to classify data quality aspects exist, e.g. [1], [2], [3], as well as R software implementations [4]. This abstract focuses on the recently updated dataquieR package.

State of the art: Among the most prominent DQA tools in R based on a DQ framework are DQAstats by Kapsner et al. and dataquieR. In version 1.0.13, dataquieR’s concept coverage was 18 out of 34 indicators, which is one of the highest, but still far from 100% [4].

DQA tools typically create reports (e.g., PDF or HTML files). Usually, these are not interactive, so inspecting single outputs may be unsatisfying: zooming may be insufficient, and finding the most relevant results in linear documents can be cumbersome. However, such documents have few preconditions for sharing the results as no web-server is needed. dataquieR employed rmarkdown by Xie to create HTML documents because of its flexibility. Nonetheless, it is a generic implementation, slowing down the report generation time without good options to improve speed. Furthermore, dataquieR used parallel computing only for selected indicators, resulting in a complicated process.

Concept: dataquieR now expands the previous HTML approach for reporting with improved interactive dashboards. A re-implementation of the report rendering allowed to include more features of the DT package by Xie, e.g., exporting tables as Excel™. Functions for reproducibility have also been added. Graphical results are stored as ggplot2 objects; to present them in a report, dataquieR now employs plotly by Sievert. This results in highly interactive figures and enables zooming, panning, hiding specific parts of the figure, and exporting them as single graphics in several formats, including vector graphics.

With the new release, dataquieR’s concept coverage increased from 52.9% to 70.6%, without increasing computation time. Most prominently, for each report output, the respective indicator function call is now included as R code, allowing to re-generate the respective output outside the reporting pipeline, which poses a huge advantage for more user specific assessments.

Implementation: In the report pipeline, the previous dataquieR version called some functions with more than one outcome variable at once, generating complicated, encapsulated result lists. Now, the way dataquieR calls data quality indicator functions has been rewritten so that dataquieR calls each function for each outcome variable separately. As a consequence, the result is a simple list structure (with one entry for each indicator function and outcome variable) and all functions can run in parallel. This modification allowed for more generic rendering functions featuring, e.g., plotly support. Alongside this, the report rendering re-implementation with htmltools by Cheng, instead of rmarkdown, resulted in a leaner reporting engine.

Lessons learned: Improving the visualization of data quality indicators and adding new functionality to reuse the code enhances the usability, reproducibility, and speed of dataquieR’s pipeline.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.