gms | German Medical Science

Background: Multimodal Neuro-Monitoring advances the treatment of patients through continuous monitoring of different parameters and generates high volumes of electronically available data [1]. One possibility to analyze this “tsunami” of data is Visual Analytics (VA) which is the field of analytical reasoning facilitated by interactive visual interfaces [2]. However, reproducibility of such analyses is of critical manner [3].

Objectives: Investigation of the reproducibility of an established VA method and collection of challenges and lessons learned.

Methods: The reproducibility, generalization and extension of a VA method as baseline is shown by abstracting the baseline method to pseudo-code, re-implementation as open tool and comparison of both methods. For this, we chose an established visualization method for neuromonitoring data which had been developed in Matlab(R) and published by Güiza et al. [4]. This method enables insight into the time and intra-cranial pressure components of signals in Neuro-intensive patients by correlating a longitudinal parameter with the overall outcome of the patient population within a single visualization.

To reproduce this method an abstraction was made through the development of a “Pseudo-Code” representation which was verified together with the original author of the method. The method was then reimplemented with the open-source language R and checked using the requirements regarding reproducibility of algorithms as stated in [5]. Additionally, for checking the generalizability, the method was reamed, so that it can also be used in other use-cases with other parameters. Finally, a validation of the reimplemented method’s robustness was done on different preprocessing mechanisms to ensure its feasibility.

Challenges and Lessons Learned were collected during the whole process and discussed in the team for their relevance and possible implications.

Results: The resulting visualizations of both methods were successfully approved through visual juxtaposition by the original authors of [4] and independent clinical experts. In addition, the application of the reamed method to other neuro-monitoring parameters demonstrated the feasibility of the extension.

The following recommendations were derived from the lessons learned:

• Do establish a plan beforehand and ensure transparency of all steps
• Do check the implementation of the VA methods.
• Do check results with domain experts, to avoid spurious correlations.
• Do define data properties and requirements for the raw data as well as the transformed analysis data, to ensure plausible results.
• Do establish clear data staging and versioning, as well as technical requirements on the limits of the data.
• Do collaborate directly with technical and domain experts, to ensure good understanding of context, methods, data and tools.
• Do ensure traceability of all steps along the whole process.

Conclusion: The objectives for the reproducibility, generalizability and extension of a selected visual analytics method were achieved by abstraction of the base VA method to pseudo-code, re-implementation in an open-source R-language, reaming of the method for other parameters and scenarios, as well as the feasibility analysis and juxtaposition of results. The challenges and lessons learned clearly demonstrate the effort required to increase reproducibility, generalizability and extendibility.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.