gms | German Medical Science

64. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

08. - 11.09.2019, Dortmund

Reproduceable Visual Analytics of Multimodal Neuro-Monitoring Data: Challenges and Lessons Learned from a Data Science Perspective

Meeting Abstract

  • Lukas Huber - UMIT, Hall in Tirol, Austria
  • Werner Hackl - UMIT, Hall in Tirol, Austria
  • Bogdan-Andrei Ianosi - UMIT, Hall in Tirol, Austria
  • Verena Rass - Medical University of Innsbruck, Innsbruck, Austria
  • Fabian Guiza Grandas - KU Leuven, Leuven, Belgium
  • Geert Meyfroidt - UZ Leuven, Leuven, Belgium
  • Raimund Helbok - Medical University of Innsbruck, Innsbruck, Austria
  • Elske Ammenwerth - UMIT - Private Universität für Gesundheitswissenschaften, Med. Informatik und Technik Tirol, Hall in Tirol, Austria

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 64. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Dortmund, 08.-11.09.2019. Düsseldorf: German Medical Science GMS Publishing House; 2019. DocAbstr. 30

doi: 10.3205/19gmds039, urn:nbn:de:0183-19gmds0394

Published: September 6, 2019

© 2019 Huber et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at



Background: Multimodal Neuro-Monitoring advances the treatment of patients through continuous monitoring of different parameters and generates high volumes of electronically available data [1]. One possibility to analyze this “tsunami” of data is Visual Analytics (VA) which is the field of analytical reasoning facilitated by interactive visual interfaces [2]. However, reproducibility of such analyses is of critical manner [3].

Objectives: Investigation of the reproducibility of an established VA method and collection of challenges and lessons learned.

Methods: The reproducibility, generalization and extension of a VA method as baseline is shown by abstracting the baseline method to pseudo-code, re-implementation as open tool and comparison of both methods. For this, we chose an established visualization method for neuromonitoring data which had been developed in Matlab(R) and published by Güiza et al. [4]. This method enables insight into the time and intra-cranial pressure components of signals in Neuro-intensive patients by correlating a longitudinal parameter with the overall outcome of the patient population within a single visualization.

To reproduce this method an abstraction was made through the development of a “Pseudo-Code” representation which was verified together with the original author of the method. The method was then reimplemented with the open-source language R and checked using the requirements regarding reproducibility of algorithms as stated in [5]. Additionally, for checking the generalizability, the method was reamed, so that it can also be used in other use-cases with other parameters. Finally, a validation of the reimplemented method’s robustness was done on different preprocessing mechanisms to ensure its feasibility.

Challenges and Lessons Learned were collected during the whole process and discussed in the team for their relevance and possible implications.

Results: The resulting visualizations of both methods were successfully approved through visual juxtaposition by the original authors of [4] and independent clinical experts. In addition, the application of the reamed method to other neuro-monitoring parameters demonstrated the feasibility of the extension.

The following recommendations were derived from the lessons learned:

  • Do establish a plan beforehand and ensure transparency of all steps
  • Do check the implementation of the VA methods.
  • Do check results with domain experts, to avoid spurious correlations.
  • Do define data properties and requirements for the raw data as well as the transformed analysis data, to ensure plausible results.
  • Do establish clear data staging and versioning, as well as technical requirements on the limits of the data.
  • Do collaborate directly with technical and domain experts, to ensure good understanding of context, methods, data and tools.
  • Do ensure traceability of all steps along the whole process.

Conclusion: The objectives for the reproducibility, generalizability and extension of a selected visual analytics method were achieved by abstraction of the base VA method to pseudo-code, re-implementation in an open-source R-language, reaming of the method for other parameters and scenarios, as well as the feasibility analysis and juxtaposition of results. The challenges and lessons learned clearly demonstrate the effort required to increase reproducibility, generalizability and extendibility.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.


Schmidt JM, De Georgia M. Multimodality monitoring: informatics, integration data display and analysis. Neurocritical care. 2014 Dec 1;21(2):229-38. DOI: 10.1007/s12028-014-0037-1 External link
Thomas JJ, Cook KA. A visual analytics agenda. IEEE computer graphics and applications. 2006 Jan 1(1):10-3. DOI: 10.1109/MCG.2006.5 External link
Baker M. 1,500 scientists lift the lid on reproducibility. Nature News. 2016 May 26;533(7604):452. DOI: 10.1038/533452a External link
Güiza F, Depreitere B, Piper I, Citerio G, Chambers I, Jones PA, Lo TY, Enblad P, Nillson P, Feyen B, Jorens P. Visualizing the pressure and time burden of intracranial hypertension in adult and paediatric traumatic brain injury. Intensive care medicine. 2015 Jun 1;41(6):1067-76. DOI: 10.1007/s00134-015-3806-1 External link
Vandewalle P, Kovacevic J, Vetterli M. Reproducible research in signal processing. IEEE Signal Processing Magazine. 2009 Apr 17;26(3):37-47. DOI: 10.1109/MSP.2009.932122 External link