gms | German Medical Science

62. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

17.09. - 21.09.2017, Oldenburg

Measures for Subjective Evidence – A comparative mini-review

Meeting Abstract

Suche in Medline nach

  • Daniela R. Recchia - Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Deutschland
  • Thomas Ostermann - Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Deutschland

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 62. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Oldenburg, 17.-21.09.2017. Düsseldorf: German Medical Science GMS Publishing House; 2017. DocAbstr. 060

doi: 10.3205/17gmds057, urn:nbn:de:0183-17gmds0578

Veröffentlicht: 29. August 2017

© 2017 Recchia et al.
Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). Lizenz-Angaben siehe



Introduction: In the past years, the development of the so-called evidence-based practice has developed rapidly in almost all areas of the life sciences. With a steadily growing number of clinical trials in the past decades, the connection to the personal experience knowledge of experts has been put aside to some extent. With the advent of person centered health care the quantification of subjective evidence became important again. A comprehensive analysis of this theme from the statistical point of view, therefore, appears to be more than necessary. The central issue is how evidence can be represented mathematically and how this experience of experts and empirical data can be modeled. In this work a short summary of already existing measures of subjective evidence together with a short comparison will be presented.

Methods: This work presents an brief comparative overview of the most common measures.From a total of 1.409 hits, the following measures of subjective evidence were extracted:

  • Dempster-Shafer Evidence (DSE)
  • Kullback-Leibler divergence (KL)
  • Cochran-Weiss-Shanteau (CWS) approach
  • Howard’s Surprise index (s-value)
  • Palm’s concept of subjective evidence (PSE) .

Results: An overall comparison between the most used measures of evidence found that they perform though differently since the maximum information received results in a maximum of the computed measures of evidence (s-value) but on the other side the minimum (PSE). Kullback-Leibler divergence at some point is equal to the novelty from Palm; this happens when the subjective novelty (PSE) refers to a novelty gain between both theoretical and personal expectations what is called as information gain. In conclusion our analysis shows that PSE has very good advantages in comparison to the others: it can be used for both continuous and discrete data, it does not require a hypothesis test and is very simple to calculate without the restrictions from KL and DSE.

Discussion: Translating subjectivity into numbers through a mathematical model to some extend reinvents the concept of information theory in terms of a new utility in health services research. Although the idea to measure expertise is not new, only a few suggestions for its quantification have been made so far. In conclusion Palms subjective evidence shows to be a very consistent measure supporting the personal believes of experts when they are in agreement with a gold standard reference and if they are not and their believes are far away from reality, the measure would also reflect this divergence. The power of persuasion of an expert can now be quantified and presented by the evidence model, which might lead to a more comprehensible and therefore negotiable culture of decision making.

Die Autoren geben an, dass kein Interessenkonflikt besteht.

Die Autoren geben an, dass kein Ethikvotum erforderlich ist.


Rakowsky KU. Fundamentals of the Dempster-Shafer theory and its applications to system safety and reliability modelling. RTA. 2007;3-4:173-185.
Shanteau J, Weiss DJ, et al. Performance-based assessment of expertise: How to decide if someone is an expert or not. European Journal of Operational Research. 2002;136:253-263.
Kullback S, Leibler RA. On Information and Sufficiency. The Annals of Mathematical Statistics. 1951;22(1):79-86.
Howard JV. Significance Testing with No Alternative Hypothesis: A Measure of Surprise. Erkenntnis. 2009;(70):253-270
Palm G. Information and Surprise in Brain Theory. Neue Konzepte der Hirnforschung. Delfin, Suhrkamp Verlag; 1996. S. 153-173.
Palm G. Evidence, Information, and Surprise. Biological Cybernetics. 1981;42:57-68.
Recchia DR, Ostermann T, et al. Surprise, p-value, s-value and a Diagnostic Procedure to Detect Not Informative Experiments. ICNAAM, Greece, (Printing); 2015.
Ostermann T, Recchia DR, et al. Von der klinischen zur personalisierten Evidenz: ein Methodenvorschlag. Deutsche Zeitschrift für Onkologie. 2014;46:163-166.