gms | German Medical Science

21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin e. V.

Deutsches Netzwerk Evidenzbasierte Medizin e. V.

13. - 15.02.2020, Basel, Schweiz

Heterogeneity of systematic reviews with meta-analyses of pharmacological, surgical and radiotherapeutic interventions in patients with advanced cancer – a meta-epidemiological study (SCOPE)

Meeting Abstract

  • Waldemar Siemens - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland
  • Guido Schwarzer - Faculty of Medicine and Medical Center, University of Freiburg, Institute of Medical Biometry and Statistics, Freiburg, Deutschland
  • Miriam S. Rohe - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland
  • Joerg J. Meerpohl - Medical Center-University of Freiburg, Institute for Evidence in Medicine (for Cochrane Germany Foundation), Freiburg, Deutschland
  • Gerhild Becker - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland

Nützliche patientenrelevante Forschung. 21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin. Basel, Schweiz, 13.-15.02.2020. Düsseldorf: German Medical Science GMS Publishing House; 2020. Doc20ebmPP8-01

doi: 10.3205/20ebm104, urn:nbn:de:0183-20ebm1044

Veröffentlicht: 12. Februar 2020

© 2020 Siemens et al.
Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). Lizenz-Angaben siehe



Background/research question: Taking heterogeneity in meta-analyses into account is crucial for drawing conclusions, e.g. for treatment guidelines. An analysis of prediction intervals (PIs) and the way of considering heterogeneity in meta-analyses of advanced cancer patients has not yet been performed and was therefore the aim of this study.

Methods: We conducted a meta-epidemiological study (CRD42019134904). Medline, the Cochrane Database of Systematic Reviews and Web of Science were searched (2009-2019). We included systematic reviews with meta-analysis of at least four randomized controlled trials with pharmacological, surgical or radiotherapeutic interventions in advanced cancer patients. The first statistically significant meta-analysis of a health-related outcome presented in the respective review was chosen for the analyses.

Heterogeneity was assessed by 95% PIs representing where the true study effects are to be expected for 95% of similar (exchangeable) future studies. We analyzed whether PIs included the null effect, e.g. Odds Ratio (OR)=1, or the opposite effect in a meta-analysis (the opposite effect of an OR=2 is OR=0.5) [1], [2].

Further, we assessed how review authors considered heterogeneity in the discussion or as subgroup analysis in the meta-analyses, in which the 95% PI included the null effect (categories: “exclusively statistically”, “PICO-scheme”, “risk of bias”, “PICO-scheme and risk of bias”).

Results: A total of 5608 hits were screened and 261 were included.

The opposite treatment effect was included by the 95% PI in 99 (37.9%) meta-analyses with a statistically significant treatment effect.

In the meta-analyses where the 95% PI included the null effect (N=197, 75.5%), heterogeneity was considered exclusively statistically in 35 (17.8%), by PICO-scheme in 93 (47.2%), by risk of bias in 10 (5.1%), and by PICO-scheme and risk of bias in 59 (29.9%) meta-analyses. Heterogeneity was considered more thoroughly in Cochrane Reviews and reviews that adhered to the PRISMA statement (p<0.001).

Conclusion: The 95% PIs indicate that about one third of future similar studies of the statistically significant meta-analyses may include the opposite treatment effect, i.e. many patients in these studies may experience negative treatment effects. Due to these patient-relevant implications, we strongly encourage review authors to consider PIs and heterogeneity in future systematic reviews and meta-analyses.

Competing interests: The authors have no conflict of interests to declare.


IntHout J, Ioannidis JP, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016 Jul 12;6(7):e010247. DOI: 10.1136/bmjopen-2015-010247 Externer Link
Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009 Jan;172(1):137-159