gms | German Medical Science

21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin e. V.

Deutsches Netzwerk Evidenzbasierte Medizin e. V.

13. - 15.02.2020, Basel, Schweiz

The impact on statistical significance when switching to random-effects and to the Hartung-Knapp method in systematic reviews of advanced cancer patients – sensitivity analyses from a meta-epidemiological study (SCOPE)

Meeting Abstract

  • Waldemar Siemens - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland
  • Guido Schwarzer - Faculty of Medicine and Medical Center, University of Freiburg, Institute of Medical Biometry and Statistics, Freiburg, Deutschland
  • Miriam S. Rohe - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland
  • Joerg J. Meerpohl - Medical Center-University of Freiburg, Institute for Evidence in Medicine (for Cochrane Germany Foundation), Freiburg, Deutschland
  • Gerhild Becker - Medical Center, University of Freiburg, Faculty of Medicine, University of Freiburg, Clinic for Palliative Care, Freiburg, Deutschland

Nützliche patientenrelevante Forschung. 21. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin. Basel, Schweiz, 13.-15.02.2020. Düsseldorf: German Medical Science GMS Publishing House; 2020. Doc20ebmS1-V2-03

doi: 10.3205/20ebm007, urn:nbn:de:0183-20ebm0074

Veröffentlicht: 12. Februar 2020

© 2020 Siemens et al.
Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). Lizenz-Angaben siehe http://creativecommons.org/licenses/by/4.0/.


Gliederung

Text

Background/research question: In biomedical research, the assumptions of the random-effects model are often more appropriate than of the fixed-effect model [1]. Further, it has been suggested to replace the standard method in random-effects meta-analyses by the Hartung-Knapp method [2].

This project aims to examine the impact on statistical significance when using the random-effects model and the Hartung-Knapp-method, respectively.

Methods: A meta-epidemiological study (CRD42019134904) was conducted. Three databases were searched from 2009-2019: Medline, the Cochrane Database of Systematic Reviews and Web of Science. We included systematic reviews with meta-analysis of at least four randomized controlled trials with pharmacological, surgical or radiotherapeutic interventions in advanced cancer patients. The first statistically significant meta-analysis of a health-related outcome presented in the respective review was chosen for the analyses.

Analyses were performed with the statistical software R.

Results: A total of 5608 hits were screened. Finally, 261 systematic reviews were included. A fixed-effect model was used in 138 (52.9%) and a random-effects model in 123 (47.1%) meta-analyses.

Six of 138 fixed-effect meta-analyses (4.4%) lost statistical significance when recalculating them. When switching from a fixed-effect to a random-effects model, 19/132 (14.4%) of the meta-analyses were no longer statistically significant.

Using the Hartung-Knapp method instead of the standard method for random-effects meta-analyses resulted in 43/123 (35.0%) meta-analyses that were no longer statistically significant.

Conclusion: A relevant number of meta-analyses lost statistical significance when applying up-to-date meta-analysis methods. This implicates that many meta-analyses cited in medical literature and treatment guidelines, e.g. to recommend drugs, would probably come to a different conclusion when applying the random-effects model or the Hartung-Knapp method. We strongly advocate the use of up-to-date methods and sensitivity analysis when presenting results from meta-analyses.

Competing interests: The authors have no conflict of interests to declare.


References

1.
Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009 Jan;172(1):137-159
2.
Jackson D, Law M, Rücker G, Schwarzer G. The Hartung-Knapp modification for random-effects meta-analysis: A useful refinement but are there any residual concerns? Stat Med. 2017 Nov 10;36(25):3923-3934. DOI: 10.1002/sim.7411 Externer Link