gms | German Medical Science

53. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

15. bis 18.09.2008, Stuttgart

Meta-analyses with rare outcomes should use adequate methods – A case study from cardiac surgery

Meeting Abstract

Search Medline for

  • Oliver Kuss - Institut für Medizinische Epidemiologie, Biometrie und Informatik, Medizinische Fakultät, Martin-Luther-Universität Halle-Wittenberg, Halle (Saale), Deutschland
  • Jochen Börgermann - Klinik für Herz- und Thoraxchirurgie, Universitätsklinikum Jena, Jena, Deutschland

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 53. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds). Stuttgart, 15.-19.09.2008. Düsseldorf: German Medical Science GMS Publishing House; 2008. DocMBIO1-2

The electronic version of this article is the complete one and can be found online at: http://www.egms.de/en/meetings/gmds2008/08gmds049.shtml

Published: September 10, 2008

© 2008 Kuss et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Outline

Text

Introduction

Meanwhile it is well understood that meta-analyses with rare binary (e.g., adverse events or safety) outcomes should not use the standard fixed-effects (FE) or random-effects (RE) model for inference on treatment effects. This has been theoretically proven [1], [2] and empirically confirmed in several simulation studies [3], [4], [5], [6], [7]. The Mantel-Haenszel (MH) method [8] has been recommended instead in these sparse data situations. However, these recommendations are frequently overlooked. For example, the three largest systematic reviews [9], [10], [11] on the comparison of off-pump and on-pump surgery in coronary artery bypass surgery, one of the most hotly debated and polarizing issues in cardiac surgery [12], all use the standard methods. This is especially problematic as the outcomes most relevant to the patient, postoperative death, stroke, myocardial infarction, or renal failure are observed very rarely, in about 1-2% of the patients.

Material and methods

In a systematic review with the common requirements (prespecified inclusion criteria, reproducible search strategy, two blinded reviewers etc.) we collected all randomized controlled trials (RCTs) comparing off-pump and on-pump surgery in coronary artery bypass surgery. We combined treatment estimates from the single studies for ten clinical outcomes (postoperative death, stroke, myocardial infarction, atrial fibrillation, RBC transfusion, inotrope requirement, wound infection, reoperation for bleeding, renal failure, and IABP support) by computing summary odds ratios [95% confidence intervals] from the FE model, the RE model, and the MH method.

Results

Our initial search yielded 106 publications, of which 57 met the inclusion criteria. These included information from 51 RCTs, 223 single study odds ratios, and 4943 patients. For clinical outcomes which are observed only rarely, the FE and the RE odds ratios always underestimate the treatment effect, in some cases considerably. For example, the FE and the RE odds ratios for postoperative death were 0.94 [0.58, 1.50], but 0.64 [0.36, 1.15] from the MH approach. For postoperative stroke the FE and the RE odds ratios were 0.71 [0.41, 1.23], but 0.50 [0.25, 1.00] from the MH approach.

Discussion

Current meta-analyses for the comparison of off-pump and on-pump surgery in coronary artery bypass surgery underestimate the benefit of the off-pump method, especially for the clinical outcomes which are most relevant to the patient. This is due to the inadequate application of the standard statistical methods. The application of the Mantel-Haenszel method would yield a more realistic picture.


References

1.
Breslow N. Odds Ratio Estimators When the Data Are Sparse. Biometrika 1981; 68 : 73-84.
2.
Greenland S, Robins JM. Estimation of common effect parameter from sparse follow-up data. Biometrics 1985; 41: 55–68.
3.
Sankey SS, Weissfeld LA, Fine MJ, Kapoor W. An assessment of the use of the continuity correction for sparse data in meta-analysis. Commun Stat – Simul C 1996; 25: 1031-1056.
4.
Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 2004; 23: 1351-75. Erratum in: Stat Med 2006; 25:2700.
5.
Lui KJ. A Monte Carlo evaluation of five interval estimators for the relative risk in sparse data. Biometrical J 2006; 48: 131-143.
6.
Kuss O, Gromann C. An exact test for meta-analysis with binary endpoints. Methods Inf Med 2007; 46:662-8.
7.
Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 2004; 23 (9): 1351-1375
8.
Mantel N, Haenszel W. Statistical Aspects of the Analysis of Data from Retrospective Studies of Disease. J Natl Cancer I 1959; 22:719-48.
9.
Wijeysundera DN, Beattie WS, Djaiani G et al. Off-pump coronary artery surgery for reducing mortality and morbidity: meta-analysis of randomized and observational studies. J Am Coll Cardiol 2005; 46: 872-82.
10.
Cheng DC, Bainbridge D, Martin JE, Novick RJ, Evidence-Based Perioperative Clinical Outcomes Research Group. Does off-pump coronary artery bypass reduce mortality, morbidity, and resource utilization when compared with conventional coronary artery bypass? A meta-analysis of randomized trials. Anesthesiology 2005; 102: 188-203.
11.
Sedrakyan A, Wu AW, Parashar A, Bass EB, Treasure T. Off-pump surgery is associated with reduced occurrence of stroke and other morbidity as compared with traditional coronary artery bypass grafting: a meta-analysis of systematically reviewed trials. Stroke 2006; 37: 2759-69.
12.
Sellke FW, DiMaio JM, Caplan LR et al. Comparing on-pump and off-pump coronary artery bypass grafting: numerous studies but few conclusions: a scientific statement from the American Heart Association council on cardiovascular surgery and anesthesia in collaboration with the interdisciplinary working group on quality of care and outcomes research. Circulation 2005; 111: 2858-64.