gms | German Medical Science

66. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 12. Jahreskongress der Technologie- und Methodenplattform für die vernetzte medizinische Forschung e. V. (TMF)

26. - 30.09.2021, online

Comparison of Variations and Extensions of the Beta-Binomial to Standard Meta-analysis Methods: A Simulation Study

Meeting Abstract

Suche in Medline nach

  • Tim Mathes - Witten/Herdecke (University), Köln, Germany
  • Oliver Kuß - German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine University Düsseldorf, Institute for Biometrics and Epidemiology, Düsseldorf, Germany

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 66. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 12. Jahreskongress der Technologie- und Methodenplattform für die vernetzte medizinische Forschung e.V. (TMF). sine loco [digital], 26.-30.09.2021. Düsseldorf: German Medical Science GMS Publishing House; 2021. DocAbstr. 154

doi: 10.3205/21gmds088, urn:nbn:de:0183-21gmds0884

Veröffentlicht: 24. September 2021

© 2021 Mathes et al.
Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). Lizenz-Angaben siehe



Introduction: In previous studies the standard beta-binomial (BB) model has shown good statistical properties for meta-analyses of binary outcomes, especially in challenging situations with a low number of studies or events [1], [2]. The standard BB-model (CR) assumes a common correlation (rho) of observations within study groups across treatment. Recently, we have introduced a common-beta model (CB) and a common-beta model with an additional multiplicative overdispersion term (CBO).

Our objective was to compare these models to standard methods for estimating the summary odds-ratio.

Methods: We compared the BB models to the following standard methods for meta-analysis in a simulation study:

Generalized linear mixed model with a fixed intercept and random treatment effects (GLFR)

Random-effects model using a refined version of Hartung-Knapp-Sidik-Jonkman (RHKSJ) 95% confidence intervals (CIs) that is the modification is only applied if the 95%CIs of the original HKSJ-method are smaller than Wald-type CIs. Heterogeneity variance estimated with the Paule-Mandel method.

The parameters of the simulation were informed by actually performed meta-analyses [3]. We considered scenarios with median study numbers of 8 and 3, respectively, which mirrors non-Cochrane and Cochrane meta-analyses [3], [4]. Results are only reported separately for the scenarios if the results differ. To compare the methods we considered median bias and mean coverage to the 95%CI.

In addition, we performed a meta-regression on the simulated data-set to analyse the impact of the number of studies in meta-analyses, event probability in the control group, and heterogeneity on coverage of the 95%CIs.

Results: None of the methods had problems with convergence. Median bias was small for all methods. Only GLFR fell relevantly below the nominal coverage probability. Meta-regression suggested that the risk of unsatisfying coverage for GLFR heavily increases with increasing heterogeneity. In the Cochrane scenario, the risk of not satisfying coverage increased with increasing event probability for the CR and increased with decreasing event probability for the RHKSJ. Coverage of CBO was least affected by different data situations.

Discussion: This study confirms previous findings that beta-binomial models work well for meta-analyses in the case of (very) low event probability and (very) few studies in the meta-analysis. Our simulation suggests that the performance of CBO is least sensitive for changes in event probability and heterogeneity. A practical advantage of the new BB models is that they are easy to implement because they can be estimated with standard software packages for count panel data from econometrics. RHKSJ also performed satisfactorily but the probability of coverage below the nominal level increased in very challenging data situations. One reason for this finding is probably, that in contrast to the other models, this model requires a continuity correction, when there are studies with zero events.

The finding that coverage probability of GLFR decreases with increasing heterogeneity of the treatment effect was also observed in previous simulations and raises some concerns for applying this model in practice [5].

Conclusion: BB models should be considered for meta-analyses of few studies and in the case of low event probabilities.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.


Mathes T, Kuss O. A comparison of methods for meta-analysis of a small number of studies with binary outcomes. Research synthesis methods. 2018;9(3):366-81.
Kuss O. Statistical methods for meta-analyses including information from studies without any events—add nothing to nothing and succeed nevertheless. Statistics in medicine. 2015;34(7):1097-1116.
Turner RM, et al. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. Int J Epidemiol. 2012;41:818–827.
Page MJ, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLOS Med. 2016;13:e1002028.
Jackson D, et al. A comparison of seven random-effects models for meta-analyses that estimate the summary odds ratio. Statistics in medicine. 2018;37(7):1059-1085.
Mathes T, Kuss O. Beta-binomial models for meta-analysis with binary outcomes: Variations, extensions, and additional insights from econometrics. Research Methods in Medicine & Health Sciences. 2021:2632084321996225.