gms | German Medical Science

Introduction: In modern evidence based medicine, meta-analytic methods have become a powerful tool to guide objective decision-making by allowing for the formal, statistical combination of information to merge data from individual experiments to a joint result. A wide range of meta-analysis problems may be approached using a simple normal random-effects model, the normal-normal hierarchical model (NNHM) [1]. The random-effect parameter, the between-study heterogeneity, plays a central role in this context. Especially when the analysis is based on few studies, which is a common problem not only for rare diseases, consideration of external a-priori information on heterogeneity may be helpful [2]. Computational simplifications (using the bmeta R package) helped to speed up computations for Bayesian standard random-effects meta-analysis dramatically. Our aim is to use this opportunity to compare the long-run behaviour of Bayesian procedures with a selection of popular frequentist methods.

Methods: We used Monte Carlo simulations to explore frequentist properties of Bayesian estimators for different priors. We investigated a range of scenarios (heterogeneities, numbers of studies), to compare bias, MSE and coverage of Bayesian and classical frequentist estimators of the main effect, i.e., the combined treatment effect. The different approaches are illustrated using an application in pediatric transplantation [3].

Results: The results from a Bayesian approach to random-effects effects meta-analysis perform as expected, yielding sensible, coherent inference based on the provided data and prior information, also for small numbers of studies included. Most frequentist estimators of the between-trial heterogeneity show a substantial fraction of zero estimates. Although a range of different (frequentist) heterogeneity estimators exist, we found there to be relatively small differences between their general performance, while it makes a substantial difference how the uncertainty in the estimate is considered in the estimation of the main effect. Without adjustment, e.g. using the Knapp-Hartung method [4], the estimation error for the main effect will exceed the nominal level.

Discussion: While Bayesian and frequentist approaches are fundamentally different, some aspects of their results are analogous to a certain degree and may be compared to each other. In the Bayesian meta-analysis framework, in addition to the (NNHM) sampling model, the prior distribution for the two unknowns in the model needs to be specified. Results based on different prior distributions then constitute answers to different questions. The different frequentist methods on the other hand provide different, sometimes contradictory, answers to essentially the same question. While frequentist methods commonly rely on asymptotic properties, this is usually not necessary for Bayesian results once the sampling model is specified. This is especially evident in the case of meta-analysis where not only in the context of rare diseases results are commonly based on small samples. In case of little information, the use of plausible weakly informative priors is recommended [5].

Acknowledgement: This work received funding from the EU through InSPiRe (FP HEALTH 2013 - 602144).