gms | German Medical Science

MAINZ//2011: 56. GMDS-Jahrestagung und 6. DGEpi-Jahrestagung

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V.
Deutsche Gesellschaft für Epidemiologie e. V.

26. - 29.09.2011 in Mainz

Bayesian model evaluation for longitudinal count data in clinical trials: application to vertigo data

Meeting Abstract

Search Medline for

  • Christine Adrion - IBE, Ludwig-Maximilians-Universität, München
  • Ulrich Mansmann - IBE, Ludwig-Maximilians-Universität, München

Mainz//2011. 56. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds), 6. Jahrestagung der Deutschen Gesellschaft für Epidemiologie (DGEpi). Mainz, 26.-29.09.2011. Düsseldorf: German Medical Science GMS Publishing House; 2011. Doc11gmds098

doi: 10.3205/11gmds098, urn:nbn:de:0183-11gmds0984

Published: September 20, 2011

© 2011 Adrion et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.



Background: In clinical trials with longitudinal counts as primary endpoint, interest often focuses on the entire profile (e.g., difference in slope between groups) instead of single time point analysis. Accordingly, the primary efficacy analysis of longitudinal counts is based on a careful pre-defined statistical model which has to be specified in the statistical section of the protocol or the statistical analysis plan. This poses challenges for model choice and model validation since the presumed model may affect the conclusion about the key scientific parameters. Besides deciding on the random effects and fixed effects structure, there are different distributions which can be assumed to model the count efficacy data: Poisson, zero-inflated, negative binomial (NB), or normal scale after a variance-stabilizing transformation. As frequentist strategies for model assessment and model diagnosis are cumbersome in random effects settings and have several limitations, it is of interest to explore Bayesian instruments which provide the needed decision support.

Material and Methods: We perform a simulation study to investigate the discriminatory power of Bayesian tools for model criticism and apply them to an open trial on vertigo attacks. A novel Bayesian approach based on integrated nested Laplace approximation (INLA) proposed by Rue et al. [1] is used for fitting generalized linear mixed models (GLMMs). With this approach the posterior marginal distributions are accurately approximated in a fully automated way. To challenge the model INLA methodology enables the computation of leave-one-out predictive measures without reanalyzing the posited model. We evaluate rival GLMMs for count outcome according to the DIC, PIT, log marginal likelihood, and by using proper scoring rules which allow ranking and validating various model alternatives.

Results: The ability of the logarithmic scoring rule and DIC to discriminate between Poisson and NB mixed-effects model was illustrated in a simulation study for a longitudinal NB data generating process which mimics characteristics of the clinical trial. Simulation procedure assumed varying degrees of overdispersion and sample sizes. Under the given conditions, it becomes obvious that the naïve choice of a random effects Poisson model is often inappropriate for real-life count outcomes [2].

Discussion: We conclude that Bayesian methods are not only appealing for inference but notably provide better insight into different aspects of model performance, such as forecast verification, calibration checks, or model selection techniques. The mean of log score provides a robust tool for model ranking which is not sensitive to sample size.


Rue H, Martino S, Chopin N. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 2009;71(2):319-92.
Adrion C, Mansmann U. Bayesian model evaluation for longitudinal count data in a clinical trial setting: application to vertigo data. 2011 [under review].
Committee for Medicinal Products for Human Use (CHMP), European Medicines Agency (EMA). Guideline on Missing Data in Confirmatory Clinical Trials. EMA/CPMP/EWP/1776/99 Rev. 1. London, UK. Date for coming into effect: 1 January 2011. Available from: External link
Czado C, Gneiting T, Held L. Predictive model assessment for count data. Biometrics. 2009;65(4):1254-61.