gms | German Medical Science

49. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds)
19. Jahrestagung der Schweizerischen Gesellschaft für Medizinische Informatik (SGMI)
Jahrestagung 2004 des Arbeitskreises Medizinische Informatik (ÖAKMI)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie
Schweizerische Gesellschaft für Medizinische Informatik (SGMI)

26. bis 30.09.2004, Innsbruck/Tirol

Investigation on the improvement of predictors by bootstrap model averaging

Meeting Abstract (gmds2004)

  • presenting/speaker Norbert Holländer - University Hospital of Freiburg, Freiburg, Deutschland
  • Nicolte Augustin - University of Glasgow, Glasgow, UK
  • corresponding author Willi Sauerbrei - University Hospital of Freiburg, Freiburg, Deutschland

Kooperative Versorgung - Vernetzte Forschung - Ubiquitäre Information. 49. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds), 19. Jahrestagung der Schweizerischen Gesellschaft für Medizinische Informatik (SGMI) und Jahrestagung 2004 des Arbeitskreises Medizinische Informatik (ÖAKMI) der Österreichischen Computer Gesellschaft (OCG) und der Österreichischen Gesellschaft für Biomedizinische Technik (ÖGBMT). Innsbruck, 26.-30.09.2004. Düsseldorf, Köln: German Medical Science; 2004. Doc04gmds113

The electronic version of this article is the complete one and can be found online at: http://www.egms.de/en/meetings/gmds2004/04gmds113.shtml

Published: September 14, 2004

© 2004 Holländer et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Outline

Text

Analysing medical studies variable selection methods are often applied in order to select one final model, which includes only prognostic factors or covariates with an influence on the outcome. However, often several models fit the data equally well, but might lead to different predictions for individual patients. Focusing on the selected model only this model selection uncertainty is ignored. Furthermore, parameter estimates obtained in the selected model may be biased and the length of confidence intervals can be underestimated substantially.

We account for model selection uncertainty by averaging over a set of possible models using weights estimated from bootstrap resampling [3], [1]. Results are compared to those obtained in the full model, by common backward selection strategies and bootstrap averaging [2]. For illustration we use an example from the literature [e.g. [4]] on the prediction of the percentage of body fat. In the framework of the linear regression model the different approaches are compared in a simulation study. We consider a model with 10 covariates (7 of them with an influence on the outcome), but will also present results from investigations with a larger number of noise variables (e.g. 25 covariates, 18 noise variables). It is shown that both, common variable strategies and bootstrap based averaging methods, lead to a better point prediction of the outcome variable than the full model, if the full model contains a lot of noise. However, wheras the length of confidence intervals is underestimated substantially with common variable selection strategies we obtained correct estimates of confidence intervals with bootstrap averaging methods.


References

1.
Augustin NH, Sauerbrei W, Schumacher M. Incorporating model selection uncertainty into prognostic factor model prediction. 2002; FDM-Preprint No. 76 (http://www.fdm.uni-freiburg.de/)
2.
Breiman L. Bagging predictors. Machine Learning 1996; 26: 123-140.
3.
Buckland ST, Burnham KP, Augustin NH. Model selection: an integral part of inference. Biometrics 1997; 53: 603-618.
4.
Hoeting JA, Madgan D, Rafferty AE, Volinsky CT. Bayesian model averaging: A tutorial. Statistical Science 1999; 14: 382-417.