gms | German Medical Science

50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds)
12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie (dae)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie
Deutsche Arbeitsgemeinschaft für Epidemiologie

12. bis 15.09.2005, Freiburg im Breisgau

Article

Send article

Resampling plans for detecting overoptimism of flexible event history models

Meeting Abstract

Search Medline for

  • Thomas Gerds - Universität Freiburg, Freiburg
  • Martin Schumacher - Universität Freiburg, Freiburg

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. Deutsche Arbeitsgemeinschaft für Epidemiologie. 50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds), 12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie. Freiburg im Breisgau, 12.-15.09.2005. Düsseldorf, Köln: German Medical Science; 2005. Doc05gmds280

The electronic version of this article is the complete one and can be found online at: http://www.egms.de/en/meetings/gmds2005/05gmds300.shtml

Published: September 8, 2005

© 2005 Gerds et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.

Outline

Text

It is a well known fact that data intensive statistical modelling produces unreliable predictions, unless reasonable large samples are available. In the context of event history analysis we are concerned about survival probabilities that are predicted by using neural nets, regression trees, or smoothing splines, or are resulting from extensive model building including variable selection and functional form diagnostic. Some mechanisms of protection against overtimism, such as penalizing terms, have been proposed separately for such methods. However, the prediction error, which is known as Brier score in the context of survival analysis ([1], [2]) is a generally applicable tool to assess and compare prognostic scores.

We investigate if resampling methods such as the bootstrap and crossvalidation can be used to adjust the prediction error and to detect overoptimism. For this we systematically generalize resampling plans such as k-fold crossvalidation and the .632 bootstrap estimator, as proposed by Efron for binary outcome [3], [4]. These methods are adapted to censored data. The study is supported by simulations with moderate sample sizes and further illustrated by using data.

Outline

References

1.
Schumacher M, E Graf, T Gerds (2003). How to assess prognostic models for survival data: A case study in oncology. Methods of Information in Medicine 42, 564-571.
2.
Graf, E, WF Sauerbrei, C Schmoor, M Schumacher (1999). Assessment and comparison of prognostic classification schemes for survival data. Statistics in Medicine 18, 2529-2545.
3.
Efron, B. (1983). Estimating the error rate of a prediction rule: Improvement on cross-validation. Journal of the American Statistical Association 78, 316-331.
4.
Efron, B. (1986). How biased is the apparent error rate of a prediction rule? Journal of the American Statistical Association 81, 461-470.