gms | German Medical Science

Introduction: When developing a prediction model, it is crucial to choose methods carefully. This is particularly important for survival endpoints, as the standard practice in regression analysis of predicting the mean of a distribution may not be suitable. To guide the process, the TRIPOD statement outlines a framework for specifying and reporting various aspects, including outcome, predictors, missing data, model specification, and validation [1].

Methods: We discuss advantages and disadvantages of different ways to handle each aspect and we present the choices we made for developing and validating a prediction model for kidney survival of patients with the Autosomal Recessive Polycystic Kidney Disease. Patient data was collected within an observational registry study [2]. The selection of the discussed methods is based on personal experience and unsystematic literature searches.

Results: In terms of predicting patient outcomes, several options exist, including a relative risk score, a complete survival distribution, or something in between. The choice between these options is dependent on the prediction objective. Additionally, choosing the right model is not straight forward, as the widely used Cox model is easy to apply in practice, but machine learning approaches, such as random survival forests [3], may provide more accurate predictions due to their adaptability. When a small set of predictors is desired, variable selection techniques are necessary. Here, the performance of different predictor sets has to be estimated in an unbiased way, e.g. by cross validation. Selecting an appropriate metric to evaluate model performance is also crucial and must align with the prediction objective. For example, concordance can be used as metric if the objective is to predict the order of events correctly [4]. Technical issues, such as pooling Kaplan-Meier curves, are essential to consider when handling missing values using multiple imputation [5]. When splitting a dataset into development and validation sets, it is important to ensure both groups are representative, which can be accomplished by stratified splitting. If strata are too small to split, we recommend splitting by minimization. Model validation should be prespecified and can occur within the development process via cross-validation or on a separate validation dataset. The latter option provides the possibility to make data driven choices during the development, without introducing bias in the validation process.

Discussion: Developing and validating a survival prediction model involves several factors that must be taken into account. To exemplify these elements, we provide a real clinical case study.

The authors declare that they have no competing interests.

The authors declare that a positive ethics committee vote has been obtained.