gms | German Medical Science

Background: In trials a huge amount of clinical information is routinely collected per subject, consuming time and resources. Only a small fraction of these data is used in standard analyses, typically for stratification and regulatory assessment purposes. The popularity of Big Data and Artificial Intelligence (AI) approaches across different fields encourages adopting and adapting AI-based methods for extracting deeper insights from clinical data. This work developed multivariate predictive models for treatment efficacy with time-to-event outcomes (overall survival, OS), using only baseline measurements.

Methods: Different methods have been applied to understand if there is valuable information not accounted for by standard methodology. In particular, traditional Cox models using typical low-dimensional, standard clinical predictor variables were compared to machine learning methods (random forests, RF) and neural networks architectures (NN), using the totality of collected baseline data from a randomized, double blind, placebo-controlled, multicenter phase III trial. 573 subjects were included; 450 variables were analyzed for 499 of them. Pre-processing of data was required to make all the predictors comparable, to deal with missing values and to transform the dataset in the most convenient format for NN implementation. Models' hyperparameters were tuned through a grid search approach to optimize performances; test sets, out-of-bag measures and cross-validation techniques were used to reduce overfitting.

Results: Results showed a substantial improvement in the percentage of variance explained, which increased from 15.9% [13.5 -17.9] (median [Q1-Q3]) for the Cox model to 28.5% [28.3-28.7] for RF; NN outperformed RF, reaching 33.4% [31.8-35.5]. The inclusion of a secondary outcome (the minimum tumor volume percentage change from baseline) in a multi-output NN allowed explaining an additional 2% of OS-related variance.

Conclusions: This work was the first attempt to deeply investigate the predictive power of all variables collected in trials; different approaches have been adopted and models should be easily applicable to other databases (given in the appropriate format). Despite the possible residual unknown amount of overfitting, the study highlighted that worthy information related to subject clinical outcome is already contained in baseline measurements. In addition, the better performances obtained with RFs and NNs algorithms, in our opinion, are mainly due to the flexibility of these new methods and their ability to naturally model the non-linear relations and higher-order interactions which characterize biological and medical systems [1]. The advantage of AI-based approaches in term of performance and exploitation of data compensates higher complexity and longer time needed for predictions.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.