gms | German Medical Science

Objectives: Classic statistical approaches in clinical study design require tens of thousands of subjects to be included for the investigation of multivariate analysis, which requires large multi-center studies or non-permissible amount of time to collect the required data. Machine Learning (ML) as part and Artificial Intelligence promises to make predictions on smaller data sets in the order of a few thousand cases. Therefore, this study evaluates 7 standard ML learning classifiers to determine the feasibility of a data-driven approach to predict non-responders in patients with non-unions (NU) in fracture healing.

Methods: The dataset comprises 617 patients with NU who reached or failed healing out of a single center data set of 1140 NU (2009 - 2016). The class distribution of responders (R) vs. non-responders (NR) was 198:419 given the definition of NR being that the patients did not respond to initial treatment after six months requiring follow-up surgery. The dataset includes 160 features of which we have selected 7 (according to literature) for this feasibility study: infection, NU classification according to Weber/Cech, number of soft tissue surgeries, use of autologous bone graft during surgery, ASA classification, cardiovascular disease and diabetes mellitus. The data were collected pre-treatment and thus could be used as predictors of R and NR. Data were divided into a training and test data-set after under-sampling the R class to give a 50:50 distribution in the training data-set to avoid class imbalance bias. The following standard ML classifiers were applied using the python pandas framework and the scikit-learn library: logistic regression, support vector machine (SVM), decision tree (DT), k-nearest neighbor (kNN) classifier, random forest classifier, AdaBoost and gradient boost models. As performance measure, we calculated precision and recall for predicting the two respective classes.

Results and Conclusion: The testing data comprises 80 cases (46 R vs 34 NR). The kNN, SVM, DT and random forest classifier performed with recall rates under 60%. Logistic regression showed a precision for R of 72% and a recall of NR of 76%. The best performing classifier was AdaBoost with 79% precision for R and 82% recall for NR.

The results demonstrate that a high-quality dataset can lead to model predictions of NR for treatment of NUs in fracture healing with recall rates of just over 80%. This suggest that given the NR group being twice as large as the R group in the original data set, the lower recall for R has little impact vs the overall classification in a prospective study. The AdaBoost classifier could already deliver clinical impact with respect to a more personalized treatment planning as the majority of patients that will not respond to the first treatment protocol ca be identified before starting treatment.