gms | German Medical Science

Introduction: Clinicians often use cutpoints or decision-thresholds to decide e.g. whether or not a patient with a depression score of, say, 20 on the BDI needs treatment for her or his depression. Since these cutpoints are so important for clinical decision making, a large number of studies use Receiver-operating characteristic methods to empirically establish the optimal cutpoints for diagnostic tools. Optimal cutpoints are those cuptpoints that maximize – or minimize – a specific metric, e.g. maximize the Youden Index, or minimize the absolute difference between sensitivity and specificity.

State of the art : At present optimal cutpoints are almost always determined “empirically”, i.e. the metric that is optimized is calculated at all scores of the diagnostic tool that were found in the sample and the score that is associated with the best metric is determined as the optimal cutpoint. A number of R-packages [1] and web-based Shiny interfaces [2] exist to help applied researchers use these empirical methods to determine optimal cutpoints. In contrast to this, the biometric literature has shown that optimal cutpoints can be estimated more efficiently using distribution based or kernel based methods [3], [4]. Some recent studies found that the variability of the latter methods is lower than that of empirical methods, particularly in small samples [5].

Concept: We developed the R package cutpointr [6] to improve on the empirical method to determine optimal cutpoints and to allow for tidy and efficient estimation of optimal cutpoints. It improves existing solutions in at least four dimensions. First, in addition to empirical methods, cutpointr povides kernel estimation and distribution-based methods to estimate optimal cutpoints. Second, cutpointr was designed to use a large number of the metrics that have been put forward to date and enables the user to very easily define their own functions for metrics. Third, cutpointr provides a parallelizable routine to estimate cutpoints’ variability and various in- and out-of-bag performance metrics. This is important because only parallelization and efficient calculation of optimal cutpoints enable large-scale simulation studies and resampling methods to determine cutpoint variability. Fourth, cutpointr provides several convenient built-in plotting functions, e.g. for ROC curves and precision recall plots.

Implementation: Cutpointr is distributed as an R-package [6]. Cutpointr follows current tidy programming practices to allow for efficient estimation, use in simulation studies, and interplay with functions from the tidyverse. A shiny-interface is currently under development.

Lessons Learned: We found that using the tidy framework for package development made it necessary to extensively adapt existing functions. At the same time, this is the main reason why cutpointr compares favorably to existing solutions with regard to usability.

Die Autoren geben an, dass kein Interessenkonflikt besteht.

Die Autoren geben an, dass kein Ethikvotum erforderlich ist.