gms | German Medical Science

Introduction: Inappropriate handling of missing values can lead to biased results in diagnostic studies [1]. While there are well-known methods to handle missing values in the reference standard [2], missing values in the index text are often ignored or replaced by single imputation [3], [4]. One reason for this may be low awareness of methods to handle missing values in the index test and the lack of systematic comparison of these methods [5]. Therefore, this simulation study compares the performance of methods for estimating the area under the ROC curve (AUC) of a continuous index test with missing values in a diagnostic study.

Methods: We simulated data including a continuous index test with missing values, a binary reference standard, and 3 covariates, one binary and two continuous. We varied the following factors: the sample size, the true AUC, the prevalence of the target condition, the percentage of missing values, the correlation between the index test and the covariates, and the missingness mechanism (missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR)). The methods for estimating the AUC were chosen from a previous review of methods for handling missing values in diagnostic studies [5]. They comprised complete case analysis, several multiple imputation (MI) approaches, single imputation with empirical likelihood confidence intervals, robust inverse probability weighting, a convolution-based approach, and a kernel-based approach. To determine performance, we examined bias, root mean squared error (RMSE), coverage, and power.

Results: Preliminary results show that bias and RMSE increase and coverage decreases with a higher proportion of missing values, and with MAR and MNAR compared to MCAR. MI approaches and the robust inverse probability method tend to perform better than the other methods under MAR. In the case of MNAR, all methods are rather biased, especially with many missing values. The complete case analysis shows a high coverage probability under MCAR and MAR while the MI approaches and the robust inverse probability method seem to have good coverage probabilities under MAR and MNAR. Under MNAR and a few missing values, the convolution-based and kernel-based estimates are less biased than the other methods and have a high coverage probability. In all other cases, however, they have a much lower coverage probability and are more biased.

Discussion: The choice of the “best” method depends, among others, on the missingness mechanism and the percentage of missing values. If MCAR is a reasonable assumption, a complete case analysis may lead to valid results, but the statistical power is decreased. However, MI approaches or robust inverse probability may be a better choice for MAR.

Conclusion: Our results emphasize the necessity of a thorough data examination and consideration of the missingness mechanisms. Sensitivity analysis should be conducted to explore the robustness of the results under different missingness assumptions.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.