gms | German Medical Science

The definition of sensitivity and specificity is based on the assumption of populations composed of diseased and non-diseased individuals and considers the probabilities of correctly classifying the persons of these two populations by the diagnostic instrument to be assessed. The terms "diseased" and "non-diseased" can be interpreted in a broader sense, e.g., as distinction between two subgroups or sublevels of a disease or in a prognostic context. In any case, the existence of an independent criterion - the so-called gold standard - is necessary to judge about the correctness of the result of the diagnostic instrument.

A typical aim of diagnostic trials is to compare sensitivities and specificities of different diagnostic instruments (called ‘modalities’ here), if possible applied to the same individuals. Things can be more complex in situations with, e.g., imaging procedures where several raters have to interpret the images. Another aspect to be addressed is given by the fact that in real world the populations of diseased and non-diseased individuals are not homogeneous. Depending on the severity of the disease of interest or on other findings in the person, some of them are easier to classify than others. All that can be modelled very well with generalized mixed linear models with random patient effects and fixed effects for rater and modality, possibly including interactions. Further (co)variables can be included, but that is not considered here.

To appropriately interpret the results of the comparisons of the modalities, the user wants to have estimates not only of the differences in sensitivity and specificity, but also for the sensitivities and specificities themselves together with corresponding confidence intervals.

Such estimates could be derived from the same models with the invers link function delivering consistent results over the whole procedure. But the use of generalized linear models implies that statistical averaging over readers or differently ‘complicated’ patients in the model is not done on the original probability scale but on a logit scale or another scale with nonlinear relation to the original probabilities. In typical applications that gives much larger weights to those subgroups with very good classification results. Similar effects have been discussed in the context of meta-analysis [1].

That problem can be overcome by omitting a link function and using linear mixed models for the binary data. But then one has problems with the restricted range of possible values for probabilities. A solution might be given by the nonparametric models of Zapf [2] and Rooney [3].

This problem is illustrated in a few simulation series fit to real world data from imaging techniques. Various link functions are considered together with a linear model and with the nonparametric approach of Zapf and Rooney. Pros and cons of the methods are discussed.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.