Article
Problems in the assessment of diagnostic test validity of already established biomarkers – a simulation study using the example of 14-3-3 in sporadic Creutzfeldt-Jakob disease
Search Medline for
Authors
-
André Karch
- Helmholtz-Zentrum für Infektionsforschung, Braunschweig, DE
-
Inga Zerr
- Universitätsmedizin Göttingen, Göttingen, DE
-
Annika Müller-Heine
- Medizinische Hochschule Hannover, Hannover, DE
| Published: | August 27, 2013 |
|---|
Outline
Text
Background: Whereas a definite diagnosis of neuro-degenerative diseases can be obtained in post-mortem autopsies only, diagnosis during lifetime is less valid and typically based on a combination of diagnostic criteria, including cerebrospinal (CSF) biomarkers. A classic example is the use of proteins 14-3-3, which has been included in the diagnostic criteria for Creutzfeldt-Jakob disease in 1999. In recent years there has been much discussion about the diagnostic validity of 14-3-3, since the range of differential diagnoses has changed over time and since other CSF markers have become available. However, when trying to assess diagnostic accuracy of proteins 14-3-3 in Phase IIIb studies, researchers have faced two major problems:
a) Studies in cases defined by lifetime diagnostic criteria might result in overestimation of diagnostic validity of 14-3-3 (since 14-3-3 is part of these criteria)
b) Restriction to autopsy-confirmed cases (about 25%) might cause selection bias, since decision about initiation of post-mortem autopsy is directly dependent on 14-3-3 (since patients with consistent clinical history and 14-3-3 results are less likely to be autopsied) resulting in a underestimation of diagnostic validity of 14-3-3
Methods: Therefore, we performed a series of simulation studies to estimate how both approaches might have biased the diagnostic validity of 14-3-3 in already performed diagnostic Phase IIIb studies using different sets of assumptions. We assumed different baseline validities of 14-3-3 (40-90% for specificity, 70-95% for sensitivity) as well as of the established lifetime diagnostic criteria (50-95% and 80-98%), investigated the effect of varying autopsy rates (15-75% overall) and the effect of differences in autopsy rates between 14-3-3 positive and 14-3-3 negative cases as well as 14-3-3 positive and negative non-cases.
Results: As the main result of our study we could show, that under extreme assumptions both approaches tend to bias point estimates of diagnostic validity in the a-priori considered way up to 30%. However, when using more realistic assumption sets, restriction to autopsy-confirmed cases is still leading to severe bias in the respective simulations (up to 30%), whereas inclusion of patients diagnosed with lifetime diagnostic criteria is overestimating diagnostic accuracy only marginally (up to 5%).
Conclusion: In the present simulation study, we address the important issue of selection bias caused by the use of a post-mortem gold standard in prospective diagnostic studies. We show that using a less valid lifetime gold standard instead of a rarely available post-mortem gold standard leads to more valid estimates of diagnostic validity in Phase 3b studies which investigate the diagnostic validity of an already established biomarker test.
