Article
Nonparametric Limits of Agreement for small to moderate sample sizes – a simulation study
Search Medline for
Authors
-
Maria Elisabeth Frey
- Charles River Laboratories Copenhagen A/S, Dept. of Toxicology, Lille Skensved, Denmark
-
Hans Christian Petersen
- University of Southern Denmark, Dept. of Mathematics and Computer Science, Odense M, Denmark
-
Oke Gerke
- Odense University Hospital, Dept. of Nuclear Medicine, Odense C, Denmark; University of Southern Denmark, Dept. of Clinical Research, Odense C, Denmark
| Published: | February 26, 2021 |
|---|
Outline
Text
Background: The assessment of agreement in method comparison and observer variability analysis of quantitative measurements is usually done by the Bland-Altman Limits of Agreement, where the paired differences are implicitly assumed to follow a normal distribution. Whenever this assumption does not hold, the 2.5% and 97.5% percentiles are obtained by quantile estimation. In the literature, empirical quantiles have been used for this purpose.
Methods: Databases such as JSTOR (Journal Storage), ScienceDirect, the online journal platform “Taylor & Francis Online” and those maintained by PubMed/Medline in the NCBI (National Center for Biotechnology Information) were searched for quantile estimator, nonparametric quantile estimator, nonparametric kernel quantile estimator, subsampling quantile estimator, and new quantile estimator. Fourteen nonparametric quantile estimators were chosen, three of which are sample quantile estimators, four are subsampling quantile estimators, two are Kernel quantile estimators, and five are other quantile estimators. In a simulation study, sample sizes between 30 and 150 and different distributions of the paired differences (normal; normal with 1%, 2%, and 5% outliers; exponential; and lognormal) were employed. The performance of the 14 estimators in generating prediction intervals was measured by their respective coverage probability for one newly generated observation.
Results: The simplest quantile estimator, employing the smallest and largest observation as estimates for the 2.5% and 97.5% percentiles, respectively, for n=30 outperformed all the other estimators, whereas only one other sample quantile estimator (a weighted average of two order statistics) achieved the nominal 95% level in all distributional scenarios for the remaining sample sizes of n=50, 80, 100, and 150. The Harrell-Davis subsampling estimator and estimators of the Sfakianakis-Verginis type achieved at least 95% coverage for all the investigated distributions for sample sizes of at least n=80, except in the case of the exponential distribution, where the coverage probability was at least 94%.
Conclusion: Simple sample quantile estimators based on one and two order statistics can be used for deriving nonparametric Limits of Agreement. For sample sizes exceeding 80 observations, more advanced quantile estimators, such as the Harrell-Davis and estimators of Sfakianakis-Verginis type, which use all the observed differences are equally applicable, but may be considered intuitively more appealing than simple sample quantile estimators that are based on only two observations per quantile.
The authors declare that they have no competing interests.
The authors declare that an ethics committee vote is not required.
