Artikel
RobotReviewer can be supportive in risk of bias assessment in nursing RCTs
Suche in Medline nach
Autoren
-
Julian Hirt
- Institute for Applied Nursing Science, Department of Health, FHS St.Gallen, Schweiz; International Graduate Academy, Institute for Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Deutschland
-
Jasmin Meichlinger
- Institute for Applied Nursing Science, Department of Health, FHS St.Gallen, Schweiz
-
Petra Schumacher
- Institute for Nursing Science, Department for Nursing Science and Gerontology, UMIT – Health & Life Sciences University, Österreich
-
Gerhard Mueller
- Institute for Nursing Science, Department for Nursing Science and Gerontology, UMIT – Health & Life Sciences University, Österreich
| Veröffentlicht: | 25. September 2020 |
|---|
Gliederung
Text
Background and current state of (inter)national research: The online tool RobotReviewer enables an automated risk of bias assessment. An evaluation study including 1180 RCTs from different health topics yielded a sensitivity between .28 and .76 and a specificity between .72 and .90 for detecting a low risk in selection, performance, and detection bias. The performance of the RobotReviewer for nursing-related RCTs is unclear.
Questions and objectives: To evaluate the prognostic validity and reliability of RobotReviewer’s risk of bias assessment in nursing-related RCTs.
Methods or hypothesis: We conducted a retrospective diagnostic study including RCTs from Cochrane reviews that contained nurs* in the title field. The index test was the RobotReviewer assessing randomisation, allocation concealment, and blinding. The reference test represented the human assessment of included Cochrane Reviews. Sensitivity, specificity, predictive values, and accuracy were analysed.
Results: The selection process yielded 190 RCTs in 23 Cochrane reviews. Sensitivity ranged from .44 to .88 and specificity from .48 to .95. Positive predictive value was highest for allocation concealment (.79) and lowest for blinding assessors (.25). Cohen’s Kappa was moderate for randomization (.52), allocation concealment (.60), and for blinding of personal/patients (.43). Blinding of outcome assessors had only slight agreement (.04).
Practical implications: This evaluation of the RobotReviewer’s performance in nursing-related RCTs yielded moderate agreement with the human assessment of Cochrane reviews for randomisation and allocation concealment as well as an adequate sensitivity for detecting low risk of selection bias. Therefore, the use of the RobotReviewer for the risk of bias assessment in nursing-related RCTs can be supportive, but should be supervised by human assessment.
