gms | German Medical Science

Systematic reviews have traditionally been a labor-intensive and time-consuming research synthesis product. To aid in the rapid translation and adoption of scientific findings into society, the production of systematic reviews must become faster. Rapid production of systematic reviews will require incorporating more computation tools and automation of tasks informed by machine learning. More rapid production may also involve different workflows to the traditional steps of review production. Automation tools must be equivalent or superior to current methods if widely adopted. In this talk, the discussion will focus on the multifactorial reasons for the slow adoption of machine learning tools in systematic reviews. Given the importance of quality review products, the current absence of trust in automation and set-up challenges are major adoption barriers. What do theories of diffusion of innovations tell us about building trust related to automation tools? Also, do attitudes to automation tools in systematic reviews differ across disciplines; therefore, adoption of automation may be more acceptable and rapid in non-health areas like agriculture and food production. I will also briefly discuss advances in automation of systematic reviews and the tools available or being developed for each step of systematic review production, from protocol development to report generation. For tasks such as citation screening, tools are available but have slowly been adopted, perhaps because of lack of trust familiarity, concern about rejection at the peer review stage, and lack of evidence of equivalence (or superiority). For other steps, tools are available but not widely known such as knitr for dynamic report generation with R. Major efforts are still underway to develop reliable tools for data extraction and the risk of bias.

Keywords: automation, information retrieval