gms | German Medical Science

Background: Horizon scanning projects systematically search a wide range of heterogeneous sources, including news data, funding calls, peer-reviewed literature, or clinical trial registrations, to detect signs of innovation. This culminates in a highly topic-dependent and unstandardised data mix, which has implications for efficiency and time. To address some of these challenges, we created two proof-of-concept tools (SCANAR and AIDOC) that aim to facilitate information-retrieval and data-sifting processes involved in horizon scanning projects.

Methods: An interdisciplinary team of data scientists, information specialists, horizon scanning and evidence synthesis specialists contributed to the design and testing of these two tools. They were programmed in Python, and user interfaces were built with Streamlit. The information specialist provided advice on sources for the tools to search and fields for information retrieval. To support news scans, SCANAR (Search Companion for Advanced News Article Retrieval) interacts with the Google News API, enabling users to perform systematic searches with self-supervised ranking and full-text scraping. AIDOC (Artificial Intelligence Document Organiser and Classifier) is a flexible, semantic similarity-based application using neural-network embeddings to prioritise spreadsheet entries, using binary labels or categorical data samples in a human-in-the-loop system.

Results: Both tools were piloted in horizon scans conducted within the NIHR Innovation Observatory. SCANAR was used for retrieving news articles on several scans. It increased the breadth of information retrieved for each project without compromising time or resource capacity. Additionally, SCANAR provided automated search documentation which helped to enhance the reporting of news scanning methods. AIDOC was used to re-order information in spreadsheets for two scans (bioengineering innovations and funding scan). It provided a semi-automated way for prioritising information which increased efficiency in screening and selection of relevant signs of innovation. AIDOC proved particularly useful in the rapid response bioengineering scan where time was extremely limited. We retroactively evaluated AIDOC on two datasets and found that on average, it prioritised 95% of all relevant records into the first half of screened records.

Conclusion: Health innovation signals may arise from a heterogeneous range of sources which poses a challenge for the delivery of comprehensive and timely horizon scanning projects. Two novel tools, SCANAR and AIDOC, demonstrated that artificial intelligence and self-supervised ranking contributes to expanding the scope of horizon scanning searches with potential to increase screening speed and ultimately reduce workload.