gms | German Medical Science

Background/research question: Clinical practice guidelines (CPGs) assist healthcare professionals in making evidence-based decisions. As a strategy to overcome barriers to their implementation in Physiotherapy (PT), the Guideline Assistant for Evidence-based Practice (GAEP), a knowledge translation tool based on a Large Language Model (LLM), was developed.

This study aimed to 1) Investigate German physiotherapists’ (PTs) experiences in using the digital tool 2) Explore how they imagine its utilization in their clinical context and 3) Analyse potential perception differences between outpatient and inpatient PT settings.

Methods: Six focus group interviews were conducted, with three in a university hospital inpatient setting and three in outpatient PT practices. The interviews were recorded, transcribed, and analysed using both inductive and deductive methods. Additionally, quantitative data was assessed using the User Experience Questionnaire (UEQ) and the System Usability Scale (SUS), as well as a self-developed questionnaire.

Results: The experience of the 20 participating PTs (11 outpatient and 9 inpatient) with the tool was positive, although participants criticized its prolonged response time. Its utilization was believed to be determined by the availability of time and the degree of workplace digitization. PTs highlighted the tool's potential to support answering clinical questions during daily practice. There were no considerable differences in experience across settings, whereas utilization perspectives varied depending on the clinical setting. The quantitative analysis revealed a score of 77.58% for the SUS and results above average in all five domains (Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, Novelty) of the UEQ. Compared to previous prototypes, GAEP performed better across all three questionnaires, highlighting its perceived usability for enhancing evidence-based practice.

Conclusion: LLM-based tools like GAEP might have the potential to enhance PTs’ guideline adherence. For successful implementation, it is essential to assess the specific needs of each setting. Although the results of the questionnaires are promising, future research should focus on validating the tool’s accuracy in a broader way, as well as comparing and refining its overall usability in terms of speed and content.

Competing interests: None to declare.