gms | German Medical Science

Introduction: The integration of powerful deep learning methods based on state-of-the-art large language models to reliably analyze unstructured clinical texts in clinical routine remains limited. Providing such competitive models enables improvement over the current situation, by actively involving physicians in the development and utilization of these methods. To support clinical routine, we employ a model to automatically classify text segments of clinical documents, enabling more efficient information extraction tasks, such as medication extraction. We present PETGUI, an intuitive and user-friendly graphical user interface for Pattern-Exploiting Training (PET), a state-of-the-art semi-supervised prompting method for text classification tasks optimized for few-shot learning scenarios [1]. This is particularly useful in low-resource environments such as the German clinical domain.

Methods: As demonstrated in prior experiments, PET significantly reduces manual annotation efforts and computational costs, by transforming text classification tasks into cloze-style phrases [2]. PET experiments involve numerous, sometimes complex preparatory steps. We mitigate this hurdle by introducing PETGUI, which offers an intuitive graphical user interface to PET experiments and does not demand any technical knowledge. PETGUI enables non-technical users to manage the preparation, training, and evaluation of a PET text classification task in an end-to-end manner.

We developed this application using FastAPI, a modern open-source Python web framework, with direct input from physicians within an agile development environment. Two physicians thoroughly tested our app as an early-stage prototype using a self-developed questionnaire (https://tinyurl.com/a95mbhrr). PETGUI is compatible with any Slurm computing infrastructure and can be efficiently utilized with just a single graphical processing unit. PETGUI is open-source and available via GitHub (https://tinyurl.com/252hcfza). For an easy installation we recommend Docker and offer corresponding installation guidelines.

Results: We received positive feedback from testers, who reported on the successful execution of preparatory, training and evaluation steps. The testers appreciated the user-friendly interface of PETGUI. Furthermore, we received encouraging feedback from testers that they can actively influence the text classification performance by adjusting the PET hyperparameters. Training time depends on the computational setup and can be improved with additional graphical processing units. Additionally, our users triggered improvements with respect to user feedback regarding error handling. Users described the installation procedure as intuitive. Nevertheless, in future versions we plan to make the installation prerequisites more flexible.

Discussion and conclusion: With the rise of large language models and few-shot learning methods such as PET, powerful deep learning methods have become crucial in data-scarce domains. We involved domain experts, i.e. physicians, in the evaluation phase with self-developed questionnaires and offered an intuitive graphical interface, PETGUI, capable of bridging the gap between the complex usability of machine learning methods and their practical clinical application. With PETGUI, we emphasize the value of granting physicians access to state-of-the-art methods for tailoring models closely to actual clinical needs. Moreover, PETGUI demonstrates that physicians are highly motivated and willing to be involved in the development of deep learning models from the onset. The overall positive reception and constructive suggestions for future versions highlight the potential of our app in the clinical domain.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.