gms | German Medical Science

Introduction: Artificial Intelligence (AI) is becoming increasingly important in many areas of everyday life. For this reason, the European Union (EU) has created the AI Act, which covers issues that need to be considered when developing and deploying AI systems. Among other things, this legal framework classifies AI systems such as those used in hospitals, which are part of critical infrastructure, in the high-risk group. [1] This emphasizes the importance of proper handling of AI systems in medical care. Addressing this topic even before the AI Act was published, an exploratory literature review was conducted to find out what needs to be considered when using AI systems in hospitals.

Methods: To identify relevant scientific literature addressing the appropriate use of AI systems in clinical context a literature review was performed using the terms Artificial Intelligence (/AI) governance health, Artificial Intelligence (/AI) principles health, Artificial Intelligence (/AI) principles clinical care, Artificial Intelligence (/AI) governance clinical care, Machine Learning (/ML) governance health, Machine Learning (/ML) principles health, Machine Learning (/ML) governance clinical care and Machine Learning (/ML) principles clinical care. The identified papers were filtered out according to the suitability of the title and the abstract. Moreover, they had to focus on the application of AI-based algorithms in hospital care and include principles and regulations for the introduction of AI-based algorithms in hospitals.

Results: A total of 3,188 publications were identified. During the development of this work, an additional 20 publications were identified. After filtering the identified publications, 34 publications were considered for this work. The following categories were identified by the authors during the review: transparency and documentation, human autonomy, handling of data, education and further training, and AI committee.

Figure 1 [Fig. 1] shows the identification process according to the PRISMA scheme. Table 1 [Tab. 1] shows the included papers.

Discussion: The identified papers mostly addressed the same problems such as lack of transparency, biased trainings data, lack of documentation, and others. However, there were topics in the literature with a consensus for instance in favor of an AI board. Furthermore, also contradicting or indecisive views in managing some topics appeared such as accountability. The AI Act addresses many of the points found in the literature. It proposes solutions for contradictive topics as e. g. accountability but also deals with topics such as transparency and AI board [1].

Conclusion: The findings of the literature review are somewhat reflected in the AI Act created by the EU. This mainly concerns the topics transparency, responsibility, documentation, AI board as well as data and data governance [1]. This underlines the importance of research in this area of science and shows that the research carried out prior to the introduction of the AI Act is in some ways aligned and addresses some of the same important issues.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.