gms | German Medical Science

Note: This abstract was already published [1]. The quotation symbol “” indicates original, unchanged phrases of that publication.

Background: “Although AI algorithms and applications become more and popular in the healthcare sector, only few institutions have an operational AI strategy. ... A commonly agreed AI auditing framework that provides best practices and tools could help speeding up the adoption process. In this paper, we first highlight important concepts in the field of AI auditing and then restructure and subsume them into an ML auditing core criteria catalog.”

???????Methods: “We conducted a scoping study where we analyzed sources being associated with the term ‘Auditable AI’ in a qualitative way. ... The literature base was compared using inductively constructed categories. Afterwards, the findings were reflected on and synthesized into a resulting ML auditing core criteria catalog.”

Results: The catalog consists of 30 questions that are grouped around the categories: “Conceptual Basics, Data & Algorithm Design and Assessment Metrics”. The category Conceptual Basics consists of:

• AI Opportunities vs. AI Risks
• Risk Management
• Methodology
• Audit Process
• Quality Assurance

Data & Algorithm Design covers:

• Data Properties
• Algorithm Design

And the last category Assessment Metrics concerns questions of:

• Qualitative Assessment
• Quantitative Assessment

Each question relates to a certain paragraph in the results section, where the context of the question is established. They follow the same topical sequence as the related paragraphs. For example, question 6) of the ML auditing core criteria catalog asks: “Are the implications in case the ML use case falls in the 'high risk' category of the EU AI Act understood?”

The context is established in the paragraph about “Legal Acts/Policies and Standards”, where it says that “the EU AI Act aims at specifying an audit process including conformity assessments ... (as cited in [2]). The legislative draft mandates for 'unacceptable' and 'high risk' applications to describe the type of information AI producers have to provide, the form of the information and the addressees of the ML algorithm in order to allow a pre-market assessment (as cited in [3], p. 4).” In March 2024 the European Parliament has passed the EU AI Act, which has been approved before in the EU council by EU member countries [4].

The questions are balanced in terms of breadth and width to provide an operationalizable starting point for diverse stakeholders.

Discussion: “Our consensus-based ML auditing criteria catalog is intended as a starting point for the development of evaluation strategies by specific stakeholders. We believe it will be beneficial to healthcare organizations that have been or will start implementing ML algorithms. Not only to help them being prepared for any upcoming legally required audit activities, but also to create better, well-perceived and accepted products. Potential limitations could be overcome by utilizing the proposed catalog in practice on real use cases to expose gaps and to further improve the catalog. Thus, this paper is seen as a starting point towards the development of a framework, where essential technical components can be specified.”

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.