gms | German Medical Science

Introduction: Artificial intelligence (AI) methods, especially machine learning (ML), experiences a recent boost in medical applications, including diagnostics, treatment and post treatment evaluation [1], [2]. Algorithmic ML models promise a better performance and speed compared to human individuals or expert groups for those cases.

State of the Art

This research project is embedded in the field of Auditable AI, which is related to Explainable AI. Recently, two popular initiatives have been brought forth. The first addresses the question of who is performing the audit and how the process should look like [3]. The second is an AI regulatory framework proposal that has a stronger emphasis on risk assessment [4]. In the area of Explainable AI, there exist key concepts and tools like model diagnostics or experiment databases [5]. Those aim at ML designers, researchers or users to empirically investigate algorithms on different tasks and data in a collaborative manner.

Having identified an algorithm that works for a specific use case in conjunction with a specific patient population is an achievement. However, for potential users, more ingredients are required to avoid a black box effect and to increase trust in AI’s decision outcomes [6].

Concept: At the Medical Data Integration Center of LMU University (MeDIC^LMU), for our framework, we want to emphasize current and future medical use cases we have in scope. Additionally, we will consider those, where the LMU Klinikum has special medical expertise.

Our ML auditing catalog utilizes general auditing principles like being evidence based, being objective and having transparent criteria. The three components in scope of the framework are: medical use case, algorithm and the patient (population) data. As of current planning, the auditing will be conducted on a logically meaningful combination of the aforementioned components.

The genesis of the catalog shall be threefold: Definition of dimensions, standard auditing procedures and consolidation into result. So far, the following relevant dimensions have been identified: Maturity, Transparency, Clinical Acceptance, Risk, Performance and Interpretability.

Implementation: First, we do a fit and gap analysis of existing frameworks, leading to the MeDIC^LMU auditing criteria catalog. Second, we specify, experiment with and iteratively implement a minimum viable product (MVP). Finally, we validate the MVP at MeDIC^LMU in the LMU Klinikum. Important elements of the MVP include an assessment dashboard that clarifies the intended use, assumptions, internal structures, training data and audit results of the selected component combination. In a second version, we want to automate the assessment steps of the MVP.

Lessons Learned: The described hypothetical design of the ML auditing criteria and procedures is work in progress. Limited access to algorithms, data and proprietary studies can pose a challenge that might require creative solutions. Essential are also defined minimum quality requirements for each of the three components, as entry criteria to our framework.

Our intended contribution aims at making ML model predictions being used more often, more effectively and more safely in the healthcare market. We want to raise trust and transparency on AI/ML model’s decision, especially for clinicians or healthcare researchers.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.