gms | German Medical Science

Kongress Medizin und Gesellschaft 2007

17. bis 21.09.2007, Augsburg

Assessing the reliability of workflow performance indicators: citing clinical imaging as an example

Meeting Abstract

  • Martin Lang - Siemens AG Medical Solutions (+ Lehrstuhl für Medizinische Informatik, Erlangen + Imaging Science Institute, Tübingen), Erlangen
  • Thomas Bürkle - Lehrstuhls für Medizinische Informatik, Erlangen
  • Nanda Kirpekar - New York University Medical Center, Department of Radiology, New York
  • Susanne Laumann - Siemens AG Medical Solutions, Image and Knowledge Management, Erlangen
  • Hans-Ulrich Prokosch - Lehrstuhls für Medizinische Informatik, Erlangen

Kongress Medizin und Gesellschaft 2007. Augsburg, 17.-21.09.2007. Düsseldorf: German Medical Science GMS Publishing House; 2007. Doc07gmds229

The electronic version of this article is the complete one and can be found online at:

Published: September 6, 2007

© 2007 Lang et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.



Introduction and problem statement: Clinical imaging services depend on cost-intensive imaging devices. Increasing commercial pressure and limited financial resources enforce workflow improvement efforts, short report turnaround times, high device utilizations and the adherence to delivery dates [1], [2], [3].

Objectives: Objective of this project is the assessment of the reliability of the key performance indicators, which are used to control a radiology department.

Methods: Working environment is the radiology department of the New York University Medical Center (NYUMC). Timestamps and other data were directly drawn from the radiology information system (RIS). This work focuses on the reliability of the performance indicators "device utilization" (indicator PI1) and "planning accuracy" (indicator PI2). Device utilization is the percentage of a shift a device is utilized. Planning accuracy is the deviation of the actual exam duration from its scheduled preset. Examination begin and end timestamps are used to compute those indicators. To obtain reliable data from the RIS the following checks were applied:

  • CI1: Percentage of exams having the exam begin timestamp entered
  • CI2: Percentage of exams having the exam completion tracked
  • CI3: Percentage of exams having the exam begin occurred at least 2min before the exam completion
  • CI4: Percentage of exams having the exam duration ranging from 2min to 5 times the pre-set duration for that exam type

Descriptive statistics and data visualization techniques such as bar-, 3D-surface-, or contour charts (2D projection of 3D-surface charts) etc. were used to gain insights in the reliability of the indicators.

Results: Our examination revealed that the quality of the timestamps varies during the week. Example given, quality decreases on average by 10% on Thursday and Friday afternoons. Additionally, we detected a lack of precise semantics for some of the measures. We will discuss these findings and their impact on assessing and improving current and future clinical workflows using widespread and established performance indicators.


Cavallaro A, Braitinger S, Weseloh D. Benchmarking radiologischer Prozesse – Möglichkeiten zur Workflow-Optimierung. J Fortschr Röntgenstr. 2004;176.
Pipino L, Lee YW, Wang RY. Data Quality Assessment. J Communications of the ACM. 2004;45:211-8.
Lang M, Kirpekar N, Bürkle T, Laumann S, Prokosch HU. Results from Data Mining in a Radiology Department: The relevance of data quality. Proc MedInfo. 2007; to appear.