gms | German Medical Science

53. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

15. bis 18.09.2008, Stuttgart

Software-Engineering in Multidisciplinary Teams in Health Informatics: Quality Assurance with GREME Reviews

Meeting Abstract

Search Medline for

  • Christa Weßel - Institute of Occupational, Social and Environmental Medicine, Johannes Gutenberg-University Mainz, Germany
  • Sandra Geisler - RWTH Aachen University, Aachen, Germany
  • Uta Christoph - RWTH Aachen University, Aachen, Germany

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 53. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds). Stuttgart, 15.-19.09.2008. Düsseldorf: German Medical Science GMS Publishing House; 2008. DocMI20-1

The electronic version of this article is the complete one and can be found online at:

Published: September 10, 2008

© 2008 Weßel et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.




Quality Management (QM) is an integral part of process models for software engineering. Due to the neglect of quality assurance (QA) measures, such as reviews, software projects face a high risk to fail or be postponed [1], [2]. Software-engineering teams and companies look for effective and efficient QA measures that can easily be integrated in their projects. Inspections (technical reviews) are used for crucial modules and documents and are performed by IT experts. Other less formalized review types, that integrate domain experts and users, can and should also be used for the QM in a software engineering project. Health informatics with its multidisciplinary approach in software-engineering and maintenance can facilitate the development and use of alternative review methods.


Reviews are time consuming procedures and require experience in moderation and in product centred software assessment. Common review types are inspections (technical reviews), team reviews, walkthroughs, pair programming, peer desk check, pass around and ad hoc review. The level of formalization follows a descending order: from inspections, that are well defined and formalized, down to ad hoc reviews, which are casual reflections and discussions during the development process [1], [2]. A GREME Review (Group discussion and Email-based Evaluation) combines team review and pass around. The purpose is to facilitate structured, comprehensible and economical quality assurance during software engineering for non-crucial modules and documents.


A research team developed and used the GREME Review during its work on a web-based information system on hospitals [3]. GREME can be applied in the different phases of software engineering: analysis, design, and implementation. The possible products to be reviewed may comprise user interfaces, documentations, manuals, user guides and working processes. The review team includes moderator, author (developer) and peers, that is IT and domain experts (health professionals, economists, others) and users.

GREME follows a defined workflow, which can be tailored to the need of a particular software engineering project: prototype releases alternate with pass arounds and moderated group discussions (team review). The discussions are integrated in routine meetings, for instance weekly department meetings, and do not exceed 30 minutes, to reduce the workload. The team discusses the current prototype, identifies errors and open requirements and proposes solutions. From the emails, submitted by the peers in the pass around cycles, comprehensible insights are gained through qualitative text analysis performed by the moderator. The author uses the results of the analysis for the identification of must, should and can measures.

The work in a group, the review of a product, and the use of qualitative research methods require both professional expertise and several social skills from the group members. They have to be open minded for the approach and new insights. Furthermore, they have to collaborate reliably, give feedback and take part in the discussions in a polite, open and supportive way. The persons involved have to bear in mind, that the subject of the review is the product and not the developer’s performance [1], [2], [4].

The moderator has to be able to guide the group discussions neutrally and goal-oriented. Reflections on the group dynamics and the ability to focus on the contributions of the group members support this [1], [2], [5]. To perform the text analysis of the email feedback the moderator needs basic knowledge in the collection of qualitative data and in qualitative text analysis. Examples on introductory literature and training programs are for instance [6], [7]. The text analysis can be performed computer supported (for instance with [8]).


GREME was used for the work on several modules that were required for the online release of a web-based information system on hospitals [3] (1) re-engineering of the web frontend for the presentation of hospital data in manner of tables, (2) development of the questionnaire content for the web-based evaluation, (3) implementation of the homepage for the web-based information system on hospital data. Review costs are generally based upon working time. Table 1 [Tab. 1] summarizes the GREME steps. Table 2 [Tab. 2] gives an overview of the effort (working hours) on the development, the GREME steps and the ratio of GREME effort to development time. Accounting all modules the overall fraction of GREME lies between 16 to 29 percent. This corresponds to the values of 10 to 50 percent reported on standardized reviews [1], [2].


GREME is intended as a further method for quality assurance in software engineering and user-centred development in order to meet the need for adaptable and cost effective QA methods [1], [2]. The examples show that GREME can be used to review non-crucial products economically, efficiently and effectively. GREME was easy to integrate in an ongoing software engineering project. The efficient and effective communication fostered the fast recovery of new insights. The team detected errors and aberrations at an early stage, designed solutions and assessed the quality of a product. The effort for the participants was foreseeable. This and the across-the-board communication of IT and domain experts led to a high acceptance of GREME. The adherence to delivery dates was high. Performed by a multidisciplinary team GREME facilitates the integration of IT knowledge and user perspective [1]. The work beyond established development teams and professional areas can strengthen the identity of an organization and foster organizational learning [1], [4].


The work presented here was done at the Department of Medical Informatics at RWTH Aachen University. In the meantime the main author of this paper, who was head of the ISG Research Group from 2002 until 2007, started a sabbatical. The authors thank the other members of the ISG Research Group for their work on and with GREME, and the head of the department, Prof. Dr. Dr. Spitzer, for his encouragement.


Wiegers KE. Peer Reviews in Software: A Practical Guide. Boston, Addison Wesley 2001.
Ludewig J, Lichter H. Software Engineering - Grundlagen, Menschen, Prozesse, Techniken. Heidelberg, dpunkt.verlag, 2007.
Weßel C, Weymann F, Spreckelsen C. A Framework for the Web-based Multi-method Evaluation of a Web-based Information System on Hospitals. 51. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. Leipzig, 10.-14.09.2006. Düsseldorf, Köln: German Medical Science; 2006. Doc 06gmds015. Available at: External link
DeMarco T, Lister T. Peopleware: Productive Projects and Teams. 2nd edition. New York, Dorset House Publishing Company 1999.
Seifert JK. Visualization, Presentation, Moderation: a Practical guide to successful presentation and the Facilitation of Business Processes. 2nd edition. NJ, Wiley 2002.
Bortz J, Döring N. Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler. Berlin, Springer 2003.
Weßel C, Weymann F, Spreckelsen C. Streamlining Qualitative Research Methods for Medical Informatics - A Methodological Approach. Accepted paper as poster on the XX International Congress of the European Federation for Medical Informatics (MIE 2006), August 27–30, 2006, Maastricht, The Netherlands.
Verbi. MAXqda – The Art of Text Analysis. [visited on 2008-01-20). External link