gms | German Medical Science

GMS Journal for Medical Education

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 2366-5017

Design and implementation of virtual patients for learning of clinical reasoning

short report Clinical Reasoning

Search Medline for

  • corresponding author Sören Huwendiek - University of Bern, Medical Faculty, Institute for Medical Education, Department for Assessment and Evaluation, Bern Switzerland External link

GMS J Med Educ 2019;36(4):Doc33

doi: 10.3205/zma001241, urn:nbn:de:0183-zma0012413

This is the English version of the article.
The German version can be found at:

Received: September 21, 2018
Revised: March 8, 2019
Accepted: April 15, 2019
Published: August 15, 2019

© 2019 Huwendiek.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at


Introduction: Virtual Patients (VP) are electronic interactive patient cases. The aim of this PhD project was to explore how to improve the design and implementation of VPs to foster learning of clinical reasoning.

Methods: This PhD report is based on five consecutive studies. Using focus groups among clerkship students, we explored design features of VP. A modified Delphi study among VP experts was used to establish a VP design typology. Validity evidence was established for a questionnaire to evaluate VP design from the student perspective. In student focus groups, we explored features on how to implement VP into a clerkship. Further, we explored students’ perception of different exam formats, in an assessment of a clerkship which includes learning with VP, by focus groups, and examined whether their psychometric properties differ.

Results: Aspects to improve VP design:

using instructional design criteria such as ensuring an appropriate level of difficulty, authenticity, interactivity, feedback, and focusing on relevant learning points,
implementing virtual coaching on clinical reasoning into the VP, such as asking for discriminating and confirming features, and
providing theory-guided instruments for systematic improvements such as the developed VP typology and VP design questionnaire.

Aspects to improve VP implementation:

sequencing VP and other educational activities according to complexity, and aligning
instruction and
assessment with the use of VP.

Conclusion: Our results are in line with insights outside of VP research. Our studies demonstrate how VP can be designed, systematically further improved, and implemented to foster learning of clinical reasoning.

Keywords: virtual patients, implementation, blended learning, elearning, evaluation


Clinical reasoning is generally seen as a crucial component of nearly everything doctors do in practice [4]. It guides them to make correct diagnoses and reach appropriate treatment decisions for their patients. As such, a physician’s clinical reasoning skills are highly important and relevant for patient outcomes and patient safety [3]. However, studies have indicated that clinical reasoning is currently not well taught [7]. According to the literature, so-called Virtual Patients (VPs) are particularly suited to foster learning of clinical reasoning [2]. Virtual Patients are online cases in which the learner, takes on the role of physician, is required to make all decisions him/herself and subsequently receives feedback. Ellaway et al. [5] define VP as “interactive computer simulation[s] of real-life clinical scenarios for the purpose of medical training, education, or assessment”. Two aspects have been found to be essential regarding learning with electronic learning tools such as VPs: their design and their implementation.

Literature on how to design and implement Virtual Patients to foster clinical reasoning was lacking when we embarked on this research. Therefore, at the outset of our research project, we sought evidence on how to improve learning with VPs in order to foster clinical reasoning in medical students. The main research question was as follows:

How can the design and implementation of Virtual Patients be improved in order to foster learning, particularly with respect to clinical reasoning?


Three of the five studies (studies 1,4,5) were conducted in an authentic educational practice setting, specifically a paediatric clinical clerkship. While in these three studies students in their paediatric clerkship were the main target of our investigation, in two studies (studies 2 and 3) VP experts and medical teachers, national and international, were strongly involved in the development of both the VP typology and the VP design questionnaire. Different research methods were used. In specific:

Study 1 explores students’ perceptions of the ideal features of Virtual Patient design to foster their learning, focusing on clinical reasoning. Fifth year students (n=104) were exposed to at least eight VPs varying in design, and discussed Virtual Patient design in focus groups [11].

Study 2 describes the development of a Virtual Patient typology based on the literature, a review of existing VP systems, results from study 1, and an international consensus process among Virtual Patient experts, applying a modified Delphi technique [9]. The virtual patient expert team consisted of all six authors of this paper.

Study 3 reports on the development and validation of a questionnaire to evaluate Virtual Patient design, drawing on published evidence, input from an international VP expert team (from eVIP project, see and including the insights gained from studies 1 and 2. Three sources of validity evidence were examined:

response process and
internal structure [8].

Study 4 seeks evidence to guide the implementation of Virtual Patients. Students’ perceptions of different scenarios aimed at developing clinical reasoning skills were explored using focus groups (39 from 116 exposed students). In these scenarios, at least ten Virtual Patients were integrated with other educational activities during a paediatric clerkship [10].

Study 5 focuses on two different types of assessment question and explores their educational impact on the learning of clinical reasoning with Virtual Patients. After students (n=377) underwent clinical clerkships and corresponding exams (n=11), they discussed in focus groups (n=8) the perception and impact of Key-Feature Problems with Long Menu (KFP) and context-rich Single Best Answer questions (crSBA). The study additionally compares the psychometric characteristics of the two formats [12].


Study 1

Five student focus groups revealed 10 principles of VP design. To facilitate learning, Virtual Patients should be

encompass an appropriate level of difficulty,
be interactive,
provide specific feedback,
utilize different, appropriate media,
direct students’ focus to relevant learning points,
foster the recapitulation of key learning points,
be authentic regarding the web-based interface and
tasks, and encompass
questions and explanations that enhance clinical reasoning.

After exposure to – from the student perspective – well-designed VPs, students felt very well prepared for reasoning in real patients, and better prepared than with any other previously experienced method. Students perceived the identified design principles as conducive to their learning.

Study 2

We synthesised 19 factors concerning four categories identified as relevant to VP design: general (title, description, language, identifier, provenance, typical study time); educational (educational level, educational modes, coverage, objectives); instructional design (path type, user modality, media use, narrative use, interactivity use, feedback use); technical (originating system, format, integration and dependence).

Study 3

The short VP design questionnaire comprises one global score and six questions distributed between three factors: authenticity of patient encounter and consultation, cognitive strategies in the consultation, coaching during consultation. Content analysis was reasonably supported by the theoretical foundation and the Virtual Patient experts (n=9) from the eVIP project []. The think-aloud studies and the analysis of free text comments supported the questionnaire’s validity. The exploratory factor analysis, encompassing 2547 student evaluations of 78 Virtual Patients from three countries yielded a three-factor model showing a reasonable fit with the data. To reliably evaluate a Virtual Patient on all three factors, a minimum of 200 student responses are required.

Study 4

The analysis of eight focus group interviews revealed six themes which were deemed by students to be important for the optimal implementation of VPs:

continuous and stable online access,
increasing complexity, adapted to students’ knowledge,
VP-related workload offset by elimination of other activities,
optimal sequencing (e.g.: lecture – 1 to 2 VP(s) – tutor-led small group discussion – real patient),
optimal alignment of Virtual Patients and educational activities, and
inclusion of VP topics in assessment.
Study 5

The analysis of 8 focus groups revealed four themes: Compared to the context-rich Single Best Answer format, Key-Feature Problems with Long Menu were seen as

more realistic,
more difficult and
more motivating for the intense study of clinical reasoning with Virtual Patients.

Moreover, overall, they

showed good acceptance when taking into account some preconditions like offering an additional free-text comment field in case students did not find a suitable answer in the long menu.

According to the statistical analysis, there was no difference in difficulty; however, a higher reliability (G coefficient) was found for Key-Feature Problems with Long Menu, even when corrected for testing time (reliability per testing time 1 hour: KFP 0,84 vs crSBA 0,79). The Key-Feature Problems showed higher correlations with the OSCE results (OSCE-KFP: 0,54; OSCE-crSBA: 0,41). For the study of clinical reasoning with Virtual Patients, students perceived the Key-Feature Problems as more motivating. The inclusion of Key-Feature Problems with Long Menu into summative clerkship exams seems to offer positive educational effects and no psychometric drawbacks.

Summary and Discussion

Studies 1-3 suggest that the following three main aspects are particularly relevant for optimising VP design to foster learning of clinical reasoning:

use of instructional design criteria, i.e. ensuring appropriate difficulty, authenticity, interactivity, feedback and focus on relevant learning points,
implementation of virtual coaching on clinical reasoning into the VP, i.e. asking for discriminating and confirming features, and
provision of validated instruments to enable systematic further improvements, such as the developed VP typology and VP design questionnaire.

The empirically derived VP typology provides a common reference point for studies or reports on Virtual Patients. The VP design questionnaire has the potential to provide valid information about Virtual Patient design, contingent on a large number of responses per VP.

Studies 4-5 suggest the following three main aspects as being especially relevant for VP implementation:

sequencing VPs and other educational activities, including bedside teaching with real patients, according to complexity, and
aligning instruction and
assessment with the use of VPs.

Our findings are in accordance with current theories and insights outside of VP research e.g. on instructional design [6], on how to foster learning of clinical reasoning [1], and on instructional design theories regarding curriculum development [13].

The main implications of our research for educational practice are as follows: To optimally foster clinical reasoning, Virtual Patients should be well designed, systematically improved through validated instruments, and well implemented.

The major strengths of this dissertation include the authentic study settings, the involvement of different stakeholders in the studies, and the combination of different research methodologies.

Limitations of this dissertation are that three studies were done at only one centre, that it did not include a study on the longitudinal implementation of VPs, or one on the transfer of learning to real patients using objective measures. Further research is necessary to deepen the understanding of Virtual Patient design and implementation from a multicentre perspective and in different contexts. Further, the efficacy and effectiveness of VP should be investigated more in depth. This should include longitudinal studies and measurements of the impact on patient outcomes.

A piece of advice

Ensure that you have blocked off sufficient time for your PhD work, choose your topic and supervisors wisely, and enjoy this time.

Biographical note

Sören Huwendiek graduated from medical school at Heidelberg University in Germany, where he also worked for 10 years as a physician (finally paediatrician and paediatric rheumatologist) and as a medical educator. He gained a Masters of Medical Education degree from Bern University and this PhD in Health professions education from Maastricht University. Since 2012, he has been Head of the Department of Assessment and Evaluation of the Institute of Medical Education (IML) in Switzerland. In 2018, he was promoted to Associate Professor for Medical Education.

The defense took place at the University of Maastricht on 25 November 2016.

Supervisors: Prof. Dr. D.H.J.M Dolmans (University of Maastricht) and Prof. Dr. C.P.M. van der Vleuten (University of Maastricht). Co-supervisor: Dr. B. de Leng (University of Münster).


My sincere gratitude goes to my great supervisors Diana Dolmans, Cees van der Vleuten and Bas de Leng from Maastricht University resp. Münster University and Burkhard Tönshoff and Georg F. Hoffmann from Heidelberg University. I also thank the many other people who contributed to and supported me in this PhD.

Competing interests

The author declares that he has no competing interests.


Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006;355(21):2217-2225. DOI: 10.1056/NEJMra054782 External link
Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010;85(10):1589-602. DOI: 10.1097/ACM.0b013e3181edfe13 External link
Croskerry P. A universal model of diagnostic reasoning. Acad Med. 2009;84(8):1022-1028. DOI: 10.1097/ACM.0b013e3181ace703 External link
Durning S, Artino AR Jr, Pangaro L, van der Vleuten CP, Schuwirth L. Context and clinical reasoning: understanding the perspective of the expert's voice. Med Educ. 2011;45(9):927-938. DOI: 10.1111/j.1365-2923.2011.04053.x External link
Ellaway R, Poulton T, Fors U, McGee JB, Albright S. Building a virtual patient commons. Med Teac. 2008;30(2):170-174. DOI: 10.1080/01421590701874074 External link
Grunwald T, Corsbie-Massay C. Guidelines for cognitively efficient multimedia learning tools: educational strategies, cognitive load, and interface design. Acad Med. 2006;81(3):213-223. DOI: 10.1097/00001888-200603000-00003 External link
Huang G, Reynolds R, Candler C. Virtual patient simulation at US and Canadian medical schools. Acad Med. 2007;82(5):446-451. DOI: 10.1097/ACM.0b013e31803e8a0a External link
Huwendiek S, De Leng BA, Kononowicz AA, Kunzmann R, Muijtjens AM, Van Der Vleuten CP, Hoffmann GF, Tönshoff B, Dolmans DH. Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning. Med Teach. 2015;37(8):775-782. DOI: 10.3109/0142159X.2014.970622 External link
Huwendiek S, deLeng B, Zary N, Fischer MR, Ruiz JG, Ellaway R. Towards a Typology of Virtual Patients. Med Teach. 2009;31(8):743-748. DOI: 10.1080/01421590903124708 External link
Huwendiek S, Duncker C, Reichert F, de Leng BA, Dolmans D, van der Vleuten CP, Haag M, Hoffmann GF, Tönshoff B. Learner preferences regarding integrating, sequencing and aligning virtual patients with other activities in the undergraduate medical curriculum: a focus group study. Med Teach. 2013;35(11):920-929. DOI: 10.3109/0142159X.2013.826790 External link
Huwendiek S, Reichert F, Bosse HM, de Leng BA, van der Vleuten CP, Haag M, Hoffmann GF, Tönshoff B. Design principles for virtual patients: a focus group study among students. Med Educ. 2009;43(6):580-588. DOI: 10.1111/j.1365-2923.2009.03369.x External link
Huwendiek S, Reichert F, Duncker C, de Leng BA, van der Vleuten CP, Muijtjens AMM, Bosse HM, Haag M, Hoffmann GF, Tönshoff B, Dolmans D. Electronic Assessment of Clinical Reasoning in Clerkships: A Mixed-Methods Comparison of Long-menu Key-Feature Problems with Context-Rich Single Best Answer Questions. Med Teach. 2017;39(5):476-485. DOI: 10.1080/0142159X.2017.1297525 External link
van Merrienboer JJ, Sweller J. Cognitive load theory in health professional education: Design principles and strategies. Med Educ. 2010;44(1):85-93. DOI: 10.1111/j.1365-2923.2009.03498.x External link