gms | German Medical Science

GMS Zeitschrift für Medizinische Ausbildung

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 1860-3572

Can the 'Assessment Drives Learning' effect be detected in clinical skills training? - Implications for curriculum design and resource planning

research article medicine

  • author Beate Buss - University of Heidelberg, Centre for Psychosocial Medicine, Department of General Internal and Psychosomatic Medicine, Heidelberg, Germany
  • author Markus Krautter - University of Heidelberg, Department of Nephrology, Heidelberg, Germany
  • author Andreas Möltner - University of Heidelberg, Medical Hospital, Centre of Excellence for Assessment in Medicine, Heidelberg, Germany
  • author Peter Weyrich - University Hospital of Tübingen, Department of Internal Medicine IV (Diabetes, Endocrinology, Nephrology and Clinical Chemistry), Tübingen, Germany
  • author Anne Werner - University Hospital of Tübingen, Department of Internal Medicine VI (Psychosomatic Medicine and Psychotherapy) Tübingen, Germany
  • author Jana Jünger - University of Heidelberg, Centre for Psychosocial Medicine, Department of General Internal and Psychosomatic Medicine, Heidelberg, Germany
  • corresponding author Christoph Nikendei - University of Heidelberg, Centre for Psychosocial Medicine, Department of General Internal and Psychosomatic Medicine, Heidelberg, Germany

GMS Z Med Ausbild 2012;29(5):Doc70

doi: 10.3205/zma000840, urn:nbn:de:0183-zma0008407

This is the original version of the article.
The translated version can be found at: http://www.egms.de/de/journals/zma/2012-29/zma000840.shtml

Received: October 21, 2011
Revised: June 4, 2012
Accepted: September 30, 2012
Published: November 15, 2012

© 2012 Buss et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Abstract

Purpose: The acquisition of clinical-technical skills is of particular importance for the doctors of tomorrow. Procedural skills are often trained for the first time in skills laboratories, which provide a sheltered learning environment. However, costs to implement and maintain skills laboratories are considerably high. Therefore, the purpose of the present study was to investigate students’ patterns of attendance of voluntary skills-lab training sessions and thereby answer the following question: Is it possible to measure an effect of the theoretical construct related to motivational psychology described in the literature – ‘Assessment drives learning’ – reflected in patterns of attendance at voluntary skills-lab training sessions? By answering this question, design recommendations for curriculum planning and resource management should be derived.

Method: A retrospective, descriptive analysis of student skills-lab attendance related to voluntary basic and voluntary advanced skills-lab sessions was conducted. The attendance patterns of a total of 340 third-year medical students in different successive year groups from the Medical Faculty at the University of Heidelberg were assessed.

Results: Students showed a preference for voluntary basic skills-lab training sessions, which were relevant to clinical skills assessment, especially at the beginning and at the end of the term. Voluntary advanced skills-lab training sessions without reference to clinical skills assessment were used especially at the beginning of the term, but declined towards the end of term.

Conclusion: The results show a clear influence of assessments on students’ attendance at skills-lab training sessions. First recommendations for curriculum design and resource management will be described. Nevertheless, further prospective research studies will be necessary to gain a more comprehensive understanding of the motivational factors impacting students’ utilisation of voluntary skills-lab training in order to reach a sufficient concordance between students’ requirements and faculty offers, as well as resource management.

Keywords: OSCE, skills-lab, assessment-driven learning, curriculum development, resource management


Introduction

The acquisition of clinical-technical skills is of particular importance for the doctors of tomorrow. Skills-labs for the training of clinical-technical skills were first established in the early nineteen seventies at the University of Illinois [1] and at the University of Maastricht [2]. Since then, they have been spreading worldwide [3], [4], [5]. Skills-lab facilities offer students a structured acquisition of clinical-technical competencies under standardised and safe conditions during training with each other [6], with simulators or mannequins [7], and with standardized patients [8]. Skills-lab training is found to be well accepted in student communities [9] and increases the frequency of performed skills on wards [10]. Undergraduate technical skills training guided by student tutors is a very satisfactory didactic approach and has been successfully established at medical faculties in addition to faculty staff-led training [11], [12]. Although skills-lab training implicates high learning potential [13], [14], [15], it requires extensive resources, e.g. staff, simulators, infrastructure and consumables. Thus, studying medicine requires much more money than studying any other discipline. In 2008, the German Federal Statistical Office ascertained that the current costs per student to earn a degree in medicine lie at 218 900 Euro [16]. This calculation is comparable with costs in the United Kingdom [17]. In the light of these facts, the resource management topic will become increasingly relevant for curriculum design, especially in the field of simulation, in order to use financial resources efficiently and in an evidence-oriented manner [18].

In the current literature, however, there are no studies assessing the utilisation of skills-lab training sessions with regard to a subsequent performance assessment that follows constructive alignment principles [19] on the equivalent stage of Miller's pyramid [20]. As a first step to administer educational and financial resources in skills-lab training more efficiently, a retrospective analysis of attendance patterns in voluntary skills-lab training sessions was conducted at the Medical Faculty of Heidelberg. The analysis was related to the well-documented construct known as 'assessment drives learning' [21], [22]. We hypothesized that an OSCE (objective structured clinical examination [23], [24]) assessment enhances students' utilisation of voluntary skills-lab training sessions that cover OSCE-relevant topics compared to skills-lab sessions that do not cover OSCE-relevant topics.


Method

Educational framework conditions

The interdisciplinary longitudinal skills-lab curriculum at the Medical Faculty of the University of Heidelberg offers skills training sessions from the first term up to the end of the final year. Internal medicine skills-lab training is embedded in this longitudinal curriculum and takes place in the 6th/7th term of medical training as part of the obligatory internal medicine rotation. Lectures (8 hours per week), seminars (8 hours per week), sessions of problem-based learning (2 hours per week), training of communication skills with standardised patients (1.6 hours per week) as well as on-ward training (2 hours per week) are further didactic elements within students' internal medicine education. The didactic skills-lab training approach is characterized by the integration of Peyton's Four-Stage Approach [25], as well as by role-play with context-relevant learning scenarios, constructive peer feedback and structured feedback by faculty staff [26].

The internal medicine skills-lab training offered by different internal medicine sub-disciplines is composed of three different types of courses:

1.
Mandatory basic skills-lab training sessions: offered once a week with different units, each lasting 90 minutes. The sessions are conducted in small groups of up to six students and taught by faculty staff (student/tutor ratio=6:1).
2.
Voluntary basic skills-lab training sessions: offered twice a week, with each session lasting 90 minutes. The learning objectives are derived from the obligatory skills-lab training sessions and are relevant for the OSCE assessment. The average student/tutor ratio is 5:1. The voluntary basic skills-lab training is conducted by faculty staff, as well as well-trained student tutors.
3.
Voluntary advanced skills-lab training sessions: start in week two of the term and are offered as single units of 90 minutes. They are related to a specific topic. Voluntary advanced skills-lab training sessions offer the possibility of acquiring more complex clinical-technical skills. The learning goals of these training sessions go beyond those of the regular obligatory basic skills-lab training sessions. Voluntary advanced skills-lab training sessions are held by faculty staff only. The average student/tutor ratio is 5:1.

The following table 1 [Tab. 1] shows typical skills-lab training sessions.

The mandatory/voluntary ratio of classes attended by students within the internal medicine rotation is approx. 22 hours mandatory to at least 6 hours voluntary sessions, thereof mandatory basic skills-lab training sessions for approx. 4 hours per week and voluntary advanced skills-lab training sessions for approx. 2 hours per week.

Study design

In order to address the research question described above, a retrospective descriptive analysis of student skills-lab attendance was conducted. All 6th and 7th term medical students (n=340) who attended the obligatory half-year internal medicine rotation at the University of Heidelberg, Medical Hospital, and thereby had the possibility to attend skills-lab training, were included in the study.

The attendance at the different skills-lab training sessions was registered by the administrative skills-lab staff. In order to obtain a substantial sample size, participants of the voluntary skills lab training sessions of both the winter term 2009/2010 and the summer term 2010 were registered.

Statistical analysis

For the statistical analysis, the Statistical Package for the Social Sciences (SPSS, Version 17.0 SPSS Inc., Chicago, IL, USA) software tool was used. The sign test was used to identify the overall difference between numbers of attendances at voluntary basic or advanced skills-lab training sessions. In order to identify potential time effects with regard to the different use of the two types of skills-lab training sessions, differences between mean numbers of attendances at voluntary basic and advanced skills-lab training sessions were also analysed with the non-parametric sign test for weeks 2 to 10, the time span during which both types of voluntary skills-lab training sessions were offered. For all tests, p<0.05 was considered statistically significant.


Results

Sample

The researched sample included n=340 students (134 male, 206 female; age: 24.70±3.32 years). 941 attendances at voluntary skills-lab training sessions over both terms and for both types of skills-lab training (voluntary basic and voluntary advanced) were taken into consideration for the statistical analysis.

Attendance at voluntary skills-lab training sessions

The mean number of attendances per student at voluntary skills-lab training showed a significantly higher average value for voluntary basic skills-lab training sessions (1.84±1.91 participations) than for voluntary advanced skills-lab training sessions (0.92±1.17 participations; p<0.001).

Figure 1 [Fig. 1] illustrates the average registered number of attendances per student at voluntary skills-lab training sessions during weeks 2 to 10. While it appears that voluntary basic skills-lab training sessions were in high demand, especially in the first weeks (weeks 3 to 5) and increasingly at the end of the internal medicine rotation (weeks 7 to 10) with a peak at week 9 (shortly before OSCE assessment), the number of attendances at advanced skills-lab training sessions continuously decreased and finally went down to zero in week 10.


Discussion

Rudland et al. [27] showed that an OSCE does not have an influence on medical students' workplace learning. However, the authors discussed the high potential of collaborative learning strategies for OSCE preparation. The study presented in this article represents the first publication reporting the effect of OSCE assessment on students' utilisation of voluntary skills-lab training offers. Voluntary basic skills-lab training sessions, which cover OSCE-relevant topics, are highly utilised throughout the rotation, with an increase in attendances towards the end of the term, which is completed with a twelve-station OSCE. This 'assessment drives learning' effect on skills-lab utilisation probably acts like a “hidden curriculum” [28]. Nevertheless, in our opinion, in future resource planning, it must be considered as a notable factor influencing the utilisation of voluntary skills-training offers. Similarly, the significant decrease in advanced skills-lab training session attendances towards the end of the internal medicine rotation seems to confirm that a strong relationship between OSCE assessment and students' patterns of participation does exist. The results emphasize the importance of a constructive alignment [19] of skills-lab training sessions with their assessment and a corresponding offer of training abilities.

The results have clear implications for curriculum design and resource management. Most of the capacity of training staff and material should be allocated to the last weeks of the term for OSCE-relevant skills-lab training sessions. In the middle of the term, OSCE-relevant skills-lab training sessions should be provided, but with fewer resources and time slots. Voluntary advanced skills-lab training sessions should be reduced at the end of the term. This rescheduling of voluntary skills-lab training offers permits the reduction of costs for administration and learning materials.

Besides such rescheduling, accompanying offers should be discussed in order to manage training resources efficiently, e.g. in-course assessments in which students can test their clinical-technical skills during their learning process and before the OSCE assessment. This approach corresponds with the results of Duvivier et al. [29], who discussed the high potential of deliberate practice to train clinical skills and the necessity of observation and feedback in this context. A core prerequisite is the students' detection of their own weaknesses as a motivational factor. In the literature, in-course assessments have been discussed for many years due to their high potential to improve the learning outcome [30].

Beyond the presented results, which obviously follow the hypothesis that “assessment drives learning”, the peak at the beginning of the term for voluntary basic skills-lab training attendances might not be so easy to explain through this construct. This finding might mainly be based on characteristic processes of self-directed learning [31]. Comparable to the data patterns of attendance at voluntary advanced skills-lab training sessions, the utilisation of voluntary basic skills-lab training sessions increased up to week 3 and went down to an absolute minimum in week 6. Thus, the first few weeks of the voluntary skills-lab training may be understood as a stage of orientation, where students figure out their learning needs and formulate objectives. Mid-term may be understood as a phase of selection, during which resources and learning strategies are identified and selected. Other learning fields such as on-ward training should be focused on in this time span. At the end of the term, the implementation of strategies and evaluation of learning outcomes are at the forefront. The process of self-directed learning is reflected in our data and should be explored further.

Beside this, influences of other learning strategies, such as patient-contact learning, should be considered. Bokken et al. [32] showed that students feel responsible towards real patients and thereby recognize the necessity of a stronger motivation for self-study before the first patient contact. At the medical faculty of the University of Heidelberg, internal medicine skills-lab training is accompanied by ward-based education, and according to our data, especially the peak for voluntary skills-lab sessions at the beginning of the term suggests a relationship between students' perceived responsibilities for patients on ward. The patient-centred learning approach should be discussed further regarding its influence on medical students' learning behaviour in voluntary skills-lab training sessions.

A limitation of our current study is that it does not provide information on the causal structure of students' voluntary skills-lab training session attendance. The study does not analyse whether students who attend voluntary skills-lab training sessions constitute a defined group that differs in motivational aspects, pre-training procedural skills abilities or in self-efficacy measures. Moreover, intrinsic motivational factors and lifestyle factors, such as having to undertake paid employment, which are identified in the literature as relevant for student attendance or non-attendance at university classes (lectures and tutorials) [33], [34], were not taken into consideration. In the current literature, this field is insufficiently explored and should be investigated further in prospective research studies. The presented study was a first step on the path to cost-effective skills-lab training curriculum design.

In summary, it can be stated that the presented study shows the “assessment drives learning”-driven effect for the first time for voluntary skills-lab training session attendance. Thereby, the study provides substantial data to elucidate voluntary clinical-technical skills learning behaviour of medical students in relation to a subsequent OSCE assessment and allows for curriculum adaptations. By taking the results of our study into account, costs for administration and consumables might be reduced. Nevertheless, in order to gain a deeper understanding of students' needs, further studies, especially regarding motivational aspects and specific performance characteristics of skills-lab training participants, will be necessary.


Notes on contributors

  • B. Buß, Ph.D. is at the University of Heidelberg Medical Hospital, and responsible for organisational and research aspects in skills-lab training at the Medical Hospital
  • M. Krautter M.D. is at the University of Heidelberg Medical Hospital, and involved in Medical Education Research in the field of skills-lab training and final-year student education
  • A. Möltner, Ph.D. is at the Centre of Excellence for Assessment in Medicine, responsible for the evaluation of medical examinations and the statistical analysis of research projects
  • P. Weyrich, M.D. is at the University of Tübingen Medical Hospital, and responsible for the interdisciplinary skills-lab (DocLab) programme at the Faculty of Medicine as well as for the medical education of final-year students.
  • A. Werner, M.D. is at the University of Tübingen Medical Hospital, and responsible for the standardized patient programme at the Faculty of Medicine as well as for undergraduate and graduate medical education.
  • J. Jünger, M.D., MME is at the University of Heidelberg Medical Hospital, and responsible for the medical education programme at the Medical Hospital.
  • C. Nikendei, M.D., MME is at the University of Heidelberg Medical Hospital, and responsible for skills-lab training and education of final-year students at the Medical Hospital.

Competing interests

The authors declare that they have no competing interests.


References

1.
Sajid A, Lipson LF, Telder V. A simulation laboratory for medical education. Med Educ. 1975;50(10):970-975 .DOI: 10.1097/00001888-197510000-00007 External link
2.
Van Dalen J, Bartholomeus P. Training clinical competence in a skills laboratory. In: Bender W, Hiemstra RJ, Scherpbier AJ, Zwiestra RP (Hrsg). Teaching and Assessing Clinical Competence. Groningen: Stichting TICTAC; 1990. p.135-140.
3.
Remmen R, Scherpbier A, van der Vleuten C, Denekens J, Derese A, Hermann I, Hoogenboom R, Kramer A, Van Rossum H, Van Royen P, Bossaert L. Effectiveness of basic clinical skills training programmes: a cross-sectional comparison of four medical schools. Med Educ. 2001;35(2):121-128. DOI: 10.1046/j.1365-2923.2001.00835.x External link
4.
Nikendei C, Weyrich P, Jünger J, Schrauth M. Medical education in Germany. Med Teach. 2009;31(7):591-600. DOI: 10.1080/01421590902833010 External link
5.
Reznick RK, MacRae H. Teaching Surgical Skills - Changes in the Wind. N Engl J Med. 2006;355(25):2664-2669. DOI: 10.1056/NEJMra054785 External link
6.
Weyrich P, Schrauth M, Nikendei C. Peer-assisted learning: a planning and implementation framework. Guide supplement 30.4 - practical application. Med Teach. 2008;30(4):444-445.
7.
Okuda Y, Bryson EO, DeMaria S Jr, Jacobson L, Quinones J, Shen B, Levine AL. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76(4):330-343. DOI: 10.1002/msj.20127 External link
8.
Kneebone R, Kidd J, Nestel D, Asvall S, Parakeva P, Darzi A. An innovative model for teaching and learning clinical procedures. Med Educ. 2002;36(7):628-634. DOI: 10.1046/j.1365-2923.2002.01261.x External link
9.
Das M, Townsend A, Hasan MY. The views of senior students and young doctors of their training in a skills laboratory. Med Educ. 1998;32(2):143-149. DOI: 10.1046/j.1365-2923.1998.00182.x External link
10.
Remmen R, Derese A, Scherpbier A, Denekens J, Hermann I, van der Vleuten C, Van Royen P, Bossaert L. Can medical schools rely on clerkships to train students in basic clinical skills? Med Educ. 1999;33(8):600-605. DOI: 10.1046/j.1365-2923.1999.00467.x External link
11.
Weyrich P, Schrauth M, Kraus B, Habermehl D, Netzhammer N, Zipfel S, Jünger J, Riessen R, Nikendei C. Undergraduate technical skills training guided by student tutors-analysis of tutors' attitudes, tutees' acceptance and learning progress in an innovative teaching model. BMC Med Educ. 2008;8:18. DOI: 10.1186/1472-6920-8-18 External link
12.
Tolsgaard MG, Gustafsson A, Rasmussen MB, Høiby P, Müller CG, Ringsted C. Student teachers can be as good as associate professors in teaching clinical skills. Med Teach. 2007;29(6):553-557. DOI: 10.1080/01421590701682550 External link
13.
Peeraer G, Scherpbier AJ, Remmen R, de Winter BY, Hendrickx K, van Petegem P, Weyler J, Bossaert L. Clinical Skills Training in a skills lab compared with skills training in internships: comparison of skills development curriculum. Educ Health (Abingdon). 2007;20(3):125.
14.
Jünger J, Schäfer S, Roth C, Schellberg D, Friedman Ben-David M, Nikendei C. Effects of Basic Clinical Skills Training on OSCE performance among medical students: a group control design study. Med Educ. 2005;39(10):1015-1020.
15.
Lynagh M, Burton R, Sanson-Fisher R. A systematic review of medical skills laboratory training: where to from here? Med Educ. 2007;41(9):879–887. DOI: 10.1111/j.1365-2923.2007.02821.x External link
16.
Wolters M, Buschle N. Hochschulen auf einen Blick. Ausgabe 2011. Wiesbaden: Statistisches Bundesamt; 2011.
17.
Brown CA, Lilford RJ. Selecting medical students. BMJ. 2008;336(7648):786. DOI: 10.1136/bmj.39517.679977.80 External link
18.
Ker J, Hogg G, Maran N. Cost-effective simulation. In: Walsh K (Hrsg). Cost Effectiveness in Medical Education. Oxford, New York: Radcliffe Publishing; 2010. p.61- 71.
19.
Biggs J, Tang C. Teaching for Quality Learning at University: What the students does. 3rd. Maidenhead: Open University Press/McGraw Hill; 2007.
20.
Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9):63–67. DOI: 10.1097/00001888-199009000-00045 External link
21.
Thistlethwaite J. More thoughts on 'assessment drives learning'. Med Educ. 2006;40(11):1149–1150. DOI: 10.1111/j.1365-2929.2006.02638.x External link
22.
McLachlan JC. The relationship between assessment and learning. Med Educ. 2006;40(8):716–717. DOI: 10.1111/j.1365-2929.2006.02518.x External link
23.
Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J. 1975;1(5955):447-451. DOI: 10.1136/bmj.1.5955.447 External link
24.
Wass V, van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001;357(9260):945-949. DOI: 10.1016/S0140-6736(00)04221-5 External link
25.
Peyton JW. Teaching and learning in medical practice. Silver Birches: Manticore Europe; 1998.
26.
Nikendei C, Kraus B, Schrauth M, Weyrich P, Zipfel S, Herzog W, Jünger J. Integration of role-playing into technical skills training: a randomized controlled trial. Med Teach. 2007;29(9):956-960. DOI: 10.1080/01421590701601543 External link
27.
Rudland J, Wilkinson T, Smith-Han K, Thompson-Fawcett M. "You can do it late at night or in the morning. You can do it at home, I did it with my flatmate." The educational impact of an OSCE. Med Teach. 2008;30(2):206-211. DOI: 10.1080/01421590701851312 External link
28.
Snyder B. The hidden curriculum. Cambridge: MIT Press; 1971.
29.
Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, Van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC Med Educ. 2011;11:101. DOI: 10.1186/1472-6920-11-101 External link
30.
Wood T. Assessment not only drives learning, it may also help learning. Med Educ. 2009;43(1):5–6. DOI: 10.1111/j.1365-2923.2008.03237.x External link
31.
Slusarski SB. Enhancing selfdirection in the adult learner: Instructional techniques for teachers and trainers. New Dir Adult Cont Educ. 1994;64:71–79. DOI: 10.1002/ace.36719946411 External link
32.
Bokken L, Rethans JJ, van Heurn L, Duvivier R, Scherpbier A, van der Vleuten C. Students' Views on the Use of Real Patients and Simulated Patients in Undergraduate Medical Education. Acad Med. 2009;84(7):958-963. DOI: 10.1097/ACM.0b013e3181a814a3 External link
33.
Massingham P, Herrington T. Does Attendance Matter? An Examination of Student Attitudes, Participation, Performance and Attendance. JUTLP. 2006;3(2):82-103.
34.
Westrick SC, Helms KL, McDonough SK, Breland ML. Factors Influencing Pharmacy Students' Attendance Decisions in Large Lectures. Am J Pharm Educ. 2009;73(5):83. DOI: 10.5688/aj730583 External link