gms | German Medical Science

GMS Journal for Medical Education

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 2366-5017

Entrustable Professional Activities – Visualization of Competencies in Postgraduate Training. Position Paper of the Committee on Postgraduate Medical Training of the German Society for Medical Education (GMA)

position paper medicine

Search Medline for

  • corresponding author Pascal O. Berberat - Technische Universität München, Fakultät für Medizin, Medizindidaktisches Centrum für Ausbildungsforschung und Lehre, TUM MeDiCAL, München, Deutschland
  • author Sigrid Harendza - Universitätsklinikum Hamburg-Eppendorf, III. Medizinische Klinik und Poliklinik, Hamburg, Deutschland
  • author Martina Kadmon - Chirurgische Universitätsklinik Heidelberg, Heidelberg, Deutschland
  • Gesellschaft für Medizinische Ausbildung, GMA-Ausschuss für Weiterbildung

GMS Z Med Ausbild 2013;30(4):Doc47

doi: 10.3205/zma000890, urn:nbn:de:0183-zma0008909

This is the English version of the article.
The German version can be found at: http://www.egms.de/de/journals/zma/2013-30/zma000890.shtml

Received: September 9, 2013
Revised: September 25, 2013
Accepted: October 22, 2013
Published: November 15, 2013

© 2013 Berberat et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Authors

The three authors contributed equally to this paper.


Introduction

Quality and structure of postgraduate medical training and educational experiences in undergraduate and postgraduate training play a major role in career decisions of young medical graduates [14], [19], [23], [28]. They consider structured mandatory and challenging postgraduate training programmes delivered by experienced trainers with high training competencies as important factors of attractiveness and good starting of their clinical career [23].

The present postgraduate training in Germany widely differs between the medical disciplines. Best-practice examples of a structured competency-based postgraduate training that may serve as models for other clinical fields exist for anaesthesiology [24], family medicine [25], urology [11] and surgery [18]. However, a national comprehensive and binding curricular structure ensuring comparable competencies independent of training site is missing. Certification examinations in all medical disciplines follow an unstructured oral assessment format outside of the clinical context. Thus, it neither adequately assesses clinical competencies of medical doctors at the time of certification nor does it represent professional tasks [Musterweiterbildungsordnung 2003 der Bundesärztekammer (accessed February 12, 2013)].

The evaluation of postgraduate training launched by the German Medical Association in 2009 and 2011 revealed mediocre overall satisfaction of the trainees [Bundesrapport 2009. Evaluation der Weiterbildung in Deutschland (accessed February 12, 2013)], [Bundesrapport 2011. Evaluation der Weiterbildung in Deutschland (accessed February 12, 2013)]. Forty-five percent of the respondents rated postgraduate training between satisfactory and failed on a school grade scale. More than a third evaluated the training culture in their work environment and the support in acquiring professional competencies as only satisfactory to deficient. The most worrying result was the lack of training in evidence-based medicine and its transfer to the delivery of patient care, which 69% of respondents rated as only satisfactory or below. As for transparency of postgraduate training, only one third of the respondents obtained a structured plan and defined outcome objectives in written form, 42% received no training schedule at all [Bundesrapport 2011. Evaluation der Weiterbildung in Deutschland (accessed February 12, 2013)]. There was a slight improvement between the two evaluation cycles in 2009 and 2011 suggesting the necessity of a further decisive and perceptible action plan to advance the present postgraduate training culture in Germany.

Currently, a vigorous discussion on the future of postgraduate medical training in Germany is ongoing among political stakeholders, the German Medical Association, the State Chambers of Physicians, scientific medical societies, professional medical organizations and all physicians in charge of postgraduate training supervision. On the 115th annual conference of the German Physicians Board 2012 the managing-committee of the German Medical Association was requested to work out concrete propositions for new Postgraduate Professional Education Regulations, which represent the reality of health care provision today and define professional competencies as essential outcomes of postgraduate training [15].

The suggestion to entirely omit guiding numerical values for medical procedures in favour of competencies was not supported by the majority. However, there is agreement that the new regulations should include the following fundamentals:

In order to facilitate an exchange of ideas the German Medical Association has established the "WIKI-BÄK-Platform" in December 2012 to allow scientific medical societies and professional medical organizations to place their propositions and thus contribute to the new regulations for certification between February and April 2013 [Bundesärztekammer (accessed February 12, 2013)].

In Austria the present situation in postgraduate medical training appears similar to that in Germany: evaluation reveals only mediocre overall satisfaction of the trainees. There is an ongoing discussion, based on the reform of undergraduate training in 2002 and the increased mobility of young doctors, that postgraduate training urgently needs new and more structured curricula. They should pursue the agreed and planned competency-based education of the final year of undergraduate training and follow a modular structure with internship, common trunk and major/elective subjects.

A slightly different situation is reported form Switzerland: all postgraduate training programs were accredited in 2011, and the national postgraduate training regulation was revised in 2013. An independent institute, the Swiss Institute for Postgraduate Training (SIWF), is responsible. Regarding the orientation towards professional competencies as outcomes of postgraduate training the present national regulation defines general outcomes for all programs. These are listed in detail in an associated catalogue of general learning objectives based on the CanMEDS roles and integrating other frameworks like the ACGME General Competencies [Lernzielkatalog – Allgemeine Lernziele für die Weiterbildungsprogramme (accessed October 19, 2013)]. Moreover different postgraduate training programs have implemented workplace-based assessments aiming at monitoring the achievement of professional competencies [20].

The present situation represents an ideal chance for considering and implementing new concepts of postgraduate training which would stress the continuum of undergraduate and postgraduate training as well as continuous professional development [17]. The Society for Medical Education (GMA) should have a significant share in the discussion and promotion of the ongoing process of developing an interdisciplinary evidence-based postgraduate education strategy for Germany in the 21st century. The Committee on Postgraduate Medical Education of the GMA has elucidated crucial questions and formulated essential stimuli supporting the current process [4]. This position paper is aimed at presenting successful competency-based frameworks for postgraduate medical education and appreciating prospects of integrating these frameworks in the current German reforms.


Competency-based frameworks for postgraduate medical training

Competency-based medical education, both in undergraduate and postgraduate training, is today considered an important prerequisite for adequate delivery of patient care in the 21st century [2], [10], [30]. Epstein and Hundert propose that professional competence may be defined as "the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served" [6]. Thus, professional competence builds on a basis of cognition, clinical skills and personal attitudes paired with the willingness and motivation to apply these skills in concrete medical contexts for adequate problem solving toward a humane patient care and to resume responsibility [6], [32], [Wildt J. The shift from teaching to learning. 2007 (accessed January 10, 2013)]. Not the time spent in clinical rotations or the numbers of medical procedures, but the demonstration of competence in delivering medical care represents the essence of competency-based postgraduate training [9], [16], [30].

Two major competency-based postgraduate training frameworks rest on this outcome-oriented fundament, the CanMEDS Physician Competency Framework and the Outcome Project of the Accreditation Council for Graduate Medical Education (ACGME) [ACGME Outcome project – General competencies documentation (accessed January 13, 2013)], [7], [8], [12], [27]. Apart from a structured postgraduate training curriculum the integration of workplace-based assessment formats, e.g. mini-clinical evaluation exercise, clinical encounter cards, direct observation of procedural skills, case-based discussion with the provision of feedback seem to be an adequate tool to support the development of clinical competencies of trainees in the course of postgraduate medical training [22]. The integration of entrustable professional activities (EPAs) in training and formative assessment may close the gap between the theory of competency-based training and the patient-centred practice in the clinical context [21], [30]. Thus, EPAs have become the core of postgraduate trainings in various disciplines [1], [3],[21].


Entrustable professional activities (EPAs)

EPAs can be based on the Canadian Medical Education Directions for Specialists (CanMEDS) framework [8] which defines competencies a physician should attain summarized in seven roles: medical expert (the central role), communicator, collaborator, manager, health advocate, scholar and professional. Not the time spent in training is the important outcome measure but rather the attainment of competence. As each EPA defines a real professional activity that can be entrusted to a resident and as each EPA is linked to several competencies which are most crucial to a specific EPA we follow the suggestion by Mulder et al. to refer to the seven CanMEDS roles as ‘domains of competence’ [21]. Each CanMEDS role or ‘domain of competence’, respectively, is underpinned by knowledge, skills and attitude [21] and can be divided further into facets of competence [34]. The ‘communicator’ role, for example, which is referred to as ‘communication’ domain [21], includes several facets of competence: ‘verbal communication with colleagues and supervisors’, ‘empathy and openness’, ‘adapted informing of patients’, ‘respecting privacy and autonomy of the patient’ and others [34], and in turn each facet of competence includes different aspects of knowledge, skills and attitudes.

Based on the CanMEDS framework a range of EPAs for the different stages of residency training can be appointed by defining the specific domains of competence relevant for each EPA [21]. For every step in residency training each EPA is ‘a critical part of professional work that can be identified as a unit to be entrusted to a trainee once sufficient competence has been reached’ [29], [30]. Using this framework, representative EPAs for the different stages of residency training can be defined and as they are linked to explicit domains of competence they also provide a base for observation and assessment in clinical practice [21].

A three-step approach resulting in an EPA-based workplace curriculum for physician assistant training has been proposed by Mulder et al. [21]. This approach can be adapted for residency training in a similar way:

1.
Selection of EPAs for a specific residency training,
2.
Description of the EPAs,
3.
Plan training and assessment of EPAs (see table 1 [Tab. 1]).

This outline will lead to observable levels of proficiency in executing each EPA and schedule EPAs and level of competence over the course of the training. The levels of proficiency could be adapted for residency training as suggested by Wijnen-Meijer et al. for medical graduates [33]:

1.
the resident is not able to do this,
2.
the resident is able to do this under direct supervision,
3.
the resident is able to do this if supervision is available,
4.
the resident is able to do this independently,
5.
the resident is able to supervise others in performing this activity (see table 2 [Tab. 2]).

Furthermore, the following four factors of influence on entrustment decisions have been identified and need to be kept in mind when planning a curriculum for residents based on EPAs: characteristics of the resident, the attending, the clinical context, and the critical task [26].

In the professional workplace of every resident in a hospital a mutual EPA for their first year of postgraduate training could be “running a ward in a multidisciplinary team”, which may serve as an example to illustrate the underlying principle for selection of EPAs for an EPA- based workplace curriculum. This EPA is a fairly broad one, which includes multiple smaller EPAs each one being linked to several domains of competence. Table 3 [Tab. 3] shows a blueprint underpinning the EPAs with their included domains of competence. After the EPAs are identified they need to be described in the next step. Mulder et al. suggest to provide a title and to describe the content of each EPA, to select the underpinning domains of competence, to specify the required knowledge and skill and to describe the assessment methods [21]. As an example, two more elementary EPAs from the set of EPAs in table 3 are given in tables 4 [Tab. 4] and 5 [Tab. 5].

For the achievement of the EPA “running a ward in a multidisciplinary team” the expected level of entrusted independence must be defined for each underpinning EPA. While for the EPA “leading a ward round” level 4 (the resident is able to do this independently) may be expected, for the EPA “taking a history and performing a physical examination of XXX-patients” level 5 (the resident is able to supervise others in performing this activity) seems adequate as a resident may supervise medical students in their practice year performing this task. The overall achievement of the EPA “running a ward in a multidisciplinary team” at level 3 (the resident is able to do this if supervision is available) can be expected at an early time of residency, while level 4 may be reached at a later stage of residency. Rather than the time spent on a certain EPA the achieved level of independence is the driving force for entrustment; this can differ between residents according to their individual behaviour and competencies [31]. However, an exemplary timeframe when certain levels for certain EPAs are expected during residency training may guide and help to visualize the longitudinal dimension of EPAs (see table 6 [Tab. 6]).

In the actual planning of an EPA-based residency curriculum a few aspects need particular attention independently of a specific discipline. To distinguish EPAs from general learning objectives or skills it can be helpful to complete the sentence “Tomorrow the resident will be allowed to …” [21]. For example, one can be allowed to perform a pleural puncture under direct supervision – this would be EPA performance level 2 – whereas one cannot be allowed to “communicate respectfully” which would be a skill needed to achieve informed consent from the patient for this procedure. As described above and shown in table 1 [Tab. 1], EPAs can vary in complexity and scope. A complex EPA, e.g. “running a ward in a multidisciplinary team”, can consist of several less complex EPAs. To be able to define the number of EPAs needed for a residency training one should think of the requirements of a resident close to taking the board exam. When considering this it will be noticeable that certain EPAs will be relevant for all disciplines, some EPAs will be of interdisciplinary relevance and a special set of EPAs will be only relevant for a specific discipline.

The description of the individual EPAs and their underpinning domains of competence including knowledge, skills and attitudes should be explicit enough for observed assessment. Assessment may include direct observation and feedback (e.g. following a ward round) or “indirect observation” by assessing the quality of patient reports and discharge letters. Another assessment method could be a developmental portfolio in combination with progress interviews where milestones for reaching level 4, i.e. independent work, can be marked. This assessment method can be used to adjust the curriculum according to the progress of the individual resident [5].

It is essential to mix assessment of directly observed behavior with judgments of trainees’ performance over a certain period of time. Whereas specific objective workplace-based assessments tools, such as mini-clinical evaluation exercise (mini CEX) or objective-structured assessment of technical skills (OSATS) may be used to assess directly observed behaviour, multisource feedback (MSF) may serve to asses performance over time [21], [22]. First experiences show that a ratio 1:5 to 1:10 between assessment of directly observed behavior and performance judgments over time may be feasible [26]. All the more the few tasks, which should be assessed by direct observation, must be chosen wisely to capture the most important activities for effective and safe entrustment decisions. Finally, the process and tools for assessment of entrustment decisions should be clearly defined, structured and transparent to supervisors and trainees. Training of supervisors will be crucial, particularly with respect to assessment and entrustment decisions, as most clinicians may not be familiar with short structured observations and the culture of regular written feedback [21].


The German perspective

The authors feel confident that the definition and use of professional competencies as essential outcomes of postgraduate training could be the one central driving force to reform and further develop the German, but also the Austrian and Swiss postgraduate medical education systems.

As already mentioned earlier, major competency-based postgraduate training frameworks, such as CanMEDS and the ACGME Outcome Project, were defined some years ago [ACGME Outcome project – General competencies documentation (accessed January 13, 2013)], [7], [8], [12] and have already been integrated in a number of countries in daily postgraduate training [30]. In Germany the CanMEDS framework builds the backbone of the National Competence-Based Learning Objectives Catalogue for Undergraduate Medical Education (NKLM), which is being developed by a nation-wide initiative led by the Association of Medical Faculties (MFT) and the Association for Medical Education (GMA) [13]. However, many clinicians still feel uncomfortable or even reject these competency-based frameworks [30]. Many see it as “theoretical constructs” with some academic but not much practical value: unclear and artificial in terminology, complicated not self-explanatory in structure, with unrealistic assessment methods etc. Consequently, well-meant initiatives may never reach implementation in clinical training.

Similar apprehensions may lead the present discussion concerning the call for new concepts of postgraduate training by the 115th annual conference of the German Physicians Board 2012. We face hesitation to omit the present regulations consisting listings of general clinical learning topics and numerical values for medical procedures in favour of competencies [Bartmann F-J. Musterweiterbildungsordnung – Sachstand zur Novellierung. 2013 (accessed June 29, 2013)]. We must answer two central questions if competence-based frameworks are meant to reach wide acceptance and implementation in residency reality:

1.
How can we align a competence-based curriculum with the day-to-day patient care of residents and their supervisors?
2.
How can we integrate regular and valid, and feasible assessment strategies depicting the various physician roles in all fields of clinical practice?

We believe that the present concept of EPAs can “bridge the gap between theory and clinical practice” and may be used as a framework for implementation of the new regulations for certification (MWBO) in the clinical residency programs of all medical disciplines. A rough proposal of a workflow including representative steps towards implementation of such a framework is shown in table 7 [Tab. 7]. Such a process would integrate the national development of a new competence-based framework and faculty development measures to support local implementation.

Furthermore, any new initiative in postgraduate medical education should consider the current development in undergraduate medical education. New regulations of certification should be aligned with the newly developed NKLM. The integration of EPAs in the practical year of the undergraduate medical curriculum in close matching with the NKLM, would provide an optimal transfer and continuity between under- to postgraduate education [17].

In summary, the EPA concept may have the potential for wide acceptance in the clinical world and shows perspectives, which currently used curricular frameworks lack:

1.
Focus on relevant and daily clinical activities including the continuum from single procedures to general competencies.
2.
Obvious and self-explanatory levels of entrustment enabling the definition of guiding timeframes but also the consideration of individual learning curves.
3.
Feasible and transparently structured assessment systems consisting of a mixture of objective measures and entrustment decisions.

The Commission of Postgraduate Medical Training of the German Society for Medical Education works on exemplary EPA curricula development and pursues supporting activities and tools to further introduce and promote the concept on different levels of German postgraduate medical education.


Competing interests

The authors declare that they have no competing interests.


References

1.
Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011;11:96-103. DOI: 10.1186/1472-6920-11-96 External link
2.
Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med. 2002;77(5):361–367. DOI: 10.1097/00001888-200205000-00003 External link
3.
Chang A, Bowen JL, Buranosky RA, Frankel RM, Gosh N, Rosenblum MJ, Thompson S, Green ML. Transforming primary care training – patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2013;28(6):801-809. DOI: 10.1007/s11606-012-2193-3 External link
4.
David DM, Euteneier A, Fischer MR, Hahn EG, Johannink J, Kulike K, Lauch R, Lindhorst E, Noll-Hussong M, Pinilla S, Weih M, Wenneke V. Die Zukunft der ärztlichen Weiterbildung in Deutschland – Positionspapier des Ausschusses Weiterbildung der Gesellschaft für Medizinische Ausbildung (GMA). GMS Z Med Ausbild. 2013;30(2):Doc 26. DOI: 10-3205/zma000869 External link
5.
Dekker H, Driessen E, Ter Braak E, Scheele F, Slaets J, Van der Molen T, Cohen-Schotanus J. Mentoring portfolio use in undergraduate and postgraduate medical education. Med Teach. 2009;31(10):903-909. DOI: 10.3109/01421590903173697 External link
6.
Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287(2):226-235. DOI: 10.1001/jama.287.2.226 External link
7.
Frank JR. The CanMEDs 2005 physician competency framework. Better standards. Better physicians. Better care. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2005. Zugänglich unter/available from: http://www.royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/resources/publications/framework_full_e.pdf (accessed January 13, 2013) External link
8.
Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29(7):642-647. DOI: 1080/01421590701746983 External link
9.
Frank JR, Snell LS, Ten Cate O, Holmboe ES, Carraccio C, Swing SR, Harris P, Glasgow NJ, Campbell C, Dath D, Harden RM, Iobst W, Long DM, Mungroo R, Richardson DL, Sherbino J, Silver I, Taber S, Talbot M, Harris KA. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-645. DOI: 10.3109/0142159X.2010.501190 External link
10.
Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, Fineberg H, Garcia P, Ke Y, Kelley P, Kistnasamy B, Meleis A, Naylor D, Pablos-Mendez A, Reddy S, Scrimshaw S, Sepulveda J, Serwadda D, Zurayk H. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376(9756):1923–1958. DOI: 10.1016/S0140-6736(10)61854-5 External link
11.
Füllhase C, Werner S, Kurpick O, Fichtner J. Das gemeinsame Logbuch "Urologische Weiterbildung" von GeSRU, DGU, BDU. Ein Meilenstein für die urologische Weiterbildung in Deutschland. Urologe. 2008;47(3):348-356. DOI: 10.1007/s00120-008-1628-2 External link
12.
Green ML, Aagaard EM, Caverzagie KJ, Chick DA, Holmboe E, Kane G, Smith CD, Iobst W. Charting the road to competence: Developmental milestones for internal medicine residency training. J Grad Med Educ. 2009;1(1):5-20. DOI: 10.4300/01.01.0003 External link
13.
Hahn EG, Fischer MR. Nationaler Kompetenzbasierter Lernzielkatalog Medizin (NKLM) für Deutschland: Zusammenarbeit der Gesellschaft für Medizinische Ausbildung (GMA) und des Medizinischen Fakultätentages (MFT). GMS Z für Med Ausbild. 2009;26(3):Doc35. DOI: 10.3205/zma000627 External link
14.
Hauer KE, Durning SJ, Kernan WN, Fagan MJ, Mintz M, O'Sullivan PS, Battistone M, DeFer T, Elnicki M, Harrell H, Reddy S, Boscardin CK, Schwartz MD. Factors associated with medical Students' career choice regarding internal medicine. JAMA. 2008;300(10)1154-1164. DOI: 10.1001/jama.300.10.1154 External link
15.
Hibbeler B. Anreize statt Zwang. Dtsch Ärztebl. 2012;109(22-23):A1143-A1144.
16.
Iobst WF, Sherbino J, Ten Cate O, Richardson DL, Dath D, Swing SR, Harris P, Mungroo R, Holmboe ES, Frank JR, for the international CBME collaborators. Competency-based medical education in postgraduate medical education. Med Teach. 2010;32(8):651-656. DOI: 10.3109/0142159X.2010.500709 External link
17.
Kadmon M, Ganschow P, Gillen S, Hofmann HS, Braune M, Johannink J, Kühn P, Buhr HJ, Berberat PO. Der kompetente Chirurg – Brückenschlag zwischen der Ausbildung im Praktischen Jahr und der chirurgischen Weiterbildung. Chirurg. 2013. [Epub ahead of print]. DOI: 10.1007/s00104-013-2531-y External link
18.
Krones CJ, Binnebösel M, Stumpf M, Schumpelick V. Praxisnahe Weiterbildung – Das Aachener Modell. Chirurg. 2010;81:7-13. DOI: 10.1007/s00104-009-1760-6 External link
19.
Laponis R, O'Sullivan PS, Hollander H, Cornett P, Julian K. Educating generalists: factors of resident continuity clinic associated with perceived impact on choosing a generalist career. J Grad Med Educ. 2011;3(4):469-474. DOI: 10.4300/JGME-D-10-00227.1 External link
20.
Montagne S, Kurmann J, Jucker-Kupper P, Beyeler C, Bauer W. Einführung des Arbeitsplatz-basierten Assessments (AbA) durch die Fachgesellschaften. Erste Erfahrungen in der Psychiatrie und Psychotherapie. Schw Ärztez. 2013;94(6):207-210.
21.
Mulder H, ten Cate O, Daalder R, Berkvens J. Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training. Med Teach. 2010;32(10):e453-e459. DOI: 10.3109/0142159X.2010.513719 External link
22.
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007;29(9):855-871. DOI: 10.1080/01421590701775453 External link
23.
Reed CE, Vaporciyan AA, Erikson C, Dill MJ, Carpenter AJ, Guleserian KJ, Merrill WH. Factors dominating choice of surgical specialty. J Am Coll Surg. 2010;210(3):319-324. DOI: 10.1016/j.jamcollsurg.2009.11.016 External link
24.
Schmidt GN, Fiege M, Goetz AE. Weiterbildung in der Anästhesiologie. Umsetzung am Universitätsklinikum Hamburg-Eppendorf. Anaesthesist. 2011;60(4):366-374. DOI: 10.1007/s00101-010-1836-1 External link
25.
Steinhäuser J, Chenot JF, Roos M, Ledig T, Joos S. Competence-based curriculum development for general practice in Germany: a stepwise peer-based approach instead of reinventing the wheel. BMC Res Notes. 2013;6:314. DOI: 10.1186/1756-0500-6-314 External link
26.
Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate O. When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010;85(9):1408-1417. DOI: 10.1097/ACM.0b013e3181eab0ec External link
27.
Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007;29(7):648-654. DOI: 10.1080/01421590701392903 External link
28.
Tambyraja AL, McCrea CA, Parks RW,Garden OJ. Attitudes of medical students toward careers in general surgery. World J Surg. 2008;32(6):960-963. DOI: 10.1007/s00268-008-9529-5 External link
29.
Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176-1177. DOI: 10.1111/j.1365-2929.2005.02341.x External link
30.
Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542-547. DOI: 10.1097/ACM.0b013e31805559c7 External link
31.
Ten Cate O, Snell L, Carraccio C. Medical competence: the interplay between individual ability and the health care environment. Med Teach. 2010;32(8):669-675. DOI: 10.3109/0142159X.2010.500897 External link
32.
Weinert FE. Vergleichende Leistungsmessung in Schulen - eine umstrittene Selbstverständlichkeit. In: Weinert FE (Hrsg). Leistungsmessung in Schulen. Weinheim: Beltz; 2002. S.17-31
33.
Wijnen-Meijer M, Van der Schaaf M, Booji E, Harendza S, Boscardin C, Van Wijngaarden J, Ten Cate TJ. An argument-based approach to the validation of UHTRUST: can we measure how recent graduates can be trusted with unfamiliar tasks? Adv Health Sci Educ Theory Pract. 2013. [Epub ahead of print] DOI: 10.1007/s10459-013-9444-x External link
34.
Wijnen-Meijer M, van der Schaaf M, Nillesen K, Harendza S, Ten Cate O. Essential facets of competence that enable trust in graduates: a Delphi study among Physician educators in the Netherlands. J Grad Med Educ. 2013;5:46-53. DOI: 10.4300/JGME-D-11-00324.1 External link