gms | German Medical Science

Introduction: In 2013 a formative competency-based written Progress test (144 MCQs) was developed. Items were constructed based on a two-dimensional blueprint. The first axis consisted of eight clusters of medical disciplines according to the German State Exam; the second axis consisted of five competency domains: theoretical clinical, practical clinical, communicative, scientific and professional behavior competencies. For each combination, the number of items was fixed corresponding to the blueprint [1]. The first progress test was carried out in November 2013; the second one will be run in November 2014. On this second progress test, 20 MCQs will be substituted with situational judgment items.

To provide meaningful feedback for students and faculties regarding test results on competency domains, sufficient reliability and discriminant validity of these domains are required. The aim of this study was to empirically analyze these properties of the competency domains.

Methods: Reliabilities of the competency domains were estimated by the coefficient “glb” [2]. To establish the discriminant validity of the domains, a discriminant analysis of principal components [3] of the data was conducted (“one against the rest” and multiclass discriminant analysis).

Results: All domains showed a high reliability for the 2013 data. The discriminant validity of the scientific competency domain was high. Theoretical clinical and practical clinical competencies could not be separated from each other, but differed significantly as a group from the other domains. Communication and professional domains showed a moderate independence and were significantly different from clinical and scientific competencies.

Results of the second progress test in November 2014 will also be presented.

Discussion: Items of the clinical competency domains, communication/professional competence, as well as scientific competence could be verified as three reliable and distinct clusters. With improved classification of the items, all five competency areas can be proved to be different constructs.