gms | German Medical Science

Introduction: Key feature (KF) questions can be used to assess higher-order cognitive skills. In these exams, students are presented with a clinical case scenario and asked to provide information on diagnostic tests, differential diagnoses and the treatment of choice. According to recent research, the act of reproducing learned material in a productive test enhances later retention of the tested information, a phenomenon known as the ‘direct testing effect’. This study assessed whether repeatedly testing diagnostic and therapeutic decision-making in formative KF examinations fosters the acquisition of these higher order cognitive skills in medical students.

Methods: In winter 2013/14, a total of 106 fourth-year medical students participated in a randomised-controlled cross-over trial in which they attended weekly e-learning sessions. Per session, four clinical cases were presented. Students in the intervention group completed five KF questions per case while students in the control group simply read the cases with no interspersed questions. Group assignments were switched every week so that all students spent the same time on ‘question cases’ and ‘reading cases’, respectively. In a final formative KF examination, all students were tested on 30 items, 15 of which they had been exposed to in question cases during e-learning sessions (intervention items) while the other 15 items had been addressed in reading cases (control items).

Results: In the exit exam, students scored significantly higher on intervention than control items (9.1±0.4 versus 7.2±0.4 out of 15 points; p<0.001; see Table 1 for results per item [Tab. 1]). A linear regression analysis revealed that this effect was independent of age, sex, overall performance levels, and – most importantly – time on task.

Discussion/conclusion: Reading clinical cases with interspersed KF questions was more effective than reading cases without such questions. This is the first study to report evidence of a significant testing effect with regard to higher order cognitive skills [1], [2], [3], [4].