gms | German Medical Science

• Olaf Kuhnigk, MD, MME (Bern), is head of the office of the vice-dean of education and specialist in psychiatry at the Medical Faculty of Hamburg University, Germany.
• Julia Schreiner is a psychologist and works at the office of the vice-dean of education at the Medical Faculty of Hamburg University, Germany.
• Sigrid Harendza, MD, MME (Bern), is full professor for internal medicine and educational research and development at the Medical Faculty of Hamburg University, Germany.

Until the 1970s it was assumed that medical school faculty members possessed teaching expertise once they had acquired the knowledge of their discipline, even though they never received any formal “teacher training” [19](. With the increasing demand upon medical school faculty members to be effective teachers, successful researchers and skilful clinicians, the need to gain teaching skills became more evident [3]. Nowadays, teacher training is regarded as an important factor within faculty development programmes for the promotion of successful learning in medical schools [4]. However, medical schools often find it challenging to identify their faculty’s needs and to develop the right strategies to provide support appropriate to the nature of the curriculum [30]. With education at the centre of academic medicine, there is an acknowledged need for comprehensive faculty development and teaching scholar programmes [36], and a broad range of ideas, tips and programmes to promote excellence in medical teaching has been established and discussed [16], [27], [34].

Although teacher training programmes have been shown to improve short-term outcomes in participants [13] and are often rated by participants as being highly useful [21], evidence for long-term change in participants’ teaching skills or students’ perception of improvement of teaching after a training programme is limited [1], [25]. One study demonstrated an improvement in didactic skills directly and up to one year after a didactic training [18]. Another study showed that didactic training lead to a long lasting improvement of lecturing skills [17]. It seems important to address professional academic skills, often neglected in teacher training [33], as well as institutional goals and priorities in general faculty development [36]. Furthermore, studies investigating the reasons why some clinical teachers do not attend centralised teacher training activities have concluded that many organisational issues as well as teachers’ personal and professional needs should be addressed when designing a programme [35].

According to Steinert, teacher training in the 21^st century will need to broaden its focus by using diverse learning methods based on established learning theories, fostering partnerships and collaboration, and rigorously evaluating interventions to keep pace with the changes in medical curricula [33]. To address these demands, the Medical Faculty of Hamburg University in Germany, whose interns and residents as well as assistant, associate and full professors are all involved in teaching, introduced a teacher training workshop in 2006 for Hamburg Medical School. The main focus of this workshop is on didactic methods, problem-based learning (PBL), objective structured clinical examination (OSCE) and tandem coaching. The three-day training, still being offered three to six times a year for 20 participants, became part of the general faculty development programme for academic staff seeking to become assistant or associate professors at our medical faculty. To some extent, workshop participation was also included as a term in the employment contract for new faculty members.

Four years into the programme the aim of this study was to evaluate, whether from the participants’ perspective the teacher training workshops had been helpful in achieving sufficient competency and sustained change in their didactic skills. In particular, participants’ current self-assessment of certain didactic competencies that had been addressed in the training was requested and compared with their self-assessment regarding these skills at the time of the training. Participants were also asked to state their personal reasons for participation in the workshop. With a view to further developing the programme, we were interested in whether the recollection of the educational content of the workshops correlates with the sustainability of a certain competency from the participants’ viewpoint.

In 2006 a teacher training workshop was established at the Medical Faculty of Hamburg University in Germany as part of the faculty development programme. Each training covered the topics “didactic methods”, “tandem coaching”, “PBL”, and “OSCE”, and lasted for three full days (Thursday to Saturday). An overview of the workshop’s content including its learning objectives, concepts and the respective activities of the trainers and participants is given in table 1 [Tab. 1]. The main focus of the workshop lies on active involvement of the participants to achieve the learning objectives. Considering the important effects of the “hidden curriculum” on the learners’ perception [14], the workshops took place at a conference hotel outside of Hamburg where participants and trainers stayed for two nights. Costs and expenses were covered by the medical faculty. Full participation was rewarded with 32 CME points by the State Physicians’ Board of Lower Saxony.

Recruiting and description of the sample: between June 2006 and December 2009 a total of 322 faculty members participated in 18 trainings. At the beginning (t0) and at the end (t1) of each training participants were asked to complete a self-assessment questionnaire regarding their teaching competencies in the above-mentioned topics. Of the 322 original participants, 36 had left the Hamburg Medical School at the time of this study, 313 participants, whose addresses (including new addresses) could be verified, received the same competency self-assessment questionnaire as at the time of the training (t2) and an evaluation questionnaire by post in 2010. The questionnaires included a 6-point Likert scale (with 1 being the lowest value and 6 being the highest value) as well as dichotomous questions and questions with multiple selections. By means of an exploratory factor analysis, we reduced the original statements about different self-assessed educational competencies from the questionnaires to four well-interpretable scales. This solution explains a total of 77.5% variance and makes it possible to unambiguously attribute every item to one factor, respectively. The generated scales with satisfactory reliability were entitled didactic methods (α=.884), tandem coaching (α=.857), PBL (α=.889) and OSCE (α=.833).

Resulting scores of competencies in the Likert-scaled self-assessment questionnaire were treated as continuous interval-scaled variables. The descriptive parameters frequency, arithmetic mean (M) and standard deviation (SD) were calculated. In addition, several comparisons of the means were drawn to identify differences between groups, i.e. two-sample and paired-sample t-tests as well as a one-way analysis of variance with multiple comparisons according to Bonferroni, respectively and where appropriate, which includes an adjustment of the family-wise error rate [7]. An exploratory factor analysis based on the Principal Component Analysis was conducted to create scales, combined with the calculation of Cronbach’s alpha to check for internal consistency [7]. We tested with a critical p value of 0.05. If needed, an adjustment of critical p values in terms of the Bonferroni correction was applied. Effect sizes were calculated by Cohen’s d [11]: ≥0.2=small effect;≥0.5=moderate effect;≥0.8=large effect. All data were analysed with PASW Statistics Version 18.0.3.

Of the 313 participants of the faculty development programme who had received the questionnaire, 184 returned it, corresponding to a return rate of 58.8% (see Table 2 [Tab. 2]). There was no significant difference in the return rate regarding the individual years of the training.

When asked for their personal reasons to participate in the faculty development workshop, 152 (83%) of the 184 participants answered that they participated because they wanted to improve their didactic skills (see Figure 1 [Fig. 1]). The questionnaire allowed multiple answers from a given list, and only 55 participants marked “improvement of didactic skills” as the one and only reason for participation.

Participation in order to be able to apply for an assistant professorship was checked by almost 50% of the participants (n=84), while participation because it had been mandatory in the employment contract was checked less frequently (n=39, i.e., 21%). Other reasons for participation were, for instance, “networking with other teachers from the faculty of medicine” or “objective in the personal target and performance agreement with the head of department”. Table 3 [Tab. 3] shows the distribution of personal reasons for participation within the different cohorts.

Since personal reasons for participation in the faculty development workshop differed, we were interested in corelations between the different groups with their self-assessed level of competence in the follow-up questionnaire (see Figure 2 [Fig. 2]). No correlations of personal reasons for participation and initial the self-assessed for the different competencies were discovered.

For the results regarding self-evaluation in the different aspects of the training, only 119 complete data sets (64.7%) of the 184 participants were available. This was due to missing data on one of the three questionnaires from some participants or owing to loss of the original training questionnaires during a move of the dean of education’s office in 2008.

Participants’ ratings regarding their different educational competencies – didactic methods, tandem coaching, PBL and OSCE – before and after the workshop and at the respective time of follow-up (i.e. one to four years after the training) are shown in Figure 3–6 [Fig. 3], [Fig. 4], [Fig. 5], [Fig. 6]. A significant increase for all competency scales was seen in all cohorts when the pre-workshop questionnaire (t0) was compared with the questionnaire directly after the workshop (t1) and with the questionnaire at the time of the follow-up (t2). Effect sizes were large for all four educational competencies in all cohorts except for “tandem coaching” in cohort 2007, 2008 and 2009, where effect size was only moderate. A significant decrease was found between the self-assessment directly after the workshop compared to the self-assessment at the time of the follow-up for all four educational competencies in the respective cohorts (see Figure 3–6 [Fig. 3], [Fig. 4], [Fig. 5], [Fig. 6]).

A significant decrease with moderate effect sizes in “didactic methods” between t1 and t2 was only observed in cohorts 2007 (T(20)=2.45, p<0.05, d=0.64), 2008, (T(37)=4.60, p<0.001, d=0.79), and 2009 ((T35)=4.08, p<0.001, d=0.67). For “tandem coaching”, a significant decrease with small effect sizes was only seen in cohorts 2008 and 2009 (T(37)=1.37, p<0.05, d=0.21; T(35)=2.20, p<0.05, d=0.23). For PBL, the self-assessed competency decreased between t1 and t2 significantly in all cohorts with small to moderate effect sizes except for cohort 2006: (T(23)=1,5, p=0.07, d=0.21; 2007: T(20)=1.91, p<0.05, d=0.40; 2008; T(37)=4.38, p<0.0001, d=0.75; 2009: T(35)=3.36, p<0.001, d=0.36). OSCE also showed a significant decrease in all cohorts with moderate to large effect sizes (2006: T(23)=2.95, p<0.01, d=0.57; 2007: T(20)=4.36, p<0.001, d=1.23; 2008: T(37)=7.16, p<0.0001, d=1.27; 2009: t(35)=3.99, p<0.001, d=.72).

Since self-assessment regarding OSCE seemed to have the largest decrease in competency compared with the self-assessment in the three other didactic competencies, we compared the means in OSCE competency of all 158 participants who had answered the question in the t2 evaluation irrespective of whether they were currently involved in OSCEs (see Figure 7A [Fig. 7]). A significantly higher self-assessment regarding OSCE skills was found in faculty members who were involved in OSCEs at the time of follow-up (t2). When the 120 participants who were not involved in OSCEs and the 38 participants who were involved in OSCEs at the time of follow-up were compared regarding their recollection of the OSCE section during the faculty development workshop, there was also a significant difference (see Figure 7B [Fig. 7]).

The faculty development workshop at the Medical Faculty of Hamburg University in Germany was designed to meet the needs of faculty members who taught in the newly developed medical curriculum. Yet, apparently, only one-third of the faculty members participated solely because they wanted to improve their didactic skills. At least two-thirds of the participants had other reasons for attendance which were not intrinsically motivated, such as needing the workshop certificate to apply for an associate professorship or being obligated to participate by their employment contract. Interestingly, the different kinds of personal reasons for attending the workshop were not associated with significant differences in the self-assessment of the didactic competencies at the time of the follow-up. This supports the postulation by Steinert et al. that it is time to move beyond ‘volunteerism’ concerning participation in faculty development programmes [34], as our study shows that the outcome of participants who attended the programme for mandatory reasons was not worse than that of voluntary participants. It has been demonstrated that, on the one hand, inexperienced teachers benefited greatly from a teacher training workshop [2] and, on the other hand, teachers with greater experience profited more from faculty development [23]. Hence, the approach of the Medical Faculty of Hamburg University to make the training a mandatory part of the employment contract, and of the application for an associate professorship, seems to be a step in the right direction as this appears to be beneficial for both groups.

Compared to the self-assessment before the faculty development workshop (t0), values in all four educational skills at the time of follow-up were still significantly higher in all cohorts irrespective of the time difference (one to four years) between the training and this study. We took this as a positive sign of the training’s long-lasting effect. However, it has been shown that extended programmes like a seminar series result in greater improvements as far as the retention of didactic skills is concerned and in additional benefits like the creation of networks [28]. Interestingly, in our participants the smallest decrease of competency in the different didactic skills between t1 and t2 was observed in the 2006 cohort, which had the longest interval between training and time of the follow-up questionnaire. Among all cohorts, the 2006 one had the highest percentage (62.2%) of participants mentioning “improvement of didactic skills” as a personal reason to participate in the faculty development workshop. A recent meta-analysis found that motivation appears to affect learning and study behaviour as an independent variable [22], which could also be a possible explanation for our finding as the teachers are in a student’s role during the training.

Considering that faculty development workshops should meet the participants’ needs to be most effective [24], we were wondering whether there was a difference in self-assessment of certain didactic skills – in this case, OSCE – between teachers who are involved in working with this assessment format and those teachers who do not currently use it. Our study indicated a significantly higher self-assessment in OSCE skills in teachers who still used this didactic format at the time of follow-up (t2). Designing faculty development formats in a task-centred way with an emphasis on immediacy of application to improve their outcome has been proposed previously [10] and can be confirmed by our findings. This result is an important finding towards further developing our teacher training workshop along the lines of an even more learner-centred approach where participants will be encouraged to apply what they have learnt to their own practice, as this was found to have an important impact on teaching and learning [29].

Several limitations of this study need to be mentioned. Nine addresses of former participants of the faculty development workshop who are no longer teaching at the Medical Faculty of Hamburg University could not be retrieved, which may lead to a small bias in the sample. A second limitation is due to the fact that it was not possible to use all 184 returned questionnaires of the follow-up (t2) for longitudinal comparisons because several questionnaires from t0 and t1 – or some answers on these questionnaires – were missing. Hence, only 119 complete data sets were available for the comparison of the self-assessment of the educational competencies at all three points in time, reducing the good return rate of 58.79% to 38.0%, which still seemed acceptable to carry on with the investigation. Another limitation is the lack of a control group because didactic skills can also be acquired by teaching practice which can not be excluded as confounding variable in our setting. Furthermore, the main assessment of the didactic competencies was based on self-evaluation. Measuring students’ learning outcomes may produce more solid data on the improvement of a teacher’s didactic skills. As it is very difficult to assess students’ learning outcome in relation to teaching quality after a teacher training workshop, self-assessment of the teachers has been used as a surrogate parameter in other studies quite successfully [8] and was therefore chosen in our approach. Additionally, the main focus of our workshop was on active involvement of the participants which has been shown to lead to better evaluation of courses and to increased knowledge retention [12]. Especially in the first year of the training (2006) many participants had been actively involved in curricular change in their departments. This may also have played a role in the self-assessment of our participants and might have lead to a long lasting impression as far as the taught contents are regarded.

In summary, participants’ self-assessment of didactic skills up to four years after attending a faculty development workshop is still significantly higher than before the workshop. The long-term outcome in feeling competent using certain didactic skills seems to be independent of the original personal reason for participation in this workshop. Active use of certain didactic skills correlates with feeling competent in applying these skills. Our study supports further development of teacher training workshops with a strong learner-centred approach, as meeting participants’ needs is correlated with a better outcome of feeling competent in acquired teaching skills in the long run.

The authors thank all participants in the faculty development programme who took the time to answer our questionnaires. The faculty development programme is supported by a grant from the Medical Faculty of the University of Hamburg.

The authors declare that they have no competing interests.