gms | German Medical Science

GMS Journal for Medical Education

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 2366-5017

Publication activities relating to digital teaching and learning in the GMS Journal for Medical Education – a descriptive analysis (1984–2020)

article medical education research

  • corresponding author Christin Kleinsorgen - University of Veterinary Medicine Hannover, Foundation, Centre for E-Learning, Didactics and Educational Research (ZELDA), Hannover, Germany
  • Andrea Baumann - University of Tübingen, Faculty of Medicine, The Competence Center for University Teaching in Medicine Baden-Württemberg, Faculty, Tübingen, Germany
  • Barbara Braun - Medical Faculty Mannheim of the University of Heidelberg, Studies and teaching development, digital teaching, Mannheim, Germany
  • Jan Griewatz - University of Tübingen, Faculty of Medicine, The Competence Center for University Teaching in Medicine Baden-Württemberg, Faculty, Tübingen, Germany
  • Johannes Lang - Justus-Liebig University Gießen, Faculty of Medicine, Division for Study and Teaching, Gießen, Germany
  • Holger Lenz - LMU Munich, University Hospital, Institute for Medical Education, Munich, Germany
  • Johanna Mink - University Hospital Heidelberg, Department of General Medicine and Health Services Research, Heidelberg, Germany
  • Tobias Raupach - University Hospital Bonn, Institute for Medical Education, Bonn, Germany
  • Bernd Romeike - University Medical Center, Academic Dean's Office, Division of Medical Education, Rostock, Germany
  • Thomas Christian Sauter - University Hospital Bern, Inselspital, Department of Emergency Medicine, Bern, Switzerland
  • Achim Schneider - Ulm University, Medical Faculty, Office of the Dean of Studies, Ulm, Germany
  • Daniel Tolks - Leuphana University Lüneburg, Centre for Applied Health Promotion, Lüneburg, Germany; Bielefeld University, Faculty of Medicine, WG Digital Medicine, Bielefeld, Germany
  • Inga Hege - LMU Munich, University Hospital, Institute for Medical Education, Munich, Germany; University of Augsburg, Medical Education Sciences, Augsburg, Germany

GMS J Med Educ 2022;39(5):Doc59

doi: 10.3205/zma001580, urn:nbn:de:0183-zma0015806

This is the English version of the article.
The German version can be found at: http://www.egms.de/de/journals/zma/2022-39/zma001580.shtml

Received: March 29, 2022
Revised: July 15, 2022
Accepted: August 22, 2022
Published: November 15, 2022

© 2022 Kleinsorgen et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/.


Abstract

Aims and objectives: Digital teaching, learning and assessment have been part of medical education and continuing education for decades. The objective of this review paper is to highlight developments and perspectives in these areas in the GMS Journal for Medical Education (GMS JME).

Methodology: In the spring of 2020, we conducted a systematic literature search of the Journal for Medical Education (JME) and analysed the articles with regard to different categories such as article type, digital tools used or mode of data collection.

Results: Of the 132 articles analysed, 78 were digital interventions (53 of which were exploratory-descriptive), 28 were project descriptions, 16 were surveys of needs or equipment and 10 were concept papers. About one-third of the studies and project reports each dealt with virtual patients or case-based learning, whereas no articles were published on trends such as serious games or virtual reality. Overall, our analysis shows that in many respects, the studies on digital teaching were more broadly based, especially between 2006 and 2010, after which this trend tended to decline again.

Conclusions: Our analysis shows that publications in the JME consider some key aspects of digital teaching in medical education and continuing education, such as educational videos or virtual patients. The variability of information and methods of presentation advocate the use of guidelines to optimise the quality of scientific papers. Furthermore, clues for future research topics and experimental study designs are identified.

Keywords: literature review, digital teaching, e-learning, digital learning, digital assessment, GMS Journal for Medical Education


1. Introduction

Digital teaching and learning are gaining increasing attention, but it is by no means a new approach; they have been known under various terms in university education and continuing education since the 1960s. Alternatively to digital teaching, terms such as e-learning, computer-based learning (CBL) or, since the development of the Internet, web-based training (WBT) have been and are used [1]. In spite of this long period of time, a review shows that there are often quality deficits in the design, data collection and ethical aspects of implementing studies on digital teaching and learning [2]. Since the 1960s, the format of digital teaching has evolved from computer applications to web-based training modules to Massive Open Online Courses (MOOC) [3] or simulations in virtual worlds. Current trends include adaptive learning or learning applications based on Artificial Intelligence (AI), Learning Analytics as well as Big Data [4]. The target groups of digital learning in the health professions have also evolved and now include not only students, trainees, teachers and clinicians, but also patients and relatives [5]. A study by Matthes et al. (2017) already showed the increasing importance of medical education in general with an analysis of articles in the GMS Journal for Medical Education (GMS JME) [6]. However, as this study covered all topics of medical education, no conclusions can be drawn regarding specific topics such as digital learning and teaching.

Therefore, the aim of our analysis was to investigate to what extent and how changes and increasing availability of digital tools and techniques are also reflected in the implementation of studies and curricular use in faculties and schools in JME since the first publication of the journal.

From this, conclusions can be drawn about what the focus is in the publication of research on digital teaching in the German-speaking world and which applications have been less used or researched. Based on this, future directions, further developments and recommendations can be derived.


2. Methods

2.1. Search strategy

Our search strategy was divided as follows: articles from 1984 (first edition) to 2011 inclusive were searched manually on the JME website due to the lack of indexing. For volumes from 2012 onwards, we were able to use the search function in PubMed with the following query:

((“GMS journal for medical education”[Journal]) OR (“GMS journal for medical education” [Journal])) AND ((e-learning) OR (elearning) or (game) OR (gamification) OR (inverted) OR (flipped) OR (blended) OR (audience response) OR (((digital) OR (app) OR (computer) OR (computer-based) or (online) or (virtual) or (mobile) OR (technology) OR (web-based) OR (multimedia)) AND ((learning) OR (assessment) OR (teaching) OR (education)))).

All articles that met the following inclusion criteria were included in the evaluation:

  • Research papers and project reports on the topic of digitisation in education, training and continuing education in health care professions
  • Publication date between 1984 (first issue) and May 2020

Excluded were:

  • Editorials, abstracts, reviews, position papers and book or film reviews
  • Articles not directly related to teaching, learning or assessment, e.g. about purely administrative support

The search and extraction of articles took place in spring 2020.

The assessment of the articles with regard to inclusion or exclusion was carried out independently by two authors. In case of initially divergent assessments, a consensus was reached in a discussion.

2.2. Coding

We developed the categories for coding in an inductive-deductive process. First, categories were developed based on the literature and the authors’ broad experience with digital teaching. The category "type of article" was developed inductively based on the articles, as it became apparent during the coding process that the original division into “study” and “project” was insufficient. After consultation, a coding instruction manual (see attachment 1 [Attach. 1]) with examples and explanations was created to instruct the coding. This includes several sections with information on authors, study or project design, digital tools used and type of results (see table 1 [Tab. 1]). The manual was discussed several times, piloted with five articles and subsequently adapted with regard to some category descriptions. The articles were each assigned to two authors for individual coding, then all coding was merged and discrepancies between the two coders were marked. Discrepancies were discussed, and in case of doubt, one or more persons were consulted until consensus was reached.


3. Results

In total through our combined search strategy, we found 221 articles, 89 of which we did not consider for further analysis in the following step due to the exclusion criteria. Thus, 132 articles were included in the further analysis.

3.1. Article type and publication year

Among the 132 articles analysed were 78 studies (59.1%), 28 project descriptions (21.2%), 16 surveys of needs or equipment (12.1%) and 10 concept papers (7.6%). The first article on experiences with the use of an audiovisual program for cardiac screening [7] dates from 1986. The trend in the proportion of studies on digital interventions shows an increase over the studied period (see figure 1 [Fig. 1]).

Articles on concept development

The 10 articles on concept development dealt with considerations on very different topics that were current at the time. The concepts ranged from technical possibilities such as intranet technology [8], computer-based examinations [9] and social media [10], legal and qualitative aspects of digital teaching [11], [12] to considerations of how curricula could be restructured through digitisation [13], [14] or how existing digital tools could be further developed [15].

Articles on needs and equipment survey

Eight of the 16 articles (50.0%) on the needs and equipment survey addressed the use of computers and computer labs [16], [17], digital tools in general [18], [19] and specific formats such as wikis [20]. Five articles surveyed current offers and existing equipment for digital teaching [21], [22]. Two articles surveyed a need for digital teaching in each faculty [23], [24] and one article dealt with the survey on the implementation of quality management in the field of digital teaching [25].

Project descriptions

The total of 28 project descriptions dealt with both the development of digital tools and the integration and use of digital media at the faculties (n=13 each, 43.3%). Four projects (13.3%) were also or exclusively concerned with the creation of digital content. A trend was not discernible over the studied time period.

Studies

Based on the classification of Ringsted et al. (2011) [26], the majority of the 78 studies were exploratory-descriptive ones (n=53, 68.0%), in addition to 14 experimental studies (18.0%) and 11 cohort studies (14.0%).

Of the 25 experimental or cohort studies, 18 (72.0%) compared different settings. Of these, nine studies (40.9%) compared different digital tools or features; eight studies (36.4%) compared the use of a digital program with an analogue or face-to-face setting; three studies (13.6%) compared a blended learning setting with presence sessions; and two studies had a control group without any intervention. There was no discernible trend over the study period.

3.2. Methodology & target groups
Data collection

Sixty-two of 78 digital studies (79.5%) used questionnaires for data collection, digital platform usage data, and tests or exams were each used by 27 studies (34.6%). Fewer of the studies used interviews (n=9, 11.5%) or focus groups (n=6, 7.7%). One article did not identify how data were collected, one evaluated quality circle meetings [27] and another evaluated written paper feedback [28]. Out of a total of 78 studies, data were collected prospectively in 39 (50.0%) and retrospectively in 37 (47.4%) studies. In two studies, data were obtained both prospectively and retrospectively. There was no discernible trend over the study period.

Among the 16 needs and equipment surveys, questionnaires were used in 11 articles (68.8%), three articles (18.8 %) described an analysis of digital repositories, catalogues or websites, and in two articles, the survey instrument was not clearly specified.

Target groups

Students were the main target group in 66 studies (84.6%). Teachers were addressed as a target group in 11 studies (14.1%), whereas trainees or persons attending a didactic training course were only considered in 3.85 % each (n=3). Other target groups were, for example, parents [29] or prospective students [30]. Figure 2 [Fig. 2] shows the distribution over the study period.

Of the 11 articles on surveys of needs or equipment that were conducted with questionnaires and the two articles with unknown data collection instruments, students were the most frequently surveyed group (n=10, 76.9%), followed by teachers and institutes in three articles each (23.1%), and students, clinicians and participants in continuing education in one article each (7.7%).

In terms of professions, 65 out of the 78 studies (78.3%) involved human medicine, seven involved dentistry (8.4%) and six involved veterinary medicine (7.2%). Interprofessional studies were described in two articles from 2010 and 2011 (2.4%). Other professions included psychology [31] or medical-technical assistants in radiography [32], [33]; there were no studies from nursing. There was no clear trend over the study period; human medicine was the predominant profession, with percentages ranging from 66.7% (2006-2010) to 88.2% (2016-2020) or 100% (1986-2000).

In total, the studies were conducted in 26 different specialised fields or subjects. The most common specialised fields included internal medicine (n=10, 12.2%), studies in interdisciplinary or multidisciplinary settings (n=10, 12.2%) and biochemistry (n=9, 11.0%). Other specialist fields included general medicine, anatomy, surgery as well as cross-cutting areas such as clinical skills, gender medicine and communication training. There was no discernible trend over the study period.

3.3. Study design

Sixty-four (82.1%) of the 78 studies were performed at one single-site, 13 (16.7%) were multi-centric, and in one article (3.2%), the location of study could not be classified. An increase in multi-centric studies was evident over the study period.

Data evaluation was quantitative in 52 studies (66.7%), qualitative in five studies (6.4%) and both quantitative and qualitative in 20 studies (25.6%). One study could not be classified (1.4%). While quantitative studies predominated until 2005, a qualitative study was published for the first time in 2009 [34], and as of 2016, studies with combined survey instruments predominate (see figure 3 [Fig. 3]).

In the 16 survey of needs or equipment, data analysis was predominantly quantitative (n=15, 93.8%); only one article described a combination of quantitative and qualitative methods.

Kirkpatrick

Fifty-one of the 78 study findings (65.4%) were based on the reaction level, capturing, for example, satisfaction with a program, 34 (43.6%) were based on the learning level, five (6.4%) were based on the behavioural level and six (7.7%) were based on the results level. The Kirkpatrick level was not evident in six articles (7.7%). There is no discernible trend over the studied time period; there is a wide variation between the time periods (see figure 4 [Fig. 4]).

3.4. Digital implementation
Activities

The studies most frequently examined virtual patients/cases (VPs) (n=26, 33.3%) and courses in learning management systems (LMS) (n=15, 19.2%). In contrast, no studies were found on serious games, MOOCs or virtual reality (VR) scenarios. Some activities (e.g. instructional videos, video recordings and visual materials) were described rather less frequently, especially up to 2010 and thereafter. Otherwise, no trends are discernible over the studied period (see figure 5 [Fig. 5]).

Among the 28 project descriptions, VPs (n=13, 46.4%) also predominated in individual activities, followed by educational videos (n=8, 28.6%) and video recordings (n=7, 25.0%). A total of 11 projects (39.3%) dealt more cross-cuttingly with e-learning platforms and repositories, for example. There was one project report on a serious game, but no project descriptions on MOOCs, e-portfolios, blogs, virtual reality (VR), gamification and audience response systems (ARS).

Study setting

Most frequently, the studies examined blended learning settings, i.e. a balance between presence and digital teaching (n=32, 41.0%); 27 studies (34.6%) examined pure e-learning settings, 11 (14.1%) examined the use of digital tools or programs in presence teaching and nine (11.5%) dealt with computer-based assessment. Five articles (6.4%) described video recording, e.g. for the purpose of feedback in communication training. Two articles (2.6%) did not identify the setting. Figure 6 [Fig. 6] shows the chronological course; a discernible trend is not evident here.

In terms of access and availability, programs which were available online played a dominant role with 74.4% (n=58), increasing to 100 % since the first online study in 1999. In contrast, offline applications (e.g. in form of CD-ROMs) played an increasingly minor role with 19.2% (n=15), especially since 2011. Studies on mobile devices were also hardly considered, with only 2.6% (n=2). Access was not evident in four articles (5.1%).

The proportion of synchronous and asynchronous study settings was roughly equally distributed over the entire study period, however, with a decrease in synchronous and an increase in asynchronous settings.

ICAP Activities

In the digital studies during the studied time period, active learning settings were most common with 44.9% (n=35), followed by constructive with 24.4% (n=19). The ICAP level was not evident in six articles (7.7%). There was a tendency for passive learning settings to decrease over the studied time period, whereas active ones tended to increase. No trend can be identified for constructive and interactive learning settings (see figure 7 [Fig. 7]).

Fifty-five of the 78 (70.5%) digital studies aimed to teach knowledge and skills, 12 (15.4%) aimed to teach skills and five (6.4%) aimed to teach attitudes; the competence level was not specified in six articles (7.7%). There was no discernible trend over the investigated period.

In addition to the coded aspects, overall it was noticeable when reviewing the articles that it was not always the case that research questions were always clearly formulated, and some studies were not well theoretically grounded.


4. Discussion

Against the background of increasing digitisation in medical education and healthcare, we analysed the published articles in the period from the first publication of the JME to May 2020 regarding developments in this area.

Overall, no general increase in articles on digital teaching could be identified in the period under consideration, but rather a continuous increase in studies since 1986 as well as a striking accumulation of project reports in the period between 2006-2010. The latter is probably related to the publication of a conference volume in JME on a symposium on “E-Learning in Medicine” in 2006.

Similarly, as Matthes et al. (2017) demonstrated in an analysis of all JME articles between 2007-2015 [6], exploratory (descriptive) studies also predominate in the topic of digital teaching.

Despite the already available evidence, almost 60% of non-descriptive studies compared digital to non-digital settings or to control groups with no intervention. Cook et al. (2009) argued already in 2009 that further studies comparing digital and traditional learning settings or no intervention do not contribute anything new to the research because digital teaching has already been shown to be as effective as traditional teaching and more effective than no teaching at all [35], [36]. Instead, Cook et al. (2009) see research gaps regarding when digital teaching should be used and how to make it most effective. To this end, more randomised controlled trials examining different digital deployment scenarios in terms of their effectiveness would also be desirable in JME and would advance the state of current research. Even though studies with purely qualitative designs or even studies with qualitative as well as quantitative survey methods increased over the period of investigation, complementary mixed-method designs could actually only rarely be identified [37].

A total of 65.4% of the studies measured the response of the study participants compared to approximately 50% internationally [38], [39], and 43.6% measured the learning outcome by analysing usage data or using knowledge tests. Future studies could focus even more on the behavioural and outcome levels, which have not been given much consideration so far.

With regard to the target groups or study participants, there were more studies from the non-human medical professions and interprofessional studies, especially in the period 2006-2015, but human medicine predominated overall. Surprisingly, no interprofessional studies on digital teaching were published in the period 2016-2020 despite the special issue “interprofessional education” in 2016. Among the study participants, there was a diversification, especially in the years 2006-2010, with teachers, trainees and participants in didactic and professional development courses. Students were the predominant target group in the JME, both in studies (84.6%) and in surveys of needs or equipment (76.9%). In an international comparison, the focus here in German-speaking countries seems to be more on education; in a review by Curran et al. (2017), only 56% of studies were related to education [36]. Studies show that online learning is also suitable for training interprofessional collaboration and, despite technical challenges, addresses geographical and time barriers, among others, and enables more flexible learning [40], [41], [42]. In the future, we believe that a stronger focus on studies on digital teaching and learning in interprofessional settings and with a greater diversity of study participants would be desirable in order to address this important aspect.

Regarding the digital formats and tools used, the focus of the studies and projects was over 30% on case-based learning/virtual patients. Studies or projects with social media were not considered very much, despite a special issue on this topic in 2013. Also, no studies or project reports on other past and current trends such as MOOCs, serious games or virtual reality learning scenarios were published in the JME. One explanation could be that these topics are published in special journals such as “simulation in healthcare”. Nonetheless, it is also possible that such formats do not (yet) play a major role in medical education in German-speaking countries. One reason could be the comparatively high effort and costs in developing such formats, especially compared to other simulation formats such as virtual patients (VPs) [43]. This could also be the reason why mobile applications have hardly been considered in studies even in recent years. While it can be assumed that most online applications were also playable on mobile devices, specific apps or studies targeting the use of mobile devices were not described.

More than 70% of the studies targeted knowledge transfer, including that through the use of formats such as VPs that are not primarily used for knowledge transfer [44]. Nevertheless, due to the now wide variety and availability of digital formats and tools, skills, collaboration and communication competencies can also be taught through digital and blended learning based teaching.

In conclusion, as also described by Nicoll et al. (2018), some studies lacked or did not clearly describe details such as research questions, theoretical foundation, study population or data collection. This makes study results difficult to interpret or transfer [2]. For future research, we recommend theoretical foundation, e.g. of instructional design frameworks, and description of the study using guidelines such as an extension proposed for simulations by Cheng et al. (2016) [45].

Limitations

By limiting our search to one medical didactic journal in the German-speaking area, some relevant articles published for example in medical journals or other international medical didactic journals could certainly not be considered. However, in the analysis, we were also explicitly interested in presenting the developments in JME and drawing conclusions from them.


5. Conclusions

Overall, our analysis shows that in many respects, studies on digital teaching were more broadly based, especially between 2006 and 2010, after which this trend tended to decline again. Nonetheless, the articles analysed were submitted before the COVID-19 pandemic, so the resulting changes in the field of digital teaching had not yet been taken into account. We hope that even after the pandemic, many ideas of digital implementation, many of which have already been published as project reports in a COVID-19 special issue [46], will be sustainably implemented, further developed and researched. To this end, we would like our recommendations, based on the analysis, to provide clues for future research topics and experimental study designs. It may also be possible to bring these further to the fore through the publication of special issues on special digital aspects.


Funding

This Open Access publication was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 491094227 “Open Access Publication Funding” and the University of Veterinary Medicine Hannover, Foundation.


Competing interests

The authors declare that they have no competing interests.


References

1.
Regmi K, Jones L. A systematic review of the factors – enablers and barriers – affecting e-learning in health sciences education. BMC Med Educ. 2020;20(1):91. DOI: 10.1186/s12909-020-02007-6 External link
2.
Nicoll P, MacRury S, van Woerden HC, Smyth K. Evaluation of Technology-Enhanced Learning Programs for Health Care Professionals: Systematic Review. J Med Internet Res. 2018;20(4):e131. DOI: 10.2196/jmir.9085 External link
3.
Hoy MB. MOOCs 101: An Introduction to Massive Open Online Courses. Med Ref Serv Q. 2014;33(1):85-91. DOI: 10.1080/02763869.2014.866490 External link
4.
Pelletier K, McCormack M, Reeves J, Robert J, Arbino N, Al-Freih M, Dickson-Deane C, Guevara C, Koster L, Sanchez-Mendiola M, Skallerup Bessette L, Stine J. 2022 EDUCAUSE Horizon Report Teaching and Learning Edition. Boulder, CO: EDUC22; 2022. Zugänglich unter/available from: https://library.educause.edu/resources/2022/4/2022-educause-horizon-report-teaching-and-learning-edition External link
5.
Pottier E, Boulanouar L, Bertrand M, Estrade A, Croiset A, Martineau C, Plantec JY, Escourou B, Ritz P. A MOOC About Bariatric Surgery Improves Knowledge and Promotes Patients' Soft Skills. Obes Surg. 2020;30(4):1600-1604. DOI: 10.1007/s11695-019-04143-5 External link
6.
Matthes J, Giesler M, Wagner-Menghin M, Himmelbauer M, Preusche I, Schüttpelz-Brauns K. Publication activity in medical education research: A descriptive analysis of submissions to the GMS Zeitschrift für Medizinische Ausbildung in 2007-2015. GMS J Med Educ. 2017;34(3):Doc32. DOI: 10.3205/zma001109 External link
7.
Rentschler HE, Burkhard CP. Erfahrungen mit dem Einsatz eines audiovisuellen Programms zur Herzuntersuchung. Med Ausbild. 1986;3(1):21-28. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/1986.html External link
8.
Dugas M, Demant T. Intranet-Technologie für die Medizinische Ausbildung am Beispiel Klinische Chemie. Gesundheitswesen (Suppl Med Ausbild). 1998;15(Suppl1):27-30. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/1998.html External link
9.
Frey P. Computerbasiert prüfen: Möglichkeiten und Grenzen. GMS Z Med Ausbild. 2006;23(3):Doc49. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2006-23/zma000268.shtml External link
10.
Hempel G, Neef M, Rotzoll D, Heinke W. Medizinstudium 2.0 dank Web 2.0?! – Risiken und Chancen am Beispiel des Leipziger Medizinstudiums. GMS Z Med Ausbild. 2013;30(1):Doc11. DOI: 10.3205/zma000854 External link
11.
von Gerlach R, Ehlers JP. Einfluss des neuen Urheberrechtsgesetzes auf die Erstellung, den Einsatz und die Verbreitung von Computerlernprogrammen. GMS Z Med Ausbild. 2005;22(4):Doc209. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2005-22/zma000209.shtml External link
12.
Scholz W, Fassnacht U, Öchsner W, Stracke S, Waldmann UM, Freidl R, Liebhardt H. Ulmer Qualitätskriterienkatalog für medizinische Lernprogramme. GMS Z Med Ausbild. 2006;23(1):Doc17. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2006-23/zma000236.shtml External link
13.
ten Cate TJ, Heijlman J, Donnison JD, Wijngaard PRA. Strategy for faculty wide introduction of computer aided instruction in medical education. Med Ausbild. 1990;7(1):77-80. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/1990.html External link
14.
Tolks D, Schäfer C, Raupach T, Kruse L, Sarikas A, Gerhardt-Szép S, Klauer G, Lemos M, Fischer MR, Eichner B, Sostmann K, Hege I. An Introduction to the Inverted/Flipped Classroom Model in Education and Advanced Training in Medicine and in the Healthcare Professions. GMS J Med Educ. 2016;33(3):Doc46. DOI: 10.3205/zma001045 External link
15.
Hege I, Kononowicz AA, Berman NB, Lenzer B, Kiesewetter J. Advancing clinical reasoning in virtual patients – development and application of a conceptual framework. GMS J Med Educ. 2018;35(1):Doc12. DOI: 10.3205/zma001159 External link
16.
Schimmelpfenning K, Schmidt J.Untersuchung über jene Anlässe, die Medizinstudenten behindern, Lehrfilme einer Mediothek freiwillig zu nutzen. Med Ausbild. 1997;14(1):43-45. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/1997.html External link
17.
Hövelmann R, Droslhagen S. Computerunterstütztes Selbststudium in der Medizin (CUSS). Gesundheitswesen (Suppl Med Ausbild). 2000;17(Suppl1):42-44. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/2000.html External link
18.
Vogelsang M, Rockenbauch K, Wrigge H, Heinke W, Hempel G. Medical Education for “Generation Z”: Everything online?! – An analysis of Internet-based media use by teachers in medicine. GMS J Med Educ. 2018;35(2):Doc21. DOI: 10.3205/zma001168 External link
19.
Tenhaven C, Tipold A, Fischer MR, Ehlers JP. Gibt es eine „net-Generation“ in der Tiermedizin? Eine Vergleichende Untersuchung zur Nutzung von Internet und Web 2.0 bei Studierenden und der Tierärzteschaft. GMS Z Med Ausbild. 2013;30(1):Doc7. DOI: 10.3205/zma000850 External link
20.
Kolski D, Arlt S, Birk S, Heuwieser W. Nutzung und Akzeptanz von Wiki-Systemen bei Studierenden der Tiermedizin. GMS Z Med Ausbild. 2013;30(1):Doc10. DOI: 10.3205/zma000853 External link
21.
Ehlers JP, Frikerl J, Liebichl HC, Stolla R. PC-Ausstattung und -nutzung von Studierenden der Tiermedizin im Vergleich zu Schülern der 12. Klasse. Gesundheitswesen (Suppl Med Ausbild). 2002;19(Suppl2):124-126. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/2002.html External link
22.
Schochow M, Steger F. State of Digital Education Options in the areas of Medical Terminology and the History, Theory and Ethics of Medicine. GMS Z Med Ausbild. 2015;32(2):Doc17. DOI: 10.3205/zma000959 External link
23.
Tiebel M, Dieterich P, Tiebel O, Hesse R, Kunath H, Dieter P. E-Learning an der Medizinischen Fakultät Dresden. GMS Z Med Ausbild. 2005;22(4):Doc216. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2005-22/zma000216.shtml External link
24.
Liebhardt H, Blasel P. Dokumentation der Zielgruppen und Bedarfsanalyse. GMS Z Med Ausbild. 2006;23(1):Doc19. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2006-23/zma000238.shtml External link
25.
Abrusch J, Marienhagen J, Böckers A, Gerhardt-Szép S. Quality management of eLearning for medical education: current situation and outlook. GMS Z Med Ausbild. 2015;32(2):Doc20. DOI: 10.3205/zma000962 External link
26.
Ringsted C, Hodges B, Scherpbier A. 'The research compass': an introduction to research in medical education: AMEE Guide no. 56. Med Teach. 2011;33(9):695-709. DOI: 10.3109/0142159X.2011.595436 External link
27.
von Hübbenet J, Steiner S. Professionelles Lernen: Durch elektronisch unterstütztes Qualitätsmanagement Behandlungsstrategien optimieren und Versorgungsleistungen verbessern. Gesundheitswesen (Suppl Med Ausbild). 2000;17(Supp2):116-119. Zugänglich unter/available from: https://gesellschaft-medizinische-ausbildung.org/publizieren/zma-archiv/2000.html External link
28.
Weyers S, Noack T, Rehkämper G. Psychosocial aspects of donation and the dissection course: An extra-curricular program with the objective of assisting students confront issues surrounding gross anatomy lab. GMS Z Med Ausbild. 2014;31(2):Doc16. DOI: 10.3205/zma000908 External link
29.
Hoffmann K, Schultz JH, Conrad C, Hancke R, Lauber H, Schönemann J, Kraus B, Bosse HM, Huwendiek S, Hoffmann GF, Herzog W, Jünger J, Nikendei C. Kommunikationsschulung mittels "Standardisierter Eltern" im Fachbereich der Pädiatrie: Effekte auf die Selbst- und Fremdeinschätzung kommunikativer Kompetenzen - eine Studie im Kontrollgruppen-Design. GMS Z Med Ausbild. 2007;24(2):Doc113. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2007-24/zma000407.shtml External link
30.
Wilcken B, von Berg S, Baltersee N, Carl T, Wagels R, Ehlers JP. Entwicklung neuer Kommunikationswege - Einsatz und Nutzen von Foren in der Tiermedizin. GMS Z Med Ausbild. 2008;25(4):Doc103. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2008-25/zma000588.shtml External link
31.
Kazubke E, Schüttpelz-Brauns K. Review of multiple-choice-questions and group performance - A comparison of face-to-face and virtual groups with and without facilitation. GMS Z Med Ausbild. 2010;27(5):Doc68. DOI: 10.3205/zma000705 External link
32.
Druhmann C, Hohenberg G. Erfolgreiches Lernen in einem Blended Learning-Szenario im Vergleich mit der Präsenzausbildung - am Beispiel einer MTA-Ausbildung der Fachrichtung Radiologie. GMS Z Med Ausbild. 2009;26(4):Doc43. DOI: 10.3205/zma000636 External link
33.
Niewald M, Hohenberg G, Ziegler R, Rübe C. Die Integration eines computerbasierten Anatomie-Lernprogramms im Curriculum der Ausbildung Medizinisch-technischer Assistenten der Fachrichtung Radiologie. GMS Z Med Ausbild. 2009;26(4):Doc40. DOI: 10.3205/zma000633 External link
34.
Glenewinkel C, Rockenbauch K. Gesprächsführung in der medizinischen Ausbildung im Lehr- und Lerncurriculum: Eine qualitative Analyse der Sicht Studierender auf den Leipziger Gesprächsführungskurs [Communication training in medical education in the teaching and learning curriculum: A qualitative analysis of the students' view on the communication training programme in Leipzig]. GMS Z Med Ausbild. 2009;26(3):Doc33. DOI: 10.3205/zma000625 External link
35.
Cook DA. The failure of e-learning research to inform educational practice, and what we can do about it. Med Teach. 2009;31(2):158-162. DOI: 10.1080/01421590802691393 External link
36.
Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300(10):1181-1196. DOI: 10.1001/jama.300.10.1181 External link
37.
Sale JE, Lohfeld LH, Brazil K. Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Qual Quant. 2002;36(1):43-53. DOI: 10.1023/A:1014301607592 External link
38.
Curran V, Matthews L, Fleet L, Simmons K, Gustafson DL, Wetsch L. A Review of Digital, Social, and Mobile Technologies in Health Professional Education. J Contin Educ Health Prof. 2017;37(3):195-206. DOI: 10.1097/CEH.0000000000000168 External link
39.
Cook DA, Levinson AJ, Garside S, Dupras D, Erwin PJ, Montori VM. Instructional Design Variations in Internet-Based Learning for Health Professions Education: A Systematic Review and Meta-Analysis. Acad Med. 2010;85(5):909-922. DOI: 10.1097/ACM.0b013e3181d6c319 External link
40.
Reeves S, Freeth D. New forms of information technology, new forms of collaboration? In: Leathard A, editor. Interprofessional Collaboration: from policy to practice in Health and Social Care. London: Routledge; 2003.
41.
MacNeill H, Telner D, Sparaggis-Agaliotis A, Hanna E. All for one and one for all: understanding health professionals' experience in individual versus collaborative online learning. J Contin Educ Health Prof. 2014;34(2):102-111. DOI: 10.1002/chp.21226 External link
42.
Reeves S, Fletcher S, McLoughlin C, Yim A, Patel KD. Interprofessional online learning for primary healthcare: findings from a scoping review. BMJ Open. 2017;7(8):e016872. DOI: 10.1136/bmjopen-2017-016872 External link
43.
Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N, Tudor Car L, Carlstedt-Duke J, Car J, Zary N. Virtual Patient Simulations in Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(7):e14676. DOI: 10.2196/14676 External link
44.
Kononowicz AA, Zary N, Edelbring S, Corral J, Hege I. Virtual patients - what are we talking about? A framework to classify the meanings of the term in healthcare education. BMC Med Educ. 2015;15:11. DOI: 10.1186/s12909-015-0296-3  External link
45.
Cheng A, Kessler D, Mackinnon R, Chang TP, Nadkarni VM, Hunt EA, Duval-Arnould J, Lin Y, Cook DA, Pusic M, Hui J, Moher D, Egger M, Auerbach M; International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE) Reporting Guidelines Investigators. Reporting Guidelines for Health Care Simulation Research: Extensions to the CONSORT and STROBE Statements. Simul Healthc. 2016;11(4):238-248. DOI: 10.1097/SIH.0000000000000150 External link
46.
Tolks D, Kuhn S, Kaap-Fröhlich S. Teaching in times of COVID-19. Challenges and opportunities for digital teaching. GMS J Med Educ. 2020;37(7):Doc103. DOI: 10.3205/zma001396 External link
47.
Chi MT, Wylie R. The ICAP framework: linking cognitive engagement to active learning outcomes. Educ Psychol. 2014;49(4):219-243. DOI: 10.1080/00461520.2014.965823 External link
48.
Kirkpatrick D. Revisiting Kirkpatrick’s four level model. Training Dev. 1996;50:54-59. DOI: 10.2307/4613292 External link