gms | German Medical Science

Scientific rigour and interdisciplinary competence-oriented teaching are the hallmarks of modern medical degree programs. The Master Plan 2020, formulated in 2017 by the Federal Ministry of Health, the Federal Ministry of Research and Education, and state representatives, explicitly calls for the mandatory inclusion of systematic training in scientific concepts and methods within a reformed medical curriculum [1]. Graduates are expected to use scientific skills competently.

These recommendations are based on, among other things, the expert statement of the German Council of Science and Humanities [2]. This statement calls for sequential courses on scientific literacy as a central component of the curriculum, also given international developments, and emphasizes the relevance of skills-oriented student-centred teaching formats. Core topics of the physician as a scientist refer to scientific attitude and information evaluation, application of scientific knowledge to patient care, lifelong learning with critical reflection, and the ability to teach. The core topics also explicitly include skills in statistical methods and publication practice.

The new National Competence-Based Learning Objectives Catalogue for Medicine has implemented the recommendations [3], [4]. Medical-scientific competencies (Chapter VIII.1) are differentiated in terms of learning and teaching (VIII.1.2, VIII.1.5), evidence-based medicine (EbM, VIII.1.3), but also practical research training (VIII.1.6) and individual research projects (VIII.1.7), as well as the basic knowledge (VIII.1.4.1) and practical skills (VIII.1.4.2) required for this purpose [3], [4]. The Institute for Medical and Pharmaceutical Examination Questions (IMPP) has already included many of these learning objectives as examination items for the second state examination [5].

Subject-specific differences in science influence how research methods and content, attitudes, and the concept of science as such are taught [6]. These differences need to be consciously perceived. They challenge the alignment of methodological science education and individually supervised project work. On the other hand, training in evidence-based medicine (EbM) is mandatory for all medical students and requires and deepens clinical epidemiology and biometry knowledge [7], [8].

The medical data sciences epidemiology, medical biometry, and medical informatics are essential as basic subjects of clinical trial methodology in providing scientific qualifications, including EbM and the ability to conduct research responsibly. With this project presentation, we would like to contribute to an exchange on implementation possibilities in science education within the medical curriculum.

The Medical Faculty of the University of Oldenburg, founded in 2012, implements a Z-curriculum as a model study program [9]. The Z-curriculum describes the abolition of the traditional separation between pre-clinical and clinical topics in separate phases of the curriculum. Scientific competencies are taught within the framework of a longitudinal research curriculum. Methodological basics were taught via a lecture block with statistical content in year 1, then via a seminar accompanying individual research projects (occasional lectures over three years, attendance not mandatory). Three increasingly extensive research papers in freely selectable subjects with freely selectable lecturers served as proof of achievement. All research projects were assigned to cross-section area 1 (Q1, epidemiology, biometrics, and medical informatics). However, research skills and knowledge from epidemiology, biometry and medical informatics were neither required nor taught or guided in a structured way nor checked professionally within these projects.

With time, problems with this approach became apparent. For example, optimizing seminar content for project work from different subjects requires intense supervisor cooperation. Without direct examination or implementation relevance, lecture series or lecture blocks are not helpful for sustainable learning success with application competence. Lecture attendance could have been better.

From around 2015, tasks from Q1 were introduced in the module final examinations, and from 2017 a compensatory examination Q1 was introduced for admission to the state examination in the event of failed subject-specific exams. The testing of basic knowledge from research and evidence-based medicine, based on the Berlin EbM questionnaire [10], showed considerable learning deficits. Thus, 15 of 35 first-year students required the Q1 compensation exam, only five sat on the first attempt, and only one was successful in the first attempt, so makeshift short-acting revision courses had to be set up.

We have since developed an independent curriculum of epidemiology and biometrics (Figure 1 [Fig. 1]). Journal clubs were first introduced in year 4 to address topics in evidence-based medicine and epidemiologic research relevant to physician practice. At the same time, fundamental methods in epidemiology and biometry were separated from general science topics. They are now taught in eight lectures with six accompanying exercises in the third semester. This position in the curriculum corresponds to the location of the revised learning objectives and competence levels in the NKLM 2.0 [3].

The introduction of a data analysis project (DAP) in the fourth semester was motivated by the following aims:

• To improve the sustainability of previously learned predominantly theoretical research skills from Q1 subjects through direct application. Following the ICAP model, interactive and constructive elements of the DAP extend the active and passive learning phases of exercises and lectures [11].
• To improve structured research training by singling out and targeting specific competencies from the data sciences under the guidance of experts in the relevant subjects.

The overall aim of the DAP is thus to provide medical students with fundamental supervised-application-level competencies in attitudes, methods, and skills as required from our subjects for student research projects and mapped in the NKLM. The project is designed to teach research tools. Publication of results is explicitly excluded to reduce the burden on students and supervisors. Furthermore, we consider the inflation of un(der)financed selectively published small projects problematic as a potential result of mandatory student research. Adding to research waste would collide with teaching research integrity and quality [12], [13].

The DAP is divided into four phases, extending over one semester during the curriculum. Students work in tandems as a measure to promote teamwork. Tandem work also enables the implementation of quality assurance through double programming or code review. The students can choose the partner and the topic, and there is also a choice for language and statistical software (German, English, SPSS, R).

After an informational and introductory session, the students partner up and formulate a preliminary research question based on information about available variables. The first meeting with a supervisor is designed to ensure the feasibility of the planned project and the comparability by severity and effort and to avoid the duplication of projects. Students then prepare an exposé to discuss their plans with a supervisor.

Phase 2 involves formulating a detailed statistical analysis plan (“SAP”). In addition to the formulation of the study question, specific information is required on the type of data, the analysis design, required variables, planned steps for plausibility testing of the data, description of the study population and further statistical analyses, information on the expected number of cases, quality assurance measures, and information on the ethics vote and data protection. Only after discussion, revision, and resubmission of the final SAP will data access be granted. Two supervisor meetings are scheduled for this planning phase.

In the subsequent analysis phase, the tandems work largely independently. We offer an additional “intensive week” when rooms are available for work on-site or online for better exchange between different tandems and with direct supervisor access for general questions.

The project concludes with the writing up of the results. Again, a discussion of a first draft is granted before submitting a final version. Optionally, a learning reflection can be submitted pseudonymously once the project is completed but before marks are received.

The learning objectives of the DAP per phase are shown in Table 1 [Tab. 1]. Based on mapping with the learning objectives of the NKLM 2.0, all learning objectives from the field of science over the first study phase, which require expertise in epidemiology and biometry, are covered ([14], page 14, Tab. 2) (VIII.1.4.1 and 4.2 in the NKLM 2.0). We introduce most of this content in the previous semester through lectures and exercises. By the DAP, the competence level is then extended from basic understanding to supervised execution.

In addition, learning objectives are touched upon that are not primarily assigned to epidemiology and biometry and only need to be taught in the DAP for the first time if they have not yet been covered by other courses in the research pathway up to that point (Table 1 [Tab. 1], marked with *).

We currently offer five different data sets. Three of them originate from our surveys and use different designs, such as a parallel survey of children and their parents, which allows agreement analysis, and a survey of diseased persons with healthy controls. Students may calculate scores on quality of life, depression and anxiety, medication adherence, children’s social behaviour, and socioeconomic status and examine their association with other characteristics. Data sets from clinical registries of collaborating colleagues include information on tumour stage, comorbidity, genotype and survival. Exemplary analyses include Kaplan-Meier curves.

It is planned that additional data sets can be contributed; for example, laboratory data will be a valuable extension of the selection. Datasets need to include variables with different scale levels, useful to describe the study population and offer various research questions.

Lectures, seminars, material production, lecturer training and technical implementation are the professor’s responsibility. Apart from the exercises, research assistants supervise about eight (part-time) to 14 (full-time) tandems as tutors. Meeting times are contingent and should focus on methodological questions of planning and evaluation. The department’s data manager supports technical questions (data access, preparation of data sets, protected interfaces, and software issues, including reference management).

Students receive extensive material made available via the Stud-IP platform through a web-based learning system (Courseware) (Figure 2 [Fig. 2]) [15]. The materials, including form templates, explanatory documents, videos, links, and literature recommendations, are continuously developed. Separate group assignments and time restrictions are available to control the selective visibility of documents. This platform is also used to submit drafts and finished work, make appointments, generate announcements, and ask general questions (chat function, “blubber”).

Exposé (10%), SAP (50%) and final report (40%) are first independently evaluated by the supervisor and one other person, usually the division head or senior biometrician, and then consented. Secondary scoring and consensus discussion are used to train staff and ensure comparable standards. Students can earn bonus points for doing and writing the project in English, using R for analyses, and submitting a reflection on their learning process.

Student feedback is mainly obtained through these voluntary pseudonymized learning reflections. Summative student evaluation of the course by the university or faculty is also possible. However, we consider reflections more helpful for strengthening the students’ sense of responsibility required to master sustainable science skills. They are also a helpful source of information for us to improve the course.

The faculty currently assigns two credit points (CP) for the DAP, corresponding to a student time commitment of 60 hours. However, the student effort is more like three CPs. Five lectures per two teaching units are credited to the division head, up to one week as an internship (intensive week), and 0.3 SWS per supervised tandem to the tutors. Videos, platform maintenance, other materials, second opinions, and supervision are not accounted for.

All data sets are available in the department only in de facto anonymized form. Students can access the data via a protected interface and cannot copy it to their computers. Access is only granted to those variables that are necessary for the respective project and have been specifically requested. In addition, 80% of the data sets are allocated in each case, randomly selected and provided with newly randomized identification numbers.

The procedure was presented to the responsible ethics committee. An ethics vote was unnecessary since the data were anonymized and not researched in the true sense of the word (Prof. F. Griesinger, Med. Ethics Committee of the University of Oldenburg, request 2018-085, dated July 26, 2018).

The project was first offered in 2018 as a “small research paper” and chosen by eight students who served as pilots. It became mandatory for all 40 students in a cohort starting in the summer semester 2019. Since 2020, the number of students has doubled from year 1, with 120 participants starting each year since 2023; the long-term goal is to have 200 students studying in Oldenburg each year. Further project developments over time primarily increase support efficiency, especially since no information is available on whether personnel resources will be adjusted to teaching tasks in the future. In addition, we are continuously working on standards to optimize processes and improve learning success.

From the beginning, the DAP was perceived as a worthwhile joint team effort. The following individual points were raised or needed adjustment:

• The credit for the DAP does not represent the actual effort of either the students or the faculty. However, all employees consider the project essential and put other work on hold to make it work.
• The requirement of an SAP was initially considered critical in terms of feasibility. However, we have the impression that it is in this phase that both the need for support and the learning effects are highest.
• Learning effects are also evident for the lecturers. Students’ written accounts of methods and results document much more clearly than oral exams or multiple choice (MC) questions where and how misunderstandings and misconceptions can occur.
• It is gratifying to see a gleam in the student’s eyes when they have bought into their choice of a topic or to read outstanding papers that easily exceed the level of many a medical dissertation or health science master’s thesis in terms of competencies from epidemiology and biometry.
• Initially, it was envisaged that the final report would just be the SAP conversed to past tense and supplemented by the results and discussion. We changed this, however, to a final report in the style of a “real” publication. We recommend the STROBE statement and JAMA author guidelines for this purpose, with cutbacks in the introduction and discussion. This makes the papers much more pleasant to read and evaluate. The reports have also become shorter.
• It took some time to decide how to proceed regarding sample size estimation. This concept is considered too advanced for this stage. Also, the main task on the clinician’s side is not in calculating a sample size estimation but in the provision of all necessary information. The search efforts to find out about relevant differences and to back up other decisions are beyond the scope of the DAP. We now request a flow chart of the expected case numbers and a statement on whether formal sample size estimation or power analysis would be required if this were a true project.
• The distinction between violated inclusion criteria, implausible data and outliers seems difficult to grasp, particularly since our datasets previously underwent exceptional quality assurance methods and data cleaning. We now include self-generated implausible data to make the problem more tangible.
• In the in-presence intensive phases, tandems worked closely together rather than dividing tasks. We find this positive in terms of an intensive learning experience. However, it does increase the time needed to complete the DAP. Unfortunately, the general curricular student workload does not allow us to offer more intensive days covering all DAP phases.
• To enable non-German-speaking staff to participate in teaching and to promote internationalization, we are increasingly offering exercises and DAP in English. For example, all lectures in the introductory phase were made available digitally in English in 2022. Currently, seven of 40 tandems work entirely in English, and an additional partially (only the final report will be in English).
• Students who choose R as their software still need more support. It seems that more than the five software exercise sessions currently offered before the DAP are required. Nine out of 40 tandems are currently working with R. Since we are very interested in offering development opportunities to more interested students, additional workshops on R are being planned.
• Also problematic is the grasp of epidemiological concepts. This concerns the correct citation of frequency data in the introduction (required) and somewhat brash handling of concepts such as representativeness or bias in discussing the results (explicitly NOT required). Since 2023, the work on the introduction, including the search for correct frequency data, has been brought forward from the final report to the exposé to avoid it getting lost in the final spurt.
• Timing collisions with exam periods and science internships at the end of the semester are problematic and regularly manifest themselves in quality lapses between SAP and the final report.
• The project was implemented with minor modifications during the pandemic. However, the DAP in 2021 turned out unsatisfactory concerning work quality (no excellent marks) and could have been more pleasant regarding the feedback from the learning reflexions (see below).
• Otherwise, we typically see a good spread of grades between 1 (very good) and 4 (sufficient), with an average of around 2 (good). About one tandem per cohort is invited to a four-way interview to avert non-passes, and about one student per year drops out and defers to the following year.
• The quality of the journal clubs in year 4 increased notably with the introduction of intensified training in year 2, although we still encounter occasional groups where active student participation remains a problem. Initially, no real text work was possible due to a general lack of basic knowledge in clinical epidemiology.

Some of the first learning reflections we received were so enthusiastic that we used pseudonymization more consistently from the following year to encourage more critical voices. A systematic evaluation through qualitative methods is pending. The following observations are commonly encountered:

• Students are mostly positive about partnering and developing their skills in the project.
• The support provided by the scientific staff and the provision of documents is also predominantly described as positive and helpful.
• Most students assume that the project prepares well for further research projects and that these can now be better envisioned.
• The student feedback could be more explicit regarding the impact on motivation for further research.
• Fears and uncertainties before starting the DAP are expressed almost regularly and are usually diffuse (I did not know what to expect, worried if I could do it, etc.).
• Reflections of the 2021 “pandemic” DAP cohort (online only, including lectures and exercises starting in 2000) were exceptional because there was hardly any self-reflection on learning processes. Instead, students reported issues of increased sensitivity, such as feeling offended by the tone of written materials or by assumed preconceptions of team members. There were also more than usual complaints about the project, including issues such as offering too many elective options and reports on inter-personal problems.
• Recurring critical topics in all years are the high time requirements, difficulties in using SPSS, which are often attributed to too much time lapse between the exercises and the DAP, and technical problems with data access via the remote server.

We succeeded in implementing the learning objectves of the NKLM and our ideas of good teaching. All staff members are committed, lecture attendance is good, and students show high engagement in their projects almost across the board. We have yet to formally prove our impression of intensive learning effects through early independent writing and combining different learning formats, including e-learning. However, these formats are also described positively in the literature [16], [17]. For many students, it is a remarkably steep step from performing supervised exercises during the winter term to self-reliant planning and application of newly learnt research skills, including problem-solving, illustrated, among other things, when using SPSS.

A central point for the future success of the DAP will be to what extent the elaborate project work can be continued with increasing student numbers. Currently, we manage 40 tandems simultaneously in a core team with three scientific staff (2 FTE) of epidemiology and biometrics plus a postdoctoral fellow of a cooperating clinical department and support by our technical staff. In 2024, the number of tandems will increase to about 60, perspectively over further years, to 100.

One measure will be the participation of additional staff from other divisions. Support by tutors will need to be increasingly restricted, standardization and detail of specifications will be further improved, dual assessments of trained employees will be limited to random samples and digital support will be further expanded.

There is also potential for optimization concerning the integration into the general medical curriculum. Synergy will likely occur if further basic research essentials, to be delivered by colleagues from ethics, medical informatics and sociology, are brought forward to the first study stage per NKLM 2.0. Unfortunately, we do not have a concept for the methodological supervision of student research work in later years, which will take up and complement the intensive basic training. These issues are not subject to our teaching responsibilities, although they should ideally be well aligned.

In addition, a formal evaluation of the long-term effects, such as on quality and supervision efforts in ethics applications and research papers and the basic scientific understanding of the graduates, is pending. We seek cooperation partners to evaluate the learning reflections for process evaluation qualitatively.

We took advantage of the unique situation of a model degree program with longitudinal pathways and initially low student numbers to develop a curriculum that might not have been realistic in this form in large conventional degree programs. Given the positive impressions, extensive standardization and digital support, we hope the format will remain feasible and can be further developed as student numbers continue to grow.

In summary, such a DAP is an essential bridge between the teaching of basic skills from data science and their application in student projects from other disciplines.

AT thanks Sarah Rose, Calgary, who introduced her to having to do a data analysis project in tandem many years ago. It inspired the current curriculum. We wish to thank all students who contributed to the further development of the DAP with comments and feedback. In addition, thanks go to the staff in the secretariat of the Department for Health Services Research, Mr Saß and Ms Garten, among others, for processing the anonymous reflections and administrative help, as well as the staff in the Dean of Studies Office, for example, Ms Gronewold, Ms König, and Mr Jerominek, who, under conditions that are not always easy, make a friendly and competent effort to provide support at all times.

The authors declare that they have no competing interests.