gms | German Medical Science

GMS Zeitschrift für Medizinische Ausbildung

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 1860-3572

Evidence based medicine at medical school: a three week course on the basis of problem based learning: EBM course based on PBL

Erlernen evidenz-basierter Medizin in der medizinischen Ausbildung. Ein drei Wochen Blockkurs auf der Grundlage des problemorientierten Lernens

Originalarbeit/research article Humanmedizin

  • corresponding author Olaf Kuhnigk - Universitätsklinikum Hamburg-Eppendorf, Prodekanat für Lehre, Hamburg, Germany
  • author Aenne M. Böthern - Universitätsklinikum Hamburg-Eppendorf, Prodekanat für Lehre, Hamburg, Germany
  • author Volker Schoder - proDERM Institut für angewandte dermatologische Forschung GMBH, Statistik & Datenmanagement, Schenefeld/Hamburg, Germany Externer Link
  • author Monika Bullinger - Universitätsklinikum Hamburg-Eppendorf, Zentrum für Psychosoziale Medizin, Institut und Poliklinik für Medizinische Psychologie, Hamburg, Germany
  • author Sigrid Harendza - Universitätsklinikum Hamburg-Eppendorf, Medizinische Klinik, Hamburg, Germany

GMS Z Med Ausbild 2007;24(4):Doc187

Die elektronische Version dieses Artikels ist vollständig und ist verfügbar unter: http://www.egms.de/de/journals/zma/2007-24/zma000481.shtml

Eingereicht: 13. Mai 2007
Überarbeitet: 13. August 2007
Angenommen: 18. September 2007
Veröffentlicht: 14. November 2007

© 2007 Kuhnigk et al.
Dieser Artikel ist ein Open Access-Artikel und steht unter den Creative Commons Lizenzbedingungen (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.de). Er darf vervielfältigt, verbreitet und öffentlich zugänglich gemacht werden, vorausgesetzt dass Autor und Quelle genannt werden.


Abstract

Profound conceptual knowledge in evidence based medicine is required as basic competence for practising physicians. Therefore contents of courses in medical statistics have moved from mere biostatistical methods to practice oriented concepts like EbM. To train these competences a course for medical statistics was developed and implemented. In 2003 and 2004 a class of medical students (n=56) in a parallel problem based curriculum at Hamburg Medical School participated in their fifth semester at the three week problem based learning (PBL) course "The scientific approach". Course quality and learning objectives were evaluated by questionnaire and exams. Results show that students feel to have reached the learning objectives, feel enabled to understand and to critically appraise relevant literature, and feel well prepared for their own scientific work. Course contents were judged from satisfying to good. The following items were rated very positively: learning objectives, clinical orientation of the course, integration of subjects, positive learning climate, and content and organization of the course. Exam results were above average. Positive evaluation and exam results recommend this course as blueprint for teaching and learning evidence based biostatistics in medical education.

Keywords: curriculum development, evaluation, evidence based medicine, medical statistics, problem based learning, teaching methods, undergraduate medical education

Zusammenfassung

Einleitung: Profundes konzeptionelles Wissen in der „Evidenzbasierten Medizin“ stellt heutzutage eine Basiskompetenz für klinisch tätige Ärzte dar. Als Konsequenz daraus entwickelten sich Lehrinhalte der studentischen Ausbildung in der Biomathematik von der reinen Lehre statistischer Methoden zu praxisorientierten Konzepten, wie dem der evidenzbasierten Medizin. Zur Vermittlung dieser Fähigkeiten wurde ein neues Blockkurskonzept unter modernen lerndidaktischen Erkenntnissen entwickelt und erprobt.

Methode: Zwei Studienjahrgänge (n=56) des Modellstudiengangs Medizin der Universität Hamburg in Deutschland nahmen 2003 und 2004 im fünften Hochschulsemester an einem dreiwöchigen Blockkurs „Wissenschaftliches Arbeiten“ auf der lerndidaktischen Grundlage des problemorientierten Lernens teil. Kursqualität und Lernziele wurden mittels Fragebögen und Prüfungen evaluiert.

Ergebnisse: Die Evaluationsergebnisse zeigten, dass die Studierenden aus eigener Sicht die definierten Lernziele erreicht haben: Sie fühlten sich in der Lage relevante Literatur zu verstehen und kritisch zu beurteilen. Darüber hinaus fühlten sie sich auf eigenes wissenschaftliches Arbeiten vorbereitet. Die einzelnen inhaltlichen Ausbildungsabschnitte wurden insgesamt als befriedigend bis gut beurteilt. Besonders positiv wurden die Lernziele, die klinische Ausrichtung, die Vernetzung der einzelnen Fächer, die positive Lehr- und Lernatmosphäre sowie die inhaltliche Ausgestaltung und organisatorische Struktur des Blockkurses evaluiert. Die Prüfungsergebnisse fielen überdurchschnittlich gut aus.

Zusammenfassung: Die positiven Evalutions- und Prüfungsergebnisse sprechen dafür, dass der Kurs „Wissenschaftliches Arbeiten“ als „Blueprint“ für das Erlernen wesentlicher Fähig- und Fertigkeiten der evidenzbasierten Medizin für Studierende der Medizin dienen kann.

Schlüsselwörter: Evidenz-basierte Medizin, Curriculum Entwicklung, Evaluation, Medizinische Statistik, problemorientiertes Lernen, Unterrichtsmethoden, Studierendenausbildung


Introduction

Teaching statistical analysis without interpretation of data is currently regarded as major faux pas in medical education [1]. The ability to make scientifically-founded decisions about therapeutic strategies on the basis of published studies has recently become a core competency of physicians, which medical students should have achieved by graduation from medical school. As a consequence, concepts of clinical epidemiology - not only methods - need to be taught to enable students and physicians, respectively, to perform their work on the basis of evidence-based medicine (EBM) [2], [3]. According to the basic concepts of EBM [4] three main topics of EBM need to be addressed [5] to plan an EBM course for students:

1.
individual clinical experience of the physician (= internal evidence),
2.
values and requests of the patient, and
3.
current status of scientifically-founded clinical medicine (= external evidence).

The first topic can be addressed by employing examples from clinical practice in EBM courses for students. Regarding the second topic, it is evident that students need to be educated in communication skills throughout their studies to be able to perceive the patient as an individual and to treat him as a partner in decision making. The third topic is addressed by the following study. The course "The Scientific Approach" was designed to enable students to gather scientific information in clinical medicine (=external evidence) and to be able to use and to critically appraise this information.


Methods

Goals of the study

The following learning objectives for the course were defined:

Upon successful completion of the course, students will be able to:

  • Find relevant references in literature for clinical questions.
  • Read, understand, and critically appraise clinical studies with special regard to biostatistical aspects.
  • Discuss methodological questions with statisticians.

At the end of the course students were asked to complete a questionnaire-based evaluation. In particular the evaluation set out to assess the following questions:

  • Were the above mentioned learning objectives reached?
  • Is the problem based course "The Scientific Approach" suited for learning basic biostatistics and EBM?
  • Is the ability of students to work scientifically higher after finishing the course than before starting the course?
  • Do the students enjoy their participation in the course "The Scientific Approach"?

Design of the course "The Scientific Approach"

The course "The Scientific Approach" was placed in the fifth semester in a parallel track PBL-curriculum without distinction between "pre-clinical" and "clinical" contents. The course was developed on the principles of problem based learning [6]. An idealized paper case on hormone replacement therapy represented the central starting point for learning. In addition to the above mentioned learning objectives regarding EBM, students were to learn to critically value hormone replacement therapy for postmenopausal women. Students were grouped into three PBL-tutorials and were asked to work on the PBL-case “Night sweat in Hamburg” (see figure 1 [Fig. 1]), a hormone replacement therapy case, with three different foci: the estimation of the individual patient's risk for developing breast cancer, cardiovascular diseases, and bone fractures. In general, the case was approached using the "seven steps" technique [6].

The course was divided into eight sections:

1.
Introduction to the medical library: Students participated in a guided tour through the medical library. A presentation followed by an introduction to the structure of the library (e.g. books and journals available on site, how to borrow books etc.). Different areas and facilities were shown and explained (e.g. reading areas, workstations for literature research, where to copy literature etc.).
2.
Introduction to literature search and data bases: Students were introduced to the most common literature databases (e.g. PubMed) and to literature search. They participated in literature search exercises and learned about online journals and interlending.
3.
Introduction to presentation techniques and media: Students learned about presentation techniques for scientific data and how to use PowerPoint for the preparation of a presentation.
4.
Introduction to the paper case "hormone replacement therapy": Different teachers from the Institute of Medical Biometrics and Epidemiology provided an introduction to the specialities of the PBL-case from the biostatistician view.
5.
Weekly PBL tutorials: Three PBL-tutorials supervised by tutors took place on the basis of the seven steps approach for PBL-tutorials for each group.
6.
Seminar about clinical studies: Methods and statistical design of clinical studies were taught in two seminars.
7.
Clerkship in a scientific laboratory: Optional clerkships in a scientific laboratory were offered for two days a week by different scientific laboratories.
8.
Exam: Students presented results of their literature searches several times during the course. In groups of seven, students presented the final results of their three weeks of work and discussed them with the students of the other groups and with biostatistics teachers. Marks for the groups of seven were awarded to the students for PowerPoint presentations and discussion of their work in these groups at the end of the course. At least two biostatistics teachers gave the marks (one=worst to four=best) and feedback to the groups based on strengths and weaknesses of both their results and presentation (see figure 2 [Fig. 2]).

Four different institutions of Hamburg Medical Faculty participated in teaching at the course. Students received a total number of 36.44 teaching hours (one teaching session = 45 minutes) (see table 1 [Tab. 1]).

Course evaluation was a questionnaire that included general questions about structure and organisation of the course and a second part with specific questions about the different subjects and institutes. Questionnaires were distributed immediately after completion of the course. Two different scales were used: 6-point-Likert scale: 1=agree very strongly, 2=agree strongly, 3=agree, 4=disagree, 5=disagree strongly, 6=disagree very strongly and 6-point-scale according to school grades: 1=very good, 2=good, 3=satisfactory, 4=fair, 5=inadequate, 6=insufficient). Students' performance was rated with an exam as described above.

Sample size and statistical analysis

All 56 students from two years of the parallel track PBL curriculum (cohort 1, year 2003, n=21; cohort 2, year 2004, n=35) participated in the course. 100% of the questionnaires were returned. 67% of the students were female, 33% male. The average age was 25 years. Results from both cohorts are presented together because no significant differences between both cohorts were found for any of the variables. Besides distribution of frequencies and arithmetic means rank correlations according to Spearman were calculated [7]. Arithmetic mean was chosen because it is the usual measure for calculating the average of "school marks" and in this specific case no common superior statistical alternatives were readily available.


Results

General evaluation

All students answered the questions in the different sections of the general evaluation in a positive manner. The main objective of the course was to provide students with the ability to work scientifically. Most students felt a lack of knowledge about scientific working procedures before the course but were quite satisfied with their achievements in this field after the course. This view resulted in an overall increase of one point for all items of the questionnaire. While "good" results were achieved in the areas "literature search", "functioning of the library" and "use of PowerPoint" despite an improvement by one point, only "satisfactory" results were seen in the areas "biostatistics" and "scientific work" (see Figure 3 [Fig. 3]).

Students were asked to rate the overall gain of their knowledge after the course ("I gained a lot of knowledge in this course"). Most students answered this question positively (see Figure 4 [Fig. 4]).

There was strong indication that an increase in knowledge through the course was independent of previous knowledge (rho=0.04).

Eighty-nine percent of the students had "fun" taking part in the course (n=49; "agree strongly", "quite agree", "agree"). Items "I had fun participating in this course" and "I gained a lot of knowledge in this course" showed a strong correlation (rho=0.55, p<0.001). The item "fun" also seemed to be independent of previous knowledge (rho=0.02). It should be noted that an item for overall satisfaction with the course was not included in the questionnaire for the first cohort but was added to the evaluation questionnaire for the second cohort ("I am satisfied with the complete course"). All students of the second cohort were satisfied with the complete course.

Almost all students agreed that the course had helped them to "find relevant literature" (see Figure 5 [Fig. 5]) and to "understand medical literature" (see Figure 5 [Fig. 5]). All students agreed that the course had enabled them to critically appraise medical literature (see Figure 5 [Fig. 5]). The item "The course has prepared me for my own scientific work" was answered mostly positive by 96% of the students (see Figure 4 [Fig. 4]).

The rate of satisfaction with the topic „hormone replacement therapy“ was 87%. Analysis of this topic in three groups with a different focus was classified as suitable by 93% of the students. Ninety-six percent of the students regarded the learning objectives as appropriate with respect to content and course time.

The number of teaching hours was described as appropriate by 82% of the students, 18% would have liked more hours of instruction. The concept of "The Scientific Approach" as block course was accepted by 95% of the students. The fifth semester was found suitable for this course by 73% of the students, while 20% would have liked to participate in this course in a later semester. High satisfaction (>89%) was noted regarding the structural organization of the course, course manual and schedule (see Figure 6 [Fig. 6]).

Exams were regarded as appropriate by 98% of the students. One student rated the exam as too easy.

Specific evaluation and exam results

The five institutes participating in the course "The Scientific Approach" were all rated "good" or "satisfactory" in the specific evaluation. For the most part, the number of teaching sessions was found appropriate. The evaluation of the "biostatistics" section will serve as an example for the items in the section specific evaluation. “Biostatistics” received an overall average mark of 2.5 (SD=0.8) (see Figure 7 [Fig. 7]). Seventy-five percent of the students were satisfied with the number of hours taught, while 18% would have liked more and 6% fewer teaching sessions.

Students regarded their knowledge before the course as "sufficient" (M=4.1, SD=1.0) and their knowledge after the course as "satisfactory" (M=2.9, SD=0.8). Ninety-five percent of the students agreed that the part "biostatistics" helped them to become prepared for their own scientific work. For their "competency as a physician" the students marked the biostatistics classes as "good" (M=2.4, SD=0.9). Students of cohort 1 (year 2003) received four grade points (the maximum) in their exams while students of cohort 2 (year 2004) reached a mean of 3.25 of four possible grade points.


Discussion

The concept of the block course "The Scientific Approach" was based on the theoretical requirements of EBM. National and international experiences from the literature about EBM and biostatistics courses were used in the planning [8], [2]. The development of the course was carried out according to the theory of "A Six-Step approach to curriculum development" by Kern et al. [9]. The whole course concept was based on the teaching and learning approach of "problem based learning" which proved to be successful in clinical medicine. Advantages of PBL for education in medical statistics are also described [10].

The main feature of PBL is the paper case [6]. The case used in our course introduced the medical question of hormone replacement therapy. This topic is very relevant for medical practice and was adapted to the interest and knowledge of third year medical students. The students’ high level of satisfaction with the content of the paper case (87%) demonstrates that this topic met the students' needs. Working in three groups with different clinical questions to consider for the case was approved by the students (93%), and 94% regarded the block course as (very) suitable as an introduction to scientific work.

The course "The Scientific Approach" was placed in the fifth semester. Due to the concept of the PBL-curriculum students were challenged with clinical questions starting in the first semester. Hence students feel comfortable with this concept before they continue their studies with practical clinical courses in the traditional curriculum in year four and five. Seventy-three percent of the students felt that it was right to take this course in the fifth semester, while 20% would have liked to take it at a later point in their studies. The course seems to be appropriately placed in the fifth semester from a clinical point of view even though some students probably would have liked to take such a course while working on their medical thesis. According to Evans [10] the position of a course in biostatistics in a medical curriculum can not be determined independent of its contents. Calculations and graphs should be taught in the first years while the place for clinical questions and further aspects of statistics is in the later years of the curriculum. Baker [11] demands that biostatistics should be taught in the clinical phase of medical studies. There is consensus amongst all references that medical statistics should be taught at medical school. Another argument in this favour is the observation that students´ increase in knowledge from statistics classes is greater than physicians´ [12]. If basic statistical knowledge is gained at medical school the use of statistics in medical practice is easier for most physicians [13].

The course "The Scientific Approach" had 36 teaching sessions in three weeks, including a guided tour through the medical library. Eighty-two percent of the students felt the number of hours was appropriate for the course, and 18% would have liked a greater number of teaching sessions. This desire for more teaching sessions mainly referred to classes in biomathematics; an astonishing result because Altman and Bland [13] found that biomathematics was not amongst students' favourite courses. Depending on the previous statistical knowledge of the students courses in biostatistics can be designed in a very flexible way. When compared to North American medical schools, the 36 teaching sessions in the course "The Scientific Approach" rank above average in terms of course time. The content of biostatistics and evidence based medicine is taught in 122 of 126 accredited North American medical schools in mandatory courses, while only 107 of these medical schools provide separate medical statistics teaching sessions [14]. A mean of 13 hours (SD=8.6) of biostatistics and 20 hours (SD=20.2) of evidence-based medicine is provided. It must be taken into account that no data is provided about the design of statistics teaching sessions in the USA. Furthermore, no data on teaching formats nor on the institutes that participate in the teaching is provided [14].

Students felt after participation in the course "The Scientific Approach" that they had reached the defined learning objectives. These positive evaluation results are not consistent with the usual evaluation results of biostatistics courses. Peters [15] summarizes this with “the risk of a little cynical exaggeration, what characterizes the teaching of statistics to undergraduate medical students is that they all get it, while few of them want it”. This statement cannot be confirmed by the student evaluation of the course in this study. In particular, students in our course found the participation of a clinician extremely helpful in explaining clinical contexts. The use of external expertise is one of the recommendations for statistics courses given by Evans [10]. An amazing result of our study is that the majority of students (>89%) stated that they had "fun" participating in this course. Ziv [16] demonstrated that teaching in a good atmosphere and with humour resulted in better exam results. The atmosphere in our biostatistics class was mostly rated as good. Students were also requested to actively participate. The presentation of teaching contents and motivation of the teachers may have added to this positive result as demonstrated by Simpson [17].

Besides course evaluation, exam results are another way of controlling the quality of a course. The broad learning objectives of our course required a special kind of exam since exams should be designed to fit the teaching and learning objectives of a course [18]. For our course a special exam was developed where students had to talk about the results of their work in a computer based presentation. Ninety percent of the students found this kind of exam appropriate. Most other educational programs where statistics are taught rely on multiple choice questions to measure knowledge in clinical epidemiology and "critical appraisal knowledge" [19]. Linzer et al. [20] could demonstrate that a free interview was valid to test for the above mentioned criteria. Newcombe [21] showed that multiple choice questions were not useful if a distinction was made between quality of process and quality of results since attitudes changed considerably while quality of academic performance remained similar. Since the exam was especially developed for the course "The Scientific Approach" no comparison can be made with exam results of other national or international courses in biostatistics. Also, the course features, number of teaching sessions, learning objectives, and kind of exams differed significantly and were often ill defined or not at all. Although a comparison of our exam with exams from other studies was not possible, the exam was an important part of our course because they motivate students to learn [10]. Furthermore, adults connect their readiness to learn with the requirements of real life and orientate their attitude towards learning on real life and job requirements [22]. These principles are taken into account using problem based learning in medical education [23], [24] and were the basic concept of our course. Since students were able to identify the topic of hormone replacement in the paper case as relevant to clinical practice they seemed to be motivated to tackle the learning objectives of this course. This relevant and basic medical topic served its goal in orienting the teaching towards the primary target group of students who want to do clinical work after graduation from medial school [25].

It has to be taken into account when discussing the results of this study that the evaluation consisted of a self rating questionnaire and that the marks given by the biostatisticians to the students for their final computer-based presentations were given to groups of seven, rather than to individuals. Quantitative evaluation by questionnaire contains the basic risk that participants can consciously manipulate the results. However, since evaluation of the course "The Scientific Approach" was anonymous, one can assume that the students did not have a personal interest in consciously manipulating the results. Exams were done in groups of seven students meaning that validity and reliability of the exams could not be tested. Since all students of cohort one attainted four out of four possible grade points in the exam, one can assume that teachers highly valued the individual increase in knowledge as well as the use of this knowledge. It is likely that not all students deserved four grade points in the exam if one had measured mere outcome criteria and learning progress had been the outcome variable. Furthermore, it is possible that weaker and less interested students profited from their stronger colleagues. This problem has to be noted in the discussion about marks for group work.

Students of cohort two scored a mean of 3.25 of four possible grade points. This course was taught by a different teacher demonstrating that the course concept works independently of the person who is in charge. Achievements of both cohorts were valued as good by teachers and course directors. The high number of learning objectives and their great differentiation did not allow for individual tests of each learning objective. Furthermore, it was not a blinded study regarding the investigators which is usually requested of such studies [26]. A randomization of the cohorts into two groups, each with differently designed courses, was not possible by decision of the ethics committee. This made the interpretation of evaluation data more difficult. However, randomized studies in the field of medical education are currently refused for ethical reasons which raise methodological problems on an international basis and is also partly criticised [27].


Conclusion

Professional competence, high quality of care, and patient centered treatment are major learning objectives in medical education. To reach these goals EBM can serve as a method for lifelong learning and as a helpful tool to find the right treatment decision together with well informed patients [28]. The good evaluation data and enthusiastic student participation in this course together with the satisfactory exam results form a good basis for the medical students' transition into work life after graduation. Students seem to be well prepared for the challenge to critically appraise medical literature.


Acknowledgements

The authors thank Nicole Griepentrog for her help with administration and organization of the course and its evaluation.

The authors thank the "Bundesministerium für Bildung und Forschung", the "Behörde für Wissenschaft und Gesundheit" and the University of Hamburg for funding the "Modellstudiengang Medizin" including the course "The Scientific Approach".


References

1.
Sackett DL, Strauss SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based Medicine. How to Practise and Teach EBM 2nd edn. Edinburgh: Churchchill-Livingstone; 2000.
2.
Morris RW. Does EBM offer the best opportunity yet for teaching medical statistics? Stat Med. 2002;21(7):969-977; discussion 979-981, 983-984.
3.
Armstrong GK, Walters SJ. Discussion: Does EBM offer the best opportunity yet for teaching medical statistics. Stat Med. 2002;21(7):979-981.
4.
Smeeton N. Statistical education in medicine and dentistry. Statistician. 1997;46:521-527.
5.
Türp JC, Antes G. Was versteht man unter "Evidenzbasierter Medizin". Dtsch Zahnarztl Z. 2001;56:74.
6.
Wood DF. Problem based learning. BMJ. 2003;326(7384):328-330.
7.
Bortz J. Statistik für Sozialwissenschaftler. Heidelberg: Springer-Verlag; 1999.
8.
Dixon RA. Medical statistics: content and objectives of a core course for medical students. Med Educ. 1994;28(1):59-67.
9.
Kern DE, Thomas PA, Howard DM, Bass EB. Curriculum Development For Medical Education, A Six-Step Approach. Baltimore, London: Johns Hopkins University Press; 1998.
10.
Evans SJ. Statistics for medical students in the 1990s: How should we approach the future? Stat Med. 1990;9(9):1069-1075.
11.
Baker MR. A student's view on medical statistics. Teaching of Statistics. Alexandria VA: American Statistical Association; 1989.
12.
Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. CMAJ. 1998;158(2):177-181.
13.
Altman DG, Bland JM. Improving doctors understanding of statistics. J R Stat Soc Ser A Stat Soc. 1991;154:223-267.
14.
Barzansky B, Etzel SI. Educational programs in US medical schools 2002-2003. JAMA. 2003;290(9):1025-1031.
15.
Peters TJ. Comment on Appleton "What statistics should we teach medical undergraduates and graduates". Stat Med. 1990;9:1023-1027.
16.
Ziv A. Teaching and learning with humour: Experiment and replication. J Exp Educ. 1988;57:5-15.
17.
Simpson JM. Teaching statistics to non-specialists. Stat Med. 1995;14(2):199-208.
18.
Astin J, Jenkins T, Moore L. Medical students perspective on the teaching of medical statistics in the undergraduate medical curriculum. Stat Med. 2002;21(7):1003-1006.
19.
Green ML. Evidence-based medicine training in graduate medical education: past, present and future. J Eval Clin Prac. 2000;6(2):121-138.
20.
Linzer M, Brown JT, Frazier LM, De ong ER, Siegel WC. Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills: a randomizes controlled trail. JAMA. 1988;260(17):2537-2541.
21.
Newcombe RG. Evaluation of statistics given to medical undergraduates. Stat Med. 1990;9(9):1045-1055.
22.
Knowles M. The adult learner. A neglected species. Houston: Gulf Publishing Company; 1984.
23.
Neame RL, Powis DA. Toward independent learning: Curricular design for assisting students to learn how to learn. J Med Educ. 1981;56(11):886-893.
24.
Barrows HS. Problem-based, self-directed learning. JAMA. 1983;250(22):3077-3080.
25.
Appleton D.R. What statistics should we teach medical undergraduates and graduates? Stat Med. 1990;9(9):1013-1021.
26.
Green ML. Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-medicine: a critical review of curricula. Acad Med. 1999;74(6):686-694.
27.
Sackett DL, Parkes J. Teaching critical appraisal: no quick fixes (editorial). CMAJ. 1998;158(2):203-204.
28.
Kunz R, Ollenschläger G, Raspe HH, Jonitz G, Kolkmann FW, Cox M. Lehrbuch Evidenzbasierte Medizin in Klinik und Praxis. Köln: Deutscher Ärzteverlag; 2007.