gms | German Medical Science

GMS Journal for Medical Education

Gesellschaft für Medizinische Ausbildung (GMA)

ISSN 2366-5017

Pilot evaluation of an objective structured assessment of technical skills tool for chest tube insertion

article Assessment

  • author Mirco Friedrich - University of Heidelberg, Department of General, Visceral, and Transplantation Surgery, Heidelberg, Germany
  • author Julian Ober - University of Heidelberg, HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg, Germany
  • author Patrick Haubruck - University of Heidelberg, HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg, Germany
  • author Christian Bergdolt - University of Heidelberg, HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg, Germany
  • author Thomas Bruckner - University of Heidelberg, Institute for Medical Biometry and Informatics, Heidelberg, Germany
  • author Karl-Friedrich Kowalewski - University of Heidelberg, Department of General, Visceral, and Transplantation Surgery, Heidelberg, Germany
  • author Martina Kadmon - University of Augsburg, Medical Faculty, Augsburg, Germany
  • author Beat-Peter Müller-Stich - University of Heidelberg, Department of General, Visceral, and Transplantation Surgery, Heidelberg, Germany
  • author Michael Christopher Tanner - University of Heidelberg, HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg, Germany
  • author Felix Nickel - University of Heidelberg, Department of General, Visceral, and Transplantation Surgery, Heidelberg, Germany

GMS J Med Educ 2018;35(4):Doc48

doi: 10.3205/zma001194, urn:nbn:de:0183-zma0011944

This is the English version of the article.
The German version can be found at:

Received: October 24, 2016
Revised: July 25, 2018
Accepted: August 13, 2018
Published: November 15, 2018

© 2018 Friedrich et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at


Background: Chest tube insertion is a standard intervention for management of various injuries of the thorax. Efficient clinical training of this and similar bed-side procedures is equally demanded and improvable. Here, we propose a nouveau means of assessment and feedback using an Objective Structured Assessment of Technical Skills (OSATS) tool. The modified OSATS for chest drain insertion is evaluated in a pilot trial focusing on chest drain insertion.

Methods: Participants in the pilot trial were medical students (3rd-6th year of studies, n=9), junior residents (1st-3rd post-graduate year, n=12), senior residents (4th-6th post-graduate year, n=14), and attending surgeons (n=6) from Heidelberg University. Chest drain insertions on a cadaveric porcine model were rated by experts with the modified OSATS score. Participants’ performances were videotaped and subsequently rated by two remote experts (video rating). Primary aim was to assess criterion validity of the OSATS to distinguish experience levels.

Results: Kruskal-Wallis test showed significant differences between means of scores between four groups stratified by previous experience in chest tube insertion (level 0: 22.1±3.2 vs. level 1: 26.8±2.8 vs. level 2: 35.4±2.2 vs. level 3: 41.0±2.0; p=0.002; p1,3=0.049, p0,3=0.005). However, if groups were stratified by formal professional level, no statistically significant distinction could be made using OSATS. Hence, the OSATS tool showed criterion validity for differentiation between experience levels.

Conclusion: In the pilot study, the modified OSATS for chest tube insertion was apt to standardize expert rating and could be used to measure skill and to depict different experience levels. The OSATS will help facilitate training and assessment of chest drain insertion and could therefore improve surgical training for trauma situations. According to our data, the OSATS might be integrated into modern curricula.

Keywords: chest tube insertion, education, training, assesssment, hematothorax, pneumothorax


In many countries worldwide, consequences of demographic change affect the health care systems. A rising demand for medical assistance is opposing an equally rising physician shortage and distribution problem respectively [1], [2]. With facing such circumstances, optimization of processes within the health care system already has to take place in medical education and -training. Regarding that, Halsted’s concept of ‘see one, do one teach one’ is no longer feasible nowadays [3]. Modern medical training curricula need to be most efficient and effective as well, as costs and time consumption are factors commonplace to today’s clinical setting. Regarding those challenging circumstances there is an increasing need for frameworks and clear defined and standardized learning goals in medical education [4], [5], [6], [7]. This will help to develop competency based learning and teaching models [7] which could further build a framework of specific medical training curricula [7]. Consequently, many centers and universities worldwide provide specific procedural courses and research is conducted to optimize training [8], [9]. Aghdasi et al. adduced procedural training and accurate assessment of the technical skills of surgeons as an instance of such challenges. Generally carried out by experts in their field, i.e. senior surgeons, postgraduate training is currently time-consuming and costly but not replaceable, provided that validated training methods and assessment tools such as the Objective Structured Assessment of Technical Skills (OSATS) or the Global Operative Assessment of Laparoscopic Skills (GOALS) are deployed [10], [11], [12].

With increasing demand for partially invasive bedside procedures, adequate training capacities are needed to ensure each trainee’s skills are adequate prior to patient contact. This is supported by a needs assessment of Weitz et al., where physicians emphasized direct bed-side-skills as pivotal elements of medical education [13].

Training needs to be most efficient and effective as well, as costs and time consumption are factors commonplace to today’s clinical setting

Chest tube insertion is an established minimally-invasive surgical intervention in treatment of acute trauma patients bearing injuries of lungs and thorax. Correct chest tube insertion facilitates efficient therapy without time loss or further complications that are potentially lethal due to the intervention’s vicinity to vital organs [14]. Thus, there is a need for a comprehensive assessment tool for training prior to the emergency situation that enables both versed physicians and novices to acquire the specific feedback necessary for proficiency in chest tube insertion. Early and decisive training using a tool as such might standardize handling of these emergency situations and could translate to improved trauma care and outcome. Assessment and feedback has been shown to be a valuable asset to surgical training [15]. In order to advance and standardize training success without straining staff and resources, we propose a nouveau Objective Structured Assessment of Technical Skill (OSATS) (see Table 1 [Tab. 1]) for chest tube insertion to standardize expert rating [16]. The aim of this manuscript was to evaluate the proposed OSATS score regarding its criterion validity by means of a pilot study. Therefore, the endpoint of the current pilot trial was the difference between means of scores obtained by groups differing in procedural expertise for blinded video rating.

Material and Methods

Evaluation of an Objective Structured Assessment of Technical Skills for chest tube insertion

In this pilot trial the aim was to evaluate a modified OSATS for chest tube insertion (see Table 1 [Tab. 1]). In depicting different levels of skill and allowing for individual feedback based on subscore assessment, this OSATS is meant to provide a valid tool for medical training and teaching. The modified OSATS tool for chest drain insertion was developed based on key steps of correct chest tube insertion, originally published by Hutton et al., which were modified and amended by a team of trauma and general surgeons [17]. Ten representative procedural steps can be evaluated separately using a 5-Point Likert scale complemented by detailed formulation of performance levels. Trainee’s performance in substeps adds up to a maximum total score of 50 points and a minimum score of 10 points, respectively [18].

The pilot trial was carried out in the surgical training center at the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital. Participants included in this study were medical students during their clinical years (3rd-6th year) at Heidelberg University (n=9) as well as junior residents (1st to 3rd year, n=12), senior residents (4th to 6th year, n=14) and attending surgeons (n=6) from Heidelberg University Hospital. The pilot trial for evaluation of the modified OSATS score was conducted in January 2017. Chest tube insertions were performed on a cadaveric porcine model, which was prepared in a standardized way on an operating table in the supine position. At the beginning of the study, a brief introduction to training and assessment was held for each participant. Training started after the participant’s consent was obtained and pseudonymization had taken place. Each participant was asked to perform an initial training session using a mobile training device and the validated surgical training app Touch Surgery™ (TS) (Kinosis Ltd., London, UK) [15], [19]. The TS-module “Chest Tube Insertion” served as both an initial training to this intervention and a compact recapitulation for versed participants, respectively. Subsequent to app-based training, each participant was asked to perform a chest tube insertion on the porcine model using the provided instruments. Assessment was performed on the OSATS sheet by two independent and blinded raters via videos taped during the intervention. All videos were taped in a standardized way. To ensure blinding of raters the recorded image showed only the porcine thorax as well as the hands of the participants. Raters were specifically trained surgeons of the Department of General, Visceral, and Transplantation Surgery as well as the Center for Orthopedics, Trauma Surgery and Spinal Cord Injury of Heidelberg University Hospital. The final score of each trainee was calculated as an average score of the two independent ratings.

Ethics, consent and permissions

All data for the pilot study were recorded anonymously, treated confidentially, and were evaluated by authorized staff for scientific purposes only. Participants’ names were kept separate from all study data and were not used for the study. Each participant was assigned a designated code that was used for the entire study documentation and data collection. Participation in the study was voluntary. There were no foreseeable negative consequences for participants related to participation. The participating staff of the Heidelberg surgical center is experienced in the handling of animal models and training devices. Participants would have been excluded from the study in the event that a participant’s physical or mental health had become jeopardized due to participation in the present study. Ethical approval was obtained by the local ethics committee at Heidelberg University (A 13/14). Written informed consent was obtained from each participant.

Statistical analysis

Data collection was carried out by MS Excel® 2016 (Microsoft®). Statistical analysis was carried out by SPSS Statistics Version 24.0 (IBM® Germany). For statistical analysis, mean and standard deviation in case of continuous data, with absolute and relative frequencies for categorical parameters were used to describe the distributions of all parameters of interest. For OSATS scores, no Gaussian distribution was assumed, thus Kruskal-Wallis test was used as nonparametric test to compare all 4 groups. Moreover, Intraclass Correlation Coefficient was analyzed between the two video raters. For all tests, a p-value less than 0,05 was considered statistically significant. Where found to be appropriate, graphical statistical methods were deployed to illustrate findings.


Results of the pilot study for evaluation of the objective structured assessment of technical skills for chest tube insertion

A total of 41 participants were included in the pilot study (students=9; junior residents=12; senior residents=14; attending surgeons=6). Groups were stratified according to formal professional level (0=student, 1=junior resident, 2=senior resident, 3=attending surgeon) or self-estimated level of experience (0=none, 1=limited, 2=moderate, 3=advanced) There were significant differences between means of scores between four groups stratified by previous experience in chest tube insertion (level 0: 22.1±3.2 vs. level 1: 26.8±2.8 vs. level 2: 35.4±2.2 vs. level 3: 41.0±2.0) (see Table 2 [Tab. 2] and Figure 1 [Fig. 1]). Moreover, Kruskal-Wallis test revealed significant differences of group medians (p=0.002, Kruskal-Wallis statistic 14.6; Results of Dunn-Bonferroni-Test: p0,3=0.005, p1,3=0.049) assuming non-Gaussian distribution. However, if groups were stratified by formal professional level, no statistically significant distinction could be made using OSATS (level 0: 26.2±3.9 vs. level 1: 26.8±3.4 vs. level 2: 30.6±3.0 vs. level 3: 37.4±2.6) (see Table 2 [Tab. 2] and Figure 1 [Fig. 1]). Here Kruskal-Wallis-test showed a non-significant p-value of 0.196). Furthermore, analysis of the Intraclass Correlation Coefficient between the two independent expert raters showed an excellent agreement between the two ratings (ICC=0.96, 95% CI 0.91-0.98) [20].


In the current pilot study, we evaluated the modified OSATS for chest tube insertion regarding it’s criterion validity for its use in blinded video rating. Video rating, instead of on-site direct rating, was used due to its ability to eliminate confounding factors such as age or formal professional level of the participants. Our results suggest that the OSATS was apt to standardize expert rating and could be used to depict and measure differences in skill between experience levels of surgeons, residents, and medical students. According to our results the OSATS tool is not biased by confounding factors like age or formal professional level, but reflects procedural experience only.

Frameworks are essential for modern medical students education [4], [5], [6], [7]. As claimed by David et al., modern curricula need to be standardized and internationally comparable [7]. This need does not only exist for chest tube insertion training. In other medical disciplines, e.g. laparoscopic surgery, there is also a demand for a multimodal training frameworks, which also needs feedback via standardized and objective scores [21]. The comparability of learning success can be achieved through scores like the here proposed OSATS. Based on our results, the OSATS for chest tube insertion might be an instrument which can fulfill David et al.’s claims [7]. Moreover, concerning the needs of modern medical education, we believe this approach of assessment and feedback to prove additionally beneficial to outcome when teaching today’s medical trainees [16]. Moreover, we suggest that learning and teaching with the presented OSATS could be an answer to the needs of modern medical education as it offers clear defined learning goals combined with structured and objective assessment and feedback [7]. Furthermore, it offers a readily available and cost-efficient assessment tool for training of chest tube insertion. Our idea is that participants will profit from both the technical instructions from the OSATS as well as from feedback because their improvement in skill level can be subsequently objectified by the modified OSATS tool. Therefore, the OSATS might help to maximize patients’ safety. Also, this study will help to investigate potential improvements of current training curricula by the use of feedback and may even provide a reliable assessment for procedural proficiency certification for medical trainees with the modified OSATS aiming at a safe execution of this crucial emergency procedure. The OSATS can also be used in the context of certification processes as well as entrustable professional activities [7]. We consequently aim to use this scoring tool in courses for general surgery, trauma surgery and intensive care offered to surgical residents, trauma surgeons, anaesthesiologists, and medical students. After further investigation, the present OSATS might be integrated into modern curricula as part of competency based teaching programs.

Nevertheless, some limitations of this pilot study should be noted. On the one hand this pilot study was conducted as single-center study at Heidelberg University Hospital. For this reason, the results of the current study are based on analyses with relatively small subgroups. On the other hand, the allocation of the participants to the four different subgroups, regarding their previous experience in using chest tubes, was based on the subjective self-evaluation of the participants. It is possible that there were inaccuracies between the subgroups due to under- or overestimation of the participants’ self-assessment. Further investigations and outcomes of the proposed study will increase the available knowledge about criteria to be met in order to ensure optimal surgical training – not only for trauma situations.

List of abbreviations

  • OSATS: Objective Structured Assessment of Technical Skills
  • TS: Touch Surgery
  • ICC: Intraclass Correlation Coefficient


  • Authors contributed equally: Mirco Friedrich and Julian Ober
  • Study conception and design: Nickel, Bergdolt, Ober, Friedrich, Haubruck, Müller-Stich, Tanner
  • Acquisition of data: Nickel, Ober, Friedrich, Kowalewski, Bergdolt
  • Statistical analysis: Ober, Bruckner
  • Analysis and interpretation of data: Ober, Friedrich, Nickel, Bergdolt, Tanner, Haubruck, Müller-Stich
  • Drafting of manuscript: Friedrich, Ober, Nickel, Kowalewski, Bergdolt, Bruckner
  • Critical revision: Müller-Stich, Tanner, Haubruck


The study was supported by the Heidelberg Surgery Foundation and the European Social Fund of the State Baden Wuerttemberg.

Competing interests

The authors declare that they have no competing interests.


Adler G, v. d. Knesebeck JH. [Shortage and need of physicians in Germany? Questions addressed to health services research]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2011;54(2):228-237. DOI: 10.1007/s00103-010-1208-7 External link
Kasch R, Engelhardt M, Forch M, Merk H, Walcher F, Frohlich S. [Physician Shortage: How to Prevent Generation Y From Staying Away - Results of a Nationwide Survey]. Zentralbl Chir. 2016;141(2):190-196.
Carter BN. The fruition of Halsted's concept of surgical training. Surgery. 1952;32(3):518-527.
Lomis K, Amiel JM, Ryan MS, Esposito K, Green M, Stagnaro-Green A, Bull J, Mejicano GC; AAMC Core EPAs for Entering Residency Pilot Team. Implementing an Entrustable Professional Activities Framework in Undergraduate Medical Education: Early Lessons From the AAMC Core Entrustable Professional Activities for Entering Residency Pilot. Acad Med. 2017;92(6):765-770. DOI: 10.1097/ACM.0000000000001543 External link
Brody H, Doukas D. Professionalism: a framework to guide medical education. Med Educ. 2014;48(10):980-987. DOI: 10.1111/medu.12520 External link
Irby DM, Hamstra SJ. Parting the Clouds: Three Professionalism Frameworks in Medical Education. Acad Med. 2016;91(12):1606-1611. DOI: 10.1097/ACM.0000000000001190 External link
David DM, Euteneier A, Fischer MR, Hahn EG, Johannink J, Kulike K, Lauch R, Lindhorst E, Noll-Hussong M, Pinilla S, Weih M, Wennekes V. The future of graduate medical education in Germany - position paper of the Committee on Graduate Medical Education of the Society for Medical Education (GMA). GMS Z Med Ausbild. 2013;30(2):Doc26. DOI: 10.3205/zma000869 External link
Napolitano LM, Biester TW, Jurkovich GJ, Buyske J, Malangoni MA, Lewis FR Jr; Members of the Trauma, Burns and Critical Care Board of the American Board of Surgery. General surgery resident rotations in surgical critical care, trauma, and burns: what is optimal for residency training? Am J Surg. 2016;212(4):629-637. DOI: 10.1016/j.amjsurg.2016.07.016 External link
Roach PB, Roggin KK, Selkov G Jr, Posner MC, Silverstein JC. Continuous, data-rich appraisal of surgical trainees' operative abilities: a novel approach for measuring performance and providing feedback. J Surg Educ. 2009;66(5):255-263. DOI: 10.1016/j.jsurg.2009.10.001 External link
van Hove PD, Tuijthof GJ, Verdaasdonk EG, Stassen LP, Dankelman J. Objective assessment of technical surgical skills. Br J Surg. 2010;97(7):972-987. DOI: 10.1002/bjs.7115 External link
Datta V, Bann S, Mandalia M, Darzi A. The surgical efficiency score: a feasible, reliable, and valid method of skills assessment. Am J Surg. 2006;192(3):372-378. DOI: 10.1016/j.amjsurg.2006.06.001 External link
Aghdasi N, Bly R, White LW, Hannaford B, Moe K, Lendvay TS. Crowd-sourced assessment of surgical skills in cricothyrotomy procedure. J Surg Res. 2015;196(2):302-306. DOI: 10.1016/j.jss.2015.03.018 External link
Weitz G, Twesten C, Hoppmann J, Lau M, Bonnemeier H, Lehnert H. Differences between students and physicians in their entitlement towards procedural skills education--a needs assessment of skills training in internal medicine. GMS Z Med Ausbild. 2012;29(1):Doc07. DOI: 10.3205/zma000777 External link
Menger R, Telford G, Kim P, Bergey MR, Foreman J, Sarani B, Pascual J, Reilly P, Schwab CW, Sims CA. Complications following thoracic trauma managed with tube thoracostomy. Injury. 2012;43(1):46-50. DOI: 10.1016/j.injury.2011.06.420 External link
Pape-Koehler C, Immenroth M, Sauerland S, Lefering R, Lindlohr C, Toaspern J, Heiss M. Multimedia-based training on Internet platforms improves surgical performance: a randomized controlled trial. Surg Endosc. 2013;27(5):1737-1747. DOI: 10.1007/s00464-012-2672-y External link
Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med. 1996;71(12):1363-1365. DOI: 10.1097/00001888-199612000-00023 External link
Hutton IA, Kenealy H, Wong C. Using simulation models to teach junior doctors how to insert chest tubes: a brief and effective teaching module. Intern Med J. 2008;38(12):887-891. DOI: 10.1111/j.1445-5994.2007.01586.x External link
Friedrich M, Bergdolt C, Haubruck P, Bruckner T, Kowalewski KF, Muller-Stich BP, Tanner MC, Nickel F. App-based serious gaming for training of chest tube insertion: study protocol for a randomized controlled trial. Trials. 2017;18(1):56. DOI: 10.1186/s13063-017-1799-5 External link
Kowalewski KF, Hendrie JD, Schmidt MW, Proctor T, Paul S, Garrow CR, Kenngott HG, Müller-Stich BP, Nickel F. Validation of the mobile serious game application Touch Surgery for cognitive training and assessment of laparoscopic cholecystectomy. Surg Endosc. 2017;31(10):4058-4066. DOI: 10.1007/s00464-017-5452-x External link
Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155-163. DOI: 10.1016/j.jcm.2016.02.012 External link
Nickel F, Kowalewski KF, Muller-Stich BP. [Risk awareness and training for prevention of complications in minimally invasive surgery]. Chirurg. 2015;86(12):1121-1127. DOI: 10.1007/s00104-015-0097-6 External link