Simulation vs workplace-based assessment in resuscitation: a cross-specialty descriptive analysis and comparison




Background: Simulation-based assessment can complement workplace-based assessment of rare or difficult to assess Entrustable Professional Activities (EPAs). We aimed to compare the use of simulation-based assessment for resuscitation-focused EPAs in three postgraduate medical training programs and describe faculty perceptions of simulation-based assessment.

Methods: EPA assessment scores and setting (simulation or workplace) were extracted from 2017-2020 for internal medicine, emergency medicine, and surgical foundations residents at the transition to discipline and foundations of discipline stages. A questionnaire was distributed to clinical competency committee members.

Results: Eleven percent of EPA assessments were simulation-based. The proportion of simulation-based assessment did not differ between programs but differed between transition (38%) and foundations (4%) stages within surgical foundations only. Entrustment scores differed between settings in emergency medicine at the transition level only (simulation: 4.82 ± 0.60 workplace: 3.74 ± 0.93). 70% of committee members (n=20) completed the questionnaire. Of those that use simulation-based assessment, 45% interpret them differently than workplace-based assessments. 73% and 100% trust simulation for high-stakes and low-stakes assessment, respectively.

Conclusions: The proportion of simulation-based assessment for resuscitation focused EPAs did not differ between three postgraduate medical training programs. Interpretation of simulation-based assessment data between committee members was inconsistent. All respondents trust simulation-based assessment for low-stakes, and the majority for high-stakes assessment. These findings have practical implications for the integration simulation into programs of assessment.


Metrics Loading ...

Author Biographies

Stephen Gauthier, Queen's University

Assistant Professor, Department of Medicine

Boris Zevin, Queen's University

Associate Professor and SEAMO Medical Education Scholar, Department of Surgery

Andrew K Hall, University of Ottawa

Associate Professor, Department of Emergency Medicine

Timothy Chaplin, Queen's University

Assistant Professor and Simulation Lead, Department of Emergency Medicine


Busing N, Harris K, MacLellan AM, et al. The future of postgraduate medical education in Canada. Acad Med J Assoc Am Med Coll. 2015;90(9):1258-1263. DOI:

Frank JR, Snell LS, Sherbino J. CanMEDS 2015 Physician Competency Framework. Royal College of Physicians and Surgeons of Canada; 2015.

Harris P, Bhanji F, Topps M, et al. Evolving concepts of assessment in a competency-based world. Med Teach. 2017;39(6):603-608. DOI:

Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med Teach. 2018;40(11):1102-1109. DOI:

Thoma B, Hall AK, Clark K, et al. Evaluation of a national competency-based assessment system in emergency medicine: a CanDREAM study. J Grad Med Educ. 2020;12(4):425-434. DOI:

Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR, Collaborators for the IC. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676-682. DOI:

Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Technology-enhanced simulation to assess health professionals: a systematic review of validity evidence, research methods, and reporting quality. Acad Med. 2013;88(6):872-883. DOI:

Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49(6):560-575. DOI:

McMurray L, Hall AK, Rich J, Merchant S, Chaplin T. The nightmares course: a longitudinal, multidisciplinary, simulation-based curriculum to train and assess resident competence in resuscitation. J Grad Med Educ. 2017;9(4):503-508. DOI:

Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): a tool to assess surgical competence. Acad Med J Assoc Am Med Coll. 2012;87(10):1401-1407. DOI:

Peters H, Holzhausen Y, Boscardin C, Cate O ten, Chen HC. Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Med Teach. 2017;39(8):802-807. DOI:

Russell E, Hall AK, Hagel C, Petrosoniak A, Dagnone JD, Howes D. Simulation in Canadian postgraduate emergency medicine training – a national survey. Can J Emerg Med. 2018;20(1):132-141. DOI:

Prudhomme N, O’Brien M, McConnell MM, Dudek N, Cheung WJ. Relationship between ratings of performance in the simulated and workplace environments among emergency medicine residents. Can J Emerg Med. Published online 2020:1-8. DOI:

Weersink K, Hall AK, Rich J, Szulewski A, Dagnone JD. Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring. Adv Simul. 2019;4. DOI:

Hall AK, Damon Dagnone J, Moore S, et al. Comparison of simulation‐based resuscitation performance assessments with in‐training evaluation reports in emergency medicine residents: a canadian multicenter study. AEM Educ Train. 2017;1(4):293-300. DOI:

Calhoun AW, Bhanji F, Sherbino J, Hatala R. Simulation for high-stakes assessment in pediatric emergency medicine. Clin Pediatr Emerg Med. 2016;17(3):212-223. DOI:




How to Cite

Seed JD, Gauthier S, Zevin B, Hall AK, Chaplin T. Simulation vs workplace-based assessment in resuscitation: a cross-specialty descriptive analysis and comparison. Can. Med. Ed. J [Internet]. 2023 Feb. 20 [cited 2023 Sep. 24];14(3):92-8. Available from:



Brief Reports

Most read articles by the same author(s)