Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

J. Damon Dagnone, Andrew K. Hall, Stefanie Sebok-Syer, Don Klinger, Karen Woolfrey, Colleen Davison, John Ross, Gordon McNeil, Sean Moore


Background: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.

Method: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE.  Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen’s Simulation Assessment Tool (QSAT).  Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario.  A fully crossed generalizability study was also conducted for each examination centre.  

Results: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.

Conclusions: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees.


Simulation; OSCE; competency-based; resuscitation; multi-centred

Full Text:



  • There are currently no refbacks.

Copyright (c) 2016

CMEJ ~ Canadian Medical Education Journal
Jennifer O'Brien PhD, Managing Editor, University of Saskatchewan; E-mail:

ISSN 1923-1202