Examining the accuracy of residents’ self-assessments and faculty assessment behaviours in anesthesiology

Authors

  • Melinda Fleming Queen's University
  • Danika Vautour Queen's University
  • Michael McMullen Queen's University
  • Nicholas Cofie Queen's University
  • Nancy Dalgarno Queen's University
  • Rachel Phelan Queen's University
  • Glenio Mizubuti Queen's University

DOI:

https://doi.org/10.36834/cmej.70697

Abstract

Background: Residents’ accurate self-assessment and clinical judgment are essential for optimizing their clinical skills development. Evidence from the medical literature suggests that residents generally do poorly at self-assessing their performance, often due to factors relating to learners’ personal backgrounds, cultures, the specific contexts of the learning environment and rater bias or inaccuracies. We evaluated the accuracy of anesthesiology residents’ self-assessed Global Entrustment scores and determined whether differences between faculty and resident scores varied by resident seniority, faculty leniency, and/or year of assessment.

Methods: We employed variance components modeling techniques and analyzed 329 pairs of faculty and self-assessed entrustment scores among 43 faculty assessors and 15 residents. Using faculty scores as the gold standard, we compared faculty scores with residents’ scores (xi(faculty)–xi(resident)), and determined residents’ accuracy, including over- and under-confidence.

Results: The results indicate that residents were respectively over- and under-confident in 10.9% and 54.4% of the assessments but more consistent in their individual self-assessments (rho = 0.70) than faculty assessors. Faculty scores were significantly higher (α = 0.396; z = 4.39; p < 0.001) than residents’ self-assessed scores. Being a lenient/dovish (β = 0.121, z = 3.16, p < 0.01) and a neutral (β = 0.137, z = 3.57, p < 0.001) faculty assessor predicted a higher likelihood of resident under-confidence. Senior residents were significantly less likely to be under-confident compared to junior residents (β = -0.182, z =-2.45, p < 0.05). The accuracy of self-assessments did not significantly vary during the two years of the study period.

Conclusions: The majority of residents’ self-assessments were inaccurate. Our findings may help identify the sources of such inaccuracies.

Metrics

Metrics Loading ...

Author Biographies

Melinda Fleming, Queen's University

Melinda Fleming, MD, FRCPC, is an Assistant professor and residency training program director in the Department of Anesthesiology and Perioperative Medicine at Queen’s University. Her academic research interests include postgraduate medical education, simulation in medical education, and competency based medical education.

Danika Vautour, Queen's University

Danika Vautour, MD, is a senior resident in the Department of Anesthesiology and Perioperative Medicine at Queen’s University. Her research interests include postgraduate medical education and simulation in medical education.

Michael McMullen, Queen's University

Michael McMullen MD, FRCPC, is an Assistant professor in the Department of Anesthesiology and Perioperative Medicine at Queen’s University. His clinical interests include preoperative assessment, regional anesthesia, acute pain management. His academic interests include postgraduate medical education and simulation in medical education.

Nicholas Cofie, Queen's University

Nicholas Cofie, PhD, is a Health Education Researcher and Consultant in the Office of Professional Development and Educational Scholarship in the Faculty of Health Sciences at Queen’s University. He has expertise in medical education research, program evluation, research methodology, and statistical data analysis.

Rachel Phelan, Queen's University

Rachel Phelan, MSc, is a Clinical Research Facilitator in the Department of Anesthesiology and Perioperative Medicine at Queen’s University. Her research interests include the impact of psychosocial factors on high fidelity medical simulation with respect to training and assessment methods.

Glenio Mizubuti, Queen's University

Glenio Mizubuti, MD, MSc is an Assistant professor in the Department of Anesthesiology and Perioperative Medicine at Queen’s University. His clinical interests include point-of-care ultrasound, perioperative echocardiography, regional anesthesia, and acute pain management. He is also involved with postgraduate medical education and has formal training in high-fidelity simulation in medical education.

Downloads

Published

2021-04-12 — Updated on 2021-09-15

Versions

How to Cite

1.
Fleming M, Vautour D, McMullen M, Cofie N, Dalgarno N, Phelan R, Mizubuti G. Examining the accuracy of residents’ self-assessments and faculty assessment behaviours in anesthesiology. Can. Med. Ed. J [Internet]. 2021 Sep. 15 [cited 2024 May 20];12(4):17-26. Available from: https://journalhosting.ucalgary.ca/index.php/cmej/article/view/70697

Issue

Section

Original Research

Most read articles by the same author(s)