Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency

Authors

DOI:

https://doi.org/10.36834/cmej.75250

Abstract

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training.

Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically.

Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding.

Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.

Metrics

Metrics Loading ...

Author Biographies

Heather Braund, Queen's University

Office of Professional Development and Educational Scholarship

Faculty of Health Sciences

Queen's University

Nancy Dalgarno, Queen's University

Office of Professional Development and Educational Scholarship

Assistant Professor (Adjunct)

Department of Biomedical and Molecular Sciences

Faculty of Health Sciences

Queen’s University

David R Taylor, Queen's University

Department of Medicine

Associate Professor

References

Schuwirth L, Ash J. Assessing tomorrow's learners: in competency-based education only a radically different holistic method of assessment will work. Six things we could forget. Med Teach. 2013;35(7):555-9. https://doi.org/10.3109/0142159X.2013.787140 DOI: https://doi.org/10.3109/0142159X.2013.787140

Schuwirth L, Valentine N, Dilena P. An application of programmatic assessment for learning (PAL) system for general practice training. GMS J Med Educ. 2017;34(5):Doc56.

van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205-14. https://doi.org/10.3109/0142159X.2012.652239 DOI: https://doi.org/10.3109/0142159X.2012.652239

Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315

Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645

Gaunt A, Patel A, Rusius V, Royle TJ, Markham DH, Pawlikowska T. 'Playing the game': How do surgical trainees seek feedback using workplace-based assessment? Med Educ. 2017;51(9):953-62. https://doi.org/10.1111/medu.13380 DOI: https://doi.org/10.1111/medu.13380

Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523

Gaunt A, Markham DH, Pawlikowska TRB. Exploring the role of self-motives in postgraduate trainees' feedback-seeking behavior in the clinical workplace: a multicenter study of workplace-based assessments from the United Kingdom. Acad Med. 2018;93(10):1576-83. https://doi.org/10.1097/ACM.0000000000002348 DOI: https://doi.org/10.1097/ACM.0000000000002348

Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523 DOI: https://doi.org/10.1097/ACM.0000000000001523

Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents' feedback-seeking behavior in relation to night shifts. Acad Med. 2009;84(7):910-7. https://doi.org/10.1097/ACM.0b013e3181a858ad DOI: https://doi.org/10.1097/ACM.0b013e3181a858ad

Dudek NL, Marks MB, Regehr G. Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005;80(10 Suppl):S84-7. https://doi.org/10.1097/00001888-200510001-00023 DOI: https://doi.org/10.1097/00001888-200510001-00023

Schwind CJ, Williams RG, Boehler ML, Dunnington GL. Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies? Acad Med. 2004;79(5):453-7. https://doi.org/10.1097/00001888-200405000-00016 DOI: https://doi.org/10.1097/00001888-200405000-00016

Bandiera G, Lendrum D. Daily encounter cards facilitate competency-based feedback while leniency bias persists. Cjem. 2008;10(1):44-50. https://doi.org/10.1017/S1481803500010009 DOI: https://doi.org/10.1017/S1481803500010009

Ginsburg S, Gold W, Cavalcanti RB, Kurabi B, McDonald-Blumer H. Competencies "plus": the nature of written comments on internal medicine residents' evaluation forms. Acad Med. 2011;86(10):S30-S4. https://doi.org/10.1097/ACM.0b013e31822a6d92 DOI: https://doi.org/10.1097/ACM.0b013e31822a6d92

Chou S, Cole G, McLaughlin K, Lockyer J. CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction. Med Educ. 2008;42:879-86. https://doi.org/10.1111/j.1365-2923.2008.03111.x DOI: https://doi.org/10.1111/j.1365-2923.2008.03111.x

Rida T-Z, Dubois D, Hui Y, Ghatalia J, McConnell M, LaDonna K, editors. Assessment of CanMEDS competencies in work-based assessment: challenges and lessons learned. 2020 CAS Annual Meeting.

McConnell M, Sherbino J, Chan TM. Mind the gap: the prospects of missing data. J Grad Med Ed. 2016;8(5):708-12. https://doi.org/10.4300/JGME-D-16-00142.1 DOI: https://doi.org/10.4300/JGME-D-16-00142.1

Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students' professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75(2):167-73. https://doi.org/10.1097/00001888-200002000-00016 DOI: https://doi.org/10.1097/00001888-200002000-00016

Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698-706. https://doi.org/10.1097/ACM.0000000000000809 DOI: https://doi.org/10.1097/ACM.0000000000000809

Yin R. Case study research and applications: design and methods. Thousand Oaks, CA: SAGE Publications, Inc; 2017.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-57. https://doi.org/10.1093/intqhc/mzm042 DOI: https://doi.org/10.1093/intqhc/mzm042

McEwen LA EA, Chamberlain S, Taylor D. Queen's Periodic Performance assessments: capturing performance information beyond single observations in CBME Contexts. 2nd World Summit on CBME; Basel, Switzerland2018.

Taylor DR. Feedback Friday Intro [Video]. 2017. Available from: https://www.youtube.com/watch?v=OsRB3NfL00g&t=7s [Accessed 2024/02/05]

Royal College of Physicians and Surgeons of Canada. Competence by Design: Canada's model for competency-based medical education 2022. Available from: https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psych. 2006;3(2):77-101. https://doi.org/10.1191/1478088706qp063oa DOI: https://doi.org/10.1191/1478088706qp063oa

Cofie N, Braund H, Dalgarno N. Eight ways to get a grip on intercoder reliability using qualitative-based measures. Can Med Ed J. 2022. https://doi.org/10.36834/cmej.72504 DOI: https://doi.org/10.36834/cmej.72504

Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res. 1999;9(1):26-44. https://doi.org/10.1177/104973299129121677 DOI: https://doi.org/10.1177/104973299129121677

Palaganas EC, Sanchez MC, Molintas VP, Caricativo RD. Reflexivity in qualitative research: a journey of learning. Qual Rep. 2017;22(2). https://doi.org/10.46743/2160-3715/2017.2552 DOI: https://doi.org/10.46743/2160-3715/2017.2552

Day LB, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315 DOI: https://doi.org/10.1097/ACM.0000000000003315

Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645 DOI: https://doi.org/10.1111/medu.13645

Ashenafi MM. Peer-assessment in higher education-twenty-first century practices, challenges and the way forward. Assess Eval High Educ. 2017;42(2):226-51. https://doi.org/10.1080/02602938.2015.1100711 DOI: https://doi.org/10.1080/02602938.2015.1100711

Wanner T, Palmer E. Formative self-and peer assessment for improved student learning: the crucial factors of design, teacher participation and feedback. Assess Eval High Educ. 2018;43(7):1032-47. https://doi.org/10.1080/02602938.2018.1427698 DOI: https://doi.org/10.1080/02602938.2018.1427698

Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency‐based medical education: are we addressing the concerns and challenges? Med Educ. 2015;49(11):1086-102. https://doi.org/10.1111/medu.12831 DOI: https://doi.org/10.1111/medu.12831

Sirianni G, Glover Takahashi S, Myers J. Taking stock of what is known about faculty development in competency-based medical education: a scoping review paper. Med Teach. 2020;42(8):909-15. https://doi.org/10.1080/0142159X.2020.1763285 DOI: https://doi.org/10.1080/0142159X.2020.1763285

Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751-7. https://doi.org/10.1080/0142159X.2021.1928036 DOI: https://doi.org/10.1080/0142159X.2021.1928036

Young JQ, Sugarman R, Schwartz J, O'Sullivan PS. Faculty and resident engagement with a workplace-based assessment tool: use of implementation science to explore enablers and barriers. Acad Med. 2020;95(12):1937-44. https://doi.org/10.1097/ACM.0000000000003543 DOI: https://doi.org/10.1097/ACM.0000000000003543

Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Advan Health Sci Ed. 2016;21(2):455-73. https://doi.org/10.1007/s10459-015-9614-0 DOI: https://doi.org/10.1007/s10459-015-9614-0

Braund H, Dalgarno N, McEwen L, Egan R, Reid M-A, Baxter S. Involving ophthalmology departmental stakeholders in developing workplace-based assessment tools. Can J Ophthalmol. 2019;54(5):590-600. https://doi.org/10.1016/j.jcjo.2019.01.013 DOI: https://doi.org/10.1016/j.jcjo.2019.01.013

Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190 DOI: https://doi.org/10.3109/0142159X.2010.501190

Govaerts MJ, van der Vleuten CP, Holmboe ES. Managing tensions in assessment: moving beyond either-or thinking. Med Educ. 2019;53(1):64-75. https://doi.org/10.1111/medu.13656 DOI: https://doi.org/10.1111/medu.13656

Cheung K, Rogoza C, Chung AD, Kwan BYM. Analyzing the administrative burden of competency based medical education. Can Assoc Radiol J 2021:08465371211038963. https://doi.org/10.1177/08465371211038963 DOI: https://doi.org/10.1177/08465371211038963

Downloads

Published

2024-02-08

How to Cite

1.
Braund H, Dalgarno N, O’Dell R, Taylor DR. Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency . Can. Med. Ed. J [Internet]. 2024 Feb. 8 [cited 2024 Jun. 15];15(2):14-26. Available from: https://journalhosting.ucalgary.ca/index.php/cmej/article/view/75250

Issue

Section

Original Research

Most read articles by the same author(s)