Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency
DOI:
https://doi.org/10.36834/cmej.75250Abstract
Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen’s University Internal Medicine residency program introduced a facilitated, team-based assessment initiative (“Feedback Fridays”) in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training.
Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program’s assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically.
Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding.
Conclusions: The introduction of facilitated team-based assessment in the Queen’s Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.
Metrics
References
Schuwirth L, Ash J. Assessing tomorrow's learners: in competency-based education only a radically different holistic method of assessment will work. Six things we could forget. Med Teach. 2013;35(7):555-9. https://doi.org/10.3109/0142159X.2013.787140 DOI: https://doi.org/10.3109/0142159X.2013.787140
Schuwirth L, Valentine N, Dilena P. An application of programmatic assessment for learning (PAL) system for general practice training. GMS J Med Educ. 2017;34(5):Doc56.
van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205-14. https://doi.org/10.3109/0142159X.2012.652239 DOI: https://doi.org/10.3109/0142159X.2012.652239
Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315
Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645
Gaunt A, Patel A, Rusius V, Royle TJ, Markham DH, Pawlikowska T. 'Playing the game': How do surgical trainees seek feedback using workplace-based assessment? Med Educ. 2017;51(9):953-62. https://doi.org/10.1111/medu.13380 DOI: https://doi.org/10.1111/medu.13380
Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523
Gaunt A, Markham DH, Pawlikowska TRB. Exploring the role of self-motives in postgraduate trainees' feedback-seeking behavior in the clinical workplace: a multicenter study of workplace-based assessments from the United Kingdom. Acad Med. 2018;93(10):1576-83. https://doi.org/10.1097/ACM.0000000000002348 DOI: https://doi.org/10.1097/ACM.0000000000002348
Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34. https://doi.org/10.1097/ACM.0000000000001523 DOI: https://doi.org/10.1097/ACM.0000000000001523
Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents' feedback-seeking behavior in relation to night shifts. Acad Med. 2009;84(7):910-7. https://doi.org/10.1097/ACM.0b013e3181a858ad DOI: https://doi.org/10.1097/ACM.0b013e3181a858ad
Dudek NL, Marks MB, Regehr G. Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005;80(10 Suppl):S84-7. https://doi.org/10.1097/00001888-200510001-00023 DOI: https://doi.org/10.1097/00001888-200510001-00023
Schwind CJ, Williams RG, Boehler ML, Dunnington GL. Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies? Acad Med. 2004;79(5):453-7. https://doi.org/10.1097/00001888-200405000-00016 DOI: https://doi.org/10.1097/00001888-200405000-00016
Bandiera G, Lendrum D. Daily encounter cards facilitate competency-based feedback while leniency bias persists. Cjem. 2008;10(1):44-50. https://doi.org/10.1017/S1481803500010009 DOI: https://doi.org/10.1017/S1481803500010009
Ginsburg S, Gold W, Cavalcanti RB, Kurabi B, McDonald-Blumer H. Competencies "plus": the nature of written comments on internal medicine residents' evaluation forms. Acad Med. 2011;86(10):S30-S4. https://doi.org/10.1097/ACM.0b013e31822a6d92 DOI: https://doi.org/10.1097/ACM.0b013e31822a6d92
Chou S, Cole G, McLaughlin K, Lockyer J. CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction. Med Educ. 2008;42:879-86. https://doi.org/10.1111/j.1365-2923.2008.03111.x DOI: https://doi.org/10.1111/j.1365-2923.2008.03111.x
Rida T-Z, Dubois D, Hui Y, Ghatalia J, McConnell M, LaDonna K, editors. Assessment of CanMEDS competencies in work-based assessment: challenges and lessons learned. 2020 CAS Annual Meeting.
McConnell M, Sherbino J, Chan TM. Mind the gap: the prospects of missing data. J Grad Med Ed. 2016;8(5):708-12. https://doi.org/10.4300/JGME-D-16-00142.1 DOI: https://doi.org/10.4300/JGME-D-16-00142.1
Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students' professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75(2):167-73. https://doi.org/10.1097/00001888-200002000-00016 DOI: https://doi.org/10.1097/00001888-200002000-00016
Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698-706. https://doi.org/10.1097/ACM.0000000000000809 DOI: https://doi.org/10.1097/ACM.0000000000000809
Yin R. Case study research and applications: design and methods. Thousand Oaks, CA: SAGE Publications, Inc; 2017.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-57. https://doi.org/10.1093/intqhc/mzm042 DOI: https://doi.org/10.1093/intqhc/mzm042
McEwen LA EA, Chamberlain S, Taylor D. Queen's Periodic Performance assessments: capturing performance information beyond single observations in CBME Contexts. 2nd World Summit on CBME; Basel, Switzerland2018.
Taylor DR. Feedback Friday Intro [Video]. 2017. Available from: https://www.youtube.com/watch?v=OsRB3NfL00g&t=7s [Accessed 2024/02/05]
Royal College of Physicians and Surgeons of Canada. Competence by Design: Canada's model for competency-based medical education 2022. Available from: https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psych. 2006;3(2):77-101. https://doi.org/10.1191/1478088706qp063oa DOI: https://doi.org/10.1191/1478088706qp063oa
Cofie N, Braund H, Dalgarno N. Eight ways to get a grip on intercoder reliability using qualitative-based measures. Can Med Ed J. 2022. https://doi.org/10.36834/cmej.72504 DOI: https://doi.org/10.36834/cmej.72504
Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res. 1999;9(1):26-44. https://doi.org/10.1177/104973299129121677 DOI: https://doi.org/10.1177/104973299129121677
Palaganas EC, Sanchez MC, Molintas VP, Caricativo RD. Reflexivity in qualitative research: a journey of learning. Qual Rep. 2017;22(2). https://doi.org/10.46743/2160-3715/2017.2552 DOI: https://doi.org/10.46743/2160-3715/2017.2552
Day LB, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7. https://doi.org/10.1097/ACM.0000000000003315 DOI: https://doi.org/10.1097/ACM.0000000000003315
Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. https://doi.org/10.1111/medu.13645 DOI: https://doi.org/10.1111/medu.13645
Ashenafi MM. Peer-assessment in higher education-twenty-first century practices, challenges and the way forward. Assess Eval High Educ. 2017;42(2):226-51. https://doi.org/10.1080/02602938.2015.1100711 DOI: https://doi.org/10.1080/02602938.2015.1100711
Wanner T, Palmer E. Formative self-and peer assessment for improved student learning: the crucial factors of design, teacher participation and feedback. Assess Eval High Educ. 2018;43(7):1032-47. https://doi.org/10.1080/02602938.2018.1427698 DOI: https://doi.org/10.1080/02602938.2018.1427698
Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency‐based medical education: are we addressing the concerns and challenges? Med Educ. 2015;49(11):1086-102. https://doi.org/10.1111/medu.12831 DOI: https://doi.org/10.1111/medu.12831
Sirianni G, Glover Takahashi S, Myers J. Taking stock of what is known about faculty development in competency-based medical education: a scoping review paper. Med Teach. 2020;42(8):909-15. https://doi.org/10.1080/0142159X.2020.1763285 DOI: https://doi.org/10.1080/0142159X.2020.1763285
Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751-7. https://doi.org/10.1080/0142159X.2021.1928036 DOI: https://doi.org/10.1080/0142159X.2021.1928036
Young JQ, Sugarman R, Schwartz J, O'Sullivan PS. Faculty and resident engagement with a workplace-based assessment tool: use of implementation science to explore enablers and barriers. Acad Med. 2020;95(12):1937-44. https://doi.org/10.1097/ACM.0000000000003543 DOI: https://doi.org/10.1097/ACM.0000000000003543
Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Advan Health Sci Ed. 2016;21(2):455-73. https://doi.org/10.1007/s10459-015-9614-0 DOI: https://doi.org/10.1007/s10459-015-9614-0
Braund H, Dalgarno N, McEwen L, Egan R, Reid M-A, Baxter S. Involving ophthalmology departmental stakeholders in developing workplace-based assessment tools. Can J Ophthalmol. 2019;54(5):590-600. https://doi.org/10.1016/j.jcjo.2019.01.013 DOI: https://doi.org/10.1016/j.jcjo.2019.01.013
Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190 DOI: https://doi.org/10.3109/0142159X.2010.501190
Govaerts MJ, van der Vleuten CP, Holmboe ES. Managing tensions in assessment: moving beyond either-or thinking. Med Educ. 2019;53(1):64-75. https://doi.org/10.1111/medu.13656 DOI: https://doi.org/10.1111/medu.13656
Cheung K, Rogoza C, Chung AD, Kwan BYM. Analyzing the administrative burden of competency based medical education. Can Assoc Radiol J 2021:08465371211038963. https://doi.org/10.1177/08465371211038963 DOI: https://doi.org/10.1177/08465371211038963
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Heather Braund, Nancy Dalgarno, Rachel O'Dell, David R Taylor
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
