Is Competency-Based Medical Education being implemented as intended? Early lessons learned from Physical Medicine and Rehabilitation
DOI:
https://doi.org/10.36834/cmej.77188Abstract
Background: As competency-based medical education (CBME) curricula are introduced in residency programs across Canada, systematic evaluation efforts are needed to ensure fidelity of implementation. This study evaluated early outcomes of CBME implementation in one Canadian Physical Medicine and Rehabilitation program that was an early adopter of CBME, with an aim to inform continuous quality improvement initiatives and CBME implementation nationwide.
Methods: Using Rapid Evaluation methodology, informed by the CBME Core Components Framework, the intended outcomes of CBME were compared to actual outcomes.
Results: Results suggested that a culture of feedback and coaching already existed in this program prior to CBME implementation, yet faculty felt that CBME added a framework to support feedback. The small program size was valuable in fostering strong relationships and individualized learning. However, participants expressed concerns about CBME fostering a reductionist approach to the development of competence. Challenges existed with direct observation, clear expectations for off-service training experiences, and tracking trainee progress. There was trepidation surrounding national curricular change, yet the institution-wide approach to CBME implementation created shared experiences and a community of practice.
Conclusions: Program evaluation can help understand gaps between planned versus enacted implementation of CBME, and foster adaptations to improve the fidelity of implementation.
Metrics
References
van Melle E, Hall AK, Schumacher DJ, et al. Capturing outcomes of competency-based medical education: the call and the challenge. Med Teach. 2021;43(7). https://doi.org/10.1080/0142159X.2021.1925640 DOI: https://doi.org/10.1080/0142159X.2021.1925640
Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a CBME intervention guided by program theory. J Eval Clin Pract. Vol 26; 2020. https://doi.org/10.1111/jep.13344 DOI: https://doi.org/10.1111/jep.13344
King JA, Morris LL, Fitz-Gibbon CT. How to assess program implementation.; 1987.
Hall AK, Rich J, Dagnone JD, et al. It’s a marathon, not a sprint: rapid evaluation of competency-based medical education program implementation. Acad Med. Published online 2020. https://doi.org/10.1097/ACM.0000000000003040 DOI: https://doi.org/10.1097/ACM.0000000000003040
Kouzmina E, Mann S, Chaplin T, Zevin B. An evaluation of the surgical foundations curriculum: a national study. J Surg Educ. 2021;78(3). https://doi.org/10.1016/j.jsurg.2020.10.002 DOI: https://doi.org/10.1016/j.jsurg.2020.10.002
Federation des Medecins Residents du Quebec. Year 3 of implementation of competence by design: negative impact still outweighs theoretical benefits. observations on the day-to-day reality of CBD and its progression since July 2017. July 2020. Available from: https://fmrq.qc.ca/wp-content/uploads/2022/07/fmrq-report-cbd-implementation-year-3_1.pdf. [Accessed Oct 28, 2023].
Sherbino J, Regehr G, Dore K, Ginsburg S. Tensions in describing competency-based medical education: a study of Canadian key opinion leaders. AdvHealth Sci Ed. 2021;26(4). https://doi.org/10.1007/s10459-021-10049-8 DOI: https://doi.org/10.1007/s10459-021-10049-8
Gold M, Helms D, Guterman S. Identifying, monitoring, and assessing promising innovations: using evaluation to support rapid-cycle change. Issue Brief (Commonw Fund). 2011;12.
van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94(7):1002-1009. https://doi.org/10.1097/ACM.0000000000002743 DOI: https://doi.org/10.1097/ACM.0000000000002743
Stockley D, Egan R, van Wylick R, et al. A systems approach for institutional CBME adoption at Queen’s University. Med Teach. 2020;42(8):916-921. https://doi.org/10.1080/0142159X.2020.1767768 DOI: https://doi.org/10.1080/0142159X.2020.1767768
Frank JR, Snell L, Sherbino J Editors. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada. CanMEDS 2015 Physician Competency Framework Ottawa: Royal College of Physicians and Surgeons of Canada. Published online 2015.
Royal College of Physicians and Surgeons of Canada. What is CBD? The Royal College of Physicians and Surgeons of Canada. https://www.royalcollege.ca/rcsite/cbd/what-is-cbd-e [Accessed Jan 19, 2023].
Royal College of Physicians and Surgeons of Canada. The Competence by Design (CBD) coaching model. 2017. Available from: https://www.royalcollege.ca/content/dam/documents/accreditation/competence-by-design/directory/cbd-coaching-model-handout-en.pdf [Accessed Oct 28, 2023].
Pero R, Pero E, Marcotte L, Dagnone JD. Educational consultants: fostering an innovative implementation of competency-based medical education. Med Educ. 2019;53(5). https://doi.org/10.1111/medu.13868 DOI: https://doi.org/10.1111/medu.13868
van Melle E. How do we know it works? (Closing Plenary) Presented at: World Summit on CBME. Published online August 25, 2018.
Lipscomb M. Abductive reasoning and qualitative research. Nurs Philos. 2012;13(4). https://doi.org/10.1111/j.1466-769X.2011.00532.x DOI: https://doi.org/10.1111/j.1466-769X.2011.00532.x
Timmermans S, Tavory I. Theory construction in qualitative research: from grounded theory to abductive analysis. Sociol Theory. 2012;30(3). https://doi.org/10.1177/0735275112457914 DOI: https://doi.org/10.1177/0735275112457914
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9). https://doi.org/10.1177/1049732305276687 DOI: https://doi.org/10.1177/1049732305276687
Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26(13). https://doi.org/10.1177/1049732316654870 DOI: https://doi.org/10.1177/1049732316654870
Bloch M, Lave J, Wenger E. Situated learning: legitimate peripheral participation. Man. 1994;29(2). https://doi.org/10.2307/2804509 DOI: https://doi.org/10.2307/2804509
Frank JR, Snell LS, Cate O ten, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8). https://doi.org/10.3109/0142159X.2010.501190 DOI: https://doi.org/10.3109/0142159X.2010.501190
Richardson D, Kinnear B, Hauer KE, et al. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751-757. https://doi.org/10.1080/0142159X.2021.1928036 DOI: https://doi.org/10.1080/0142159X.2021.1928036
23. Ott M, Pack R, Cristancho S, et al. “The most crushing thing”: understanding resident assessment burden in a competency-based curriculum. JGME. 2022;14(5). https://doi.org/10.4300/JGME-D-22-00050.1 DOI: https://doi.org/10.4300/JGME-D-22-00050.1
Spillane JP, Reiser BJ, Reimer T. Policy implementation and cognition: Reframing and refocusing implementation research. Rev Educ Res. 2002;72(3). https://doi.org/10.3102/00346543072003387 DOI: https://doi.org/10.3102/00346543072003387
Griffiths J, Dalgarno N, Schultz K, Han H, van Melle E. Competency-Based Medical Education implementation: Are we transforming the culture of assessment? Med Teach. 2019;41(7). https://doi.org/10.1080/0142159X.2019.1584276 DOI: https://doi.org/10.1080/0142159X.2019.1584276
Hall AK, Woods R, Frank JR. Changing the culture of residency training through faculty development – ERRATUM. CJEM. 2019;21(6). https://doi.org/10.1017/cem.2019.419 DOI: https://doi.org/10.1017/cem.2019.419
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Jessica Trier, Sussan Askari, Tessa Hanmore, Heather-Ann Thompson, Natalie Wagner, Heather Braund, Andrew Koch Hall, Laura McEwen, Nancy Dalgarno, Jeffrey Damon Dagnone
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
