The education passport: connecting programmatic assessment across learning and practice
DOI:
https://doi.org/10.36834/cmej.73871Abstract
Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an “Education Passport” that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice.
The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care.
Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.
Metrics
References
McGaghie WC, Miller GE, Sajid AW, Telder TV. Competency-based curriculum development on medical education: an introduction. Public Health Pap. 1978;(68):11-91.
Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190
van der Vleuten CP, Schuwirth LW, Driessen EW, Govaerts MJ, Heeneman S. 12 Tips for programmatic assessment. Med Teach. Nov 2014:1-6. https://doi.org/10.3109/0142159X.2014.973388
Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med teach. 2007;29(7):642-647. https://doi.org/10.1080/01421590701746983
Edgar L, Roberts S, Holmboe E. Milestones 2.0: a step forward. JGME. 2018;10(3):367. https://doi.org/10.4300/JGME-D-18-00372.1
Englander R, Flynn T, Call S, et al. Toward defining the foundation of the md degree: core Entrustable Professional Activities for entering residency. Acad Med. Oct 2016;91(10):1352-1358. https://doi.org/10.1097/acm.0000000000001204
Coalition for Physician Accountability. Initial summary report and preliminary recommendations of the undergraduate medical education to graduate medical education review committee (UGRC). https://physicianaccountability.org/wp-content/uploads/2021/04/UGRC-Initial-Summary-Report-and-Preliminary-Recommendations-1.pdf [Accessed May 15, 2021].
Santen SA, Foster KW, Hemphill RR, Christner J, Mejicano G. Simplifying the interconnected alphabet soup of medical education. Acad Med. 2022;97(5)
Gingerich A, Sebok-Syer SS, Larstone R, Watling CJ, Lingard L. Seeing but not believing: insights into the intractability of failure to fail. Med Educ. Dec 2020;54(12):1148-1158. https://doi.org/10.1111/medu.14271
Papadakis MA, Teherani A, Banach MA, et al. Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. Dec 2005;353(25):2673-82. https://doi.org/10.1056/NEJMsa052596
Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Intern Med. Jun 2008;148(11):869-76. https://doi.org/10.7326/0003-4819-148-11-200806030-00009
Jonker G, Ochtman A, Marty AP, Kalkman CJ, Ten Cate O, Hoff RG. Would you trust your loved ones to this trainee? Certification decisions in postgraduate anaesthesia training. Br J Anaesth. Nov 2020;125(5):e408-e410. https://doi.org/10.1016/j.bja.2020.07.009
Dweck CS. Mindset: The new psychology of success. Random House Digital, Inc; 2008.
Lucey CR, Hauer KE, Boatright D, Fernandez A. Medical education's wicked problem: achieving equity in assessment for medical learners. Acad Med. Dec 2020;95(12S Addressing harmful bias and eliminating discrimination in health professions learning environments):S98-s108. https://doi.org/10.1097/acm.0000000000003717
Low D, Pollack SW, Liao ZC, et al. Racial/ethnic disparities in clinical grading in medical school. Teach Learn Med. Oct-Dec 2019;31(5):487-496. https://doi.org/10.1080/10401334.2019.1597724
Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med. 2003;15(4):270-92. https://doi.org/10.1207/S15328015TLM1504_11
Warm EJ, Kinnear B, Pereira A, Hirsh DA. The residency match: escaping the prisoner's dilemma. J Grad Med Educ. Oct 2021;13(5):616-625. https://doi.org/10.4300/jgme-d-21-00477.1
Mendu ML, Kachalia A, Eappen S. Revisiting US news & world report's hospital rankings-moving beyond mortality to metrics that improve care. J Gen Intern Med. 2021:209-210. vol. 1. https://doi.org/10.1007/s11606-020-06002-x
Ried LD, Ried DB. Predicting the 2016 US News & world report rankings using a reputation and prestige model. Curr Pharm Teach Learn. Feb 2021;13(2):91-101. https://doi.org/10.1016/j.cptl.2020.09.011
Newsweek. Best Hospitals 2021 - Canada. 2021. https://www.newsweek.com/best-hospitals-2021/canada [Accessed May 9, 2021].
Fang DZ, Young CB, Golshan S, Moutier C, Zisook S. Burnout in premedical undergraduate students. Acad Psych. 2012;36(1):11-16.
Ishak W, Nikravesh R, Lederer S, Perry R, Ogunyemi D, Bernstein C. Burnout in medical students: a systematic review. Clin Teach. Aug 2013;10(4):242-5. https://doi.org/10.1111/tct.12014
Prins JT, Gazendam-Donofrio SM, Tubben BJ, van der Heijden FM, van de Wiel HB, Hoekstra-Weebers JE. Burnout in medical residents: a review. Med Educ. Aug 2007;41(8):788-800. https://doi.org/10.1111/j.1365-2923.2007.02797.x
West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. Jun 2018;283(6):516-529. https://doi.org/10.1111/joim.12752
Bynum WEt, Artino AR, Jr., Uijtdehaage S, Webb AMB, Varpio L. Sentinel emotional events: the nature, triggers, and effects of shame experiences in medical residents. Acad Med. Jan 2019;94(1):85-93. https://doi.org/10.1097/acm.0000000000002479
Bynum WEt, Varpio L, Lagoo J, Teunissen PW. 'I'm unworthy of being in this space': the origins of shame in medical students. Med Educ. Feb 2021;55(2):185-197. https://doi.org/10.1111/medu.14354
Albuquerque J, Tulk S. Physician suicide. Cmaj. May 6 2019;191(18):E505. https://doi.org/10.1503/cmaj.181687
Schneider EC, Sarnak DO, Squires D, Shah A, Doty MM. Mirror Mirror 2017. Commonwealth Fund New York; 2017.
McDonald FS, Duhigg LM, Arnold GK, Hafer RM, Lipner RS. The American board of internal medicine maintenance of certification examination and state medical board disciplinary actions: a population cohort study. J Gen Intern Med. Aug 2018;33(8):1292-1298. https://doi.org/10.1007/s11606-018-4376-z
Naji L, Singh B, Shah A, et al. Global prevalence of burnout among postgraduate medical trainees: a systematic review and meta-regression. CMAJ Open. Jan-Mar 2021;9(1):E189-e200. https://doi.org/10.9778/cmajo.20200068
Dreyfus SE. The five-stage model of adult skill acquisition. Bulletin of science, technology & society. 2004;24(3):177-181. https://doi.org/10.1177/0270467604264992
Dunning D. The Dunning–Kruger effect: On being ignorant of one's own ignorance. Advances exper soc psychol. Elsevier; 2011:247-296. https://doi.org/10.1016/B978-0-12-385522-0.00005-6
Cutrer W, Pusic M, Gruppen LD, Hammoud MM, Santen SA. The Master Adaptive Learner. Elsevier Health Sciences; 2019.
Cutrer WB, Miller B, Pusic MV, et al. Fostering the development of master adaptive learners: a conceptual model to guide skill acquisition in medical education. Acad Med. Jan 2017;92(1):70-75. https://doi.org/10.1097/acm.0000000000001323
Andrews JS, Bale JF, Jr., Soep JB, et al. Education in pediatrics across the continuum (EPAC): first steps toward realizing the dream of competency-based education. Acad Med. Mar 2018;93(3):414-420. https://doi.org/10.1097/acm.0000000000002020
Warm EJ, Held JD, Hellmann M, et al. entrusting observable practice activities and milestones over the 36 months of an internal medicine residency. Acad Med. Oct 2016;91(10):1398-1405. https://doi.org/10.1097/acm.0000000000001292
Hauer KE, Chesluk B, Iobst W, et al. Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. Aug 2015;90(8):1084-92. https://doi.org/10.1097/acm.0000000000000736
Hauer KE, Edgar L, Hogan SO, Kinnear B, Warm E. The science of effective group process: lessons for clinical competency committees. J Grad Med Educ. Apr 2021;13(2 Suppl):59-64. https://doi.org/10.4300/jgme-d-20-00827.1
Thoma B, Warm E, Hamstra SJ, et al. Next steps in the implementation of learning analytics in medical education: consensus from an international cohort of medical educators. J Grad Med Educ. Jun 2020;12(3):303-311. https://doi.org/10.4300/jgme-d-19-00493.1
Warm EJ, Mathis BR, Held JD, et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med. Aug 2014;29(8):1177-82. https://doi.org/10.1007/s11606-014-2801-5
Schauer DP, Kinnear B, Kelleher M, Sall D, Schumacher DJ, Warm EJ. Developing the expected entrustment score: accounting for variation in resident assessment. J Gen Intern Med. Apr 4 2022; https://doi.org/10.1007/s11606-022-07492-7
Nousiainen MT, Mironova P, Hynes M, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med teach. 2018;40(10):1042-1054. https://doi.org/10.1080/0142159X.2017.1421751
Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. Learning analytics in medical education assessment: the past, the present, and the future. AEM Educ Train. Apr 2018;2(2):178-187. https://doi.org/10.1002/aet2.10087
Thoma B, Caretta-Weyer H, Schumacher DJ, et al. Becoming a deliberately developmental organization: using competency based assessment data for organizational development. Med Teach. May 25 2021:1-9. https://doi.org/10.1080/0142159x.2021.1925100
Ten Cate O, Graafmans L, Posthumus I, Welink L, van Dijk M. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. May 2018;40(5):506-513. https://doi.org/10.1080/0142159x.2018.1435856
Choe JH, Knight CL, Stiling R, Corning K, Lock K, Steinberg KP. Shortening the miles to the milestones: connecting epa-based evaluations to ACGME milestone reports for internal medicine residency programs. Acad Med. Jul 2016;91(7):943-50. https://doi.org/10.1097/acm.0000000000001161
Wagner LM, Dolansky MA, Englander R. Entrustable professional activities for quality and patient safety. Nurs Outlook. May - Jun 2018;66(3):237-243. https://doi.org/10.1016/j.outlook.2017.11.001
Kelleher M, Kinnear B, Wong SEP, O'Toole J, Warm E. Linking workplace-based assessment to ACGME milestones: a comparison of mapping strategies in two specialties. Teach Learn Med. Apr-May 2020;32(2):194-203. https://doi.org/10.1080/10401334.2019.1653764
Gerstle CR. Parallels in safety between aviation and healthcare. J Pediatr Surg. May 2018;53(5):875-878. https://doi.org/10.1016/j.jpedsurg.2018.02.002
Borrelli RA. Reflections on developing an identity for the third generation nuclear engineer in the post-Fukushima society. Reflections on the Fukushima Daiichi Nuclear Accident. Springer, Cham; 2015:353-365. https://doi.org/10.1007/978-3-319-12090-4_19
Jonker CM, Riemsdijk M, Vermeulen B. Shared mental models. Springer; 132-151. https://doi.org/10.1007/978-3-642-21268-0_8
Warm EJ, Englander R, Pereira A, Barach P. Improving Learner Handovers in Medical Education. Acad Med. Jul 2017;92(7):927-931. https://doi.org/10.1097/acm.0000000000001457
Eva KW, Regehr G. "I'll never play professional football" and other fallacies of self-assessment. J Contin Educ Health Prof. Winter 2008;28(1):14-9. https://doi.org/10.1002/chp.150
Ikizler TA, Berns JS. Transformation of ABIM and What the Changes Mean to Nephrologists. Clin J Am Soc Nephrol.2018;13(1):164-166. https://doi.org/10.2215/CJN.05700517
Teirstein PS. Boarded to death--why maintenance of certification is bad for doctors and patients. N Engl J Med. Jan 8 2015;372(2):106-8. https://doi.org/10.1056/NEJMp1407422
Boyd VA, Whitehead CR, Thille P, Ginsburg S, Brydges R, Kuper A. Competency-based medical education: the discourse of infallibility. Med Educ. Jan 2018;52(1):45-57. https://doi.org/10.1111/medu.13467
Murray KE, Lane JL, Carraccio C, et al. Crossing the gap: using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. Mar 2019;94(3):338-345. https://doi.org/10.1097/acm.0000000000002535
Heath JK, Wang T, Santhosh L, et al. Longitudinal milestone assessment extending through subspecialty training: the relationship between acgme internal medicine residency milestones and subsequent pulmonary and critical care fellowship milestones. Acad Med. May 18 2021; https://doi.org/10.1097/acm.0000000000004165
Heifetz RA, Laurie DL. The work of leadership. Harvard business review. 1997;75:124-134.
Heifetz R, Linsky M. Leadership on the line, with a new preface: Staying alive through the dangers of change. Harvard Business Press; 2017.
Groupsmith. Technical Problems vs. Adaptive Challenges. 2021. https://www.sgaumc.org/files/files_library/technical_vs_adaptive_challenges.pdf [Accessed May 9, 2021].
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Eric J Warm, Carol Carraccio , Matthew Kelleher, Benjamin Kinnear, Daniel J Schumacher, Sally Santen
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.