Five ways to get a grip on applying a program evaluation model in health professions education academies

Rebecca D Blanchard; Katherine E McDaniel; Deborah L Engle

doi:10.36834/cmej.77995

Authors

Rebecca D Blanchard University of Kansas Medical Center
Katherine E McDaniel Duke University School of Medicine
Deborah L Engle Duke University School of Medicine

DOI:

https://doi.org/10.36834/cmej.77995

Abstract

The proliferation of health professions educator academies across Canada and the United States illustrates the value they hold for faculty and institutions. Yet, establishing and evaluating the efficacy of them through program evaluation can be challenging. Moreover, academy leadership often lack the time, bandwidth skillset and personnel to undertake rigorous program evaluation efforts. We outline a step-by-step guide for getting a grip on evaluating health professions educator academies. Developing a plan for program evaluation in advance of any new academy initiative helps to ensure the academy calibrates and re-calibrates to accomplish outcomes and meet stakeholder expectations. It also provides a mechanism for tracking academy impact, which strengthens requests for funding, promotes sustainability and encourages continued buy-in and support from institutional stakeholders. For all of these reasons, we present the following recommendations: apply the relevant program evaluation framework(s); identify resources for program evaluation; prepare to tell your academy’s story; list desired program outcomes; establish a data collection plan; and obtain institutional review board approval.

References

Uijtdehaage S, Ho MJ, Harvey E, Dorris CS, Huggett KN. Academies in health professions education: a scoping review. Acad Med. 2021;96(10):1476-1483. https://doi.org/10.1097/ACM.0000000000004161

Fernandez N, Audétat MC. Faculty development program evaluation: a need to embrace complexity. Adv Med Educ Pract. 2019;10:191-199. Published 2019 Apr 16. https://doi.org/10.2147/AMEP.S188164

Corral J, Guiton G, Aagaard E. The impact of an academy of medical educators on the culture of an American health sciences campus. Acad Med. 2017; 92(8):1145–1150. https://doi.org/10.1097/ACM.0000000000001508

Blanchard, Rebecca, PhD, MEd, Belforti, Raquel, DO, MS. Evaluation of a health professions teaching academy through the lens of social capital. J Contin Educ Health Prof. 2022;42(1):53-59. https://doi.org/10.1097/CEH.0000000000000357.

Thoma B, Gottlieb M, Boysen-Osborn M, et al. Curated collections for educators: five key papers about program evaluation. Cureus. 2017 May 4;9(5):e1224. https://doi.org/10.7759/cureus.1224

Fallis D, Irwin S, Cervero R, Durning S. Frameworks to guide faculty development for health professions education: a scoping review. J Contin Educ Health Prof. 2022;42(3):180-189. https://doi.org/10.1097/CEH.0000000000000376.

Alexandraki I, Rosasco RE, Mooradian AD. An evaluation of faculty development programs for clinician–educators: a scoping review. Acad Med. 2021;96(4): e10-e10e11. https://doi.org/10.1097/ACM.0000000000004009

Haas M, Triemstra J, Tam M. et al. A decade of faculty development for health professions educators: lessons learned from the Macy Faculty Scholars Program. BMC Med Educ. 2023 Mar 27;23(1):185. https://doi.org/10.1186/s12909-023-04155-x

Searle NS, Thompson BM, Friedland J, et al. The prevalence and practice of academies of medical educators: a survey of U.S. medical schools. Acad Med 85(1):p 48-56, Jan 2010. https://doi.org/10.1097/ACM.0b013e3181c4846b

Oermann MH, Reynolds SS, Granger BB. Using an implementation science framework to advance the science of nursing education. J Prof Nurs. 2022 Mar-Apr;39:139-145. https://doi.org/10.1016/j.profnurs.2022.01.014

Hamza DM, Regehr G. Eco-normalization: evaluating the longevity of an innovation in context. Acad Med. 96(11S):p S48-S53, Nov 2021. https://doi.org/10.1097/ACM.0000000000004318

Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. 3rd ed. San Francisco, CA: Berrett-Koehler; 2006.