Developing a dashboard for faculty development in competency-based training programs: a design-based research project

Authors

  • Yusuf Yilmaz McMaster University
  • Robert Carey University of Saskatchewan
  • Teresa Chan McMaster University
  • Venkat Bandi University of Saskatchewan
  • Shisong Wang University of Saskatchewan
  • Robert A Woods University of Saskatchewan
  • Debajyoti Mondal University of Saskatchewan
  • Brent Thoma University of Saskatchewan https://orcid.org/0000-0003-1124-5786

DOI:

https://doi.org/10.36834/cmej.72067

Abstract

Background: Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires frequent assessments of entrustable professional activities (EPAs). Faculty struggle to provide helpful feedback and assign appropriate entrustment scores. CBME faculty development initiatives rarely incorporate teaching metrics. Dashboards could be used to visualize faculty assessment data to support faculty development.

Methods: Using a design-based research process, we identified faculty development needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. Data was collected within the emergency medicine residency program at the University of Saskatchewan through interviews with program leaders, faculty development experts, and faculty participating in development sessions. Two investigators thematically analyzed interview transcripts to identify faculty needs that were audited by a third investigator. The needs were described using representative quotes and the dashboard elements designed to address them.

Results: Between July 1, 2019 and December 11, 2020 we conducted 15 interviews with nine participants (two program leaders, three faculty development experts, and four faculty members). Three needs emerged as themes from the analysis: analysis of assessments, contextualization of assessments, and accessible reporting. We addressed these needs by designing an accessible dashboard to present contextualized quantitative and narrative assessment data for each faculty member.

Conclusions: We identified faculty development needs related to EPA assessments and designed dashboard elements to meet them. The resulting dashboard was used for faculty development sessions. This work will inform the development of CBME assessment dashboards for faculty.

Published

2021-06-30 — Updated on 2021-09-15

Versions

How to Cite

1.
Yilmaz Y, Carey R, Chan T, Bandi V, Wang S, Woods RA, Mondal D, Thoma B. Developing a dashboard for faculty development in competency-based training programs: a design-based research project. Can. Med. Ed. J [Internet]. 2021Sep.15 [cited 2021Oct.23];12(4):48-64. Available from: https://journalhosting.ucalgary.ca/index.php/cmej/article/view/72067

Issue

Section

Major Contributions