A phenomenological study of resident and faculty experiences with learner engagement in the normalization of workplace-based assessment
DOI:
https://doi.org/10.36834/cmej.76192Abstract
Background: Workplace-based assessments (WPBA) have become integral to learner-centred medical education. As previous research has linked learner engagement to WPBA implementation, this study explores residents’ and faculty members’ experiences with learner engagement in the normalisation of WPBA practice.
Methods: Transcendental phenomenology was used as the qualitative approach, focusing on the participants’ lived experiences. A semi-structured interview guide was used to interview five faculty members and five residents who had conducted WPBA. The interviews were transcribed and analysed using phenomenological data analysis.
Results: Three themes were identified between learner engagement and WPBA conduct: (a) work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. WPBA learner engagement occurred when participants interacted with each other and with the clinical setting to facilitate teaching and learning. Both participant groups reported a desire to participate in WPBA, but time constraints at times hindered participation. The residents indicated that WPBA improved their knowledge and admitted to experiencing negative emotions during the assessment. Overall, participants recognised the reciprocal benefits of WPBA participation for their professional development.
Conclusion: The findings of the study suggest that learner engagement influences the use of WPBA. Consequently, it may be beneficial to consider the role of learner engagement to normalise WPBA application for teaching and learning in the clinical context.
References
Cate OT. Competency-based postgraduate medical education: past, present and future. GMS J Med Educ. 2017;34(5):Doc69. https://doi.org/10.3205/zma001146
Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017 Jun;39(6):609–16. https://doi.org/10.1080/0142159X.2017.1315082 DOI: https://doi.org/10.1080/0142159X.2017.1315082
Earl L, Steven Katz, Campbell G, et al. Rethinking classroom assessment with purpose in mind. Crown in right of the Governments of Alberta. 2006;112. Available from: https://www.edu.gov.mb.ca/k12/assess/wncp/full_doc.pdf [Accessed Sept 24, 2020].
Jenkins CE. The relationship between formative assessment and student engagement at Walters State Community College. ProQuest LLC. [Tennessee]: East Tennessee State University; 2010. Available from: https://dc.etsu.edu/cgi/viewcontent.cgi?article=3034&context=etd [Accessed Jun 9, 2022].
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007 Jan;29(9–10):855–71. https://doi.org/10.1080/01421590701775453 DOI: https://doi.org/10.1080/01421590701775453
Stiggins R. From formative assessment to assessment for learning: a path to success in standards-based schools. Phi Delta Kappan. 2005 Dec;87(4):324–8. https://doi.org/10.1177/003172170508700414 DOI: https://doi.org/10.1177/003172170508700414
Reschly AL, Pohl AJ, Christenson SL, SpringerLink. Student engagement effective academic, behavioral, cognitive, and affective interventions at school. 1st ed. 2020. Cham: Springer International Publishing; 2020. xix+331. https://doi.org/10.1007/978-3-030-37285-9 DOI: https://doi.org/10.1007/978-3-030-37285-9
Kahu ER. Framing student engagement in higher education. Studies Higher Educ. 2013 Jun;38(5):758–73. https://doi.org/10.1080/03075079.2011.598505 DOI: https://doi.org/10.1080/03075079.2011.598505
Fredricks JA, Blumenfeld PC, Paris AH. School engagement: potential of the concept, state of the evidence. review of educational research. 2004 Mar 1;74(1):59–109. https://doi.org/10.3102/00346543074001059 DOI: https://doi.org/10.3102/00346543074001059
Dijksterhuis MGK, Schuwirth LWT, Braat DDM, Teunissen PW, Scheele F. A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach. 2013 Aug;35(8):e1396–402. https://doi.org/10.3109/0142159X.2012.756576 DOI: https://doi.org/10.3109/0142159X.2012.756576
Barrett A, Galvin R, Scherpbier AJJA, Teunissen PW, O’Shaughnessy A, Horgan M. Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and experiences. Postgrad Med J. 2017 Mar;93(1097):138–42. https://doi.org/10.1136/postgradmedj-2015-133917 DOI: https://doi.org/10.1136/postgradmedj-2015-133917
Lörwald AC, Lahner FM, Nouns ZM, et al. The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: a systematic review and meta-analysis. Zhang Q, editor. PLoS ONE. 2018 Jun 4;13(6):1–15. https://dx.plos.org/10.1371/journal.pone.0198009 DOI: https://doi.org/10.1371/journal.pone.0198009
Spike N, Alexander H, Elliott S, et al. In-training assessment - its potential in enhancing clinical teaching. Med Educ. 2000 Oct;34(10):858–61. http://doi.wiley.com/10.1046/j.1365-2923.2000.00755.x DOI: https://doi.org/10.1046/j.1365-2923.2000.00755.x
Billett S. Toward a workplace pedagogy: guidance, participation, and engagement. Adult Educ Quart. 2002 Nov 1;53(1):27–43. https://doi.org/10.1177/074171302237202 DOI: https://doi.org/10.1177/074171302237202
Prentice S, Benson J, Kirkpatrick E, Schuwirth L. Workplace-based assessments in postgraduate medical education: a hermeneutic review. Med Educ. 2020 Nov;54(11):981–92. https://doi.org/10.1111/medu.14221 DOI: https://doi.org/10.1111/medu.14221
Andreassen P, Malling B. How are formative assessment methods used in the clinical setting? A qualitative study. Int J Med Educ. 2019 Nov 22;10:208–15. https://doi.org/10.5116/ijme.5db3.62e3 DOI: https://doi.org/10.5116/ijme.5db3.62e3
Ali J, Goh A. Student perceptions of workplace-based assessment. Clin Teach. 2017;14(5):319–24. https://doi.org/10.1111/tct.12588 DOI: https://doi.org/10.1111/tct.12588
Acai A, Li SA, Sherbino J, Chan TM. Attending emergency physicians’ perceptions of a programmatic workplace-based assessment system: the McMaster Modular Assessment Program (McMAP). Teach Learn Med. 2019 Aug 8;31(4):434–44. https://doi.org/10.1080/10401334.2019.1574581 DOI: https://doi.org/10.1080/10401334.2019.1574581
Harlen W, James M. Assessment and Learning: differences and relationships between formative and summative assessment. Assess Educ: Princ Policy Pract. 1997 Nov;4(3):365–79. https://doi.org/10.1080/0969594970040304 DOI: https://doi.org/10.1080/0969594970040304
Liang Y, Noble LM. Chinese doctors’ views on workplace-based assessment: trainee and supervisor perspectives of the mini-CEX. Med Educ Online. 2021;26(1):1869393. https://doi.org/10.1080/10872981.2020.1869393 DOI: https://doi.org/10.1080/10872981.2020.1869393
Jones A. Bridging the implementation gap: normalisation process theory in educational settings. 2017 Apr 15; Available from: https://www.researchgate.net/publication/316666078_Bridging_the_implementation_gap_normalization_process_theory_in_educational_settings
Massie J, Ali J. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv in Health Sci Educ. 2015 May 24;21(2):455–73. https://doi.org/10.1007/s10459-015-9614-0 DOI: https://doi.org/10.1007/s10459-015-9614-0
Persaud D, Cole J, Jainarine R, Khalid Z. Internal medicine residency program in Guyana: a collaborative model for sustainable graduate medical education in resource-limited settings. Front Public Health. 2017 May 29;5:112. https://doi.org/10.3389/fpubh.2017.00112 DOI: https://doi.org/10.3389/fpubh.2017.00112
Etheridge L, Boursicott K. Performance and workplace assessment. In: A practical guide for medical teachers. 4th ed. London; New York: Churchill Livingstone/Elsevier; 2013. p. 307–14. Available from: https://www.elsevier.com/books/a-practical-guide-for-medical-teachers/dent/978-0-7020-8170-5
McQueen SA, Petrisor B, Bhandari M, Fahim C, McKinnon V, Sonnadara RR. Examining the barriers to meaningful assessment and feedback in medical training. Amer J Surg. 2016 Feb;211(2):464–75. https://doi.org/10.1016/j.amjsurg.2015.10.002 DOI: https://doi.org/10.1016/j.amjsurg.2015.10.002
Husserl E. Ideas: general introduction to pure phenomenology. New York, NY: Routledge; 2012;434 p. Available from: https://www.routledge.com/Ideas-General-Introduction-to-Pure-Phenomenology/Husserl/p/book/9780415519038#:~:text=The%20publication%20of%20Ideas%20in,the%20essence%20of%20consciousness%22%20itself. [Accessed on Mar 22, 2023].
Vagle MD. Crafting phenomenological research. 2nd edition. New York: Routledge; 2018. 214 p. https://doi.org/10.4324/9781315173474 DOI: https://doi.org/10.4324/9781315173474
Streubert HS, Carpenter DR. Qualitative research in nursing: advancing the humanistic imperative. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2011.
Smith DW. Phenomenology. In: Zalta EN, editor. The Stanford encyclopedia of philosophy. 2018th ed. Stanford, California: Metaphysics Research Lab, Stanford University; 2018. Available from: https://plato.stanford.edu/archives/sum2018/entries/phenomenology/. [Accessed On Feb 18, 2021].
Englander M. The interview: data collection in descriptive phenomenological human scientific research. J Phenomenol Psychol. 2012;43(1):13–35. https://doi.org/10.1163/156916212X632943 DOI: https://doi.org/10.1163/156916212X632943
Peoples K. How to write a phenomenological dissertation: a step-by-step guide. 1 edition. Thousand Oaks, California: SAGE Publications, Inc; 2020. 192 p. Available from: https://us.sagepub.com/en-us/nam/how-to-write-a-phenomenological-dissertation/book261976
Varpio L, Ajjawi R, Monrouxe LV, O’Brien BC, Rees CE. Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking. Med Educ. 2017 Jan;51(1):40–50. https://doi.org/10.1111/medu.13124 DOI: https://doi.org/10.1111/medu.13124
Zoom Video Communications, Inc. Zoom. San Jose, California; 2021. Available from: https://zoom.us/
Fleming J. Recognizing and resolving the challenges of being an insider researcher in work-integrated learning. International J Work-Integ Learn. 2018;19(3):311–20. Available from: https://files.eric.ed.gov/fulltext/EJ1196753.pdf [Accessed on Mar 22, 2023].
Hycner RH. Some guidelines for the phenomenological analysis of interview data. Human Studies. 1985;8:279–303. https://doi.org/10.1007/BF00142995 DOI: https://doi.org/10.1007/BF00142995
Ortlipp M. Keeping and using reflective journals in the qualitative research process. The Qualitative Report. 2008 Dec 1;13(4):695–705. Available from: https://nsuworks.nova.edu/tqr/vol13/iss4/8
Javadi M, Zarea K. Understanding thematic analysis and its pitfall. J Clie Care. 2016;1(1):34–40. https://doi.org/10.15412/J.JCC.02010107 DOI: https://doi.org/10.15412/J.JCC.02010107
Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ. 2008 Aug 7;337:a1035. https://doi.org/10.1136/bmj.a1035 DOI: https://doi.org/10.1136/bmj.a1035
Cohen L, Manion L, Morrison K. Research methods in education. 8th edition. London ; New York: Routledge; 2017. 944 p. Available from: https://www.routledge.com/Research-Methods-in-Education/Cohen-Manion-Morrison/p/book/9781138209886 DOI: https://doi.org/10.4324/9781315456539
May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. SAGE Publications. 2009;43(3):535–54. https://doi.org/10.1177/0038038509103208 DOI: https://doi.org/10.1177/0038038509103208
McNaughton RJ, Steven A, Shucksmith J. Using normalization process theory as a practical tool across the life course of a qualitative research project. Qual Health Res. 2020 Jan 1;30(2):217–27. https://doi.org/10.1177/1049732319863420 DOI: https://doi.org/10.1177/1049732319863420
Wood P. Overcoming the problem of embedding change in educational organizations: a perspective from normalization process theory. Manag Educ. 2017 Jan;31(1):33–8 https://doi.org/10.1177/0892020616685286 DOI: https://doi.org/10.1177/0892020616685286
Harris P, Bhanji F, Topps M, et al. Evolving concepts of assessment in a competency-based world. Med Teach. 2017 Jun;39(6):603–8. https://doi.org/10.1080/0142159X.2017.1315071 DOI: https://doi.org/10.1080/0142159X.2017.1315071
Konopasek L, Norcini J, Krupat E. Focusing on the formative: building an assessment system aimed at student growth and development. Acad Med. 2016 Nov;91(11):1492–7. https://doi.org/10.1097/ACM.0000000000001171 DOI: https://doi.org/10.1097/ACM.0000000000001171
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Melissa Mc Donald, Fiona Muir
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
