Multisource feedback: an overview of its use and application as a formative assessment
DOI:
https://doi.org/10.36834/cmej.73775Abstract
Multisource feedback (MSF), often termed 360-degree feedback, is a formative performance assessment in which data about an individual’s observable workplace behaviors are collected through questionnaires from those interacting with the individual; data are aggregated for anonymity and confidentiality; the aggregated data, along with self-assessment if available, are provided to the individual; and the recipient meets with a trusted individual to review the data and develop an action plan. It is used along the continuum of medical education. This article provides an overview of MSF’s utility, its evidence base and cautions.
Metrics
References
Lockyer JM, Sargeant J, Introduction to MCC 360: a multi-source feedback initiative. Prepared for Medical Council of Canada, Ottawa, Canada, Nov 17, 2017. https://mcc.ca/media/MCC-360-MSF-Guidelines.pdf [Accessed Oct 12, 2021]
Lockyer J, Multi Source Feedback, in ES Holmboe, RE Hawkins, SJ Durning (eds), A practical guide to the assessment of clinical competence, 2nd Ed. Elsevier. 2018. Paperback ISBN: 9780323447348; eBook ISBN: 9780323448949. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
Kane MT. Validating the Interpretations and Uses of Test Scores. J Educ Meas. 2013; 50: 1-73. https://doi.org/10.1111/jedm.12000
Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560-75. https://doi.org/10.1111/medu.12678.
Moonen-van Loon JM, Overeem K, Donkers HH, van der Vleuten CP, Driessen EW. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 2013;18(5):1087-102. https://doi.org/10.1007/s10459-013-9450-z.
Donnon T, Al Ansari A, Al Alawi S, Violato C. The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review. Acad Med. 2014;89(3):511-6. https://doi.org/10.1097/ACM.0000000000000147.
Stevens S, Read J, Baines R, Chatterjee A, Archer J. Validation of multisource feedback in assessing medical performance: a systematic review. J Contin Educ Health Prof. 2018;38(4):262-268. https://doi.org/10.1097/CEH.0000000000000219.
Lockyer J. Multisource feedback: can it meet criteria for good assessment? J Contin Educ Health Prof. 2013;33(2):89-98. https://doi.org/10.1002/chp.21171.
Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med Teach. 2018;40(11):1102-1109. https://doi.org/10.1080/0142159X.2018.1500016.
Francois J, Sisler J, Mowat S. Peer-assisted debriefing of multisource feedback: an exploratory qualitative study. BMC Med Educ. 2018;18(1):36. https://doi.org/10.1186/s12909-018-1137-y.
Pooley M, Pizzuti C, Daly M. Optimizing multisource feedback implementation for Australasian physicians. J Contin Educ Health Prof. 2019;39(4):228-235. https://doi.org/10.1097/CEH.0000000000000267.
Arabsky S, Castro N, Murray M, Bisca I, Eva KW. The influence of relationship-centered coaching on physician perceptions of peer review in the context of mandated regulatory practices. Acad Med. 2020; 95 (11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S14-S19. https://doi.org/10.1097/ACM.0000000000003642.
Hu J, Lee R, Mullin S, Schwaitzberg S, Harmon L, Gregory P, Elkin PL. How physicians change: multisource feedback driven intervention improves physician leadership and teamwork. Surgery. 2020;168(4):714-723. https://doi.org/10.1016/j.surg.2020.06.008.
Sargeant J, Lockyer J, Mann K, et al. facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015; 90(12):1698-706. Https://doi.org/10.1097/ACM.0000000000000809.
van der Meulen MW, Boerebach BC, Smirnova A et al. Validation of the INCEPT: a multisource feedback tool for capturing different perspectives on physicians' professional performance. J Contin Educ Health Prof. 2017;37(1):9-18. https://doi.org/10.1097/CEH.0000000000000143.
House JB, Franko LR, Haque F, Cranfod JA, Santen SA, Variation in assessment of first-year medical students' interprofessional competencies by rater profession, J. Interprof. Educ. Pract. 2021 (24); 100424. ISSN 2405-4526 https://doi.org/10.1016/j.xjep.2021.100424
Lai MMY, Roberts N, Mohebbi M, Martin J. A randomised controlled trial of feedback to improve patient satisfaction and consultation skills in medical students. BMC Med Educ. 2020;20(1):277. https://doi.org/10.1186/s12909-020-02171-9.
Emke AR, Cheng S, Chen L, Tian D, Dufault C. A Novel Approach to Assessing Professionalism in Preclinical Medical Students Using Multisource Feedback Through Paired Self- and Peer Evaluations. Teach Learn Med. 2017;29(4):402-410. https://doi.org/10.1080/10401334.2017.1306446. Epub 2017 May 12. PMID: 28498003.
Prediger S, Fürstenberg S, Berberat PO, Kadmon M, Harendza S. Interprofessional assessment of medical students’ competences with an instrument suitable for physicians and nurses. BMC Med Educ. 2019;19(1):46. Https://doi.org/10.1186/s12909-019-1473-6. PMID: 30728006; PMCID: PMC6364398.
Prediger S, Schick K, Fincke F, Fürstenberg S, Oubaid V, Kadmon M, Berberat PO, Harendza S. Validation of a competence-based assessment of medical students’ performance in the physician’s role. BMC Med Educ. 2020;20(1):6. Https://doi.org/10.1186/s12909-019-1919-x. PMID: 31910843; PMCID: PMC6947905.
Lockyer JM, Violato C, Fidler H. The assessment of emergency physicians by a regulatory authority. Acad Emerg Med. 2006;13(12):1296-303. Https://doi.org/10.1197/j.aem.2006.07.030. Epub 2006 Nov 10. PMID: 17099191.
Lockyer JM, Violato C, Fidler H, Alakija P. The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool. Arch Pathol Lab Med. 2009;133(8):1301-8. Https://doi.org/10.5858/133.8.1301.
Holmboe ES, Durning SJ, Hawkins RE, Practical guide to the evaluation of clinical competence 2nd ed, Philadelphia: Elsevier 2018. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
Gregory PJ, Robbins B, Schwaitzberg SD, Harmon L. Leadership development in a professional medical society using 360-degree survey feedback to assess emotional intelligence. Surg Endosc. 2017;31(9):3565-3573. https://doi.org/10.1007/s00464-016-5386-8.
Rourke J, Bornstein S, Vardy C, Speed D, White T, Corbett P. Evaluation of and Feedback for Academic Medicine Leaders: Developing and Implementing the Memorial Method. Acad Med. 2017;92(11):1590-1594. https://doi.org/10.1097/ACM.0000000000001722.
Desmedt M, Ulenaers D, Grosemans J, Hellings J, Bergs J. Clinical handover and handoff in healthcare: a systematic review of systematic reviews. Int J Qual Health Care. 2021;33(1):mzaa170. https://doi.org/10.1093/intqhc/mzaa170
Bonds RL. SBAR Tool Implementation to advance communication, teamwork, and the perception of patient safety culture. Creative nursing, 2018; 24(2), 116–123. https://doi-org.ezproxy.lib.ucalgary.ca/10.1891/1078-4535.24.2.116
Etherington C, Wu M, Cheng-Boivin O, Larrigan S, Boet S. Interprofessional communication in the operating room: a narrative review to advance research and practice. Communication interprofessionnelle en salle d’opération: un compte rendu narratif pour faire avancer la recherche et la pratique. Can J Anaesth. 2019;66(10):1251-1260. https://doi.org/10.1007/s12630-019-01413-9
Chladek MS, Doughty C, Patel B et al. The standardisation of handoffs in a large academic paediatric emergency department using I-PASS. BMJ Open Qual. 2021;10(3):e001254. https://doi.org/10.1136/bmjoq-2020-001254.
Narayanan A, Farmer EA, Greco MJ. Multisource feedback as part of the medical board of Australia's professional performance framework: outcomes from a preliminary study. BMC Med Educ. 2018;18(1):323. https://doi.org/10.1186/s12909-018-1432-7. Erratum in: BMC Med Educ. 2019 Mar 4;19(1):73.
Medical Council of Canada, MCC 360, https://mcc.ca/assessments/mcc360/ [Accessed Feb 15, 2022].
360 PULSE Program: quality at work https://pulseprogram.com. [Accessed Feb 15, 2022].
Saedon H, Salleh S, Balakrishnan A, Imray CH, Saedon M. The role of feedback in improving the effectiveness of workplace based assessments: a systematic review. BMC Med Educ. 2012; 12:25. https://doi.org/10.1186/1472-6920-12-25.
Alofs L, Huiskes J, Heineman MJ, et al. User reception of a simple online multisource feedback tool for residents. Perspect Med Educ. 2015 (2):57-65. https://doi.org/10.1007/s40037-015-0173-0.
Yama BA, Hodgins M, Boydell K, Schwartz SB. A qualitative exploration: questioning multisource feedback in residency education. BMC Med Educ. 2018;18(1):170. https://doi.org/10.1186/s12909-018-1270-7.
G, Bandiera G. Multisource feedback and self-assessment of the communicator, collaborator, and professional CanMEDS roles for diagnostic radiology residents. Can Assoc Radiol J. 2014;65(4):379-84. https://doi.org/10.1016/j.carj.2014.04.003.
Beaudoin PL, Labbé M, Fanous A, et al. Teaching communication skills to OTL-HNS residents: multisource feedback and simulated scenarios. J Otolaryngol Head Neck Surg. 2019;48(1):8. https://doi.org/10.1186/s40463-019-0329-8.
Castonguay V, Lavoie P, Karazivan P, Morris J, Gagnon R. Perceptions of emergency medicine residents of multisource feedback: different, relevant, and useful information. Ann Emerg Med. 2019;74(5):660-669. https://doi.org/10.1016/j.annemergmed.2019.05.019.
Buis CAM, Eckenhausen MAW, Ten Cate O. Processing multisource feedback during residency under the guidance of a non-medical coach. Int J Med Educ. 2018; 23;9:48-54. https://doi.org/10.5116/ijme.5a7f.169d.
Wenghofer EF, Campbell C, Marlow B, Kam SM, Carter L, McCauley W. The effect of continuing professional development on public complaints: a case-control study. Med Educ. 2015;49(3):264-75. https://doi.org/10.1111/medu.12633.
Chaulk D, Krueger C, Stang AS. A retrospective review of physician-related patient complaints from a tertiary pediatric hospital. Pediatr Qual Saf. 2019 13;4(1):e136. https://doi.org/10.1097/pq9.0000000000000136.
Saha R, Kabanovski A, Klejman S, Margolin E, Buys YM. Patients' complaints involving ophthalmologists in the province of Ontario, Canada: a 5-year review. Can J Ophthalmol. 2020;55(3 Suppl 1):22-26. https://doi.org/10.1016/j.jcjo.2019.08.007.
Brennan N, Bryce M, Pearson M, Wong G, Cooper C, Archer J. Towards an understanding of how appraisal of doctors produces its effects: a realist review. Med Educ. 2017 Oct;51(10):1002-1013. Https://doi.org/10.1111/medu.13348.
Lockyer J, Sargeant J, Implementing multisource feedback, in D Boud and E Molloy (Eds), Feedback in Higher and Professional Education, London, Routledge, 2013. ISBN: 9781135107468. https://www.routledge.com/Feedback-in-Higher-and-Professional-Education-Understanding-it-and-doing/Boud-Molloy/p/book/9780415692298
Moonen–van Loon JM, Overeem K, Govaerts MJ, Verhoeven BH, van der Vleuten CP, Driessen EW. The reliability of multisource feedback in competency-based assessment programs. Acad Med, 2015; 90 (8), 1093-1099. Https://doi.org/10.1097/ACM.0000000000000763.
Al Khalifa K, Al Ansari A, Violato C, Donnon T. Multisource feedback to assess surgical practice: a systematic review. J surg educ, 2013; 70(4), 475–486. https://doi-org.ezproxy.lib.ucalgary.ca/10.1016/j.jsurg.2013.02.002
Andrews JJ, Violato C, Al Ansari A, Donnon T, Pugliese G. Assessing psychologists in practice: Lessons from the health professions using multisource feedback. Professional psychology: research and practice. 2013;44(4):193.
Sargeant JM, Holmboe ES, Feedback and coaching in clinical teaching and learning. ES Holmboe, RE Hawkins, SJ Durning (eds), A practical guide to the assessment of clinical competence, 2nd Ed. Elsevier. 2018. Paperback ISBN: 9780323447348; eBook ISBN: 9780323448949. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
Ashworth N, de Champlain AF, Kain N, A review of multi-source feedback focusing on psychometrics, pitfalls and some possible solutions, Springer Nature Social Sciences, 2021 I:24. https://link.springer.com/article/10.1007/s43545-020-00033-1
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Jocelyn Lockyer, Joan Sargeant
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
