Licensing exams in Canada: a closer look at the validity of the MCCQE Part II
DOI:
https://doi.org/10.36834/cmej.73894Abstract
The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
References
Kennedy B. The Part II examination: political exercise or national standard? Cmaj. 1995;152(8):1183-1184. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf
Lougheed T. Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e87-88. https://doi.org/10.36834/cmej.36630
Benusic M. Should the MCCQE II exams go ahead? In: CMAJ Blogs; 2019. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf
Bowmer MI. Response to: Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e89-91.
https://doi.org/10.36834/cmej.36722
Topps M. Why the MCC qualifying examination part II still matters. In: CMAJ Blogs; 2019. https://cmajblogs.com/why-the-mcc-qualifying-examination-part-ii-still-matters/
Sir Thomas Roddick: his work in medicine and public life. JAMA. 1940;115(9):802-803. https://doi.org/10.1001/jama.1940.02810350146035
Reda J. The new LMCC. Cmaj. 1992;146(1):10-11. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1488208/pdf/cmaj00290-0012a.pdf
Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. Jama. 2007;298(9):993-1001. https://doi.org/10.1001/jama.298.9.993
Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med teach. 2018;40(11):1102-1109. https://doi.org/10.1080/0142159X.2018.1500016
Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560-575 https://doi.org/10.1111/medu.12678
Kane M. Validating score interpretations and uses. Language Testing. 2012;29(1):3-17. https://doi.org/10.1177/0265532211417210
Bachman LF. Fundamental considerations in language testing. Oxford, England: Oxford University Press; 1990.
Boulet JR. Establishing the validity of licensing examination scores. J Grad Med Educ. 2019;11(5):527-529. https://doi.org/10.4300/JGME-D-19-00611.1
Burdick WP, Boulet JR, LeBlanc KE. Can we increase the value and decrease the cost of clinical skills assessment? Acad Med. 2018;93(5):690-692. https://doi.org/10.1097/ACM.0000000000001867
Tamblyn RA M, Bartlett G, Winslade N, et al. The Quebec-Ontario follow-up study of the association between scores achieved on the MCCQE Part II examination and performance in clinical practice. Online: Medical Council of Canada; 2009. https://h5a9c8a9.stackpathcdn.com/media/Tamblyn_Score-Association_MCCQE-Part-II_Clinical-Practice-Performance_2009.pdf
De Champlain AQ, Tian F, Ashworth N, Kain N, Wiebe D. Do national licensing examination scores predict patient complaints as well as physician Opioid and Benzodiazepine prescribing patterns? Online: Medical Council of Canada; 2018. https://h5a9c8a9.stackpathcdn.com/media/IAMRA-2018Poster-A.DeChamplain.pdf
Wenghofer E, Klass D, Abrahamowicz M, et al. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ. 2009;43(12):1166-1173. https://doi.org/10.1111/j.1365-2923.2009.03534.x
Bordage G, Meguerditchian AN, Tamblyn R. Practice indicators of suboptimal care and avoidable adverse events: a content analysis of a national qualifying examination. Acad Med. 2013;88(10):1493-1498. https://doi.org/10.1097/acm.0b013e3182a356af
McKendry RJ, Busing N, Dauphinee DW, Brailovsky CA, Boulais AP. Does the site of postgraduate family medicine training predict performance on summative examinations? A comparison of urban and remote programs. Cmaj. 2000;163(6):708-711. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC80166/pdf/20000919s00013p708.pdf
Dore KL, Reiter HI, Kreuger S, Norman GR. CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores. Adv Health Sci Educ Theory Pract. 2017;22(2):327-336. https://doi.org/10.1007/s10459-016-9739-9
Eva KW, Reiter HI, Rosenfeld J, Trinh K, Wood TJ, Norman GR. Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. Jama. 2012;308(21):2233-2240. https://doi.org/10.1001/jama.2012.36914
Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14(1):34-42. https://doi.org/10.1207/S15328015TLM1401_9
Medical Council of Canada. Annual Report 2019-2020. Ottawa; 2020. https://h5a9c8a9.stackpathcdn.com/media/MCC-Annual-Report-2019-2020.pdf
Medical Council of Canada. MCCQE Part II Annual Technical Report. Ottawa; 2019. https://h5a9c8a9.stackpathcdn.com/media/MCCQE-Part-II-Annual-Technical-Report-2019.pdf
Norcini JJ, Boulet JR, Opalek A, Dauphinee WD. The relationship between licensing examination performance and the outcomes of care by international medical school graduates. Acad Med. 2014;89(8):1157-1162. https://doi.org/10.1097/acm.0000000000000310
Tatem GB, Gardner-Gray J, Standifer B, Alexander K. While you don't see color, i see bias: identifying barriers in access to graduate medical education training. ATS Sch. 2021;2(4):544-555. https://doi.org/10.34197/ats-scholar.2020-0134PS
Low D, Pollack SW, Liao ZC, et al. Racial/Ethnic Disparities in Clinical Grading in Medical School. Teach Learn Med. 2019;31(5):487-496. https://doi.org/10.1080/10401334.2019.1597724
MacFarlane MM. When a Canadian is not a Canadian: marginalization of IMGs in the CaRMS match. Can Med Educ J. 2021;12(4):132-140. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8463232/pdf/CMEJ-12-132.pdf
Weggemans MM, van Dijk B, van Dooijeweert B, Veenendaal AG, Ten Cate O. The postgraduate medical education pathway: an international comparison. GMS J Med Educ. 2017;34(5):Doc63. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5704606/pdf/JME-34-63.pdf
Harden RM. Five myths and the case against a European or national licensing examination. Med Teach. 2009;31(3):217-220. https://doi.org/10.1080/01421590902741155
Archer J, Lynn N, Coombes L, et al. The impact of large scale licensing examinations in highly developed countries: a systematic review. BMC Med Educ. 2016;16(1):212. https://doi.org/10.1186/s12909-016-0729-7
Babla K, Crampton P, Kronfli M. National licensing examinations: what are they good for? Clin Teach. 2020;17(3):323-325. https://doi.org/10.1111/tct.13083
Schuwirth L. National licensing examinations, not without dilemmas. Med Educ. 2016;50(1):15-17. https://doi.org/10.1111/medu.12891
Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609-616. https://doi.org/10.1080/0142159x.2017.1315082
Karay Y, Schauber SK. A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Med Teach. 2018;40(11):1123-1129. https://doi.org/10.1080/0142159x.2018.1472370
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Alina Smirnova
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
