Licensing exams in Canada: a closer look at the validity of the MCCQE Part II

Authors

  • Alina Smirnova University of Calgary

DOI:

https://doi.org/10.36834/cmej.73894

Abstract

The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.

Metrics

Metrics Loading ...

References

Kennedy B. The Part II examination: political exercise or national standard? Cmaj. 1995;152(8):1183-1184. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf

Lougheed T. Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e87-88. https://doi.org/10.36834/cmej.36630

Benusic M. Should the MCCQE II exams go ahead? In: CMAJ Blogs; 2019. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf

Bowmer MI. Response to: Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e89-91.

https://doi.org/10.36834/cmej.36722

Topps M. Why the MCC qualifying examination part II still matters. In: CMAJ Blogs; 2019. https://cmajblogs.com/why-the-mcc-qualifying-examination-part-ii-still-matters/

Sir Thomas Roddick: his work in medicine and public life. JAMA. 1940;115(9):802-803. https://doi.org/10.1001/jama.1940.02810350146035

Reda J. The new LMCC. Cmaj. 1992;146(1):10-11. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1488208/pdf/cmaj00290-0012a.pdf

Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. Jama. 2007;298(9):993-1001. https://doi.org/10.1001/jama.298.9.993

Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med teach. 2018;40(11):1102-1109. https://doi.org/10.1080/0142159X.2018.1500016

Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560-575 https://doi.org/10.1111/medu.12678

Kane M. Validating score interpretations and uses. Language Testing. 2012;29(1):3-17. https://doi.org/10.1177/0265532211417210

Bachman LF. Fundamental considerations in language testing. Oxford, England: Oxford University Press; 1990.

Boulet JR. Establishing the validity of licensing examination scores. J Grad Med Educ. 2019;11(5):527-529. https://doi.org/10.4300/JGME-D-19-00611.1

Burdick WP, Boulet JR, LeBlanc KE. Can we increase the value and decrease the cost of clinical skills assessment? Acad Med. 2018;93(5):690-692. https://doi.org/10.1097/ACM.0000000000001867

Tamblyn RA M, Bartlett G, Winslade N, et al. The Quebec-Ontario follow-up study of the association between scores achieved on the MCCQE Part II examination and performance in clinical practice. Online: Medical Council of Canada; 2009. https://h5a9c8a9.stackpathcdn.com/media/Tamblyn_Score-Association_MCCQE-Part-II_Clinical-Practice-Performance_2009.pdf

De Champlain AQ, Tian F, Ashworth N, Kain N, Wiebe D. Do national licensing examination scores predict patient complaints as well as physician Opioid and Benzodiazepine prescribing patterns? Online: Medical Council of Canada; 2018. https://h5a9c8a9.stackpathcdn.com/media/IAMRA-2018Poster-A.DeChamplain.pdf

Wenghofer E, Klass D, Abrahamowicz M, et al. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ. 2009;43(12):1166-1173. https://doi.org/10.1111/j.1365-2923.2009.03534.x

Bordage G, Meguerditchian AN, Tamblyn R. Practice indicators of suboptimal care and avoidable adverse events: a content analysis of a national qualifying examination. Acad Med. 2013;88(10):1493-1498. https://doi.org/10.1097/acm.0b013e3182a356af

McKendry RJ, Busing N, Dauphinee DW, Brailovsky CA, Boulais AP. Does the site of postgraduate family medicine training predict performance on summative examinations? A comparison of urban and remote programs. Cmaj. 2000;163(6):708-711. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC80166/pdf/20000919s00013p708.pdf

Dore KL, Reiter HI, Kreuger S, Norman GR. CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores. Adv Health Sci Educ Theory Pract. 2017;22(2):327-336. https://doi.org/10.1007/s10459-016-9739-9

Eva KW, Reiter HI, Rosenfeld J, Trinh K, Wood TJ, Norman GR. Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. Jama. 2012;308(21):2233-2240. https://doi.org/10.1001/jama.2012.36914

Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14(1):34-42. https://doi.org/10.1207/S15328015TLM1401_9

Medical Council of Canada. Annual Report 2019-2020. Ottawa; 2020. https://h5a9c8a9.stackpathcdn.com/media/MCC-Annual-Report-2019-2020.pdf

Medical Council of Canada. MCCQE Part II Annual Technical Report. Ottawa; 2019. https://h5a9c8a9.stackpathcdn.com/media/MCCQE-Part-II-Annual-Technical-Report-2019.pdf

Norcini JJ, Boulet JR, Opalek A, Dauphinee WD. The relationship between licensing examination performance and the outcomes of care by international medical school graduates. Acad Med. 2014;89(8):1157-1162. https://doi.org/10.1097/acm.0000000000000310

Tatem GB, Gardner-Gray J, Standifer B, Alexander K. While you don't see color, i see bias: identifying barriers in access to graduate medical education training. ATS Sch. 2021;2(4):544-555. https://doi.org/10.34197/ats-scholar.2020-0134PS

Low D, Pollack SW, Liao ZC, et al. Racial/Ethnic Disparities in Clinical Grading in Medical School. Teach Learn Med. 2019;31(5):487-496. https://doi.org/10.1080/10401334.2019.1597724

MacFarlane MM. When a Canadian is not a Canadian: marginalization of IMGs in the CaRMS match. Can Med Educ J. 2021;12(4):132-140. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8463232/pdf/CMEJ-12-132.pdf

Weggemans MM, van Dijk B, van Dooijeweert B, Veenendaal AG, Ten Cate O. The postgraduate medical education pathway: an international comparison. GMS J Med Educ. 2017;34(5):Doc63. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5704606/pdf/JME-34-63.pdf

Harden RM. Five myths and the case against a European or national licensing examination. Med Teach. 2009;31(3):217-220. https://doi.org/10.1080/01421590902741155

Archer J, Lynn N, Coombes L, et al. The impact of large scale licensing examinations in highly developed countries: a systematic review. BMC Med Educ. 2016;16(1):212. https://doi.org/10.1186/s12909-016-0729-7

Babla K, Crampton P, Kronfli M. National licensing examinations: what are they good for? Clin Teach. 2020;17(3):323-325. https://doi.org/10.1111/tct.13083

Schuwirth L. National licensing examinations, not without dilemmas. Med Educ. 2016;50(1):15-17. https://doi.org/10.1111/medu.12891

Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609-616. https://doi.org/10.1080/0142159x.2017.1315082

Karay Y, Schauber SK. A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Med Teach. 2018;40(11):1123-1129. https://doi.org/10.1080/0142159x.2018.1472370

Downloads

Published

2022-06-07

How to Cite

1.
Smirnova A. Licensing exams in Canada: a closer look at the validity of the MCCQE Part II. Can. Med. Ed. J [Internet]. 2022 Jun. 7 [cited 2024 Nov. 5];13(4):23-9. Available from: https://journalhosting.ucalgary.ca/index.php/cmej/article/view/73894

Issue

Section

Commissioned Scientific Reports