A critical analysis of The Royal College of Physicians and Surgeons of Canada examination experience
DOI:
https://doi.org/10.36834/cmej.79678Abstract
The Royal College of Physicians and Surgeons of Canada (RCPSC) plays a leading role in specialty and subspecialty post graduate medical education (PGME) in Canada. As the RCPSC accredits PGME programs, these programs are structured to meet the RCPSC Competence by Design model and their CanMEDS roles. RCPSC Certification is required by Medical Regulatory Agencies (MRAs) across Canada as a condition of entry to independent practice. The RCPSC relies heavily on the use of high-stakes subject examinations as a key component of its Certification process. Recently, questions have been raised regarding the usefulness of such high-stakes examinations. If such examinations are to be fair and equitable, they must be designed and implemented in accordance with best practices for educational testing and the processes for implementation and grading must be transparent and fair. This paper reviews the recent literature on high-stakes examinations and best practices in examination construction, references the findings of a survey of RCPSC examination experiences conducted by the Society for Canadians Studying Medicine Abroad exploring the perception of respondents, and raises concerns regarding RCPSC examinations related to validity, reliability, and fairness. The paper concludes by recommending closer scrutiny of RCPSC examination processes by interested stakeholders and by provincial MRA's who delegate entry to practice decisions to the RCPSC.
References
1. Royal College of Physicians and Surgeons of Canada. About/History. Available from https://www.royalcollege.ca/en/about/history [Accessed Jul 5, 2024]
2. Royal College of Physicians and Surgeons of Canada. Competence by Design. Available from https://www.collegeroyal.ca/en/standards-and-accreditation/competence-by-design.html [Accessed Jul 5, 2024]
3. Royal College of Physicians and Surgeons of Canada. Standards documents. Available from https://news.royalcollege.ca/en/standards-and-accreditation/pgme-accreditation [Accessed Jul 5, 2024.
4. American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. Standards for psychological and educational testing. 2014. Available from https://www.testingstandards.net/uploads/7/6/6/4/76643089/standards_2014edition.pdf [Accessed Jul 5, 2024]
5. Tavakol M, Stewart C, Sharpe, CC. Ensuring fairness in assessment in health professions education: rapid analysis tools to detect differential item functioning across groups. IJME, 15:80-83 https://doi.org/10.5116/ijme.6694.de69.
6. Medical Council of Canada. MCCQE Part 1. Available from https://mcc.ca/examinations-assessments/mccqe-part-i/ [Accessed Jul 5, 2024]
7. Medical Council of Canada. MCCQE Part 1 annual technical report 2020. Available from https://mcc.ca/wp-content/uploads/MCCQE-Part-I_Technical-Report_2020.pdf [Accessed Jul 5, 2024]
8. Medical Council of Canada. Preparatory products. Available from https://mcc.ca/examinations-assessments/resources-to-help-with-exam-prep/preparatory-products/ [Accessed Jul 5, 2024]
9. Canada Q Bank. QBank for the MCCQE Part 1. Available from https://www.canadaqbank.com/mccqe-part1.php [Accessed Jul 5, 2024]
10. Royal College of Physicians and Surgeons of Canada. Developing multiple choice questions for the RCPSC certification examinations. 2004. Available from https://cmapsconverted.ihmc.us/rid=1P2XTLCSV-19DYZD-BDK/Wood_et_coll_2004_RCPSC_MCQ_guidelines.pdf [Accessed Jul 6, 2024]
11. Royal College of Physicians and Surgeons of Canada. Roles and responsibilities of examination boards and specialty committees with regards to examinations. Available from https://www.royalcollege.ca/content/dam/documents/accreditation/specialties/member-resources/specialty-committee/roles-responsibilities-of-examination-specialty-committees-regards-to-examinations-e.pdf [Accessed Jul 6, 2024]
12. Royal College of Physicians and Surgeons of Canada. Average pass rates by specialty or subspecialty. Available from https://www.collegeroyal.ca/en/eligibility-and-exams/exam-results/average-pass-rates [Accessed Jul 6, 2024]
13. Prat, S. Who is monitoring the gatekeepers? The Royal College an impenetrable fortress. Oct 23, 2022. Available from https://healthydebate.ca/2022/10/topic/royal-college-exam/ [Accessed Jul 6, 2024]
14. Royal College of Physicians and Surgeons of Canada. Policy on formal review of examinations Available from file:///C:/Users/Owner/OneDrive/Documents/Policy%20on%20Formal%20Reviews%20and%20Appendix.pdf [Accessed Jul 6, 2024]
15. Society for Canadians Studying Medicine Abroad (SOCASMA). Royal college of physicians and surgeons exam experiences survey summary. Available from https://socasma.com/wp-content/uploads/2023/11/RCPSC-EXAM-EXPERIENCES-SURVEY-SUMMARY.pdf [Accessed Jul 6, 2024]
16. Government of Canada. Panel on research ethics. Available from https://ethics.gc.ca/eng/tcps2-eptc2_2022_chapter2-chapitre2.html [Accessed Jul 6, 2024]
17. Royal College of Physicians and Surgeons of Canada. Exam process overview. Available from https://www.royalcollege.ca/en/eligibility-and-exams/exam-process [Accessed Jul 9, 2024]
18. Blew P, Muir JG, Naik VN. The evolving Royal College examination in anesthesiology. Can J Anesthes. 57, 2010, Pg 804-810. https://doi.org/10.1007/s12630-010-9341-1
19. Mosaic. Using disaggregated data to address the systemic discrimination experienced by international medical graduates. May 2024. Available from https://mosaicbc.org/international-medical-graduates/ [Accessed Nov 18, 2024]
20. Kutcher S, Ravalia M, Omidvar R, El-Assal K. Maximizing medical talent: how Canada can increase the supply of family doctors by 50% quickly and cost effectively. Senate Report, Oct 2024. Available from https://www.ratnaomidvar.ca/wp-content/uploads/2024/10/IMG-Report-Design-EN.pdf [Accessed Nov 18, 2024]
21. MacFarlane, M. When a Canadian is not a Canadian: Marginalization of IMGs in the CaRMS Match. Can Med Ed J. 12(4), 2021, Pg 132-140. https://doi.org/10.36834/cmej.71790
22. Royal College of Physicians and Surgeons of Canada. Information by discipline. Available from https://www.royalcollege.ca/en/standards-and-accreditation/information-by-discipline [Accessed Jul 6, 2024]
23. American Board of Medical Specialties. Available from https://www.abms.org/ [Accessed Jul 6, 2024]
24. Royal College of Physicians and Surgeons of Canada. Assessment and exam fees. Available from https://www.royalcollege.ca/en/eligibility-and-exams/exam-process/assessment-and-exam-fees#assessment [Accessed Jul 6, 2024]
25. Royal College of Physicians and Surgeons of Canada. Prepare for your Royal College Exams. Available from https://www.collegeroyal.ca/en/eligibility-and-exams/exam-preparation [Accessed Jul 6, 2034]
26. Thoma B, Monteiro S, Pardhan A, et al. Replacing high-stakes summative examinations with graduated medical licensure in Canada. Can Med Assoc J. 194(5) 2022. https://doi.org/10.1503/cmaj.211816
27. Royal College of Physicians and Surgeons of Canada. What are we doing about Canada’s health human resource crisis? Available from https://news.royalcollege.ca/en/newsroom/posts/what-are-we-doing-about-canada-s-health-human-resource-crisis-.html [Accessed Jul 6, 2024]
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Malcolm M MacFarlane
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
