Comparison of Student Performance on Internally Prepared Clerkship Examinations and NBME Subject Examinations
Background: This pilot study compared performance of University of Calgary students on internal clerkship examinations with corresponding National Board of Medical Examiners (NBME) subject examinations.
Methods: Between April and October 2007, students completed internal and NBME subject examinations following six mandatory rotations. Local faculty within each discipline set the minimum performance level (MPL) for internal examinations. Two methods of standard setting were considered for NBME exams and a sensitivity analysis was performed. Corresponding internal and NBME examination scores were compared using McNemar’s discordant pair analysis.
Results: A significant and unexpected difference in failure rate between internal and external examinations was found in all clerkships. 1.4% of students were below the MPL for internal examinations and 27.3% (modified Angoff) or 25.9% (mean Hofstee compromise) (p<0.0001 for both) for the NBME. The proportion of students below MPL for internal examinations was also below the lower limit of the Hofstee compromise (14.4%).
Conclusion: Possible explanations include leniency bias in internal standard setting, discrepant content validity between local curriculum and NBME examinations, difference in student perception of examinations, and performance bias due to unfamiliar units.
How to Cite
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.