The development of two Point of Care Ultrasound stations for Objective Structured Clinical Examinations in undergraduate medical education
DOI :
https://doi.org/10.36834/cmej.81229Résumé
Introduction: Point-of-care ultrasound (POCUS) is a valuable clinical skill that improves clinical care but requires substantial training. Validated assessment tools provide empirical evidence regarding trainee performance while also informing program-level evaluation. We developed two POCUS-specific stations for objective structured clinical examinations (OSCEs) to assess skill acquisition and inform best practices in undergraduate medical education.
Methods: A multidisciplinary group of POCUS educators identified two POCUS applications (pleural effusion and abdominal free fluid) well suited for the undergraduate level. A modified Delphi approach was used to develop POCUS-application-specific skill checklists and global rating scale. Two medical programs piloted the stations to inform reliability.
Results: Across two sites, 46 and 41 students participated in the pleural effusion and abdominal free fluid stations respectively. Checklists showed high internal reliability, with Cronbach’s alpha of 0.85 (95% CI 0.71-0.93) for the pleural effusion station and 0.87 (95% CI 0.74-0.95) for the abdominal free fluid station. Krippendorff’s alpha, a measure of inter-rater reliability, was also equally strong at 0.85 (95% CI 0.43-0.94) and 0.83 (95% CI 0.50-0.94) respectively.
Conclusion: Both POCUS OSCE stations demonstrated good internal and inter-rater reliability. Deployment of these OSCE stations at programs with integrated POCUS curricula may help refine programming and training expectations.
Téléchargements
Références
1. Diaz-Gomez JL, Mayo PH, Koenig SJ. Point-of-care ultrasonography. N Engl J Med. 2021;385:1593-1602. https://doi.org/10.1056/nejmra1916062 DOI: https://doi.org/10.1056/NEJMra1916062
2. Sena A, Alerhand S, Lamba S. Milestone approach to designing a point-of-care ultrasound curriculum for transition-to-residency programs in the United States. Teach Learn Med. 2020;33(3):270-81. https://doi.org/10.1080/10401334.2020.1814296 DOI: https://doi.org/10.1080/10401334.2020.1814296
3. Tarique U, Tang B, Singh M, Kulasegaram KM, Ailon J. Ultrasound curricula in undergraduate medical education: a scoping review. J Ultrasound Med. 2018;37(1):69-82. https://doi.org/10.1002/jum.14333 DOI: https://doi.org/10.1002/jum.14333
4. Davis JJ, Wessner CE, Potts J, Au AK, Pohl CA, Fields JM. Ultrasonography in undergraduate medical education: a systematic review. J Ultrasound Med. 2018;37(11):2667-79. https://doi.org/10.1002/jum.14628 DOI: https://doi.org/10.1002/jum.14628
5. Steinmetz P, Dobrescu O, Oleskevich S, Lewis J. Bedside ultrasound education in Canadian medical schools: a national survey. Can Med Educ J. 2016;7(1):e78-e86. https://doi.org/10.36834/cmej.36646 DOI: https://doi.org/10.36834/cmej.36646
6. Russell FM, Zakeri B, Herbert A, et al. The state of point-of-care ultrasound training in undergraduate medical education: findings from a national survey. Acad Med. 2022;97(5):723-7. https://doi.org/10.1097/ACM.0000000000004512 DOI: https://doi.org/10.1097/ACM.0000000000004512
7. Hoppmann RA, Mladenovic J, Melniker L, et al. International consensus conference recommendations on ultrasound education for undergraduate medical students. Ultrasound J. 2022;14:31. https://doi.org/10.1186/s13089-022-00279-1 DOI: https://doi.org/10.1186/s13089-022-00279-1
8. Ma IWY, Steinmetz P, Weerdenburg K, et al. The Canadian medical student ultrasound curriculum: a statement from the Canadian Ultrasound Consensus for Undergraduate Medical Education Group. J Ultrasound Med. 2020;39(7):1279-87. https://doi.org/10.1002/jum.15218 DOI: https://doi.org/10.1002/jum.15218
9. Dinh VA, Lakoff D, Hess J, et al. Medical student core clinical ultrasound milestones: a consensus among directors in the United States. J Ultrasound Med. 2016;35(2):421-34. https://doi.org/10.7863/ultra.15.07080 DOI: https://doi.org/10.7863/ultra.15.07080
10. DeBiasio C, Pageau P, Shefrin A, Woo MY, Cheung WJ. Point-of-care-ultrasound in undergraduate medical education: a scoping review of assessment methods. Ultrasound J. 2023;15(1):30. https://doi.org/10.1186/s13089-023-00325-6 DOI: https://doi.org/10.1186/s13089-023-00325-6
11. Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective. Med Teach. 2013;35(9):e1437-e46. https://doi.org/10.3109/0142159x.2013.818634 DOI: https://doi.org/10.3109/0142159X.2013.818634
12. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9):S63-S7. https://doi.org/10.1097/00001888-199009000-00045 DOI: https://doi.org/10.1097/00001888-199009000-00045
13. Patrício MF, Julião M, Fareleira F, Carneiro AV. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach. 2013;35(6):503-14. https://doi.org/10.3109/0142159x.2013.774330 DOI: https://doi.org/10.3109/0142159X.2013.774330
14. Sheppard G, Williams KL, Metcalfe B, et al. Using Kane’s framework to build an assessment tool for undergraduate medical student’s clinical competency with point of care ultrasound. BMC Med Educ. 2023;23:43. https://doi.org/10.1186/s12909-023-04030-9 DOI: https://doi.org/10.1186/s12909-023-04030-9
15. Damewood SC, Leo M, Bailitz J, et al. Tools for measuring clinical ultrasound competency: recommendations from the Ultrasound Competency Work Group. AEM Educ Train. 2019;4(Suppl 1):S106-S12. https://doi.org/10.1002/aet2.10368 DOI: https://doi.org/10.1002/aet2.10368
16. Royal College of Physicians and Surgeons of Canada. Objectives of training in the specialty of emergency medicine. Available at: https://www.royalcollege.ca/rcsite/documents/ibd/emergency_otr_e.pdf. [Accessed Dec 10, 2024].
17. Royal College of Physicians and Surgeons of Canada. Entrustable Professional Activities for anesthesiology Version 3.0. Available from: https://www.royalcollege.ca/rcsite/documents/cbd/epa-guide-anesthesiology-v3-e.pdf. [Accessed Dec 10, 2024].
18. Emergency ultrasound guidelines. Ann Emerg Med. 2009;53:550-70. https://doi.org/10.1016/j.annemergmed.2008.12.013 DOI: https://doi.org/10.1016/j.annemergmed.2008.12.013
19. Blehar DJ, Barton B, Gaspari RJ. Learning curves in emergency ultrasound education. Acad Emerg Med. 2015;22(5):574-82. https://doi.org/10.1111/acem.12653 DOI: https://doi.org/10.1111/acem.12653
20. Association of American Medical Colleges. Core EPAs guiding principles. Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide. Available at: https://store.aamc.org/downloadable/download/sample/sample_id/63/%20. [Accessed Dec 10, 2024].
21. Association of Faculties of Medicine of Canada. AFMC Entrustable Professional Activities for the Transition from Medical School to Residency. Available at: https://www.afmc.ca/wp-content/uploads/2022/10/AFMC_Entrustable-Professional-Activities_EN_Final.pdf. [Accessed Nov 30, 2024].
22. Steinmetz P, Oleskevich S, Dyachenko A, McCusker J, Lewis J. Accuracy of medical students in detecting pleural effusion using lung ultrasound as an adjunct to the physical examination. J Ultrasound Med. 2018;37:2545-52. https://doi.org/10.1002/jum.14612 DOI: https://doi.org/10.1002/jum.14612
23. Olszynski P, Russell M, Neufeld A, Malin G. The clinical ultrasonography elective in clerkship (CUSEC): a pilot elective for senior clerkship students at the University of Saskatchewan. Can Med Educ J. 2020;11(1):e144-6. https://doi.org/10.36834/cmej.61810 DOI: https://doi.org/10.36834/cmej.61810
24. Kern DE, Thomas PA, Hughes MT. Curriculum development for medical education: a six-step approach. Baltimore: Johns Hopkins University Press; 2009.
25. Bahner DP, Hughes D, Royall NA. I-AIM: a novel model for teaching and performing focused sonography. J Ultrasound Med. 2012;31(2):295-300. https://doi.org/10.7863/jum.2012.31.2.295 DOI: https://doi.org/10.7863/jum.2012.31.2.295
26. Hejri SM, Jalili M, Muijtjens AM, Van Der Vleuten CP. Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination. J Res Med Sci. 2013;18(10):887-91.
Téléchargements
Publié-e
Comment citer
Numéro
Rubrique
Licence
© Ryan Good, Juliana Wilson, Joshua C Kaine, Vijay J Daniels, Janeve Dessy, Joshua Lloyd, Gillian Sheppard, Linden Kolbenson, Irene WY Ma, Arthur Au, Paul Adam Olszynski 2025
Cette œuvre est sous licence Creative Commons Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International.
La soumission d’un manuscrit original à la revue constitue une indication qu’il s’agit d’un travail original, qu’il n’a jamais été publié et qu’il n’est pas envisagé pour publication dans une autre revue. S’il est accepté, il sera publié en ligne et ne pourra l’être ailleurs sous la même forme, à des fins commerciales, dans quelque langue que ce soit, sans l’accord de l’éditeur.
La publication d’une recherche scientifique a pour but la diffusion de connaissances et, sous un régime sans but lucratif, ne profite financièrement ni à l’éditeur ni à l’auteur.
Les auteurs qui publient dans la Revue canadienne d’éducation médicale acceptent de publier leurs articles sous la licence Creative Commons Paternité - Pas d’utilisation commerciale, Pas de modification 4.0 Canada. Cette licence permet à quiconque de télécharger et de partager l’article à des fins non commerciales, à condition d’en attribuer le crédit aux auteurs. Pour plus de détails sur les droits que les auteurs accordent aux utilisateurs de leur travail, veuillez consulter le résumé de la licence et la licence complète.
