Examining Course-Level Conceptual Connections Using a Card Sort Task: A Case Study in a First-Year, Interdisciplinary, Earth Science Laboratory Course
Keywords:card sort task, conceptual connections, assessment tools, problem-based learning
Universities are recognizing the need to prepare graduates to think conceptually and have the ability to take on complex, real-world problems. Strategies to assess conceptual knowledge are limited and often require more time and effort to complete than is accessible for most undergraduate courses. Card sorting is a very broad technique for understanding how people group concepts, but in higher education has typically been used to show a student’s development towards expert-like thinking in a discipline as a whole. However, it typically does not give much insight into how we should change our teaching. In this paper, using the novel setting of two terms of a first-year, earth and ocean science lab that uses problem-based learning (PBL), we show how one can generate a card sort that is built using course learning goals and then use the analysis to make actionable improvements to course instruction. Using a card sort designed so that the expert sort corresponds to learning goals supported by the lab activities, we found that in both offerings of the course students generally moved towards expert-like sorting with a reduction in novice-like sorting. A striking feature stood out in both terms of the course, with one question scoring significantly lower than any other expert pairings, despite a change in the wording of that question between terms. This suggests that our course materials do not promote this specific conceptual connection that we had expected and gives us a clear place to look for issues in our course material. In a broader context, our results suggest that tailoring card sort questions to material at a course level, rather than at the discipline level, can provide a manageable, routine assessment of conceptual knowledge in students, while also providing feedback on the quality of course materials.
Addison, Christopher J., James Charbonneau, Patrick Dubois. 2017. “A Novel Card Sort Activity to Measure Interdisciplinary Thinking.” International Society for the Scholarship of Teaching and Learning (ISSOTL) 2017, Calgary (International).
Bissonnette, Sarah. A., Elijah D. Combs, Paul H. Nagami, Victor Byers, Juliana Fernandez, Dinh Le, Jaren Realin, Selina Woodham, Julia I. Smith, and Kimberly D. Tanner. 2017. “Using the Biology Card Sorting Task to Measure Changes in Conceptual Expertise During Postsecondary Biology Education.” CBE Life Sciences Education, 16, no. 1 (March): 1–15. http://doi.org/10.1187/cbe.16-09-0273.
Bussolon, Stephen, Barbara Russi, and Fabio Del Missier. 2006. “Online Card Sorting: As Good as the Paper Version.” Proceedings of the 13th European conference on Cognitive ergonomics: trust and control in complex socio-technical systems, 250 (September): 113–114. http://doi.org/10.1145/1274892.1274912.
Chi, Michelene T., Paul J. Feltovich, and Robert Glaser. 1981. “Categorization and Representation of Physics Problems by Experts and Novices.” Cognitive Science, 5, no. 2 (April): 121–152. https://doi.org/10.1207/s15516709cog0502_2.
Dadd, Kelsie. A. 2009. “Using Problem-Based Learning to Bring the Workplace into the Classroom.” Journal of Geoscience Education, 57, no. 1 (January): 1–10. http://doi.org/10.5408/1.3544224.
Fernandez-Plaza, José Antonio, and Adrian Simpson. 2016. “Three Concepts or One? Students’ Understanding of Basic Limit Concepts.” Educational Studies in Mathematics, 93, no. 3 (November): 315–332. https://doi.org/10.1007/s10649-016-9707-6.
Galloway, Kelli R., Min Wah Leung, and Alison B. Flynn. 2019. “Patterns of Reactions: A Card Sort Task to Investigate Students’ Organization of Organic Chemistry Reactions.” Chemistry Education Research and Practice, 20, no. 1 (July): 30–52. http://doi.org/10.1039/c8rp00120k.
Hoskinson, Anne-Marie, Jessica Middlemis Maher, Cody Bekkering, and Diane Ebert-May. 2017. “A Problem-Sorting Task Detects Changes in Undergraduate Biological Expertise Over a Single Semester.” CBE Life Sciences Education, 16, no. 2 (October): 1–12. http://doi.org/10.1187/cbe.16-05-0175.
Irby, Stefan M., Andy L. Phu, Emily J. Borda, Todd R. Haskell, Nicole Steed, and Zachary Meyer. 2016. “Use of a Card Sort Task to Assess Students’ Ability to Coordinate Three Levels of Representation in Chemistry.” Chemistry Education Research and Practice, 17, no. 2 (January): 337–352. http://doi.org/10.1039/c5rp00150a.
Jones, Francis. 2018. “Comparing Student, Instructor, Classroom and Institutional Data to Evaluate a Seven-Year Department-Wide Science Education Initiative.” Assessment and Evaluation in Higher Education, 43, no. 2 (June): 323–338. http://doi.org/10.1080/02602938.2017.1343799.
Krieter, Felicia E., Ryan W. Julius, Kimberly D. Tanner, Seth D. Bush, and Gregory E. Scott. 2016. “Thinking Like a Chemist: Development of a Chemistry Card-Sorting Task to Probe Conceptual Expertise.” Journal of Chemical Education, 93 no. 5 (March): 811–820. http://doi.org/10.1021/acs.jchemed.5b00992.
Lapierre, Keith. R., and Alison B. Flynn. 2020. “An Online Categorization Task to Investigate Changes in Students’ Interpretations of Organic Chemistry Reactions.” Journal of Research in Science Teaching, 57, no. 1(January): 87–111. http://doi.org/10.1002/tea.21586.
Lawless, Kimberly A., Scott W. Brown, and Mark A. Boyer. 2016. “Educating Students for STEM Literacy: GlobalEd2.” In Technology, Theory, and Practice in Interdisciplinary STEM Programs, edited by Reneta D. Lansiquot, 53–82. New York: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-56739-0.
Loftus, Geoffrey R. 1985. “Evaluating Forgetting Curves.” Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, no. 2: 397–406. http://doi.org/10.1037//0278-73184.108.40.2067.
Montgomery, Homer, and Katherine Donaldson. 2014. “Using Problem-Based Learning to Deliver a More Authentic Experience in Paleontology.” Journal of Geoscience Education, 62, no. 4: 714–724. http://doi.org/10.5408/13-085.1.
Petcovic, Heather L., and Julie C. Libarkin. 2007. “Research in Science Education: The Expert-Novice Continuum.” Journal of Geoscience Education, 55, no. 4: 333–339. https://doi.org/10.1080/10899995.2007.12028060.
Peters, Michael. 2015. “Using Cognitive Load Theory to Interpret Student Difficulties with a Problem-Based Learning Approach to Engineering Education: A Case Study.” Teaching Mathematics and Its Applications, 34, no. 1 (March): 53–62. http://doi.org/10.1093/teamat/hru031.
Smith, Julia I., Elijah D. Combs, Paul H. Nagami, Valerie M. Alto, Henry G. Goh, Muryam A. A. Gourdet, Christina M. Hough, et al. 2013. “Development of the Biology Card Sorting Task to Measure Conceptual Expertise in Biology.” CBE Life Sciences Education, 12, no. 4 (December): 628–644. http://doi.org/10.1187/cbe.13-05-0096.
How to Cite
Copyright (c) 2022 Ashley B. Davidson, Christopher J. Addison, James Charbonneau
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.