Is the Course Working? An Account of Our Development of an Instrument to Measure the Science Attitudes and Skills of Undergraduate Students Outside of Science Disciplines

Ellen Watson; Sheryl Gares; Brian Rempel

doi:10.20343/teachlearninqu.13.29

Authors

Ellen Watson Brandon University https://orcid.org/0000-0002-0052-1725
Sheryl Gares University of Alberta https://orcid.org/0009-0005-8136-6921
Brian Rempel University of Alberta https://orcid.org/0000-0002-1838-5055

DOI:

https://doi.org/10.20343/teachlearninqu.13.29

Keywords:

instrument development, science skills, science attitudes, post-secondary education, general science

Abstract

After a redesign of our school year structure, our science team developed an introduction to science course focused on teaching science to non-science majors early in their post-secondary studies. The goal of this course was not to prepare students for further pursuit of science degrees; instead, we wanted to equip them with the skills and attitudes necessary to understand the scientific world in which we live. Consequently, we wondered whether these skills and attitudes were being met throughout the course; was the course working? When searching the literature, we did not identify any instrument that simultaneously and concisely measured general science skills and attitudes. Given this gap and based on our desire to measure science skills and attitudes for non-science majors at our campus, this research team developed Augustana Interdisciplinary Scientific Literacy Evaluation (AISLE) in order to provide a measurement of students’ science skills and abilities in a general science course at the post-secondary level. However, as we would come to know, this process was not as simple as might seem. The purpose of this paper is to provide an account of the development and validation of the AISLE for those who wish to use the instrument or for others in the SoTL community looking to develop similar tools. We also offer an account of using the AISLE in our course to measure students’ science skill and attitude development. In the end, our STEM-based instructional team learned that what appeared to be straight forward assessment development, was, in fact, a far more involved and complicated process.

Downloads

Download data is not yet available.

Author Biographies

Ellen Watson, Brandon University

Ellen Watson, PhD (CAN), is an associate professor at Brandon University’s Faculty of Education. She teaches courses in science education, curriculum, research methodologies, and other educational topics.

Sheryl Gares, University of Alberta

Sheryl Gares, PhD (CAN), is an associate professor at the Augustana Faculty of the University of Alberta, where she teaches in the integrative biology program in the Department of Science.

Brian Rempel, University of Alberta

Brian Rempel, PhD (CAN), is an associate professor at the Augustana Faculty of the University of Alberta, where he teaches organic chemistry in the chemical and physical science program in the Department of Science.

References

Abd-El-Khalick, Fouad, and Norman G. Lederman. 2000. “The Influence of History of Science Courses on Students’ Views of Nature of Science.” Journal of Research in Science Teaching 37 (10): 1057–95. https://doi.org/10.1002/1098-2736(200012)37:10<1057::AID-TEA3>3.0.CO;2-C. DOI: https://doi.org/10.1002/1098-2736(200012)37:10<1057::AID-TEA3>3.0.CO;2-C

Adams, Wendy K., Perkins, Katherine, Podolefsky, Noah S., Dubson, Michael, Finkelstein, Noah D., and Carl E. Wieman. 2006. “New Instrument for Measuring Student Beliefs about Physics and Learning Physics: The Colorado Learning Attitudes about Science Survey.” Physical Review Special Topics - Physics Education Research 2 (1): 1–14. https://doi.org/10.1103/PhysRevSTPER.2.010101. DOI: https://doi.org/10.1103/PhysRevSTPER.2.010101

Adams, Wendy. K., and Carl E. Wieman. 2011. “Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking.” International Journal of Science Education 33 (9): 1289–312. https://doi.org/10.1080/09500693.2010.512369. DOI: https://doi.org/10.1080/09500693.2010.512369

Bass, Kristin. M., Dina Drits-Esser, and Louisa A. Stark. 2016. “A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use.” CBE—Life Sciences Education 15 (2): 1–14. https://doi.org/10.1187/cbe.15-07-0142. DOI: https://doi.org/10.1187/cbe.15-07-0142

Blalock, Cheryl L., Michael J. Lichtenstein, Steven Owen, Linda Pruski, Carolyn Marshall, and MaryAnne Toepperwein. 2008. “In Pursuit of Validity: A Comprehensive Review of Science Attitude Instruments 1935–2005.” International Journal of Science Education 30 (7): 961–77. https://doi.org/10.1080/09500690701344578. DOI: https://doi.org/10.1080/09500690701344578

Braun, Virginia, and Victoria Clarke. 2022. Thematic Analysis: A Practical Guide. SAGE. DOI: https://doi.org/10.1007/978-3-319-69909-7_3470-2

Cary, Tawnya L., Caroline J. Wienhold, and Janet Branchaw. 2019. “A Biology Core Concept Instrument (BCCI) to Teach and Assess Student Conceptual Understanding.” CBE—Life Sciences Education 18 (3):, https://doi.org/10.1187/cbe.18-09-0192. DOI: https://doi.org/10.1187/cbe.18-09-0192

Chowning, Jeanne T. 2022. “Science Teachers in Research Labs: Expanding Conceptions of Social Dialogic Dimensions of Scientific Argumentation.” Journal of Research in Science Teaching 59 (8): 1388–415. https://doi.org/10.1002/tea.21760. DOI: https://doi.org/10.1002/tea.21760

D’Avanzo, Charlene. 2008. “Biology Concept Inventories: Overview, Status, and Next Steps.” BioScience 58 (11): 1079–85. https://doi.org/10.1641/B581111. DOI: https://doi.org/10.1641/B581111

Daker, Richard J., Sylvia U. Gattas, Moriah H. Sokolowski, Adam E. Green, and Ian M. Lyons. 2021. “First-Year Students’ Math Anxiety Predicts STEM Avoidance and Underperformance throughout University, Independently of Math Ability.” Nature Partner Journals, Science of Learning 6 (17). https://doi.org/10.1038/s41539-021-00095-7. DOI: https://doi.org/10.1038/s41539-021-00095-7

Darmaji, Darmaji, Dwi A. Kurniawan, Astalini Astalini, Rahmat Perdana, Kuswanto Kuswanto, Muhammad Ikhlas. 2020. “Do a Science Process Skills Affect on Critical Thinking in Science? Differences in Urban and Rural.” International Journal of Evaluation and Research in Education (IJERE) 9 (4): 874. https://doi.org/10.11591/ijere.v9i4.20687. DOI: https://doi.org/10.11591/ijere.v9i4.20687

DeVellis, Robert F. 2017. Scale Development: Theory and Applications, fourth edition. Thousand Oaks, California: SAGE Publications Ltd.

Halloun, Ibrahim, and David Hestenes. 1998. “Interpreting VASS Dimensions and Profiles for Physics Students.” Science and Education 7 (6): 553–77. https://doi.org/10.1023/A:1008645410992. DOI: https://doi.org/10.1023/A:1008645410992

Hestenes, David, Malcolm Wells, and Gregg Swackhamer. 1992. “Force Concept Inventory.” The Physics Teacher 30 (3): 141–58. https://doi.org/10.1119/1.2343497. DOI: https://doi.org/10.1119/1.2343497

Hunter, Anne-Barrie, Sandra L. Laursen, Elaine Seymour. 2006. “Becoming a Scientist: The Role of Undergraduate Research in Students’ Cognitive, Personal, and Professional Development.” Science Education 91 (1): 36–74. https://doi.org/10.1002/sce.20173. DOI: https://doi.org/10.1002/sce.20173

Kite, Vance, Soonhye Park, Katherine McCance, and Elsun Seung. 2021. “Secondary Science Teachers’ Understandings of the Epistemic Nature of Science Practices.” Journal of Science Teacher Education 32 (3): 243–64. https://doi.org/10.1080/1046560X.2020.1808757. DOI: https://doi.org/10.1080/1046560X.2020.1808757

Konnemann, Christiane, Christian Höger, Roman Asshoff, Marcus Hammann, and Werner Rieß. 2018. “A Role for Epistemic Insight in Attitude and Belief Change? Lessons from a Cross-Curricular Course on Evolution and Creation.” Research in Science Education 48 (6): 1187–204. https://doi.org/10.1007/s11165-018-9783-y. DOI: https://doi.org/10.1007/s11165-018-9783-y

Kopacz, Dawn M., and Zachary Handlos. 2021. “Less is More: Course Redesign and the Development of an Atmospheric Science Process Skills Assessment.” International Journal for the Scholarship of Teaching and Learning15 (2). https://doi.org/10.20429/ijsotl.2021.150212. DOI: https://doi.org/10.20429/ijsotl.2021.150212

Krause, Stephen, Birk, James, Bauer, Richard, Jenkins, Brook, and Pavelich, Michael J. 2004. “Development, Testing, and Application of a Chemistry Concept Inventory.” In 34th Annual Frontiers in Education, 103–7. Savannah, GA: IEEE. https://doi.org/10.1109/FIE.2004.1408473. DOI: https://doi.org/10.1109/FIE.2004.1408473

Maloney, David P., Thomas L. O’Kuma, Curtis J. Hieggelke, and Alan Van Heuvelen. 2001. “Surveying Students’ Conceptual Knowledge of Electricity and Magnetism.” American Journal of Physics 69 (S1): 12–23. https://doi.org/10.1119/1.1371296. DOI: https://doi.org/10.1119/1.1371296

Manarin, Karen, Christine Adams, Richard Fendler, Heidi Marsh, Ethan Pohl, Suzanne Porath, and Alison Thomas. 2021. “Examining the Focus of SoTL Literature—Teaching and Learning?” Teaching and Learning Inquiry 9 (1): 349–64. https://doi.org/10.20343/teachlearninqu.9.1.23. DOI: https://doi.org/10.20343/9.1.23

Mao, Peipei, Zhihui Cai, Jinbo He, Xinjie Chen, and Xitao Fan. 2021. “The Relationship between Attitude toward Science and Academic Achievement in Science: A Three-Level Meta-Analysis.” Frontiers in Psychology 12. https://doi.org/10.3389/fpsyg.2021.784068. DOI: https://doi.org/10.3389/fpsyg.2021.784068

Miller-Young, Janice, and Michelle Yeo. 2015. “Conceptualizing and Communicating SoTL: A Framework for the Field.” Teaching & Learning Inquiry 3 (2): 37–53. https://doi.org/10.20343/teachlearninqu.3.2.37. DOI: https://doi.org/10.20343/teachlearninqu.3.2.37

Pérez García, Marilú, Cynthia J. Luxford, Theresa L. Windus, and Thomas Holme. 2016. “A Quantum Chemistry Concept Inventory for Physical Chemistry Classes.” Journal of Chemical Education 93 (4): 605–12. https://doi.org/10.1021/acs.jchemed.5b00781. DOI: https://doi.org/10.1021/acs.jchemed.5b00781

Scharff, Lauren, Capocchiano, Holly, Chick, Nancy, Eady, Michelle, Friberg, Jen, Gregory, Diana, Loy, Kara and Maurer, Trent. 2023. “Grand Challenges for SoTL #1.” International Society for the Scholarship of Teaching and Learning. https://issotl.com/grand-challenges-for-sotl/gc-sotl-1/.

Simmons, Nicola. 2009. “Playing for SoTL Impact: A Personal Reflection.” International Journal for the Scholarship of Teaching and Learning 3 (2). https://doi.org/10.20429/ijsotl.2009.030230. DOI: https://doi.org/10.20429/ijsotl.2009.030230

Simmons, Nicola. 2020. “The 4M Framework As Analytic Lens for SoTL’s Impact: A Study of Seven Scholars.” Teaching & Learning Inquiry 8 (1): 76–90. https://doi.org/10.20343/teachlearninqu.8.1.6. DOI: https://doi.org/10.20343/teachlearninqu.8.1.6

Strzalkowski, Nicholas, and Mandana Sobhanzadeh. 2023. “Views and Value of an Undergraduate General Education on Advancing Student Attitudes and Engagement with Science.” Imagining SoTL 3 (2): 89–119. https://doi.org/10.29173/isotl687. DOI: https://doi.org/10.29173/isotl687

Supardi, Rafsanjani, Edi Istiyono, and Wirawan Setialaksana. 2019. “Developing Scientific Attitudes Instrument of Students in Chemistry.” Journal of Physics Conference Series 1223: Paper 012025. https://doi.org/10.1088/1742-6596/1233/1/012025. DOI: https://doi.org/10.1088/1742-6596/1233/1/012025

Thornton, Ronald K. and David R. Sokoloff. 1998. “Assessing Student Learning of Newton’s Laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula.” American Journal of Physics 66 (4): 338–52. https://doi.org/10.1119/1.18863. DOI: https://doi.org/10.1119/1.18863

Villafañe, Sachel M., and Jennifer E. Lewis. 2016. “Exploring a Measure of Science Attitude for Different Groups of Students Enrolled in Introductory College Chemistry.” Chemistry Education Research and Practice 17 (4): 731–42. https://doi.org/10.1039/c5rp00185d. DOI: https://doi.org/10.1039/C5RP00185D

Watson, Ellen. 2020. “The Slippery Business of Measuring Beliefs Lessons from a Failed Attempt at Developing an Instrument to Measure Teachers’ Epistemic Beliefs about Physics Knowledge.” Electronic Journal for Research in Science & Mathematics Education 24 (2): 119–40. https://ejrsme.icrsme.com/article/view/20294.

Wilson-Doenges, Georjeanna. 2015. “The State of Scale Validation in SoTL Research in Psychology.” In A Compendium of Scales for the Use of Scholarship of Teaching and Learning, edited by Rajiv S. Jhangiana, Jordan D. Troisi, Bethany Fleck, Angela M. Legg, and Heather D. Hussey, 44–54. Society for the Teaching of Psychology. https://teachpsych.org/ebooks/compscalessotp.