Knowledge Surveys: An Effective and Robust Student Self-Assessment and Learning Tool

Authors

DOI:

https://doi.org/10.20343/teachlearninqu.13.47

Keywords:

self-assessment, knowledge surveys, metacognition, metacognition and teaching, spaced practice, learning to learn

Abstract

Knowledge surveys (KS) are a student self-assessment tool consisting of an ability statement for each learning objective in a course. Students respond by rating their confidence in performing a specified skill. Pre-unit KS transparently communicate learning objectives, alert faculty to self-assessed knowledge and skills students possess as they enter a unit of instruction and support metacognitive learning through the unit’s lessons and activities. Post-unit KS allow comparison of student self-assessed learning with faculty assessments of student learning, providing feedback to support continued development of self-assessment skills, as well as metacognitive and self-regulated learning skills. This study synthesizes several semesters of KS implementation across six different engineering courses, taught by six instructors utilizing multiple assessment types (exams, technical writing, design projects) and delivery modes (in person, hybrid, online). Results indicate that student KS scores aligned with faculty assessments, and alignment improved with cycles of performance and feedback in a subset of courses with similar assessment types. Student feedback indicates value in KS as a learning guide and as a helpful addition to these courses for learning and self-assessment, with student quotes demonstrating positive student learning practices and evidence of metacognitive thinking. Faculty find KS easily manageable; they also appreciate how KS help maximize alignment within their courses and provide additional data to support student learning. Knowledge surveys are thus an effective, robust, and relatively easy-to-implement method to systematically incorporate a self-assessment component into a variety of courses.

Downloads

Download data is not yet available.

Author Biographies

Joel Sloan, US Air Force Academy

Joel Sloan (USA) is the head of the Department of Civil and Environmental Engineering at the US Air Force Academy. He earned his PhD in civil engineering from Virginia Tech.

Timothy Frank, US Air Force Academy

Timothy Frank (USA) serves at the Air Force Installation and Mission Support Center and is a former faculty member at the US Air Force Academy. He earned his PhD in civil engineering from Stanford University.

Lauren Scharff, US Air Force Academy

Lauren Scharff (USA) is the director for the Scholarship of Teaching and Learning at the US Air Force Academy. She earned her PhD in human experimental psychology from the University of Texas at Austin.

Karin Becker, US Air Force Academy

Karin Becker (USA) is the former director of the Communication Strategies Center at the US Air Force Academy. She earned her PhD in health communication and public discourse from the University of North Dakota.

References

Andrade, Heidi L. 2010. “Students as the Definitive Source of Formative Assessment: Academic Self-Assessment and the Self-Regulation of Learning.” NERA Conference Proceedings 2010: 25. https://opencommons.uconn.edu/nera_2010/25/.

Andrade, Heidi, and Anna Valtcheva. 2009. “Promoting Learning and Achievement through Self-Assessment.” Theory Into Practice 48 (1): 12–19. https://doi.org/10.1080/00405840802577544.

Ariel, Robert, and Jeffrey D. Karpicke. 2018. “Improving Self-Regulated Learning with a Retrieval Practice Intervention.” Journal of Experimental Psychology Applied 24 (1): 43–56. https://doi.org/10.1037/xap0000133.

Beauregard, Melissa S., Joel A. Sloan, and Michael Brannon. 2019. “Summarizing the Use of Knowledge Surveys to Inform Effective Learning and Teaching Practices.” Proceedings of Rocky Mountain Section of the American Society for Engineering Education’s Annual Conference, May 2019, Laramie, WY.

Becker, Karin L., and Joel A. Sloan. 2023. “Implementing Technical Communication Instruction in a Civil Engineering Course: The Value of Knowledge Surveys and Peer Review.” Journal of Civil Engineering Education 149 (4). https://doi.org/10.1061/JCEECD.EIENG-1927.

Bloom, Benjamin S., Max D. Englehart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. New York: David McKay Company.

Bourke, Roseanna. 2018. “Self-Assessment to Incite Learning in Higher Education: Developing Ontological Awareness.” Assessment & Evaluation in Higher Education 43 (5): 827–39.

https://doi.org/10.1080/02602938.2017.1411881.

Chew, Stephen. 2021. “An Advance Organizer for Student Learning: Choke Points and Pitfalls in Studying.” Canadian Psychology 62 (4): 420–27. https://doi.org/10.1037/cap0000290.

Clauss, Jon, and Kevin Geedey. 2010. “Knowledge Surveys: Students Ability to Self-Assess.” Journal of the Scholarship of Teaching and Learning 10 (2): 14–24.

Cronbach, Lee J. 1951. “Coefficient Alpha and the Internal Structure of Tests.” Psychometrika 16 (3): 297–334. https://doi.org/10.1007/BF02310555.

Falchikov, Nancy, and David Boud. 1989. “Student Self-Assessment in Higher Education: A Meta-Analysis.” Review of Educational Research 59 (4): 395–430.

Fink, Dee L. 2003. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. San Francisco: John Wiley & Sons. Inc.

Flavell, John. 1976. “Metacognitive Aspects of Problem Solving.” In The Nature of Intelligence (231–36). Hillsdale, NJ: Erlbaum.

Lindsey, Beth A., and Megan Nagel. 2013. “Assessing Students’ Metacognitive Calibration with Knowledge Surveys.” In AIP Conference Proceedings 1513 (1): 258–61). American Institute of Physics. https://doi.org/10.1063/1.4789701.

Mahlberg, Jamie. 2015. “Formative Self-Assessment College Classes Improves Self-Regulation and Retention in First/Second Year Community College Students.” Community College Journal of Research and Practice 39 (8): 772–83. https://doi.org/10.1080/10668926.2014.922134.

McDevitt, Andrew L., Manisha V. Patel, Brad Rose, and Aaron M. Ellison. 2016. “Insights Into Student Gains From Undergraduate Research Using Pre- and Post-Assessments.” BioScience 66 (12): 1070–78. https://doi.org/10.1093/biosci/biw141.

Nuhfer, Edward. 1996. “The Place of Formative Evaluations in Assessment and Ways to Reap Their Benefits.” Journal of Geoscience Education 44: 385–94. https://doi.org/10.5408/1089-9995-44.4.385.

Nuhfer, Edward, Christopher Cogan, Steven Fleischer, Eric Gaze, and Karl Wirth. 2016a. “Random Number Simulations Reveal How Random Noise Affects the Measurements and Graphical Portrayals of Self-Assessed Competency.” Numeracy 9 (1). https://doi.org/10.5038/1936-4660.9.1.4.

Nuhfer, Edward B., Christopher B. Cogan, Carl Kloock, Gregory G. Wood, Anya Goodman, Natalie Z. Delgado, and Christopher W. Wheeler. 2016b. “Using a Concept Inventory to Assess the Reasoning Component of Citizen-Level Science Literacy: Results From a 17,000-Student Study.” Journal of Microbiology and Biology Education 17 (1): 143–55. https://doi.org/10.1128/jmbe.v17i1.1036.

Nuhfer, Edward, Steven Fleischer, Christopher Cogan, Karl Wirth, and Eric Gaze. 2017. “How Random Noise and a Graphical Convention Subverted Behavioral Scientists’ Explanations of Self-assessment Data: Numeracy Underlies Better Alternatives.” Numeracy 10 (1). https://doi.org/10.5038/1936-4660.10.1.4.

Nuhfer, Edward, and Delores Knipp. 2003. “4: The Knowledge Survey: A Tool for All Reasons.” To Improve the Academy 21: (20210331). https://doi.org/10.3998/tia.17063888.0021.006.

Nuhfer, Edward B., and Delores Knipp. 2006. “Re: The Use of a Knowledge Survey as an Indicator of Student Learning in an Introductory Biology Course.” CBE—Life Sciences Education 5 (4): 313–14. https://doi.org/10.1187/cbe.06-05-0166.

Pintrich, Paul R. 2000. “The Role of Goal Orientation in Self-Regulated Learning.” In Handbook of Self-Regulation, edited by Monique Boekaerts, Paul R. Pintrich, and Moshe Zeidner, 451–502. https://doi.org/10.1016/b978-012109890-2/50043-3.

Pintrich, Paul R. 2002. “The Role of Metacognitive Knowledge in Learning, Teaching, and Assessing.” Theory Into Practice 41 (4): 219–25. https://doi.org/10.1207/s15430421tip4104_3.

Ross, John A. 2006. “The Reliability, Validity, and Utility of Self-Assessment.” Practical Assessment, Research, and Evaluation 11 (10): 1–13. https://doi.org/10.7275/9wph-vv65.

Rushton, Alison. 2005. “Formative Assessment: A Key to Deep Learning?” Medical Teacher 27 (6): 509–13. https://doi.org/10.1080/01421590500129159.

Scharff, Lauren F., Chandler-Garcia, Lynne M., Culver, Christopher, Frank, Timothy E., Post, Martiqua, Tetley, Julie, Sloan, Joel A. In Press. “Knowledge Surveys: Developing Self-Assessment, Integrating Effective Learning Strategies, and Enhancing Transparency to Support Student Success.” Book chapter accepted for TILTed Pedagogy: A Collection of Evidence-Based Assignments to Inspire Learning, edited by Shannon Sipes, Mandy Frake-Mistak, Jennifer Friberg. Elon’s Center for Engaged Learning (CEL) Open Access Book Series.

Sloan, Joel A., Melissa M. Beauregard, and M. Mark Russell. 2023. “Chapter 4: High Impact Practices in the Civil Engineering Major at the United States Air Force Academy.” In High Impact Practices in Higher Education: International Perspectives (Innovations in Higher Education Teaching and Learning, edited by Enakshi Sengupta. Emerald Publishing Ltd. https://doi.org/10.1108/S2055-364120230000051004.

Sloan, Joel A., Karin L. Becker, and Timothy E. Frank. 2022. “Implementing a Technical Communication Block in a Civil Engineering Computer Applications Course: Lessons Learned.” Proceedings of the 2022 ASEE Annual Conference, Minneapolis, MN. https://peer.asee.org/41191.

Sloan, Joel A., and Timothy E. Frank. 2023. “Student Self-Assessment Data: An Instrument for Program Assessment.” Intersection: A Journal at the Intersection of Assessment and Learning 4 (2). http://dx.doi.org/10.61669/001c.84511.

Sloan, Joel A., and Timothy E. Frank. 2024. Relationships Between Student Self-Assessment Ability and Performance. Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47937.

Sloan, Joel A., and Lauren F. Scharff. 2022. “Student Self-Assessment: Relationships Between Accuracy, Engagement, Perceived Value, and Performance.” Journal of Civil Engineering Education 148 (3). https://doi.org/10.1061/(asce)ei.2643-9115.0000063.

Sullivan, Keith, and Cedric Hall. 1997. “Introducing Students to Self‐assessment.” Assessment & Evaluation in Higher Education 22 (3): 289–305. https://doi.org/10.1080/0260293970220303.

Tanner, Kimberly D. 2012. “Promoting Student Metacognition.” CBE—Life Sciences Education 11 (2): 113–20. https://doi.org/10.1187/cbe.12-03-0033.

Taras, Maddalena. 2002. “Using Assessment for Learning and Learning From Assessment.” Assessment & Evaluation in Higher Education 27 (6): 501–10. https://doi.org/10.1080/0260293022000020273.

Taras, Maddalena. 2010. “Student Self-Assessment: Processes and Consequences.” Teaching in Higher Education 15 (2): 199–209. https://doi.org/10.1080/13562511003620027.

Wirth, Karl R., Edward B. Nuhfer, Rachel Watson, Steven C. Fleisher, and Roxanne Bailey. 2021. “Knowledge Surveys: Supporting Students Along Pathways to Self-Directed Learning With Self-Assessment.” In Self Directed Learning: An Imperative for Education in a Complex Society, edited by Elsa Metnz, Dorothy Laubscher, and Jako Olivier, 127–64. https://doi.org/10.4102/aosis.2021.bk279.06.

Wirth, Karl R., and Dexter Perkins. 2005. “Knowledge Surveys: An Indispensable Course Design and Assessment Tool.” Innovations in the Scholarship of Teaching and Learning.

Wirth, Karl R., and Dexter Perkins. 2013, October 27. “Learning to Learn.” Available from: https://www.montana.edu/rmaher/barrier_courses/Learning%20to%20Learn%20Wirth.pdf.

Zell, Ethan, and Zlatan Krizan. 2014. “Do People Have Insight Into Their Abilities? A Metasynthesis.” Perspectives on Psychological Science 9 (2): 111–25. https://doi.org/10.1177/1745691613518075.

A landscape photo of a campus buildings with a blue sky, mountains, and grass.

Downloads

Published

2025-10-31

How to Cite

Sloan, Joel, Timothy Frank, Lauren Scharff, and Karin Becker. 2025. “Knowledge Surveys: An Effective and Robust Student Self-Assessment and Learning Tool ”. Teaching and Learning Inquiry 13 (October):1–20. https://doi.org/10.20343/teachlearninqu.13.47.