Best Practice for Online Tests: How Long Do Students Actually Need?
DOI:
https://doi.org/10.20343/teachlearninqu.13.35Keywords:
student testing, test time, multiple choice, online testingAbstract
Multiple choice tests are unlikely to disappear from formal education, partly due to the ease of large-scale administration and grading and their similarity to licensing exams in various fields (e.g., nursing). Despite post-secondary instructors’ best intentions in giving students adequate time to complete multiple choice assessments, it can be difficult to judge the amount of time that is actually required by students, while attempting to maintain test integrity and minimize cheating behaviour. Further, much of the available literature on this topic focuses on students enrolled in four-year university programs, which are likely to differ from other post-secondary programs (i.e. two- and three-year diploma programs). The present study aims to quantify the amount of time students in two- and three-year programs actually used to answer multiple choice questions in a fully online, asynchronous, introduction to psychology course, as well as examine whether differences exist in the time used on two types of assessments: small quizzes with unlimited attempts and unit tests with only one attempt. Results showed that students used on average 39 seconds per question, though they used significantly more time on summative assessments (unit tests) compared to formative quizzes. These results can help guide pedagogical decisions, but it is also important to consider learner-specific characteristics which might affect how much time they use (or need) to complete multiple choice assessments.
Downloads
References
Anderson, Lorin W., David R. Krathwohl, and Benjamin S. Bloom, editor. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.
Attali, Yigal. 2016. “Effort in Low-Stakes Assessments: What Does It Take to Perform as Well as in a High-Stakes Setting?” Educational and Psychological Measurement 76 (6): 1045–58. https://doi.org/10.1177/0013164416634789.
Baron, Naomi S. 2015. Words Onscreen: The Fate of Reading in a Digital World. Oxford: Oxford University Press.
Bloom, Benjamin S., editor. 1956. Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay.
Bound, John, Michael F. Lovenheim, and Sarah Turner. 2010. “Why Have College Completion Rates Declined? An Analysis of Changing Student Preparation and Collegiate Resources.” American Economic Journal: Applied Economics 2 (3): 129–57. http://doi.org/10.1257/app.2.3.129.
Brothen, Thomas. 2012. “Time Limits on Tests: Updating the 1-Minute Rule.” Teaching of Psychology 39 (4): 288–92. https://doi.org/10.1177/0098628312456630.
Brothen, Thomas, and Cathrine Wambach. 2001. “Effective Student Use of Computerized Quizzes.” Teaching of Psychology 28 (4): 292–4. https://doi.org/10.1207/S15328023TOP2804_10.
CAST. 2018. Universal Design for Learning Guidelines Version 2.2.
https://udlguidelines.cast.org/more/downloads/#v2-2.
CAST. 2024. Universal Design for Learning Guidelines Version 3.0. https://udlguidelines.cast.org.
Durham College (n.d.). DC Students. Open Data at DC. Accessed June 15, 2024. https://durhamcollege.ca/about/office-of-research-services-innovation-and-entrepreneurship-orsie/institutional-research-and-planning/open-data-at-dc/our-students.
Education Quality and Accountability Office. 2018. “Ontario Student Achievement: Ontario’s Provincial Secondary School Report.” Toronto, ON: Queen’s Printer for Ontario.
Follwell, Tianna, and Sam Andrew. 2001. “How to End Streaming in Ontario Schools.” Ontario 360. https://on360.ca/wp-content/uploads/2021/05/ON360_EndingAcademicStreaming_v2.pdf.
Government of Ontario. 2023. “Framework for Programs of Instruction for Colleges of Applied Arts and Technology.” Ontario.ca. Accessed December 20, 2024. https://www.ontario.ca/page/framework-programs-instruction-colleges-applied-arts-and-technology.
Ha, Le Anh, Goran Marsic, and Vanya Yaneva. 2017. “Predicting Item Response Time Using Linguistic Features.” Paper Presented at the Timing Impact on Measurement in Education Conference, Philadelphia, PA.
Harik, Paul, Robert A. Fineberg, and Brian E. Clausser. 2020. “How Examinees Use Time: Examples from a Medical Licensing Examination.” In Integrating Timing Considerations to Improve Testing Practices, edited by Melissa J. Margolis and Robert A. Feinberg, 173–89. New York: Routledge.
Hutchings, Pat. 2000. “Approaching the Scholarship of Teaching and Learning.” In Opening Lines: Approaches to the Scholarship of Teaching and Learning, edited by Pat Hutchings, 1–10. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching.
Juric, D. P. 2020. “A History of Test Speededness: Tracing the Evolution of Theory and Practice.” In Integrating Timing Considerations to Improve Testing Practices, edited by Melissa J. Margolis and Robert A. Feinberg, 1–18. New York: Routledge.
Kennette, Lynne N., and Morgan Chapman. 2024. “Student and Faculty Perceptions of Ineffective Teaching Behaviours.” The Canadian Journal for the Scholarship of Teaching and Learning 15 (1). https://doi.org/10.5206/cjsotlrcacea.2024.1.15148.
Kennette, Lynne N., Kathleen Flynn, and Morgan Chapman. 2023. “Has the Pandemic Affected Students’ and Faculty’s Use and Perception of Universal Design for Learning?” Currents in Teaching and Learning 14 (2): 21–48. https://webcdn.worcester.edu/currents-in-teaching-and-learning/wp-content/uploads/sites/65/2023/01/Currents-14-02-Kennette-Flynn-Chapman-Student-and-Faculty-Use-and-Perception-of-Universal-Design-for-Learning.pdf.
Kennette, Lynne N., and Milan Jelenic. 2023. “Cheating: It Depends How You Define It.” Canadian Perspectives on Academic Integrity 5 (2): 16–33. https://doi.org/10.11575/cpai.v5i2.75649.
Kennette, Lynne N., and Bibia R. Redd. 2020. “An Exploration of Study Habits: How Do Four-Year and Two-Year Colleges Compare?” Innovation Abstracts XLII (43). https://www.nisod.org/2020/11/10/xlii_43/.
Kinnon, Emily. 2016. “(In)Equity and Academic Streaming in Ontario: Effects on Students and Teachers and How to Overcome These.” Ontario Institute for Studies in Education of the University of Toronto. https://tspace.library.utoronto.ca/bitstream/1807/72216/1/Kinnon_Emily_R_201606_MT_MTRP.pdf.
McKeachie, Wilbert J. 2002. Teaching Tips. 11th ed. Boston: Houghton Mifflin.
Ofiesh, Nicole S., and Charles A. Hughes. 2002. “How Much Time?: A Review of the Literature on Extended Test Time for Postsecondary Students with Learning Disabilities.” Journal of Postsecondary Education and Disability 16 (1): 2–16. https://files.eric.ed.gov/fulltext/EJ875992.pdf.
Paterson, Donald G. 1924. Preparation and Use of New-Type Examinations: A Manual for Teachers. Yonkers-on-Hudson, NY: World Book Company.
Pear Deck Learning. 2019. “How to Determine the Best Length for Your Assessment.” Pear Deck. https://www.peardeck.com/blog/how-to-determine-the-best-length-for-your-assessment.
People for Education. 2019. “Roadmaps and Roadblocks: Career and Life Planning, Guidance, and Streaming in Ontario’s Schools.” Toronto: People for Education. https://peopleforeducation.ca/wp-content/uploads/2019/02/Roadmaps_roadblocks_WEB.pdf.
Ruch, George M. 1924. The Improvement of the Written Examination. Chicago: Scott Foresman.
Sage, Karen, Heather Augustine, Heather Shand, Kelsey Bakner, and Sarah Rayne. 2019. “Reading from Print, Computer, and Tablet: Equivalent Learning in the Digital Age.” Education and Information Technologies 24: 2477–502. https://doi.org/10.1007/s10639-019-09887-2.
Segedin, Lauren. 2012. “Listening to the Student Voice: Understanding the School-Related Factors That Limit Student Success.” McGill Journal of Education 47 (1): 93–107. http://doi.org/10.7202/1011668ar.
Shaienks, Danielle, Tomasz Gluszynski, and Justin Bayard. 2008. “Postsecondary Education, Participation and Dropping out: Differences Across University, College and Other Types of Postsecondary Institutions.” Statistics Canada. https://eric.ed.gov/?id=ED508178.
Sherman, Thomas M., and Terry M. Wildman. 1982. Proven Strategies for Successful Test Taking. Columbus, OH: Merrill.
Swanson, David B., Susan M. Case, David R. Ripkey, Brian E. Clauser, and Michael C. Holtman. 2001. “Relationships among Item Characteristics, Examinee Characteristics, and Response Times on the USMLE Step 1.” Academic Medicine 79: S114–S116. https://doi.org/10.1097/00001888-200110001-00038.
Swanson, David B., Karen Z. Holtzman, Karen Albee, and Brian E. Clauser. 2006. “Psychometric Characteristics and Response Times for Content-Parallel Extended-Matching and One-Best-Answer Items in Relation to Number of Options.” Academic Medicine 81: S52–55. https://doi.org/10.1097/01.acm.0000236518.87708.9d.
Trauzettel-Klosinski, Susanne, Klaus Dietz, and the IReST Study Group. 2012. “Standardized Assessment of Reading Performance: The New International Reading Speed Texts IReST.” Investigative Ophthalmology & Visual Science 53 (9): 5452–61. https://doi.org/10.1167/iovs.11-8284.
van der Linden, Wim J. 2009. “Conceptual Issues in Response-Time Modeling.” Journal of Educational Measurement 46: 247–72. https://doi.org/10.1111/j.1745-3984.2009.00080.x.
Weber, Iain, and Lynne N. Kennette. 2022. “College Student Study Habits: Initial Patterns and Implications.” Reinvention: An International Journal of Undergraduate Research 7 (1): 6–10. https://cjur.ca/may-2022-volume-7-issue-1/.
Wise, Steven L., and Xiaojing Kong. 2005. “Response Time Effort: A New Measure of Examinee Motivation in Computer-Based Tests.” Applied Measurement in Education 18 (2): 163–83. https://doi.org/10.1207/s15324818ame1802_2.
Wise, Steven L., and Megan R. Kuhfeld. 2020. “A Cessation of Measurement: Identifying Test Taker Disengagement Using Response Time.” In Integrating Timing Considerations to Improve Testing Practices, edited by Melissa J. Margolis and Robert A. Feinberg, 150–64. New York: Routledge.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Lynne N. Kennette, Dawn McGuckin
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
