ComPAIR: A new online tool using adaptive comparative judgement to support learning with peer feedback
DOI:
https://doi.org/10.20343/teachlearninqu.5.2.8Keywords:
peer feedback, answer comparison, adaptive comparative judgement, online teaching tools, collaborative learningAbstract
Peer feedback is a useful strategy in teaching and learning, but its effectiveness particularly in introductory courses can be limited by the relative newness of students to both the body of knowledge upon which they are being asked to provide feedback and the skill set involved in providing good feedback. This paper applies a novel approach to facilitating novice feedback: making use of students’ inherent ability to compare. The ComPAIR application discussed in this article scaffolds peer feedback through comparisons, asking students to choose the “better” of two answers in a series of pairings offered in an engaging online context. In contrast to other peer-feedback approaches that seek to train novices to be able to provide expert feedback (such as calibrated peer review) or to crowdsource grading, ComPAIR focuses upon the benefits to be gained from the critical process of comparison and ranking. The tool design is based on the longstanding psychological principle of comparative judgement, by which novices who may not yet have the compass to assess others’ work confidently can still rank content as “better” with accuracy. Data from 168 students in pilot studies in English, Physics and Math courses at the University of British Columbia are reviewed. Though the use of ComPAIR required little classroom time, students perceived this approach to increase their facility with course content, their ability assess their own work, and their capacity to provide feedback on the work of others in a collaborative learning environment.
Metrics
References
Ambrose, S., Bridges, M.W., DiPietro, M., Lovett, M.C., Norman, M.K., & Mayer, R.E. (2010). How learning works: Seven research-based principles for smart teaching. San Francisco, CA: Jossey-Bass.
Boyd, J. (2010). The best of both worlds: The large lecture, writing-intensive course. Communication Teacher, 24(4), 229-237. DOI: 10.1080/17404622.2010.513992
Buckley, G.L, Bain, N.R., Luginbuhl, A.M., & Dyer, M.L. (2004). Adding an ‘active learning’ component to a large lecture course. Journal of Geography, 103(6), 231-237. DOI: 10.1080/00221340408978607
Burke, J and Gilmore, B. (2015). Academic Moves for College and Career Readiness, Grades 6-12. Thousand Oaks, CA: Sage.
Cho, K., & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education, 48(3), 409-426. DOI: 10.1016/j.compedu.2005.02.004
Dochy, F. J. R. C., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331-350. DOI: 10.1080/03075079912331379935
Graham, S. A., Namy, L. L., Gentner, D., & Meagher, K. (2010). The role of comparison in preschoolers' novel object categorization. Journal of Experimental Child Psychology, 107(3), 280-290. DOI:
1016/j.jecp.2010.04.017
Hafner, J. and Hafner, P. (2003). Quantitative analysis of the rubric as an assessment tool: An empirical study of student peer-group rating. International Journal of Science Education 25(12), 1509-1528. DOI:10.1080/0950069022000038268 DOI: 10.1007/s11251-013-9300-7
Jhangiani, R. (2016). The impact of participating in a peer assessment activity on subsequent academic performance. Teaching of Psychology 43(3), 180-186. DOI: 10.1177/0098628316649312
Könings, K. D., Seidel, T., Jeroen, J., & van Merriënboer, G. (2014). Participatory design of learning environments: integrating perspectives of students, teachers, and designers. Instructional Science, 42(1), DOI:10.1007/s11251-013-9305-2
Kulkarni, C. E., Bernstein, M. S., & Klemmer, S. R. (2015). PeerStudio: Rapid peer feedback emphasizes revision and
improves performance. L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ Scale, 75-84. New York, NY: ACM. DOI: 10.1145/2724660.2724670
Loibl, K., Roll, I., & Rummel, N. (2016). Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review. (2016). DOI:10.1007/s10648-016-9379-x https://link.springer.com/article/10.1007/s10648-016-9379-x
Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of Second Language Writing, 18(1), 30-43. DOI: 10.1016/j.jslw.2008.06.002
Nguyen, H., Xiong, W., & Litman, D. (2014). Classroom evaluation of a scaffolding intervention for improving peer review localization. International Conference on Intelligent Tutoring Systems, 272-282. New York: Springer International Publishing. DOI: 10.1007/978-3-319-07221-0_34
Paulus, T. M. (1999). The effect of peer and teacher feedback on student writing. Journal of Second Language Writing, 8(3), 265-289. DOI: 10.1016/S1060-3743(99)80117-9
Pollitt, Alistair (2012). The method of adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice, 19(3), 281-300. DOI: 10.1080/0969594X.2012.665354
Polya, G. (1945). How to Solve it: A New Aspect of Mathematical Method. Princeton, NJ: Princeton University Press.
Robinson, R. (2001). Calibrated Peer Review™: An application to increase student reading & writing skills. The American Biology Teacher, 63(7), 474-480. DOI: 10.1662/0002 7685(2001)063%5B0474:CPR%5D2.0.CO;2
Roll, I., Holmes, N. G., Day, J., & Bonn, D. (2012). Evaluating metacognitive scaffolding in guided invention activities. Instructional Science, 40, 691-710. DOI: 10.1007/s11251-012-9208-7
Schuler, D. & Namioka, A. (Eds.). (1993). Participatory design: Principles and practices. Boca Ratan, FL: CRC Press.
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475-522. DOI: 10.1207/s1532690xci1604_4
Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. (2004). Improving online learning: Student perceptions of useful and challenging characteristics. The Internet and Higher Education, 7(1), 59-70. DOI: 10.1016/j.iheduc.2003.11.003 http://dx.doi.org/10.1207/s1532690xci1604_4
Star, J. R., & Rittle-Johnson, B. (2009). It pays to compare: An experimental study on computational estimation. Journal of Experimental Child Psychology, 102(4), 408-26. DOI: 10.1016/j.jecp.2008.11.004
Suen, H. K. (2014). Peer assessment for massive open online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 15(3). DOI: 10.19173/irrodl.v15i3.1680
Thurstone, L. L. (1927). A Law of Comparative Judgment. Psychological Review, 34(4), 273-286. DOI: 10.1037/h0070288
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276. DOI: 10.3102/00346543068003249
Weimer, M. (2002). Learner-Centered Teaching: Five Key Changes to Practice. San Francisco, CA: Jossey-Bass.
White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3-118. DOI: 10.1207/s1532690xci1601_2
Zhu, W. (1995). Effects of training for peer response on students’ comments and interaction. Written Communication, 14(2), 492-528. DOI: 10.1177/0741088395012004004
