Chasing impact: The tale of three SoTL studies

Keywords: SoTL, impact, evaluation, innovation, innovative teaching

Abstract

This article queries the notion of impact in studies of teaching and learning located within the field of Scholarship of Teaching and Learning (SoTL). Grounded in literature focused on measuring and challenging the impact in SoTL, and primarily on the “what works” question, the author proposes a rubric by which to judge various levels and dimensions of impact achieved in SoTL-focused projects. To operationalize it, the rubric is applied to three completed projects, which while differing in their initial scope and intended outputs were united by a shared goal of improving learning by the means of innovative teaching. By using the rubric to analyze these projects’ outputs, strengths and weaknesses of each project’s design and evaluation methodology are revealed. Diverse levels and dimensions of impact are identified and discussed. The author invites scholars of teaching and learning to use, test, and critique the rubric in the context of their completed or in-progress studies.

Author Biography

Ekaterina Pechenkina, Swinburne University of Technology

Dr Katya (Ekaterina) Pechenkina is Research Fellow in Learning Transformations Unit, Swinburne University of Technology. Katya holds a PhD in anthropology from the University of Melbourne and several other degrees. She was a 2003-2004 International Research and Exchange Board fellow at the California State University Bakersfield, where she majored in sociology. Anthropologist and education researcher, Katya’s research interests encompass the discourses of technology, innovation, teaching excellence, and Indigenous experiences in higher education. Katya is an avid social media user and can be found on Twitter @DrKatya_Pech and Facebook @DrKatyaPech

References

Acai, A., Ahmad, A., Fenton, N., Graystone, L., Phillips, K., Smith, R., & Stockley, D. (2018). The 3M National Teaching Fellowship: A high impact community of practice in higher education. Teaching & Learning Inquiry, 6(2), 50-66. https://doi.org/10.20343/teachlearninqu.6.2.5

Bloch-Schulman, S. (2016). A critique of methods in the scholarship of teaching and learning in philosophy. Teaching & Learning Inquiry, 4(1), 1-15. https://doi.org/10.20343/teachlearninqu.4.1.10

Brew, A., & Ginns, P. (2008). The relationship between engagement in the scholarship of teaching and learning and students’ course experiences. Assessment & Evaluation in Higher Education, 33(5), 535-545. https://doi.org/10.1080/02602930701698959

Cabrera, D., Vartabedian, B. S., Spinner, R. J., Jordan, B. L., Aase, L. A., & Timimi, F. K. (2017). More than likes and tweets: Creating social media portfolios for academic promotion and tenure. Journal of Graduate Medical Education, 9(4), 421-425. https://doi.org/10.4300/JGME-D-17-00171.1

Chick, N. L., & Brame, C. (2015). An investigation of the products and impact of graduate student SoTL programs: Observations and recommendations from a single institution. International Journal for the Scholarship of Teaching and Learning, 9(1), article 3. https://doi.org/10.20429/ijsotl.2015.090103

Downie, J. (2016, July 13). Welcome to online open-access academia: Please mind your head [Blog post]. Retrieved from https://thesiswhisperer.com/2016/07/13/welcome-to-online-open-access-academia-please-mind-your-head/

Ebbinghaus, H. (2013). Memory: A contribution to experimental psychology. Annals of Neurosciences, 20(4), 155-156. Retrieved from http://annalsofneurosciences.org/journal/index.php/annal/article/view/540

Geertsema, J. (2016). Academic development, SoTL and educational research. International Journal for Academic Development, 21(2), 122-134. https://doi.org/10.1080/1360144X.2016.1175144

Gilpin, L. S., & Liston, D. (2009). Transformative education in the scholarship of teaching and learning: An analysis of SoTL literature. International Journal for the Scholarship of Teaching and Learning, 3(2), article 11. https://doi.org/10.20429/ijsotl.2009.030211

Hoessler, C., Britnell, J., & Stockley, D. (2010). Assessing the impact of educational development through the lens of the scholarship of teaching and learning. New Directions for Teaching and Learning, 2010(122), 81-89. https://doi.org/10.1002/tl.400

Hubball, H., & Clarke, A. (2010). Diverse methodological approaches and considerations for SoTL in higher education. Canadian Journal for the Scholarship of Teaching and Learning/ La revue canadienne sur l’avancement des connaissances en enseignement et en apprentissage, 1(1), article 2. http://dx.doi.org/10.5206/cjsotl-rcacea.2010.1.2

Hutchings, P. (2000). Introduction: Approaching the scholarship of teaching and learning. In P. Hutchings (Ed.), Opening lines: Approaches to the scholarship of teaching and learning (pp. 1-10). Menlo Park, CA: Carnegie Foundation for the Advancement of Teaching.

Hutchings, P., Borin, P., Keesing-Styles, L., Martin, L., Michael, R., Scharff, L., Simkins, S., & Ismail, A. (2013). The scholarship of teaching and learning in an age of accountability: Building Bridges. Teaching & Learning Inquiry, 1(2), 35-47. https://doi.org/10.20343/teachlearninqu.1.2.35

Hwang, A., & Arbaugh, J. B. (2009). Seeking feedback in blended learning: Competitive versus cooperative student attitudes and their links to learning outcome. Journal of Computer Assisted Learning, 25(3), 280-293. https://doi.org/10.1111/j.1365-2729.2009.00311.x

Jaarsma, A. S. (2015). On being taught. Canadian Journal for the Scholarship of Teaching and Learning/ La revue canadienne sur l’avancement des connaissances en enseignement et en apprentissage, 6(2), article 6. https://doi.org/10.5206/cjsotl-rcacea.2015.2.6

Kelley, P., & Whatson, T. (2013). Making long-term memories in minutes: A spaced learning pattern from memory research in education. Frontiers in Human Neuroscience, 7, article 589. https://doi.org/10.3389/fnhum.2013.00589

Kreber, C., & Brook, P. (2001). Impact evaluation of educational development programmes. International Journal for Academic Development, 6(2), 96-108. https://doi.org/10.1080/13601440110090749

Land, R., & Gordon, G. (Eds.). (2013). Enhancing quality in higher education: International perspectives. Milton Park, UK: Routledge.

Light, G., Calkins, S., & Cox, R. (2009). Learning and teaching in higher education: The reflective professional (2nd. ed.). Los Angeles, CA: Sage.

Locke, W. (2014). Shifting academic careers: Implications for enhancing professionalism in teaching and supporting learning. York: Higher Education Academy. https://www.advance-he.ac.uk/knowledge-hub/shifting-academic-careers-implications-enhancing-professionalism-teaching-and

McCarthy, J. (2010). Blended learning environments: Using social networking sites to enhance the first year experience. Australasian Journal of Educational Technology, 26(6), 729-740. https://doi.org/10.14742/ajet.1039

McGee, P. (2014). Blended course design: Where’s the pedagogy? International Journal of Mobile and Blended Learning, 6(1), 33-55. https://doi.org/10.4018/ijmbl.2014010103

Miller, A. (2013). Improving SoTL programs: The impact of a student sector. InSight: A Journal of Scholarly Teaching, 8(1), 44-50. Retrieved from https://files.eric.ed.gov/fulltext/EJ1016760.pdf

Oates, G., Pechenkina, E., Laurence, D., Eldridge, D., & Hunter, D. (2016a, July). Increasing academic performance and engagement through push notifications, badges and leaderboards: Is there an app for that? Paper presented at Accounting Education @ the Edges, conference of the Accounting and Finance Association of Australia and New Zealand, Gold Coast, Australia.

Oates, G., Pechenkina, E., Laurence, D., Eldridge, D., & Hunter, D. (2016b, September). Push notifications, digital badges and leaderboards: Evaluating the impact of Quitch mobile application on learning. Paper presented at Transforming Learning Conference, Melbourne, Australia.

Pechenkina, E., Lawrence, D., Oates, G., Eldridge, D., & Dan, H. (2017). Using a gamified mobile app to increase student engagement, retention and academic achievement. International Journal of Educational Technology in Higher Education, 14, article 31. https://doi.org/10.1186/s41239-017-0069-7

Pechenkina, E., Scardamaglia, A., & Gregory, J. (2018). “It’s the combination that works”: Evaluating student experiences with a multi-element blended design in first-year law. Contemporary Educational Technology, 9 (4), 405-422. https://doi.org/10.30935/cet.471019

Poole, G., & Simmons, N. (2013). Contributions of the scholarship of teaching and learning to quality enhancement in Canada. In R. Land & G. Gordon (Eds.), Enhancing quality in higher education: International perspectives (pp. 118-128). London, UK: Routledge.

Prince, M. J., Felder, R. M., & Brent, R. (2007). Does faculty research improve undergraduate teaching? An analysis of existing and potential synergies Journal of Engineering Education, 96(4), 283-294. https://doi.org/10.1002/j.2168-9830.2007.tb00939.x

Reid, D., & Pechenkina, E. (2016, October). To BYOD or not BYOD: That is the question? Investigating student preferences for mobile learning. Paper presented at Mobile Learning Futures: Sustaining Quality Research and Practice in Mobile Learning, Sydney, Australia.

Scardamaglia, A., & Pechenkina, E. (2015, September). Evaluating innovative blended delivery in a first-year law unit. Paper presented at the Transforming Learning Conference, Melbourne, Australia

Schroeder, C. (2007). Countering SoTL marginalization: A model for integrating SoTL with institutional initiatives. International Journal for the Scholarship of Teaching and Learning, 1(1), article 15. https://doi.org/10.20429/ijsotl.2007.010115

Sen, T. K. (2013). Blended and traditional teaching strategies for a final year advanced separation processes unit: An effective course delivery. In Institution of Engineers Australia (Ed.), Chemeca 2013: Challenging Tomorrow (pp. 32-38). Barton: Engineers Australia.

Shulman, L. S. (2005). Signature pedagogies in the professions. Daedalus, 134(3), 52-59. https://doi.org/10.1162/0011526054622015

Simmons, N. (2009). Playing for SoTL impact: A personal reflection. International Journal for the Scholarship of Teaching and Learning, 3(2), article 30. https://doi.org/10.20429/ijsotl.2009.030230

Simmons, N. (2016). Synthesizing SoTL institutional initiatives toward national impact. New Directions for Teaching and Learning, 2016(146), 95-102. https://doi.org/10.1002/tl.20192

Sutherland, K. A. (2015). Language. Teaching & Learning Inquiry, 3(2), 109-110. https://doi.org/10.20343/teachlearninqu.3.2.109

Trigwell, K. (2013). Evidence of the impact of scholarship of teaching and learning purposes. Teaching & Learning Inquiry, 1(1), 95-105. https://doi.org/10.20343/teachlearninqu.1.1.95

Vo, H. M., Zhu, C., & Diep, N. A. (2017). The effect of blended learning on student performance at course-level in higher education: A meta-analysis. Studies in Educational Evaluation, 53, 17-28. https://doi.org/10.1016/j.stueduc.2017.01.002

Werbach, K., & Hunter, D. (2015). The gamification toolkit: Dynamics, mechanics, and components for the win. Philadelphia, PA: Wharton Digital Press.

Werder, C., Pope-Ruark, R., Verwoord, R. (2016). Students as co-inquirers (Special section guest editors’ introduction). Teaching & Learning Inquiry, 4(2), 1-3. https://doi.org/10.20343/teachlearninqu.4.2.2

Yin, R. K. (2013). Case study research: Design and methods (5th ed.) London: Sage.

Published
2020-03-15
How to Cite
Pechenkina, Ekaterina. 2020. “Chasing Impact: The Tale of Three SoTL Studies”. Teaching & Learning Inquiry 8 (1), 91-107. https://doi.org/10.20343/teachlearninqu.8.1.7.