Threshold Concepts in the Development of Problem-solving Skills

<p>Problem-solving skills are often identified as a key component of 21<sup xmlns:m="http://www.w3.org/1998/Math/MathML" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> century education. This study collected data from students enrolled in a university-level Liberal Education science course called <i>Problems and Puzzles</i>, which introduced students to the theory and practice of problem solving via puzzles. Based on classroom observation and other qualitative data collected over three semesters, we have identified three significant changes in student behaviour at specific points in the course. These changes can be posited to reveal three underlying threshold concepts in the evolution and establishment of students’ problem-solving skills.</p>


INTRODUC TION
Problem solving is recognized among the criti cal attributes identified in Kay's 21 st century learning framework (2010). Specific problem-solving skills such as analy sis, transfer and metacognition also fig ure prominently in the revised Bloom's taxonomy (Bloom, 1956;Krathwohl, 2002) and in the facets of understanding described by Wiggins and Mc-Tighe (2005). But how might we ensure that our students learn and practice such skills?
At the post-sec ondary level, problem-solving courses are regularly offered within the confined contexts of particular disciplines such as engineering, business or medicine, or to develop specific and distinct skills in subjects like mathematics or computer science. The problem-based learning approach (Savery, 2006;Wilkerson & Gijselaers, 1996), which grew out of education for medical students, also tends to focus on student learning in a content area through the use of open-ended problems in that area that students are required to research and address. These approaches are generally disposed to using problem solving to teach content rather than using content to explicitly teach problem solving skills.
We have studied the development of problem-solving skills in a less content-focused environment, as a learning outcome rather than using "problem-based learning" to achieve other content or context specific learning objectives. The locus of our study is a Liberal Education problem-solving course for university students called Problems and Puzzles. The course approaches the study of problem solving and the development of problemsolving skills in a general way, using puzzles as a learning vehicle to help students develop a generic and transferable set of problem-solving skills. Pedagogically, we are interested in whether there are particular skills or strategies that are helpful to students at different times during the course, and in our research study we have looked for changes in student behaviour that could indicate development of strategic problem-solving skills. We also wish to explore whether observable behavioural changes are related to threshold concepts, in the sense suggested by Meyer and Land (2003). Meyer and Land define threshold concepts in a particular area of study as concepts which students must understand in a deep way in order to truly understand the area. Such threshold concepts share five key characteristics: they are irreversible, integrative, bounded, initially troublesome, and transformative. Once students attain a deep understanding of such a concept, there is no going back; the new understanding integrates all previous knowledge into a transformed understanding of the subject, and also delineates its boundaries from other related subjects. Such knowledge, and especially the process of gaining it and transforming one's understanding, can of en be difficult and troublesome for students, as it involves changing and rearranging previous conceptions and misconceptions.
The early literature on threshold concepts focuses on specific content-based concepts; for instance the main examples given in the seminal paper by Meyer and Land (2003) are opportunity cost in economics, complex numbers and limits in calculus, and signification in literary and cultural studies. More recent papers deal with a wide range of disciplines, in clud ing among many others history (Adler-Kassner, Majewski & Koshnick, 2012), en gineering (Adawi & Kabo, 2012) and project management (Hokstand et.al., 2012). Schwartzman (2010) argues that, while Meyer and Land "construe approaches to teaching within the [threshold concept] framework as entirely discipline-specific, and have eschewed those that transcend disciplinary boundaries" (p. 22), there are crucial commonalities across disciplines in how students confront and deal with difficult or troublesome knowledge. Middendorf and Pace (2004) have described a "decoding the disciplines model [for] helping students learn disciplinary ways of thinking" (p.1) which can be applied broadly across all disciplines, and Diaz, Middendorf, Pace and Shopkow describe "bottlenecks" as "places where significant numbers of students are unable to grasp basic concepts or successfully complete important tasks" (2008, p. 212). We too wish to broaden the scope of threshold concepts. Our course and our research approach are both more process-oriented than content-oriented, and use a metacognitive lens to look at threshold processes as they might apply in the general development of problem-solving skills.

THE COURSE
The Problems and Puzzles course is taken as a science elective by a broad range of students from a number of faculties (Arts and Science, Fine Arts, Management, and Education) and a wide variety of majors. As a Liberal Education course, it introduces students to the problem-solving literature from a broad perspective, in clud ing history and anthropology of problems and problem solving, some mathematical problems and approaches, cognitive and metacognitive approaches, and connections to larger "real-world" problems. It presents a number of problem-solving and metacognitive strategies, and most importantly, through out the semester provides students with significant hands-on experience tackling and solving new problems in the form of puzzles. The class meets for a fifyminute period three times a week, for a total of thirteen weeks. Although the strategies, topics and puzzles used are carefully sequenced, lecture instruction during class time is kept to a minimum, with the focus on students working collaboratively or independently in class to apply the problem-solving skills and concepts taught to the solution of actual problems presented as puzzles. In almost every class students are given time to work on a new puzzle, followed by a class "debriefing" of different solutions and strategies used.

THE STUDY
The research described here is one component of a three-year on-going mixedmethods research project. To date we have collected a variety of quantitative and qualitative data for each of the three times this course has been offered; from a total enrolment of 140 students, we had n = 100 study participants. We obtained research ethics approval from our institution for this on-going study (Protocol # 2011-084), and efforts were made to ensure appropriate confidentiality and anonymity of participants. Students were informed of the on-going research, were given the choice to participate, were able to opt out at any time, and were assured that the instructor of the course would not know who was participating. Data was collected, coded and analyzed by a research assistant, afer the course had ended and all grades were submitted.
The data we collected consists of three main types. First, we conducted in-class observations, by the course instructor, another faculty colleague and a teaching assistant, the latter two of whom each attended about two-thirds of the classes. Our quantitative data consists of students' responses to three Likert-scale surveys: a demographics and attitudes survey conducted both pre-and post-course, and thinking style (Gregorc, 1979) and learning style (Barsch, 1991) surveys. Analysis and discussion of quantitative data has been reported more specifically elsewhere (Wismath, Orr & Good, 2014;Wismath, Orr & Zhong, 2013;. Qualitative data collected from the voluntary student participants in this study forms the primary source for the component of our research described here, and includes a series of guided reflection assignments that were assigned as part of the course (and graded for completion only), and transcripts from two different post-course voluntary focus groups. The reflection assignments (collected over three semesters) formed part of the metacognitive emphasis of the course, encouraging students to reflect on their problem solving and their progress, but were also designed to provide qualitative information related to aspects we wished to investigate, such as collaborative work, metacognitive activity, and transfer of skills. To ensure that participants could speak freely about their experiences, we also had research assistants not connected in any way with the course or project conduct two small-group post-course focus-group discussions, in 2012 and 2013, using similar questions to those in the reflections.
In this paper we are reporting primarily on the qualitative data we collected, using both in-class observations and participants' oral and written comments. We describe our observation of several changes in student behaviour displayed during the course, and theorize how the patterns we observed can be linked to the general theory of threshold concepts, as discussed above. Naturally, research of this nature-based primarily on observations, reflections and surveys of students in a unique context-evokes examination of the "validity" and "reliability" of the data collection and analy sis processes (Mayan, 2009). Mayan posits that research which is necessarily contextual in nature is better defined by a "rigor" which demonstrates "why . . . the findings of a particular inquiry are worth paying attention to" (p. 100). Such rigor may be linked to the "credibility" of the descriptions and explanations presented by the research ( Janesick, 1998), and to the contextual relevance of the knowledge gained (Altheide & Johnson, 1998). The results reported and paradigms presented in this facet of our study are certainly contextually specific, but are additionally, we suggest, credible in a broader applicability.

THE PERSISTENCE THRESHOLD
The first shif in student behavior we have observed in the Problems and Puzzles course occurred noticeably afer about three weeks of classes. Until that point, observed student behavior varied widely when students were presented with a new puzzle to work on. Many students seemed unable to begin; they appeared to sit and stare blankly at the board or handout, apparently unsure where to begin, and of en waited for the instructor or teaching assistant to come by and provide help. Student comments referred to feeling "very anxious because I never know where to start", and to getting "really anxious when I see puzzles and I don't even know where to start." Others seemed to have some initial ideas about what to do, but would appear to get stuck at some point and then wait for help. By the fourth week of class, however, this behavior visibly changed: all students now started working on each puzzle on their own, appearing to have some idea of how to begin, and even if observed to be experiencing some difficulty would consult each other or their notes for help and keep working. Students appeared to move from being scared, anxious and unable to start or sustain progress, to a point where they now seemed prepared to engage with each problem, and appeared to have strategies and techniques they were confident in using. In the final reflection assignment, almost all the students alluded to this change, as they wrote about having a tool-kit of strategies, and knowing where to start and what to do when they experienced difficulties. Comments from the reflection assignments support this apparent change in behaviour.
Just having a list of problem-solving strategies has helped me immensely! I also feel I have developed the knowledge of what to do when I become stuck on a problem. Now I know that when I am stuck on a problem there are countless ways to approach it, to break it down, to simplify it or to look at it from a different perspective that may lead to me finding a solution faster, and more elegantly than I would have otherwise.
My most valuable problem solving strategy I learned was to try a strategy, but if it's not going anywhere with the puzzle, try another one, and then another. This has helped me to solve many puzzles during class time.
I can look at the problems in more ways than I used to. This makes starting a problem easier and makes it easier to get past "road blocks" that I come across because I have become more open minded about what techniques I am willing to use.
These comments illustrate the participants' realization that, while having a tool-kit of strategies is useful, mental habits of patience and persistence are important parts of problem solving, especially at points where one experiences difficulties (Polya, 1945). The strate gies students learned seemingly helped them to start towards and persist to the achieve ment of a solution; and seemingly increased their confidence in their abilities to do so. More importantly, we posit that a mental habit of persistence and willingness to keep working when confounded was developed. Our first suggested threshold concept then is the importance of persistence for success in problem solving. Almost all the students be gan the course reporting limited confidence in their abilities to tackle problems (Wis math, Orr and Zhong, 2013), and expressed a belief that the key attributes involved in successful problem solving were intelligence and creativity. While these are important, it is in fact of en persistence which leads to the most success: knowing what to do when you are stuck, and being willing to keep going rather than giving up (Polya, 1945). Gaining a deep understanding that most problems are solvable with sustained effort we suggest marks a fundamental and irreversible threshold in students' development of problemsolving skills. The "reliability" of this assertion may be suggested by the significant increase in reported levels of acquired problem-solving skill and confidence in those skills among student participants (Wismath, Orr and Zhong, 2013).

THE PROCESS OVER ANSWER THRESHOLD
A sec ond change we observed in student behaviour and comments began around the eighth week of the course, but was not as universally apparent as the first shif. Indeed, it occurred more gradually, and only about a quarter of the students showed direct indications of this change. The first indication of this sec ond change occurred afer the class worked on a particularly long and challenging puzzle, which not everyone successfully completed during class time. One student remarked that she knew the answer to the problem (a particular number) but that that answer meant nothing to her; instead she indicated a strong desire to know why that answer was correct and what that particular number had to do with the pattern she and others had observed in the problem. At this point, this student and some others appeared to have moved beyond simply being satisfied to determine a correct answer, and seemed to begin to focus more on deeper understanding, and more specifically on the problem-solving process rather than on the answer. Students at this point additionally demonstrated an interest in finding multiple solutions to a problem, and looking for faster or better solutions, rather than simply being content to find "the answer" and stop working. Two comments, one from a reflection assignment and one from the focus group, elucidate this attitude.

What is central to the knowledge learned in this class is not just "this is the answer to this puzzle," it is HOW one got to the answer. It is the process that is important; this includes the moment one is confronted with a problem and how they proceed. I guess I do like a lot of puzzles for fun because I'm nerdy like that, but I found I would like, get the puzzles before, but now I know why I'm getting them which is nice because before I would look at something and be like 'oh, okay I know this is the answer it would be the right answer but I didn't know why it was the right [answer].
Another participant in the focus group described a similar shif, based on her experience at writing puzzle solutions for regular homework assignments. Here again we suggest there is a change in attitude, from "I got an answer and I'm done" to a search for deeper understanding and self-efficacy.

its like 'I got this and that's the answer, but that was it'. And I think as the course went on to do well on the assignment you had to be able to explain yourself. And then it became 'of course I'm going to explain myself -because that is how you do a puzzle now.' [. . .] that's the way it makes most sense, because you have to explain it to get the answer, at least for me.
Another facet of this transition in orientation from answer to process is the realization that mistakes are not an indication of failure, but rather an integral part of the learning process. As one student wrote, "I now know that I can use a mistake as a learning experience." Another student comment from a focus group discussion also reports this shif: [Y]ou [. . .]

learn from your mistakes, like if that didn't work, don't stop there, find out why or use that to help try again. Whereas before [. . .] I would totally be, like, 'I'm done!'
The sec ond threshold concept we posit, therefore, is a metacognitive awareness that the process of problem solving is more important than getting an answer. Although not all the students in our study appeared to reach a deep understanding of this concept, some of them clearly demonstrated a realization that the process of thinking one's way through complex ideas is more important than getting "the right answer."

THE CAREFUL MODELING THRESHOLD
A final apparent change in student activity in class was observed to occur during the last three weeks of the semester. At this time, the instructor and other observers became aware of a marked increase in questions being asked by students as they started tackling a class puzzle. Earlier in the course, students had generally been observed to plunge directly into their work, asking questions only near the end of the class and of en apparently only if they got stuck or needed a hint. We observed by this point, however, that students started asking questions immediately afer their first or sec ond reading of the problem. These were usually clarification questions, such as: "Do you think it means this?" "Can we assume that . . .?" "Are we supposed to find this, or that?" We link this change to the distinction between "novice" and "expert" approaches to problem solving identified in the cognitive psychology literature (Pretz, Naples and Sternberg, 2003). Lesgold (1988) describes how experts spend more time at the start of a problem, determining a suitable representation and approach strategy, and less time actually working out the details later. Novices, on the other hand, tend to quickly choose a model and strategy and plunge into the problem, but of en spend more time backtracking later if their strategy does not work out or they have missed important details. Many of the participants in our study, while they did not necessarily become experts, did seem to move beyond novice status, in the sense of learning to invest careful consideration to a model at the start of a problem, using context and information given, in order to avoid wasting time later. A number of student comments on assignments and in the focus group reflect this shif.

I have started to be more careful when starting a problem and looking at what I have and want to decide what strategy would work best. I would normally jump right into a problem and start trying to fig ure it out without looking at the problem at a whole. I used to have a tendency to skim over instructions and questions (even on tests) so first and foremost, I know that carefully reading the problem is fundamental.
I have learned the importance of reading a question carefully. Knowing what the in di vidual elements of a problem are is so essential to finding a solution. I used to quickly skim problems and dismiss relevant information.
I think the most important thing I learned in the class was to really think about the problem before attempting to find solutions. When I take the time to step back and assess a situation or a problem I can make a good judgment on which process to use and it ensures I fully understand the problem. I find that this can make a big difference on finding the most efficient solution.

One of the most important problem solving skills [. . .] is to really understand the problem. I had a bad habit of reading the question and jumping in too fast. This sometimes led to me getting the wrong answer since I didn't always have the right information. I have learned to take my time and make sure I know exactly what the question is asking.
Our suggestion then is that a third threshold concept in problem-solving involves the realization of the importance of careful and complete modeling of a problem before one plunges into tackling it. Interestingly, this relates to the careful reading and understanding of a problem or puzzle which is the first step in all the many versions of problem-solving steps based on the classic four-step method by Polya (1945). This was discussed in our course in the initial lectures, and yet for most students a deep understanding of this concept was not reached until near the end of the course.

THRESHOLD CONCEPT CHARAC TERISTICS
From a pedagogical perspective, we believe that the changes in student activity and understanding which we have observed can be consciously and intentionally facilitated for students. We have also suggested that our observations reveal three key threshold concepts underlying the development of problem-solving skills. All are metacognitive concepts, dealing with an understanding of the processes of problem solving rather than a specific content, reflecting the importance of metacognition in problem solving (Flavell, 1979(Flavell, , 1987. It is worth noting that, although we have identified these concepts in an order based on our class-room observations, they do not necessarily occur in that order for in di vidual students. The first change occurred early on and for all students; the third change occurred late in the course, for about three-quarters of the students; the sec ond shif, from answer-only to process-focused, occurred more gradually over the duration of the course, and fewer than half the participants showed direct evidence of this change in approach. Further study on the inter-relationships of these concepts would be useful. Threshold concepts are irreversible. They integrate an area of study, establish boundaries distinguishing that area from other areas, are initially troublesome, and are transfor-mative for those who grasp them. We believe that the comments from our study participants do confirm these defining characteristics of threshold concepts. The three concepts suggested here appear to be irreversible, as ways of looking at problem solving: students who have mastered these concepts do not move backwards. These deeper understandings help students integrate into a new framework the vari ous problem-solving strategies and techniques they have learned. The acquisition of deeper understanding is additionally, to some extent, dependent on a troublesome process. Most students began the course nervous and afraid, and of en appeared to struggle with the open-ended nature of the in-class time. Although the course sequence and puzzles were carefully structured, with very little formal lecturing being done in class, students were expected to fully engage with their own learning and to teach themselves how to problem-solve through intensive experience. Students were observed to struggle at first with what to do or what was expected of them. One student for instance worried about whether he was supposed to hand in solutions to in-class problems or memorize them for tests. In fact, the overall process of the course put students continually in that "liminal state" described by Cousin (2006), in which they appeared to feel uncomfortable uncertainty during their continuous transformation from one way of thinking and knowing to another.
The transformative nature of the concepts, as realized during the course experience, was revealed in a number of student comments. Many of the students commented that they had previously found problems intimidating, but "now just because we have done this class we see that puzzles are sometimes fun and they are not as intimidating as they seem and that they are solvable." One student who spoke in the focus group about her math anxiety talked about now "being a problem solver", showing the transformation she had undergone: I found this class, . . . for me being a problem solver . . . helped me take a step back and kind of view it differently. . . . I'm not a math major, I don't take math classes; this is the only one I have ever taken. And I get really anxious when I see puzzles (and I don't even) know where to start. Now I . . . feel like I can -actually found a couple that actually, I can enjoy.
The importance of an increase in confidence was clearly illustrated in one participant's comments at the end of the course: I honestly think that I have developed the confidence in my problem solving abilities to be better able to explain strategies and processes. [. . .] It has taught me that if I put my mind to a task or complicated assignment and stick to it, I will eventually find the answer I am looking for. Trying this course that was completely outside my comfort zone only developed characteristics that established the self-belief that I have the ability to achieve anything I put my mind to.
The transformative nature of the mastery of the threshold concepts was particularly evident in one focus group conversation. A student remarked that "I think this class should be mandatory [. . .] for first year students, imagine how valuable this would have been in your first year of university." Another student agreed that in "first year it would have been very, very helpful." A third student then replied that "I think if I would have taken this in my first year, it would have been significantly . . . more life changing." For these students the course learning did indeed transform their academic life.

DISCUSSION AND CONCLUSIONS
Our theories about metacognitive thresholds in problem solving are based on sustained observation by three instructors over three semesters, as well as students' oral and written comments. An obvious next step for our research is to develop a more precise way to measure such shifs, perhaps through direct questions on assignments or other more specific interrogation of students. In general, we acknowledge the difficulty inherent in attempting to identify specific markers of progress in the development of problem-solving skills, but offer our results as a starting point for further study. We also note that we do not measure whether students have in fact developed as problem solvers, other than by their own report. Our quantitative survey data suggest that students experienced a significant shif in confidence and perceived ability (Wismath, Orr & Zhong, 2013;Wismath & Zhong, 2013), but this is not an objective measure of problem solving skill.
While it is clear that our society needs educated citizens who are good problem solv ers (Conference Board of Canada, 2013), it is not always clear how good generic prob lem solving skills can best be developed in our students. Our study suggests three dis tinct threshold concepts in the learning of problem-solving skills. For those teaching problem-solving skills, we suggest that mastery of these concepts can be emphasized with learned experiences structured to facilitate development. For example, the first threshold can be encouraged by providing a sequence of introductory puzzles at the right level to illustrate a number of different strategies, along with a supportive classroom environment that emphasizes effort and engagement rather than competition or grades. Modeling metacognitive discussion in class, and requiring metacognitive reflection by students on assignments, can promote an increasing awareness of problem solving as a journey and not a destination or answer only. Allowing students to fig ure out how to learn from their mistakes can also enhance an appreciation of the process of problem solving, rather than the process of getting a quick right answer.
We have suggested here a broadening of the study of threshold concepts, away from purely content-driven concepts to more metacognitive concepts, which might be used to study how students develop an understanding of processes as well as content areas.
This suggestion has been illustrated by our work on threshold processes in the development of problem solving skills. Good problem solving requires a constellation of skills, attitudes and mental approaches that, once developed, transforms the way people look at the world around them. In this sense, a course in problem solving can become "a portal, opening up a new and previously inaccessible way of thinking about something" (Meyer and Land, 2003, p. 1).

ACKNOWLEDGEMENT
This work received funding from the Teaching Development Fund of the University of Lethbridge in 2010, 2012, and 2013.

Shelly Wismath is a Professor at University of Lethbridge in the Departments of Mathematics and
Computer Science and the Liberal Education Program.