The development of problem solving rubrics to define learning progressions to support the inclusion of students with additional learning needs
In a changing society with evolving needs and occupations, problem solving is increasingly seen as an essential skill for success in many contexts, including school and the workplace. However, students with additional learning needs often lack the necessary support to be successful in this skill domain. The aim of the study described in this paper was to design rubrics for problem solving that define progressive levels of learning to support the instruction of this cohort of students. A working definition of problemsolving was used to draft observation statements in the form of rubrics that teachers could use to describe their students' skill development. Workshops were conducted with specialist teachers (n=13) who critiqued and extended these draft rubrics, and mapped them to a learning progression. These materials were then trialed with experienced special education teachers, who reviewed them and generated intervention strategies for students at different skill levels on the derived progression. The outcome of the study was a set of assessment rubrics and level descriptions that teachers could use to support the learning of students with additional needs in a skill domain often dismissed as too demanding to be included in their personal programs. Assessment rubrics based on teacher observation and judgement The study described in this paper was grounded on a review of the purpose of assessment, signalling a shift from judging student success or failure towards building a stronger understanding of where students are in their learning journey (Griffin, 2007; Masters, 2013) This has led to increased interest in the design and use of rubrics to describe student learning. For the purpose of this study, rubrics were defined as "a set of scoring guidelines that describe the characteristics of the different levels of performance used in scoring or judging a performance" (Gronlund, 1998, p. 225) The central features of rubrics were thus the ordered categories or levels of performance that comprise a description of the cognitive, affective, and psychomotor skills embedded in competent performance (Griffin) Underpinning this concept of rubrics was a criterion referenced interpretation in which an individual's achievement or competence was described in terms of the behaviour they demonstrated or the tasks they performed (Glaser, 1981) The end purpose of the use of assessment rubrics and resulting progression was seen as teachers' capacity to facilitate targeted instruction appropriate for a student's level of learning (Griffin, McGaw Care, 2012; Roberts Griffin, 2009) In Australian schools, students tend to progress with their age cohort rather than being retained at a grade or year level if they have not achieved that level's expected educational standard. As Howes (2012) pointed out, this means that teachers in most classrooms must adapt their teaching to meet the needs of students working across an ability range roughly equivalent to five ageor grade based curriculum levels. Therefore, for many students, including students with additional needs, targeted instruction without age or year level assumptions is essential (Masters, 2013) In his paper on the design and use of assessment rubrics to guide teachers' understanding of their students, and to promote targeted teaching, Griffin (1993) responded to critics who devalued the role of teacher judgement in assessment (e.g. Bailey, 1993; McCloskey, 1990; Neisworth Bagnato, 1988) His contention was that experienced teachers developed a rich store of knowledge about their students, but that this knowledge was often internalised and therefore seemingly automatic. Teachers' judgement based assessment of the things they observed their students doing or saying in everyday classroom interaction was often under valued as too subjective to form the basis of sound instructional decisions. By contrast, Griffin argued that the value of teacher observation and judgement should be The Development of Problem solving Rubrics. AARE Conference 2016 Melbourne, Victoria Page 2 of 12 reconsidered as "direct documentation" of students' learning, and favourably contrasted with the "indirect estimation" (p.3) provided by students' test performance. Yet, it should also be noted that Griffin's support for the role and value of teacher judgement was conditional on teachers having a strong knowledge of their students as learners and of the educational construct being taught. It rested, also, on teachers' use of a sound schema or framework against which to evaluate student performance. In the case of students with additional learning needs, and particularly those educated in mainstream classrooms, the latter stipulation may too often be compromised by teachers' lack of experience working with students at comparable levels of proficiency (Senate Standing Committee on Education and Employment [Senate Committee] 2016) An argument can be made, therefore, for provision of such a framework in the form of assessment rubrics to guide teachers' observations and instructional decisions. This was the purpose of the current study to develop a set of rubrics to guide teachers' decisions about how best to teach problem solving skills to their students with additional learning needs. As a result of a changing society with evolving needs and occupations, problem solving has increasingly been seen as an essential skill for success in many contexts, including school and the workplace (Glevey, 2006; Griffin et al, 2012) One source of evidence for this is its inclusion in large scale assessments, such as the Programme for International Student Assessment (PISA) and the prioritisation of the skill in curricula internationally (Organisation for Economic Cooperation and Development [OECD] 2013) and nationally (Australian Curriculum, Assessment and Reporting Authority [ACARA] 2013) However, a widespread lack of understanding of problem solving as a teachable skill (Robbins, 2011; Yetter Lee, 2007) in combination with low levels of teacher confidence and competence when working with students with additional needs (Senate Committee, 2016) meant that two of Griffin's (1993) necessary conditions for the use of teacher judgement in instructional planning were missing. To date, little is understood about the necessary instructional support for students to be successful in problem solving skills (Agran Hughes, 2005; Clarke, Goos Morony, 2007) particularly for students with additional needs (Agran, Cavin, Wehmeyer Palmer, 2006; Cote et al. 2014; Cote et al. 2010) A number of factors have influenced this situation. These include the influence of textbooks (Carnine, 1991; Hord Newton, 2014; Jitendra, 2011) teachers' attitudes (Lombardi Savage, 1994; Wehmeyer, 2007) lack of time given the volume of mandated curricula (Lang, Mastropieri, Scruggs, Porter, 2004) the hierarchical nature of mathematics learning (Fuchs Fuchs, 2005; Rojewski Schell, 1994; Thornton Jones, 1996) and the role of high stakes assessment (Clarke et al. 2007; Griffin et al. 2012) With this in mind, and to achieve the end of ensuring equitable opportunities for students with additional needs to learn skills that are the foundation for successful problem solving, rubrics were developed and trialed with specialist teachers. In this process, the following research questions were addressed: What components of problem solving need to be focused on at school for students with additional needs? To what extent can a set of problem solving rubrics provide a basis for teachers to plan strategies and targeted interventions to teach this skill?