Explicating the value of standardized educational achievement data and a protocol for collaborative analysis of this data

Abstract

Governments expect teachers to be able to make sense of and take action on data at various levels of aggregation. In our research we collaborated with 13 teachers from six primary schools and one intermediate school to use a Data Conversation Protocol to analyze and act on mathematics assessment data generated through a standardized assessment tool—the Progressive Achievement Test (PAT). Our intention was to optimize teacher use of this data for pedagogical decision making and action. At team meetings, the teachers co-constructed then refined a taken-as-shared definition for teacher data literacy for instructional action, which acted to inform and anchor our collaborative research. Data were collected in all teacher meetings and via interviews. Initial findings indicate that a ‘Data Conversation Protocol’ is helping teachers to slow down the process of considering, interpreting and making a judgement about their students’ understanding thereby opening up a space for deeper consideration of the range of possible reasons for student responses to assessment items. Students responded positively to teachers’ data informed small group teaching, gaining in understanding and confidence. Teachers considered this confidence translated to more positive engagement with mathematical ideas. Patterns and trends in student responses emerging from the teachers’ collaborative analysis of standard data supported a shift from viewing student responses as linked to student or school characteristics to critical analysis of how their teaching approaches might have contributed to student answers/misunderstandings. This finding has implications for how we might challenge assumptions about students through a willingness to engage critically with student achievement data. The importance of teachers having a rich pedagogical content knowledge as a basis for this was clearly evident.