Evaluation approaches for a national ICT initiative: the example of laptops for New Zealand teachers

Abstract

This paper will highlight the successful evaluation approaches deployed for long-term exploration of the impact of a national policy initiative as well as some of the results and outcomes. An interactive feedback process informed decision-making at the national and local level thereby enhancing both the initiative and its implementation in schools. The integration of information and communication technologies (ICT) into schools has been a focus of a plethora of policy initiatives by governments worldwide for over 20 years. The provision of laptops to teachers is one component of the New Zealand compulsory school sector ICT strategy. The evaluation research design was to use a mixed-methods approach incorporating three yearly cycles of annual nationwide surveys, regional focus groups, and school-based longitudinal case studies. Randomized sampling was used to identify 20% of the available laptop schools as potential participants in the survey. The same schools participated in each of the 3 years. A distinctive feature of this evaluation has been the regular feedback loops between the evaluators and the stakeholders. The stakeholders included the Ministry of Education and laptop suppliers as well as schools, school leaders, and teacher unions. In the current accountability, environment schools are increasingly being asked to participate in evaluation studies. In long-term studies, it is important to establish effective relationships with the schools and the policy-makers and to deploy robust evaluation methodologies. This is essential for both validity and reliability of the findings and the useability and utilization of the findings.