Automatic species identification from images for Aotearoa

Abstract

Image classification for species identification has applications in areas such as conservation and education. Given New Zealand's geographic isolation and the relatively small number of species present on its islands, there is an opportunity to apply machine learning to enable accurate automatic species identification for Aotearoa, even on mobile devices without Internet access. We present neural network-based image classification models trained to classify organisms present in New Zealand. The data for model development and evaluation, obtained from the crowd-sourcing website iNaturalist, comprises 14,991 species, including 6,216 Animalia, 6,173 Plantae, and 2,407 Fungi species, alongside a small set of observations of Bacteria, Chromista, Protozoa, and Viruses. It contains organisms observed in the natural environment as well as captive and cultivated organisms. The trained models achieve over 76% classification accuracy across all species and produce class probability estimates, calibrated using temperature scaling, that can be used to gauge confidence in their classifications. Input attribution methods can be used to interpret a model's inferences by highlighting its areas of focus on images. The models are available to the public as downloadable model files and as part of both web and mobile applications for species identification that are distributed as open-source software.