In the field of aerial and satellite remote sensing, the widespread adoption of deep learning brings new possibilities. Current approaches, however, often overlook the unique characteristics of aerial data. This study introduces a methodology that capitalizes on distinctive features, leveraging additional annotations for enhanced neural network training. Despite modest gains in classification accuracy, the synergy of enhanced explainability, automated segmentation, and targeted classification demonstrates nuanced improvements. Preliminary results showcase potential applications in land cover mapping. This work can be extented towards reducing dependency on labor-intensive human annotations through an iterative annotation and training loop.