Show simple item record  

dc.contributor.authorGrzenda, Maciejen_NZ
dc.contributor.authorGomes, Heitor Muriloen_NZ
dc.contributor.authorBifet, Alberten_NZ
dc.coverage.spatialGlasgow, UKen_NZ
dc.date.accessioned2020-12-02T00:18:07Z
dc.date.available2020-12-02T00:18:07Z
dc.date.issued2020en_NZ
dc.identifier.citationGrzenda, M., Gomes, H. M., & Bifet, A. (2020). Performance measures for evolving predictions under delayed labelling classification. In Proceedings of 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). Glasgow, UK: IEEE. https://doi.org/10.1109/IJCNN48605.2020.9207256en
dc.identifier.isbn9781728169262en_NZ
dc.identifier.urihttps://hdl.handle.net/10289/14000
dc.description.abstractFor many streaming classification tasks, the ground truth labels become available with a non-negligible latency. Given this delayed labelling setting, after the instance data arrives and before its true label is known, the online classifier model may change. Hence, the initial prediction can be replaced with additional periodic predictions gradually produced before the true label becomes available. The quality of these predictions may largely vary. Thus, the question arises of how to summarise the performance of these models when multiple predictions for a single instance are made due to delayed labels.In this study, we aim to provide intuitive performance measures summarising the performance of multiple predictions made for individual instances before their true labels arrive. Particular attention is paid to the fact that under the delayed label setting, the emphasis placed on the quality of initial predictions can vary depending on problem needs. The intermediate performance measures we propose complement existing initial and test-then-train performance evaluation when verification latency is observed. Results provided for both real and synthetic datasets show that the new measures can be used to easily rank methods in terms of their ability to produce and refine predictions before the true labels arrive.en_NZ
dc.format.mimetypeapplication/pdf
dc.language.isoen
dc.publisherIEEEen_NZ
dc.rightsThis is an author’s accepted version of an article published in the journal: Proceedings of 2020 International Joint Conference on Neural Networks (IJCNN). ©2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
dc.sourceIJCNN 2020en_NZ
dc.subjectcomputer scienceen_NZ
dc.subjectpredictive modelsen_NZ
dc.subjectdata modelsen_NZ
dc.subjectdelaysen_NZ
dc.subjecttrainingen_NZ
dc.subjecttestingen_NZ
dc.subjectlabelingen_NZ
dc.subjectMachine learning
dc.titlePerformance measures for evolving predictions under delayed labelling classificationen_NZ
dc.typeConference Contribution
dc.identifier.doi10.1109/IJCNN48605.2020.9207256en_NZ
dc.relation.isPartOfProceedings of 2020 International Joint Conference on Neural Networks (IJCNN)en_NZ
pubs.begin-page1
pubs.elements-id258072
pubs.end-page8
pubs.finish-date2020-07-24en_NZ
pubs.publication-statusPublisheden_NZ
pubs.start-date2020-07-19en_NZ


Files in this item

This item appears in the following Collection(s)

Show simple item record