Applying machine learning to subject classification and subject description for information retrieval
Cunningham, S. J. & Summers, B.(1995). Applying machine learning to subject classification and subject description for information retrieval. (Working paper 95/20). Hamilton, New Zealand: University of Waikato, Department of Computer Science.
Permanent Research Commons link: https://hdl.handle.net/10289/1097
This paper describes an experiment in applying standard supervised machine learning algorithms (C4.5 and Induct) to the problem of developing subject classification rules for documents. These algorithms are found to produce surprisingly concise models of document classifications. While the models are highly accurate on the training sets, evaluation over test sets or through cross-validation shows a significant decrease in classification accuracy. Given the difficult nature of the experimental task, however, the results of this investigation are promising and merit further study. An additional algorithm, 1R, is shown to be highly effective in generating lists of candidate terms for subject descriptions.
University of Waikato, Department of Computer Science
- 1995 Working Papers