Domain-independent automatic keyphrase indexing with small training sets
Medelyan, O. & Witten, I.H. (2008). Domain-independent authomatic keyphrase indexing with small training sets. Journal of American Society for Information Science and Technology, 59(7), 1026-1040.
Permanent Research Commons link: https://hdl.handle.net/10289/1734
Keyphrases are widely used in both physical and digital libraries as a brief, but precise, summary of documents. They help organize material based on content, provide thematic access, represent search results, and assist with navigation. Manual assignment is expensive because trained human indexers must reach an understanding of the document and select appropriate descriptors according to defined cataloging rules. We propose a new method that enhances automatic keyphrase extraction by using semantic information about terms and phrases gleaned from a domain-specific thesaurus. The key advantage of the new approach is that it performs well with very little training data. We evaluate it on a large set of manually indexed documents in the domain of agriculture, compare its consistency with a group of six professional indexers, and explore its performance on smaller collections of documents in other domains and of French and Spanish documents.