Loading...
Thumbnail Image
Item

Extracting corpus specific knowledge bases from Wikipedia

Abstract
Thesauri are useful knowledge structures for assisting information retrieval. Yet their production is labor-intensive, and few domains have comprehensive thesauri that cover domain-specific concepts and contemporary usage. One approach, which has been attempted without much success for decades, is to seek statistical natural language processing algorithms that work on free text. Instead, we propose to replace costly professional indexers with thousands of dedicated amateur volunteers--namely, those that are producing Wikipedia. This vast, open encyclopedia represents a rich tapestry of topics and semantics and a huge investment of human effort and judgment. We show how this can be directly exploited to provide WikiSauri: manually-defined yet inexpensive thesaurus structures that are specifically tailored to expose the topics, terminology and semantics of individual document collections. We also offer concrete evidence of the effectiveness of WikiSauri for assisting information retrieval.
Type
Working Paper
Type of thesis
Series
Computer Science Working Papers
Citation
Milne, D., Witten, I.H. & Nichols, D.M. (2007). Extracting corpus specific knowledge bases from Wikipedia. (Working paper series. University of Waikato, Department of Computer Science. No. 03/2007). Hamilton, New Zealand: University of Waikato.
Date
2007-06-01
Publisher
University of Waikato, Department of Computer Science
Degree
Supervisors
Rights