Loading...
Thumbnail Image
Item

Text categorization and similarity analysis: similarity measure, architecture and design

Abstract
This research looks at the most appropriate similarity measure to use for a document classification problem. The goal is to find a method that is accurate in finding both semantically and version related documents. A necessary requirement is that the method is efficient in its speed and disk usage. Simhash is found to be the measure best suited to the application and it can be combined with other software to increase the accuracy. Pingar have provided an API that will extract the entities from a document and create a taxonomy displaying the relationships and this extra information can be used to accurately classify input documents. Two algorithms are designed incorporating the Pingar API and then finally an efficient comparison algorithm is introduced to cut down the comparisons required.
Type
Working Paper
Type of thesis
Series
Computer Science Working Papers
Citation
Fowke, M., Hinze, A., & Heese, R.(2013). Text categorization and similarity analysis: similarity measure, architecture and design. (Working paper 12/2013). Hamilton, New Zealand: University of Waikato, Department of Computer Science.
Date
2013-12
Publisher
University of Waikato, Department of Computer Science
Degree
Supervisors
Rights
© 2013 Michael Fowke, Annika Hinze, Ralf Heese.