Ian Witten – Professor of Computer Science, University of Waikato
Wikipedia is a goldmine of information; not just for its many readers, but also for the growing community of researchers who recognize it as a resource of exceptional scale and utility. It represents a vast investment of manual effort and judgment: a huge, constantly evolving tapestry of concepts and relations that is being applied to a host of tasks.
This talk will introduce the process of “wikification”; that is, automatically and judiciously augmenting a plain-text document with pertinent hyperlinks to Wikipedia articles — as though the document were itself a Wikipedia article. We first describe how Wikipedia can be used to determine semantic relatedness, and then introduce a new, high-performance method of wikification that exploits Wikipedia’s 60 M internal hyperlinks for relational information and their anchor texts as lexical information, using simple machine learning.
The result amounts to a new semantic representation of text in terms of the salient concepts it mentions, where “concept” is equated to “Wikipedia article.” Wikification is a useful process in itself, adding value to plain text documents. More importantly, it supports new methods of document processing that exploit Wikipedia’s internal hyperlinks for relational information and their anchor texts as lexical information.
Presentation [ Open ] – (Requires Adobe Flash Player)
