@flint63

Automatic Knowledge Extraction from Documents

, , , and . IBM Journal of Research and Development 56 (3/4): 5:1--5:10 (2012)

Abstract

Access to a large amount of knowledge is critical for success at answering open-domain questions for DeepQA systems such as IBM Watson. Formal representation of knowledge has the advantage of being easy to reason with, but acquisition of structured knowledge in open domains from unstructured data is often difficult and expensive. Our central hypothesis is that shallow syntactic knowledge and its implied semantics can be easily acquired and can be used in many areas of a question-answering system. We take a two-stage approach to extract the syntactic knowledge and implied semantics. First, shallow knowledge from large collections of documents is automatically extracted. Second, additional semantics are inferred from aggregate statistics of the automatically extracted shallow knowledge. In this paper, we describe in detail what kind of shallow knowledge is extracted, how it is automatically done from a large corpus, and how additional semantics are inferred from aggregate statistics. We also briefly discuss the various ways extracted knowledge is used throughout the IBM DeepQA system.

Links and resources

DOI:
10.1147/JRD.2012.2186519
BibTeX key:
FanKalyanpurEtAl12ibmjrd
search on:

Comments and Reviews  
(0)

There is no review or comment yet. You can write one!

Tags


Cite this publication