Document centered approach to text normalization
SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, page 136--143. New York, NY, USA, ACM, (2000)DOI: http://dx.doi.org/10.1145/345508.345564
Abstract
In this paper we present an approach to tackle three important problems of text normalization: sentence boundary disambiguation, disambiguation of capitalized words when they are used in positions where capitalization is expected, and identification of abbreviations. The main feature of our approach is that it uses a minimum of pre-built resources, instead dynamically inferring disambiguation clues from the entire document itself. This makes it domain independent, closely targeted to each individual document and portable to other languages. We thoroughly evaluated this approach on several corpora and it showed high accuracy.
