Techreport,
On-Line New Event Detection using Single Pass Clustering
University of Massachusetts, Amherst, MA, USA, (1998)
Abstract
This paper discusses the implementation and evaluation of a new-event detection system. We focus on a strict on-line setting, in that the system must indicate whether the current document contains or does not contain discussion of a new event before looking at the next document. Our approach to the problem uses a single pass clustering algorithm and a novel thresholding model that incorporates the properties of events as a major component. A corpus containing newswire and transcribed broadcast news was analyzed using our system, and our results compared favorably to those of other systems. We develop an evaluation methodology based on a combination of techniques that allows us to infer the expected performance of our approach in the field, and to suggest avenues for future research that may lead to better performance.
