Language model-based document clustering using random walks
G. Erkan. Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, page 479--486. Morristown, NJ, USA, Association for Computational Linguistics, (2006)
DOI: http://dx.doi.org/10.3115/1220835.1220896
Abstract
We propose a new document vector representation specifically designed for the document clustering task. Instead of the traditional term-based vectors, a document is represented as an n-dimensional vector, where n is the number of documents in the cluster. The value at each dimension of the vector is closely related to the generation probability based on the language model of the corresponding document. Inspired by the recent graph-based NLP methods, we reinforce the generation probabilities by iterating random walks on the underlying graph representation. Experiments with k-means and hierarchical clustering algorithms show significant improvements over the alternative tf·idf vector representation.
Description
Language model-based document clustering using random walks
Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
%0 Conference Paper
%1 1220896
%A Erkan, Günes
%B Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
%C Morristown, NJ, USA
%D 2006
%I Association for Computational Linguistics
%K clustering document language-models
%P 479--486
%R http://dx.doi.org/10.3115/1220835.1220896
%T Language model-based document clustering using random walks
%U http://portal.acm.org/citation.cfm?id=1220896
%X We propose a new document vector representation specifically designed for the document clustering task. Instead of the traditional term-based vectors, a document is represented as an n-dimensional vector, where n is the number of documents in the cluster. The value at each dimension of the vector is closely related to the generation probability based on the language model of the corresponding document. Inspired by the recent graph-based NLP methods, we reinforce the generation probabilities by iterating random walks on the underlying graph representation. Experiments with k-means and hierarchical clustering algorithms show significant improvements over the alternative tf·idf vector representation.
@inproceedings{1220896,
abstract = {We propose a new document vector representation specifically designed for the document clustering task. Instead of the traditional term-based vectors, a document is represented as an n-dimensional vector, where n is the number of documents in the cluster. The value at each dimension of the vector is closely related to the generation probability based on the language model of the corresponding document. Inspired by the recent graph-based NLP methods, we reinforce the generation probabilities by iterating random walks on the underlying graph representation. Experiments with k-means and hierarchical clustering algorithms show significant improvements over the alternative tf·idf vector representation.},
added-at = {2009-02-11T02:39:38.000+0100},
address = {Morristown, NJ, USA},
author = {Erkan, G\"{u}ne\c{s}},
biburl = {https://www.bibsonomy.org/bibtex/2c6cdcbae097c41e5caa000a5981717b7/cdevries},
booktitle = {Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics},
description = {Language model-based document clustering using random walks},
doi = {http://dx.doi.org/10.3115/1220835.1220896},
interhash = {9ce6d7e6ecb8584ac1f538d831dd0a57},
intrahash = {c6cdcbae097c41e5caa000a5981717b7},
keywords = {clustering document language-models},
location = {New York, New York},
pages = {479--486},
publisher = {Association for Computational Linguistics},
timestamp = {2009-03-23T09:10:10.000+0100},
title = {Language model-based document clustering using random walks},
url = {http://portal.acm.org/citation.cfm?id=1220896},
year = 2006
}