Inproceedings,

Balancing volume, quality and freshness in Web crawling

, and .
Soft Computing Systems - Design, Management and Applications, page 565--572. Santiago, Chile, IOS Press Amsterdam, (2002)

Abstract

We describe a crawling software designed for high-performance, large-scale information discovery and gathering on the Web. This crawler allows the administrator to seek for a balance between the volume of a Web collection and its freshness; and also provides flexibility for defining a quality metric to prioritize certain pages.

Tags

Users

  • @chato
  • @lysander07
  • @dblp

Comments and Reviews