Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients.
This page provides a large hyperlink graph for public download. The graph has been extracted from the Common Crawl 2012 web corpus and covers 3.5 billion web pages and 128 billion hyperlinks between these pages. To the best of our knowledge, this graph is the largest hyperlink graph that is available to the public outside companies such as Google, Yahoo, and Microsoft. Below we provide instructions on how to download the graph as well as basic statistics about its topology.
Die Deutsche Gesellschaft für Informationswissenschaft und Informationspraxis e.V. (DGI) fördert die Entwicklungen der Informationswissenschaft und Informationspraxis durch die Beobachtung und Vermittlung von Grundlagen, Arbeitsmethoden und technischen Hilfsmitteln.
L. Becchetti, C. Castillo, D. Donato, S. Leonardi, and R. Baeza-Yates. The European Integrated Project Dynamically Evolving, Large Scale Information Systems (DELIS): proceedings of the final workshop, 222, page 99--113. Heinz-Nixdorf-Institut, Universität Paderborn, (February 2008)
P. Nakov, and M. Hearst. Proceedings of the Ninth Conference on Computational Natural Language Learning, page 17--24. Stroudsburg, PA, USA, Association for Computational Linguistics, (2005)
J. Abernethy, O. Chapelle, and C. Castillo. Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web, page 41--44. New York, NY, USA, ACM, (2008)
P. Singer, D. Helic, A. Hotho, and M. Strohmaier. International Conference on World Wide Web, page 1003--1013. Republic and Canton of Geneva, Switzerland, International World Wide Web Conferences Steering Committee, (2015)
M. Becker, H. Mewes, A. Hotho, D. Dimitrov, F. Lemmerich, and M. Strohmaier. International Conference Companion on World Wide Web, page 17--18. Republic and Canton of Geneva, Switzerland, International World Wide Web Conferences Steering Committee, (2016)
A. Hotho, R. Jaeschke, and K. Lerman. Semantic Web, 8 (5):
623--624(April 2017)2017 IOS Press and the authors. This is an author produced version of a paper subsequently published in Semantic Web. Uploaded in accordance with the publisher's self-archiving policy..