What would be a good way to extract headlines, dates, and authors from news articles? It seems easy to write a scraper using xpath or similar to extract this information from a single site, but I'm not sure of a more scalable solution if you're extracting from say 10,000 sites.
This is the project page for SecondString, an open-source Java-based package of approximate string-matching techniques. This code was developed by researchers at Carnegie Mellon University from the Center for Automated Learning and Discovery, the Department of Statistics, and the Center for Computer and Communications Security.
SecondString is intended primarily for researchers in information integration and other scientists. It does or will include a range of string-matching methods from a variety of communities, including statistics, artificial intelligence, information retrieval, and databases. It also includes tools for systematically evaluating performance on test data. It is not designed for use on very large data sets.
TeSSI® (Terminology Supported Semantic Indexing) is a state-of-the-art tool that improves upon the existing search and retrieval tools by extracting the meaning out of medical free text and placing the resulting medical ‘concepts’ in the document...
M. Hearst. Proceedings of the 14th Conference on Computational Linguistics - Volume 2, page 539--545. Stroudsburg, PA, USA, Association for Computational Linguistics, (1992)
P. Kluegl, M. Atzmueller, and F. Puppe. Proc. LWA 2009, Knowledge Discovery and Machine Learning Track, Darmstadt, Germany, University of Darmstadt, (2009)
P. Kluegl, M. Toepfer, P. Beck, G. Fette, and F. Puppe. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations, page 29--33. Dublin, Ireland, Dublin City University and Association for Computational Linguistics, (August 2014)
Y. Chan, and D. Roth. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, page 551--560. Stroudsburg, PA, USA, Association for Computational Linguistics, (2011)
M. Paukkeri, I. Nieminen, M. Pöllä, and T. Honkela. Coling 2008: Companion volume: Posters, page 83--86. Manchester, UK, ACL, Coling 2008 Organizing Committee, (August 2008)
B. Martins, H. Manguinhas, and J. Borbinha. Proceedings of the International Conference on Semantic Computing, page 1--9. IEEE Computer Society, (August 2008)
S. Riedel, L. Yao, and A. McCallum. Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III, page 148--163. Berlin, Heidelberg, Springer-Verlag, (2010)
P. Kluegl, M. Atzmueller, and F. Puppe. Proc. LWA 2009, Knowledge Discovery and Machine Learning Track, Darmstadt, Germany, University of Darmstadt, (2009)
Y. Zhang, N. Zincir-Heywood, and E. Milios. WIDM '05: Proceedings of the 7th annual ACM international workshop on Web information and data management, page 51--58. New York, NY, USA, ACM, (2005)
Y. Ohsawa, N. Benson, and M. Yachida. ADL '98: Proceedings of the Advances in Digital Libraries Conference, page 12. Washington, DC, USA, IEEE Computer Society, (1998)
Y. Matsuo, and M. Ishizuka. Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference, page 392-396. AAAI Press, (2003)
J. Chang, J. Boyd-Graber, and D. Blei. KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, page 169--178. New York, NY, USA, ACM, (2009)
G. Wang, Y. Yu, and H. Zhu. Proceedings of the 6th International Semantic Web Conference and 2nd Asian Semantic Web Conference (ISWC/ASWC2007), Busan, South Korea, volume 4825 of LNCS, page 575--588. Berlin, Heidelberg, Springer Verlag, (November 2007)