Scaling to very very large corpora for natural language disambiguation
M. Banko, and E. Brill. ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
, page 26--33. Morristown, NJ, USA, Association for Computational Linguistics, (2001)
Description
With a billion word corpus, your algorithm doesn't matter - and you can skip all your clever tricks.
Also, active learning works better with huge data sets to pick interesting examples from.