We observed that generally the embedding representation is very rich and information dense. For example, reducing the dimensionality of the inputs using SVD or PCA, even by 10%, generally results in worse downstream performance on specific tasks.
J. Turian, L. Ratinov, and Y. Bengio. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, page 384--394. Stroudsburg, PA, USA, Association for Computational Linguistics, (2010)