We observed that generally the embedding representation is very rich and information dense. For example, reducing the dimensionality of the inputs using SVD or PCA, even by 10%, generally results in worse downstream performance on specific tasks.
I. Jureta, S. Faulkner, and P. Schobbens. RE '06: Proceedings of the 14th IEEE International Requirements Engineering Conference (RE'06), page 116--125. Washington, DC, USA, IEEE Computer Society, (2006)
S. Staab, M. Erdmann, A. Maedche, and S. Decker. Proc. of First Workshop on the Semantic Web at the Fourth European Conference International Workshop on Research and Advanced Technology for Digital Libraries, Lisbon, Portugal 18-20 September 2000, (September 2000)
J. Filipe, M. Felici, and S. Anderson. International Conference on Human Aspects of Advanced Manufacturing:
Agility & Hybrid Automation, page 321--328. (May 2003)
S. Greenspan, J. Mylopoulos, and A. Borgida. International conference on Software engineering, page 135--147. Los Alamitos, CA, USA, IEEE Computer Society Press, (1994)