Maximally Informative Hierarchical Representations of High-Dimensional
Data
G. Steeg, and A. Galstyan. (2014)cite arxiv:1410.7404Comment: 13 pages, 8 figures. Appearing in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS) 2015.
Abstract
We consider a set of probabilistic functions of some input variables as a
representation of the inputs. We present bounds on how informative a
representation is about input data. We extend these bounds to hierarchical
representations so that we can quantify the contribution of each layer towards
capturing the information in the original data. The special form of these
bounds leads to a simple, bottom-up optimization procedure to construct
hierarchical representations that are also maximally informative about the
data. This optimization has linear computational complexity and constant sample
complexity in the number of variables. These results establish a new approach
to unsupervised learning of deep representations that is both principled and
practical. We demonstrate the usefulness of the approach on both synthetic and
real-world data.
Description
[1410.7404] Maximally Informative Hierarchical Representations of High-Dimensional Data
cite arxiv:1410.7404Comment: 13 pages, 8 figures. Appearing in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS) 2015
%0 Generic
%1 steeg2014maximally
%A Steeg, Greg Ver
%A Galstyan, Aram
%D 2014
%K gregversteeg machine-learning
%T Maximally Informative Hierarchical Representations of High-Dimensional
Data
%U http://arxiv.org/abs/1410.7404
%X We consider a set of probabilistic functions of some input variables as a
representation of the inputs. We present bounds on how informative a
representation is about input data. We extend these bounds to hierarchical
representations so that we can quantify the contribution of each layer towards
capturing the information in the original data. The special form of these
bounds leads to a simple, bottom-up optimization procedure to construct
hierarchical representations that are also maximally informative about the
data. This optimization has linear computational complexity and constant sample
complexity in the number of variables. These results establish a new approach
to unsupervised learning of deep representations that is both principled and
practical. We demonstrate the usefulness of the approach on both synthetic and
real-world data.
@misc{steeg2014maximally,
abstract = {We consider a set of probabilistic functions of some input variables as a
representation of the inputs. We present bounds on how informative a
representation is about input data. We extend these bounds to hierarchical
representations so that we can quantify the contribution of each layer towards
capturing the information in the original data. The special form of these
bounds leads to a simple, bottom-up optimization procedure to construct
hierarchical representations that are also maximally informative about the
data. This optimization has linear computational complexity and constant sample
complexity in the number of variables. These results establish a new approach
to unsupervised learning of deep representations that is both principled and
practical. We demonstrate the usefulness of the approach on both synthetic and
real-world data.},
added-at = {2017-04-17T23:24:58.000+0200},
author = {Steeg, Greg Ver and Galstyan, Aram},
biburl = {https://www.bibsonomy.org/bibtex/2c5735b69a1ff7f36975a0b54ed38bb74/marcsaric},
description = {[1410.7404] Maximally Informative Hierarchical Representations of High-Dimensional Data},
interhash = {886e7e0a53756fc1bc26de34d5e3e1f6},
intrahash = {c5735b69a1ff7f36975a0b54ed38bb74},
keywords = {gregversteeg machine-learning},
note = {cite arxiv:1410.7404Comment: 13 pages, 8 figures. Appearing in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS) 2015},
timestamp = {2017-05-05T22:14:49.000+0200},
title = {Maximally Informative Hierarchical Representations of High-Dimensional
Data},
url = {http://arxiv.org/abs/1410.7404},
year = 2014
}