Article,
A Triangular Network For Density Estimation
(2020)cite arxiv:2004.14593Comment: Universal density estimator. Supplements like code at https://github.com/lixilinx/TriNet4PdfEst.
Abstract
In this paper, triangular networks refer to feedforward neural networks with
triangular block matrices as their connection weights, and they are studied for
density estimation. A special two layer triangular monotonic neural network
unit is designed and shown to be universal approximator for invertible mappings
with triangular Jacobians based on the simple observation that positively
weighted sum of monotonically increasing functions is still monotonic. Then,
deep invertible neural networks consisting of stacked such monotonic triangular
network units and permutations are proposed as universal density estimators.
Our method is most closely related to neural autoregressive density
estimations, especially the block neural autoregressive flow. But, unlike many
autoregressive models, our designs are highly modular, parameter economy,
computationally efficient, and applicable to density estimation of data with
high dimensions. Experimental results on image density estimation benchmarks
are reported for performance comparisons.
