DIBS: Diversity inducing Information Bottleneck in Model Ensembles
S. Sinha, H. Bharadhwaj, A. Goyal, H. Larochelle, A. Garg, and F. Shkurti. (2020)cite arxiv:2003.04514Comment: Samarth Sinha* and Homanga Bharadhwaj* contributed equally to this work. Code will be released at https://github.com/rvl-lab-utoronto/dibs.
Abstract
Although deep learning models have achieved state-of-the-art performance on a
number of vision tasks, generalization over high dimensional multi-modal data,
and reliable predictive uncertainty estimation are still active areas of
research. Bayesian approaches including Bayesian Neural Nets (BNNs) do not
scale well to modern computer vision tasks, as they are difficult to train, and
have poor generalization under dataset-shift. This motivates the need for
effective ensembles which can generalize and give reliable uncertainty
estimates. In this paper, we target the problem of generating effective
ensembles of neural networks by encouraging diversity in prediction. We
explicitly optimize a diversity inducing adversarial loss for learning the
stochastic latent variables and thereby obtain diversity in the output
predictions necessary for modeling multi-modal data. We evaluate our method on
benchmark datasets: MNIST, CIFAR100, TinyImageNet and MIT Places 2, and
compared to the most competitive baselines show significant improvements in
classification accuracy, under a shift in the data distribution and in
out-of-distribution detection. Code will be released in this url
https://github.com/rvl-lab-utoronto/dibs
Description
[2003.04514] DIBS: Diversity inducing Information Bottleneck in Model Ensembles
cite arxiv:2003.04514Comment: Samarth Sinha* and Homanga Bharadhwaj* contributed equally to this work. Code will be released at https://github.com/rvl-lab-utoronto/dibs
%0 Journal Article
%1 sinha2020diversity
%A Sinha, Samarth
%A Bharadhwaj, Homanga
%A Goyal, Anirudh
%A Larochelle, Hugo
%A Garg, Animesh
%A Shkurti, Florian
%D 2020
%K bayesian uncertainty
%T DIBS: Diversity inducing Information Bottleneck in Model Ensembles
%U http://arxiv.org/abs/2003.04514
%X Although deep learning models have achieved state-of-the-art performance on a
number of vision tasks, generalization over high dimensional multi-modal data,
and reliable predictive uncertainty estimation are still active areas of
research. Bayesian approaches including Bayesian Neural Nets (BNNs) do not
scale well to modern computer vision tasks, as they are difficult to train, and
have poor generalization under dataset-shift. This motivates the need for
effective ensembles which can generalize and give reliable uncertainty
estimates. In this paper, we target the problem of generating effective
ensembles of neural networks by encouraging diversity in prediction. We
explicitly optimize a diversity inducing adversarial loss for learning the
stochastic latent variables and thereby obtain diversity in the output
predictions necessary for modeling multi-modal data. We evaluate our method on
benchmark datasets: MNIST, CIFAR100, TinyImageNet and MIT Places 2, and
compared to the most competitive baselines show significant improvements in
classification accuracy, under a shift in the data distribution and in
out-of-distribution detection. Code will be released in this url
https://github.com/rvl-lab-utoronto/dibs
@article{sinha2020diversity,
abstract = {Although deep learning models have achieved state-of-the-art performance on a
number of vision tasks, generalization over high dimensional multi-modal data,
and reliable predictive uncertainty estimation are still active areas of
research. Bayesian approaches including Bayesian Neural Nets (BNNs) do not
scale well to modern computer vision tasks, as they are difficult to train, and
have poor generalization under dataset-shift. This motivates the need for
effective ensembles which can generalize and give reliable uncertainty
estimates. In this paper, we target the problem of generating effective
ensembles of neural networks by encouraging diversity in prediction. We
explicitly optimize a diversity inducing adversarial loss for learning the
stochastic latent variables and thereby obtain diversity in the output
predictions necessary for modeling multi-modal data. We evaluate our method on
benchmark datasets: MNIST, CIFAR100, TinyImageNet and MIT Places 2, and
compared to the most competitive baselines show significant improvements in
classification accuracy, under a shift in the data distribution and in
out-of-distribution detection. Code will be released in this url
https://github.com/rvl-lab-utoronto/dibs},
added-at = {2020-03-11T20:32:47.000+0100},
author = {Sinha, Samarth and Bharadhwaj, Homanga and Goyal, Anirudh and Larochelle, Hugo and Garg, Animesh and Shkurti, Florian},
biburl = {https://www.bibsonomy.org/bibtex/2ca87c0809412a8e5b22e222b172aa518/kirk86},
description = {[2003.04514] DIBS: Diversity inducing Information Bottleneck in Model Ensembles},
interhash = {0d07193cc144a79ff01ae744b0f36d86},
intrahash = {ca87c0809412a8e5b22e222b172aa518},
keywords = {bayesian uncertainty},
note = {cite arxiv:2003.04514Comment: Samarth Sinha* and Homanga Bharadhwaj* contributed equally to this work. Code will be released at https://github.com/rvl-lab-utoronto/dibs},
timestamp = {2020-03-11T20:32:47.000+0100},
title = {DIBS: Diversity inducing Information Bottleneck in Model Ensembles},
url = {http://arxiv.org/abs/2003.04514},
year = 2020
}