Compositional Attention Networks for Machine Reasoning
D. Hudson, and C. Manning. (2018)cite arxiv:1803.03067Comment: Published as a conference paper at ICLR 2018.
Abstract
We present the MAC network, a novel fully differentiable neural network
architecture, designed to facilitate explicit and expressive reasoning. MAC
moves away from monolithic black-box neural architectures towards a design that
encourages both transparency and versatility. The model approaches problems by
decomposing them into a series of attention-based reasoning steps, each
performed by a novel recurrent Memory, Attention, and Composition (MAC) cell
that maintains a separation between control and memory. By stringing the cells
together and imposing structural constraints that regulate their interaction,
MAC effectively learns to perform iterative reasoning processes that are
directly inferred from the data in an end-to-end approach. We demonstrate the
model's strength, robustness and interpretability on the challenging CLEVR
dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy,
halving the error rate of the previous best model. More importantly, we show
that the model is computationally-efficient and data-efficient, in particular
requiring 5x less data than existing models to achieve strong results.
Description
[1803.03067] Compositional Attention Networks for Machine Reasoning
%0 Journal Article
%1 hudson2018compositional
%A Hudson, Drew A.
%A Manning, Christopher D.
%D 2018
%K deep-learning
%T Compositional Attention Networks for Machine Reasoning
%U http://arxiv.org/abs/1803.03067
%X We present the MAC network, a novel fully differentiable neural network
architecture, designed to facilitate explicit and expressive reasoning. MAC
moves away from monolithic black-box neural architectures towards a design that
encourages both transparency and versatility. The model approaches problems by
decomposing them into a series of attention-based reasoning steps, each
performed by a novel recurrent Memory, Attention, and Composition (MAC) cell
that maintains a separation between control and memory. By stringing the cells
together and imposing structural constraints that regulate their interaction,
MAC effectively learns to perform iterative reasoning processes that are
directly inferred from the data in an end-to-end approach. We demonstrate the
model's strength, robustness and interpretability on the challenging CLEVR
dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy,
halving the error rate of the previous best model. More importantly, we show
that the model is computationally-efficient and data-efficient, in particular
requiring 5x less data than existing models to achieve strong results.
@article{hudson2018compositional,
abstract = {We present the MAC network, a novel fully differentiable neural network
architecture, designed to facilitate explicit and expressive reasoning. MAC
moves away from monolithic black-box neural architectures towards a design that
encourages both transparency and versatility. The model approaches problems by
decomposing them into a series of attention-based reasoning steps, each
performed by a novel recurrent Memory, Attention, and Composition (MAC) cell
that maintains a separation between control and memory. By stringing the cells
together and imposing structural constraints that regulate their interaction,
MAC effectively learns to perform iterative reasoning processes that are
directly inferred from the data in an end-to-end approach. We demonstrate the
model's strength, robustness and interpretability on the challenging CLEVR
dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy,
halving the error rate of the previous best model. More importantly, we show
that the model is computationally-efficient and data-efficient, in particular
requiring 5x less data than existing models to achieve strong results.},
added-at = {2019-07-15T03:58:40.000+0200},
author = {Hudson, Drew A. and Manning, Christopher D.},
biburl = {https://www.bibsonomy.org/bibtex/26829c47bdd9dbe942500179784ff0cd6/kirk86},
description = {[1803.03067] Compositional Attention Networks for Machine Reasoning},
interhash = {f7a126c3124eea8b5894984114692e0c},
intrahash = {6829c47bdd9dbe942500179784ff0cd6},
keywords = {deep-learning},
note = {cite arxiv:1803.03067Comment: Published as a conference paper at ICLR 2018},
timestamp = {2019-07-15T03:58:40.000+0200},
title = {Compositional Attention Networks for Machine Reasoning},
url = {http://arxiv.org/abs/1803.03067},
year = 2018
}