Adversarial Training Methods for Semi-Supervised Text Classification
T. Miyato, A. Dai, and I. Goodfellow. (2016)cite arxiv:1605.07725Comment: Published as a conference paper at ICLR 2017.
Abstract
Adversarial training provides a means of regularizing supervised learning
algorithms while virtual adversarial training is able to extend supervised
learning algorithms to the semi-supervised setting. However, both methods
require making small perturbations to numerous entries of the input vector,
which is inappropriate for sparse high-dimensional inputs such as one-hot word
representations. We extend adversarial and virtual adversarial training to the
text domain by applying perturbations to the word embeddings in a recurrent
neural network rather than to the original input itself. The proposed method
achieves state of the art results on multiple benchmark semi-supervised and
purely supervised tasks. We provide visualizations and analysis showing that
the learned word embeddings have improved in quality and that while training,
the model is less prone to overfitting.
Description
Adversarial Training Methods for Semi-Supervised Text Classification
%0 Generic
%1 miyato2016adversarial
%A Miyato, Takeru
%A Dai, Andrew M.
%A Goodfellow, Ian
%D 2016
%K Adversarial classification text
%T Adversarial Training Methods for Semi-Supervised Text Classification
%U http://arxiv.org/abs/1605.07725
%X Adversarial training provides a means of regularizing supervised learning
algorithms while virtual adversarial training is able to extend supervised
learning algorithms to the semi-supervised setting. However, both methods
require making small perturbations to numerous entries of the input vector,
which is inappropriate for sparse high-dimensional inputs such as one-hot word
representations. We extend adversarial and virtual adversarial training to the
text domain by applying perturbations to the word embeddings in a recurrent
neural network rather than to the original input itself. The proposed method
achieves state of the art results on multiple benchmark semi-supervised and
purely supervised tasks. We provide visualizations and analysis showing that
the learned word embeddings have improved in quality and that while training,
the model is less prone to overfitting.
@misc{miyato2016adversarial,
abstract = {Adversarial training provides a means of regularizing supervised learning
algorithms while virtual adversarial training is able to extend supervised
learning algorithms to the semi-supervised setting. However, both methods
require making small perturbations to numerous entries of the input vector,
which is inappropriate for sparse high-dimensional inputs such as one-hot word
representations. We extend adversarial and virtual adversarial training to the
text domain by applying perturbations to the word embeddings in a recurrent
neural network rather than to the original input itself. The proposed method
achieves state of the art results on multiple benchmark semi-supervised and
purely supervised tasks. We provide visualizations and analysis showing that
the learned word embeddings have improved in quality and that while training,
the model is less prone to overfitting.},
added-at = {2017-10-04T16:30:37.000+0200},
author = {Miyato, Takeru and Dai, Andrew M. and Goodfellow, Ian},
biburl = {https://www.bibsonomy.org/bibtex/249a3bda7edf1c04d4b39d5b77b96e306/daschloer},
description = {Adversarial Training Methods for Semi-Supervised Text Classification},
interhash = {ba1f927e819e798a8baa0c0eeb52b317},
intrahash = {49a3bda7edf1c04d4b39d5b77b96e306},
keywords = {Adversarial classification text},
note = {cite arxiv:1605.07725Comment: Published as a conference paper at ICLR 2017},
timestamp = {2017-10-04T16:30:37.000+0200},
title = {Adversarial Training Methods for Semi-Supervised Text Classification},
url = {http://arxiv.org/abs/1605.07725},
year = 2016
}