Network Randomization: A Simple Technique for Generalization in Deep
Reinforcement Learning
K. Lee, K. Lee, J. Shin, and H. Lee. (2019)cite arxiv:1910.05396Comment: Accepted in ICLR 2020 and NeurIPS Workshop on Deep RL 2019 / First two authors are equally contributed.
Abstract
Deep reinforcement learning (RL) agents often fail to generalize to unseen
environments (yet semantically similar to trained agents), particularly when
they are trained on high-dimensional state spaces, such as images. In this
paper, we propose a simple technique to improve a generalization ability of
deep RL agents by introducing a randomized (convolutional) neural network that
randomly perturbs input observations. It enables trained agents to adapt to new
domains by learning robust features invariant across varied and randomized
environments. Furthermore, we consider an inference method based on the Monte
Carlo approximation to reduce the variance induced by this randomization. We
demonstrate the superiority of our method across 2D CoinRun, 3D DeepMind Lab
exploration and 3D robotics control tasks: it significantly outperforms various
regularization and data augmentation methods for the same purpose.
Description
[1910.05396] Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
%0 Journal Article
%1 lee2019network
%A Lee, Kimin
%A Lee, Kibok
%A Shin, Jinwoo
%A Lee, Honglak
%D 2019
%K generalization randomized reinforcement-learning
%T Network Randomization: A Simple Technique for Generalization in Deep
Reinforcement Learning
%U http://arxiv.org/abs/1910.05396
%X Deep reinforcement learning (RL) agents often fail to generalize to unseen
environments (yet semantically similar to trained agents), particularly when
they are trained on high-dimensional state spaces, such as images. In this
paper, we propose a simple technique to improve a generalization ability of
deep RL agents by introducing a randomized (convolutional) neural network that
randomly perturbs input observations. It enables trained agents to adapt to new
domains by learning robust features invariant across varied and randomized
environments. Furthermore, we consider an inference method based on the Monte
Carlo approximation to reduce the variance induced by this randomization. We
demonstrate the superiority of our method across 2D CoinRun, 3D DeepMind Lab
exploration and 3D robotics control tasks: it significantly outperforms various
regularization and data augmentation methods for the same purpose.
@article{lee2019network,
abstract = {Deep reinforcement learning (RL) agents often fail to generalize to unseen
environments (yet semantically similar to trained agents), particularly when
they are trained on high-dimensional state spaces, such as images. In this
paper, we propose a simple technique to improve a generalization ability of
deep RL agents by introducing a randomized (convolutional) neural network that
randomly perturbs input observations. It enables trained agents to adapt to new
domains by learning robust features invariant across varied and randomized
environments. Furthermore, we consider an inference method based on the Monte
Carlo approximation to reduce the variance induced by this randomization. We
demonstrate the superiority of our method across 2D CoinRun, 3D DeepMind Lab
exploration and 3D robotics control tasks: it significantly outperforms various
regularization and data augmentation methods for the same purpose.},
added-at = {2020-02-28T01:52:52.000+0100},
author = {Lee, Kimin and Lee, Kibok and Shin, Jinwoo and Lee, Honglak},
biburl = {https://www.bibsonomy.org/bibtex/2c93656c64730b44267eb6bd5ed1212ab/kirk86},
description = {[1910.05396] Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning},
interhash = {ab507c75ec54adc5905fdd211eb0f9dd},
intrahash = {c93656c64730b44267eb6bd5ed1212ab},
keywords = {generalization randomized reinforcement-learning},
note = {cite arxiv:1910.05396Comment: Accepted in ICLR 2020 and NeurIPS Workshop on Deep RL 2019 / First two authors are equally contributed},
timestamp = {2020-02-28T01:52:52.000+0100},
title = {Network Randomization: A Simple Technique for Generalization in Deep
Reinforcement Learning},
url = {http://arxiv.org/abs/1910.05396},
year = 2019
}