Momentum Contrast for Unsupervised Visual Representation Learning
K. He, H. Fan, Y. Wu, S. Xie, und R. Girshick. (2019)cite arxiv:1911.05722Comment: CVPR 2020 camera-ready. Code: https://github.com/facebookresearch/moco.
Zusammenfassung
We present Momentum Contrast (MoCo) for unsupervised visual representation
learning. From a perspective on contrastive learning as dictionary look-up, we
build a dynamic dictionary with a queue and a moving-averaged encoder. This
enables building a large and consistent dictionary on-the-fly that facilitates
contrastive unsupervised learning. MoCo provides competitive results under the
common linear protocol on ImageNet classification. More importantly, the
representations learned by MoCo transfer well to downstream tasks. MoCo can
outperform its supervised pre-training counterpart in 7 detection/segmentation
tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large
margins. This suggests that the gap between unsupervised and supervised
representation learning has been largely closed in many vision tasks.
Beschreibung
Momentum Contrast for Unsupervised Visual Representation Learning
%0 Generic
%1 he2019momentum
%A He, Kaiming
%A Fan, Haoqi
%A Wu, Yuxin
%A Xie, Saining
%A Girshick, Ross
%D 2019
%K cs.CV
%T Momentum Contrast for Unsupervised Visual Representation Learning
%U http://arxiv.org/abs/1911.05722
%X We present Momentum Contrast (MoCo) for unsupervised visual representation
learning. From a perspective on contrastive learning as dictionary look-up, we
build a dynamic dictionary with a queue and a moving-averaged encoder. This
enables building a large and consistent dictionary on-the-fly that facilitates
contrastive unsupervised learning. MoCo provides competitive results under the
common linear protocol on ImageNet classification. More importantly, the
representations learned by MoCo transfer well to downstream tasks. MoCo can
outperform its supervised pre-training counterpart in 7 detection/segmentation
tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large
margins. This suggests that the gap between unsupervised and supervised
representation learning has been largely closed in many vision tasks.
@misc{he2019momentum,
abstract = {We present Momentum Contrast (MoCo) for unsupervised visual representation
learning. From a perspective on contrastive learning as dictionary look-up, we
build a dynamic dictionary with a queue and a moving-averaged encoder. This
enables building a large and consistent dictionary on-the-fly that facilitates
contrastive unsupervised learning. MoCo provides competitive results under the
common linear protocol on ImageNet classification. More importantly, the
representations learned by MoCo transfer well to downstream tasks. MoCo can
outperform its supervised pre-training counterpart in 7 detection/segmentation
tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large
margins. This suggests that the gap between unsupervised and supervised
representation learning has been largely closed in many vision tasks.},
added-at = {2021-09-28T04:55:35.000+0200},
author = {He, Kaiming and Fan, Haoqi and Wu, Yuxin and Xie, Saining and Girshick, Ross},
biburl = {https://www.bibsonomy.org/bibtex/25f3ecfd4f5d4353bcf0ede0e0de7c378/aerover},
description = {Momentum Contrast for Unsupervised Visual Representation Learning},
interhash = {c429a043af0f17b6cf1bface57acc845},
intrahash = {5f3ecfd4f5d4353bcf0ede0e0de7c378},
keywords = {cs.CV},
note = {cite arxiv:1911.05722Comment: CVPR 2020 camera-ready. Code: https://github.com/facebookresearch/moco},
timestamp = {2021-09-28T04:55:35.000+0200},
title = {Momentum Contrast for Unsupervised Visual Representation Learning},
url = {http://arxiv.org/abs/1911.05722},
year = 2019
}