Self-supervised 3D Shape and Viewpoint Estimation from Single Images for
Robotics
O. Mees, M. Tatarchenko, T. Brox, and W. Burgard. (2019)cite arxiv:1910.07948Comment: Accepted at the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Video at https://www.youtube.com/watch?v=oQgHG9JdMP4.
Abstract
We present a convolutional neural network for joint 3D shape prediction and
viewpoint estimation from a single input image. During training, our network
gets the learning signal from a silhouette of an object in the input image - a
form of self-supervision. It does not require ground truth data for 3D shapes
and the viewpoints. Because it relies on such a weak form of supervision, our
approach can easily be applied to real-world data. We demonstrate that our
method produces reasonable qualitative and quantitative results on natural
images for both shape estimation and viewpoint prediction. Unlike previous
approaches, our method does not require multiple views of the same object
instance in the dataset, which significantly expands the applicability in
practical robotics scenarios. We showcase it by using the hallucinated shapes
to improve the performance on the task of grasping real-world objects both in
simulation and with a PR2 robot.
Description
[1910.07948] Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics
cite arxiv:1910.07948Comment: Accepted at the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Video at https://www.youtube.com/watch?v=oQgHG9JdMP4
%0 Generic
%1 mees2019selfsupervised
%A Mees, Oier
%A Tatarchenko, Maxim
%A Brox, Thomas
%A Burgard, Wolfram
%D 2019
%K 3d image reconstruction single view
%T Self-supervised 3D Shape and Viewpoint Estimation from Single Images for
Robotics
%U http://arxiv.org/abs/1910.07948
%X We present a convolutional neural network for joint 3D shape prediction and
viewpoint estimation from a single input image. During training, our network
gets the learning signal from a silhouette of an object in the input image - a
form of self-supervision. It does not require ground truth data for 3D shapes
and the viewpoints. Because it relies on such a weak form of supervision, our
approach can easily be applied to real-world data. We demonstrate that our
method produces reasonable qualitative and quantitative results on natural
images for both shape estimation and viewpoint prediction. Unlike previous
approaches, our method does not require multiple views of the same object
instance in the dataset, which significantly expands the applicability in
practical robotics scenarios. We showcase it by using the hallucinated shapes
to improve the performance on the task of grasping real-world objects both in
simulation and with a PR2 robot.
@misc{mees2019selfsupervised,
abstract = {We present a convolutional neural network for joint 3D shape prediction and
viewpoint estimation from a single input image. During training, our network
gets the learning signal from a silhouette of an object in the input image - a
form of self-supervision. It does not require ground truth data for 3D shapes
and the viewpoints. Because it relies on such a weak form of supervision, our
approach can easily be applied to real-world data. We demonstrate that our
method produces reasonable qualitative and quantitative results on natural
images for both shape estimation and viewpoint prediction. Unlike previous
approaches, our method does not require multiple views of the same object
instance in the dataset, which significantly expands the applicability in
practical robotics scenarios. We showcase it by using the hallucinated shapes
to improve the performance on the task of grasping real-world objects both in
simulation and with a PR2 robot.},
added-at = {2020-01-28T16:55:33.000+0100},
author = {Mees, Oier and Tatarchenko, Maxim and Brox, Thomas and Burgard, Wolfram},
biburl = {https://www.bibsonomy.org/bibtex/23574605f430884fcf47d5881179ec79b/kluger},
description = {[1910.07948] Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics},
interhash = {a43a5b14b17e1f27ac359e485b8801d4},
intrahash = {3574605f430884fcf47d5881179ec79b},
keywords = {3d image reconstruction single view},
note = {cite arxiv:1910.07948Comment: Accepted at the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Video at https://www.youtube.com/watch?v=oQgHG9JdMP4},
timestamp = {2020-01-28T16:55:33.000+0100},
title = {Self-supervised 3D Shape and Viewpoint Estimation from Single Images for
Robotics},
url = {http://arxiv.org/abs/1910.07948},
year = 2019
}