HackerPapers

arXiv:1703.06907cs.RO

Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World

Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel

Abstract

Bridging the 'reality gap' that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to $1.5$cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.

arXiv comments
8 pages, 7 figures. Submitted to 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017)
Journal reference
DOI
Subjects
cs.RORobotics cs.LGMachine Learning

On Hacker News

Linked by 3 identifiable commenters in 3 comments across 3 threads, from Apr 2017 to Apr 2021.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Massively Parallel Rendering of Complex Closed-Form Implicit Surfaces (2020) 1 comment

    • This is important for vision ML because sufficiently advanced simulations may comprise the real world as a special case: https://arxiv.org/abs/1703.06907 For example, gaming pixel maps have been used for semantic simulation, and have been rendered scenes…

      bravuraView on HNParent context

  2. A better way to build ML: Why you should be using Active Learning 1 comment

    • what is your thought on synthetic data vs active learning? For some domains, with privacy concerns or rarity of objects, getting labelled data for deep learning is challenging. There is decent research on sim2real i.e transferring models trained on synthetic data to real world…

      dexter89_kp3View on HNParent context

  3. Spam detection in the physical world 1 comment

    • While the application is indeed a joke, the technique for sim-to-real transfer (and the robotic system in the post) is very real: https://arxiv.org/abs/1703.06907 :).

      gdbView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.