HackerPapers

arXiv:1509.02971cs.LG

Continuous control with deep reinforcement learning

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra

Abstract

We adapt the ideas underlying the success of Deep Q-Learning to the continuous action domain. We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. Using the same learning algorithm, network architecture and hyper-parameters, our algorithm robustly solves more than 20 simulated physics tasks, including classic problems such as cartpole swing-up, dexterous manipulation, legged locomotion and car driving. Our algorithm is able to find policies whose performance is competitive with those found by a planning algorithm with full access to the dynamics of the domain and its derivatives. We further demonstrate that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs.

arXiv comments
10 pages + supplementary
Journal reference
DOI
Subjects
cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 4 identifiable commenters in 4 comments across 4 threads, from Oct 2015 to Dec 2020. Found in 2 HN submissions from Sep 2015 to Sep 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. You might not need machine learning 1 comment

    • Stagnation is a known problem in reinforcement learning and similar methods. It's very easy to get stuck at a local maximum. My favorite fun example is https://gym.openai.com/envs/BipedalWalkerHardcore-v2/ where a standard DDPG(https://arxiv.org/abs/1509.02971) will get stuck at…

      laquitionView on HNParent context

  2. AlphaGo beats the world champion Lee Sedol in first of five matches 1 comment

    • ...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy…

      ogriselView on HNParent context

  3. The first person to hack the iPhone is building a self-driving car 1 comment

    • Prototypical case of the 80/20 rule. He has implemented the happy case. But that system is nothing people realistically would want to drive their cars. What he did is impressive. But the results are not that outlandish for a talented person. 1) Hook up a computer to the CAN-Bus…

      jpfrView on HNParent context

  4. A Peek Inside Google’s Efforts to Create a General-Purpose Robot 1 comment

    • It's handy knowing they're fiddling with ROS, and the research papers show use of DNN&RL for robotics via simulated tasks with actions keying off of raw pixel inputs (paper: http://arxiv.org/pdf/1509.02971v1.pdf), and grasp paper shows ambitions on dexterous manipulation. I…

      mappingbabeljcView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.