HackerPapers

arXiv:1502.05477cs.LG

Trust Region Policy Optimization

John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel

Abstract

We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement. By making several approximations to the theoretically-justified procedure, we develop a practical algorithm, called Trust Region Policy Optimization (TRPO). This algorithm is similar to natural policy gradient methods and is effective for optimizing large nonlinear policies such as neural networks. Our experiments demonstrate its robust performance on a wide variety of tasks: learning simulated robotic swimming, hopping, and walking gaits; and playing Atari games using images of the screen as input. Despite its approximations that deviate from the theory, TRPO tends to give monotonic improvement, with little tuning of hyperparameters.

arXiv comments
16 pages, ICML 2015
Journal reference
DOI
Subjects
cs.LGMachine Learning

On Hacker News

Linked by 3 identifiable commenters in 4 comments across 3 threads, from Mar 2016 to May 2017.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Roboschool: open-source software for robot simulation 1 comment

    • These environments are often used as a testbed for reinforcement learning, e.g. https://arxiv.org/abs/1502.05477

      dpfView on HNParent context

  2. Generative Models 1 comment

    • For reinforcement learning, one of OpenAI's focus areas, the book by Sutton & Barto is still the standard reference: https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html Improved algorithms have been devised since it was written, see http://karpathy.github.io/2016/05/31/rl/…

      T-AView on HNParent context

  3. The Artificial Intelligence Revolution: A Special Report, Pt. 1 2 comments

    • See in the paper[0] Figure 3 near the end shows the neural networks used for the different scenarios. None of them have a "confidence" input or parameter. As for confidence intervals, that's why I made the distinction originally, the word is not being used in the "formal" sense…

      dr_zoidbergView on HNParent context

    • Can't edit the parent post anymore, but here I found the one interesting thing I got from reading this. The Trust Region Policy Optimization paper, barely mentioned in the article: http://arxiv.org/pdf/1502.05477v3.pdf

      dr_zoidbergView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.