RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning · HackerPapers discussions

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by homarp

Fri, 03 Feb 2023 18:10:51 GMT

https://arxiv.org/abs/1611.02779 is the closest - RL2: Fast Reinforcement Learning via Slow Reinforcement Learning

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by jimfleming

Thu, 28 Mar 2019 21:24:45 GMT

Hyperparameter optimization (including architectures) is not really meta-learning. Meta-learning, also known as "learning to learn", is more like MAML[0], RL2[1], L2RL[2], etc. 0. https://arxiv.org/abs/1703.03400 1. https://arxiv.org/abs/1611.02779 2. https://arxiv.org/abs/1611.05763

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by readonlydanger

Fri, 09 Jun 2017 19:23:51 GMT

Unfortunately, many of your assertions are inaccurate. As far as game playing goes, the ways in which AI agents beat human opponents is simply a matter of computers being able to process more information in a given period of time. The selection and evaluation criteria are actually pretty similar to human decision-making, if a bit more systematic. I can expand on this point in another comment, if you'd like. Chess and Go are not solved games, and will likely not be for some time (10+ years, for Go at least). Solving non-deterministic problems are even harder, and many likely will not be…

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by willwhitney

Thu, 23 Feb 2017 00:42:01 GMT

Handling delays (and the uncertainty they entail) is a huge challenge, and I think it'll be a rich area of research. The simplest part of the problem is that delays in action or perception also slow the propagation of reward signals, and credit assignment is still a really hard problem. Thinking further afield, future models could learn to adapt their expectations to fit the behavior of a particular opponent. This kind of metalearning is pretty much a wide open problem, though a pair of (roughly equivalent) papers in this direction recently came out from DeepMind…

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by saycheese

Fri, 20 Jan 2017 00:35:57 GMT

Note: The original title noted OpenAI's role in the paper, which maybe seen by loading the PDF and reading the author credits. https://arxiv.org/pdf/1611.02779v2.pdf

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — HN submission

Thu, 19 Jan 2017 21:01:05 GMT

RL²: Fast Reinforcement Learning via Slow Reinforcement Learning (2016)

50 points; 9 discussion replies on HN. Paper and source history. A submission is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by lucidrains

Tue, 03 Jan 2017 00:55:52 GMT

Other equally exciting papers that relates to learning to learn in DL. "Neural Architecture Search with Reinforcement Learning" https://arxiv.org/abs/1611.01578 "RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning" https://arxiv.org/abs/1611.02779 "Designing Neural Network Architectures using Reinforcement Learning" https://arxiv.org/abs/1611.02167

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by sapphireblue

Thu, 22 Dec 2016 01:05:55 GMT

Meta- reinforcement learning could prove to be such breakthrough, see [1],[2]. Also next generation ASIC accelerators (Google's TPU, Nervana) can give 10x increase in NN performance over a GPU manufactured on the same process, with another 10x possible with some form of binarized weights, e.g. BNN, XNOR-net. There are also interesting techniques to update the model's parameters in a sparse manner. So, there certainly is a lot of room left for performance improvements! 1. https://arxiv.org/abs/1611.05763 2. https://arxiv.org/abs/1611.02779

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by sapphireblue

Mon, 05 Dec 2016 14:38:56 GMT

Looks like the "UNREAL" (https://arxiv.org/abs/1611.05397), "Learning to reinforcement learn" (https://arxiv.org/abs/1611.05763) and "RL^2" (https://arxiv.org/abs/1611.02779) are state of art in pure RL for now. Finally there is a trend of using recurrent neural network as a top component of the Q-network. Perhaps we will see even more sophisticated RNNs like DNC and Recurrent Entity Networks applied here. Also we'll see meta-reinforcement learning applied to a curriculum of environments.

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by zhongwenxu

Mon, 21 Nov 2016 01:35:05 GMT

On Google and Oxford's cases, I don't think it is common to broadcast among your colleagues in different groups your proposed ideas before you actually go into it. So I think it may happen even in the same company/university. For the meta-learning papers, you may have interests to read the related work part of the RL^2 paper https://arxiv.org/pdf/1611.02779.pdf. Quoted as follows, "Our work draws inspiration from a particular line of work (Younger et al., 2001; Santoro et al., 2016; Vinyals et al., 2016), which formulates meta-learning as an optimization problem, and can thus be optimized…

Paper and discussion history. A mention is not an endorsement.

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning — mentioned by mappingbabeljc

Sat, 19 Nov 2016 09:43:03 GMT

Even more interesting - an OpenAI paper on the roughly same subject/technique came out a little earlier (RL2 fast reinforcement learning for slow reinforcement learning https://arxiv.org/pdf/1611.02779.pdf). I think parallel inventions tends to indicate that multiple people have stumbled on a similar good idea at the same time. (disclaimer: I work at OpenAI. v pleased to see these two papers emerge so close together)

Paper and discussion history. A mention is not an endorsement.