Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning · HackerPapers discussions

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning — mentioned by kastnerkyle

Wed, 08 Jun 2016 16:37:01 GMT

Montezuma's Revenge, Castle Wolfenstein (not the shooter), and puzzle games in general have a problem of long term credit assignment and sparse reward. This "intrinsic reward" approach form the paper, based on pseudo counts seems to be one way to get an intermediate reward which helps the model learn toward an overall goal (winning/progress) which happens rarely. The previous best work had to pre-define the intrinsic rewards as I understand it [0], and DeepMind has been tracking this general problem for a while [1] along with a whole bunch of earlier work from the 70s/80s/90s (cited in the…

Paper and discussion history. A mention is not an endorsement.

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning — mentioned by ogrisel

Tue, 29 Dec 2015 16:10:29 GMT

The utility function can be specified in a way to build up something that looks like an internal motivation system. This is often referred to as intrinsically motivated reinforcement learning. A recent paper by 2 Google Deepmind researchers on this topic: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning Shakir Mohamed and Danilo J. Rezende http://arxiv.org/abs/1509.08731 and a somewhat older survey paper on the same topic: How can we define intrinsic motivation? Pierre-Yves Oudeyer, Frederic Kaplan http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf

Paper and discussion history. A mention is not an endorsement.

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning — mentioned by davmre

Fri, 23 Oct 2015 00:24:46 GMT

The point is that supervised training on large networks requires millions or billions of examples. In RL, that means potentially billions of execution trajectories, which can be really expensive if you're executing on an actual, physical robot. And even then, supervised classification objectives are more informative than just a scalar reward: when a supervised system makes a wrong prediction you don't just tell it "sorry, that's wrong", you also tell it what the right prediction would have been. That extra information is really valuable for learning. In the black-box RL setting, you only see…

Paper and discussion history. A mention is not an endorsement.