IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures · HackerPapers discussions

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures — mentioned by heyitsguay

Wed, 22 Aug 2018 16:05:58 GMT

Very misleading title about a very interesting development! A link to the actual paper is here: https://arxiv.org/abs/1802.01561. In less clickbaity terms, DeepMind has developed a scalable system for distributed training of certain types of reinforcement learning agents. They use that to train an agent to solve a pretty big collection of different reinforcement learning problems.

Paper and discussion history. A mention is not an endorsement.

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures — mentioned by goolulusaurs

Wed, 30 May 2018 06:37:12 GMT

This paper is amazing, and exactly what I was thinking of posting in response to that part of the article. The amount of research Deepmind is putting out is astonishing, and even if you are paying attention it's hard to keep up with it all. Here are just a few papers I've been looking at from the last few months, maybe none of them are advancements on the level of AlphaGo & Zero but they still show significant progress in a wide variety of areas. https://arxiv.org/abs/1802.10542 https://arxiv.org/abs/1802.07740 https://www.nature.com/articles/s41586-018-0102-6 https://arxiv.org/abs/1804.09401…

Paper and discussion history. A mention is not an endorsement.

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures — mentioned by gwern

Thu, 15 Feb 2018 23:46:27 GMT

DRL is different from regular DL in that it tends towards CPU-heavy, not GPU-heavy. It's hard to saturate a single GPU/TPU since you're using tiny little NNs and only once in a while updating them based on long episodes through the environment. It might not be using GPUs/TPUs at all! If you look at the algorithm which that DM paper is based on, PPO, the original OpenAI paper & implementation (https://blog.openai.com/openai-baselines-ppo/) doesn't use GPUs, it's pure-CPU. (They have a second version which adds GPU support.) Or in a DM vein, look at their latest IMPALA which you might've…

Paper and discussion history. A mention is not an endorsement.