HackerPapers

arXiv:1603.01121cs.LG

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Johannes Heinrich, David Silver

Abstract

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.

arXiv comments
updated version, incorporating conference feedback
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.GTComputer Science and Game Theory

On Hacker News

Linked by 3 identifiable commenters in 3 comments across 3 threads, from Jan 2017 to Dec 2021. Found in 1 HN submission from Apr 2016 to Apr 2016. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Player of Games 1 comment

    • I'm more surprised David Silver isn't on it, since his background is in imperfect information games, with papers such as https://arxiv.org/abs/1603.01121 He did multiple poker papers before he was the main author of Alpha Zero.

      thomasahleView on HNParent context

  2. Is AlphaZero really a breakthrough in AI? 1 comment

    • I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information…

      sanxiynView on HNParent context

  3. Game Theory Reveals the Future of Deep Learning 1 comment

    • That paper doesn't mention game theory or the Nash equilibrium at all. Perhaps you mean [2], which does. However, the article seems to misinterpret this: the fact that that system approximates the Nash equilibrium should be expected. Any system (including humans) that performs…

      nlView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.