HackerPapers

arXiv:1712.01815cs.AI

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis

Abstract

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 14 identifiable commenters in 14 comments across 13 threads, from Dec 2017 to May 2025. Found in 1 HN submission from Dec 2017 to Dec 2017. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments13 threads

  1. Achieving Human Level Competitive Robot Table Tennis 1 comment

    • It's exactly at the level I thought was communicated by the title and introduction. "Human level competitive", "solidly amateur-level human performance", "beat 100% of beginners and 55% of intermediate players". That robot would definitely win some games in your local club…

      dmurrayView on HNParent context

  2. Ask HN: Which recent research paper blew your mind? 1 comment

    • I thought the AlphaZero paper was pretty cool: https://arxiv.org/abs/1712.01815 Not only did we get a whole new type of Chess engine, it was also interesting to see how the engine thought of different openings at various stages in its training. For instance, the Caro-Kann, which…

      dargscisyhpView on HNParent context

  3. 4000x Speedup in Reinforcement Learning with Jax 1 comment

    • According to the AlphaZero paper (https://arxiv.org/pdf/1712.01815.pdf) they ran game logic on TPUs: > Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using 5,000 first-generation TPUs to generate self-play games…

      sillysaurusxView on HNParent context

  4. Poisoning web-scale training datasets is practical 1 comment

    • >alphazero's loss function never changed Yes, it did. Alpha zero's loss function changes with every update. The loss function works on the actual outcome versus the predicted outcome, and the predicted outcome is a result of previous learning. So with every step it takes, which…

      SideQuarkView on HNParent context

  5. John Carmack: I’m going to work on artificial general intelligence 1 comment

    • Not working in this area but the abstract of the AlphaZero paper [0] seems to disagree about your /any prior knowledge/ point: "Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the…

      tastroderView on HNParent context

  6. AI winter is well on its way 1 comment

    • The results were published - https://arxiv.org/abs/1712.01815 I agree AlphaZero had fancier hardware and so it wasn't really a fair fight.

      tim333View on HNParent context

  7. Likelihood of discontinuous progress around the development of AGI 1 comment

    • There's a vector encoding "queen moves" and another encoding "knight moves". Between them, they cover all possible chess moves. Knight moves obviously are modelled by "knight moves". Edit: https://arxiv.org/pdf/1712.01815.pdf The information you want is on page 13, last…

      YeGoblynQueenneView on HNParent context

  8. Lessons from Optics, the Other Deep Learning 1 comment

    • I'm curious why you believe to be able to tell that my comment is not accurate when you yourself admit that you have no idea about deep learning? Note that I'm not saying that Google is doing something stupid or leaving potential gains on the table. What I'm saying is that their…

      yorwbaView on HNParent context

  9. AI and Deep Learning in 2017 – A Year in Review 1 comment

    • I think for most people that hurdle of understanding is immense, and isn't likely to get any better. ML/DL infrastructure will continue to improve and deploying AI systems will become more accessible as time goes on. But I don't see familiarity with (non) convex optimization…

      nealeratzlaffView on HNParent context

  10. Ask HN: What was the best CS paper you read in 2017? 1 comment

    • Programming languages Slepak et al - An Array-Oriented Language with Static Rank Polymorphism [1] Distributed systems Kiayias et al - A Provably Secure Proof-of-Stake Blockchain Protocol [2] Cognitive computing Boahen - A Neuromorph's Prospectus [3] Machine learning Silver et al…

      beefmanView on HNParent context

  11. Is AlphaZero really a breakthrough in AI? 1 comment

    • Interesting information on reasons for choosing a 1GB hash size, thanks. > A0 should have let Stockfish use its opening book though. Has it been confirmed that it didn't? As far as I can tell, DeepMind never said that Stockfish didn't use an opening book or endgame tablebase…

      vatueilView on HNParent context

  12. Google's AlphaZero Beats Stockfish In 100-Game Match 2 comments
  13. Mastering Chess and Shogi by Self-Play with General Reinforcement Learning 1 comment

    • Here's an HTML version of the paper: https://www.arxiv-vanity.com/papers/1712.01815/ Table 2 is broken, but the rest is much more readable if you're on a phone.

      bfirshView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.