Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm · HackerPapers discussions

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by dmurray

Sat, 03 May 2025 07:19:16 GMT

It's exactly at the level I thought was communicated by the title and introduction. "Human level competitive", "solidly amateur-level human performance", "beat 100% of beginners and 55% of intermediate players". That robot would definitely win some games in your local club league, except that it doesn't serve, and unless it's cheating in ways the announcement glosses over like extra cameras - DeepMind have some history here so I reserve the right to be skeptical. The only thing I'd take issue with in the abstract is "Table tennis... requires human players to undergo years of training to…

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by dargscisyhp

Mon, 24 Jul 2023 14:43:11 GMT

I thought the AlphaZero paper was pretty cool: https://arxiv.org/abs/1712.01815 Not only did we get a whole new type of Chess engine, it was also interesting to see how the engine thought of different openings at various stages in its training. For instance, the Caro-Kann, which is my weapon of choice, was favored quite heavily by it for several hours and then seemingly rejected (perhaps it even refuted it?!) near the end.

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by sillysaurusx

Fri, 07 Apr 2023 17:17:55 GMT

According to the AlphaZero paper (https://arxiv.org/pdf/1712.01815.pdf) they ran game logic on TPUs: > Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using 5,000 first-generation TPUs to generate self-play games and 64 second-generation TPUs to train the neural networks. Further details of the training procedure are provided in the Methods.

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by SideQuark

Wed, 22 Feb 2023 14:16:09 GMT

>alphazero's loss function never changed Yes, it did. Alpha zero's loss function changes with every update. The loss function works on the actual outcome versus the predicted outcome, and the predicted outcome is a result of previous learning. So with every step it takes, which are random (and the initial weights are also random), it modifies it's own loss function for future walks in the space of weights. Rerun the entire training with different initial random weights, and look at the sequence of loss functions, and you will get a different sequence. The paper…

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by tastroder

Thu, 14 Nov 2019 08:13:25 GMT

Not working in this area but the abstract of the AlphaZero paper [0] seems to disagree about your /any prior knowledge/ point: "Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case." [0] https://arxiv.org/abs/1712.01815

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by tim333

Wed, 30 May 2018 09:50:18 GMT

The results were published - https://arxiv.org/abs/1712.01815 I agree AlphaZero had fancier hardware and so it wasn't really a fair fight.

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by YeGoblynQueenne

Sun, 25 Feb 2018 16:34:30 GMT

There's a vector encoding "queen moves" and another encoding "knight moves". Between them, they cover all possible chess moves. Knight moves obviously are modelled by "knight moves". Edit: https://arxiv.org/pdf/1712.01815.pdf The information you want is on page 13, last paragraph. The previous paragraph describes the representation of the boardd.

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by yorwba

Tue, 13 Feb 2018 07:28:03 GMT

I'm curious why you believe to be able to tell that my comment is not accurate when you yourself admit that you have no idea about deep learning? Note that I'm not saying that Google is doing something stupid or leaving potential gains on the table. What I'm saying is that their methods make sense when you are able to perform enough experiments to actually make data-driven decisions. There is just no way to emulate that when you don't even have the budget to try more than one value for some hyperparameters. And since you mentioned chess: The paper https://arxiv.org/pdf/1712.01815.pdf doesn't…

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by nealeratzlaff

Tue, 02 Jan 2018 22:53:55 GMT

I think for most people that hurdle of understanding is immense, and isn't likely to get any better. ML/DL infrastructure will continue to improve and deploying AI systems will become more accessible as time goes on. But I don't see familiarity with (non) convex optimization, statistical learning, topology, etc. becoming mainstream. Without these its hard to see a reasonable path forward for AI. For me this year in DL was a lot narrower. it mostly revolved around systems becoming more human in capability. I think most of this is in the article already: We can now hopefully say that ImageNet…

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by beefman

Sat, 30 Dec 2017 19:42:37 GMT

Programming languages Slepak et al - An Array-Oriented Language with Static Rank Polymorphism [1] Distributed systems Kiayias et al - A Provably Secure Proof-of-Stake Blockchain Protocol [2] Cognitive computing Boahen - A Neuromorph's Prospectus [3] Machine learning Silver et al - Mastering Chess and Shogi [4] 1 http://www.ccs.neu.edu/home/shivers/papers/rank-polymorphism... 2 https://eprint.iacr.org/2016/889.pdf 3 https://pdfs.semanticscholar.org/3767/1e53c9949fa08d9dc150ad... 4 https://arxiv.org/pdf/1712.01815.pdf

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by vatueil

Fri, 15 Dec 2017 00:31:47 GMT

Interesting information on reasons for choosing a 1GB hash size, thanks. > A0 should have let Stockfish use its opening book though. Has it been confirmed that it didn't? As far as I can tell, DeepMind never said that Stockfish didn't use an opening book or endgame tablebase, only that AlphaZero didn't: https://twitter.com/demishassabis/status/938347604462542849 The only time opening books are mentioned in the arxiv paper is when they are described as standard components of chess engines; while it doesn't explicitly state whether Stockfish was allowed to use them in the tests, I would think…

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by wrsh07

Thu, 07 Dec 2017 22:33:42 GMT

The arxiv paper is here: https://arxiv.org/abs/1712.01815

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by heliumcraft

Thu, 07 Dec 2017 13:40:49 GMT

You can see that on page 7 of the paper https://arxiv.org/pdf/1712.01815.pdf

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — mentioned by bfirsh

Wed, 06 Dec 2017 10:08:15 GMT

Here's an HTML version of the paper: https://www.arxiv-vanity.com/papers/1712.01815/ Table 2 is broken, but the rest is much more readable if you're on a phone.

Paper and discussion history. A mention is not an endorsement.

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — HN submission

Wed, 06 Dec 2017 03:17:22 GMT

Mastering Chess and Shogi by Self-Play with General Reinforcement Learning

539 points; 270 discussion replies on HN. Paper and source history. A submission is not an endorsement.