Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model · HackerPapers discussions

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by hulium

Tue, 02 Sep 2025 21:21:45 GMT

During play, yes, obviously you need an implementation of the game to play it. But in its planning tree, no: > MuZero only masks legal actions at the root of the search tree where the environment can be queried, but does not perform any masking within the search tree. This is possible because the network rapidly learns not to predict actions that never occur in the trajectories it is trained on. https://arxiv.org/pdf/1911.08265

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — HN submission

Sat, 25 Nov 2023 10:01:06 GMT

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (2019)

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by not-my-account

Mon, 24 Jul 2023 17:42:07 GMT

totally! MuZero is my favourite[1] The super cool thing about MuZero is that it learns the dynamics of the problem, i.e. you don't have to give it the rules of the game, which makes the algorithm very general. For example, DeepMind threw MuZero at video compression and found that it can reduce video sizes by 6.28% (massive for something like YouTube)[2][3]. Curious if anyone else knows examples of MuZero being deployed outside of toy examples? [1] https://arxiv.org/pdf/1911.08265.pdf [2] https://arxiv.org/pdf/2202.06626.pdf [3] https://www.deepmind.com/blog/muzeros-first-step-from-resear...…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by notacat

Mon, 03 Jul 2023 15:09:43 GMT

AI Research Residency Program (1 year) | Zug/Zurich area | Full time | On site | 100-200k CHF/annum I am launching a new AI research organisation in the Zug/Zurich area. The goal of the organisation is to make a meaningful contribution towards solving AGI. I am looking for employees number no. 1 and 2. This will ideally be someone with a history of interesting research and a clear agenda for future research. I am personally interested in LLM's, MuZero (https://arxiv.org/abs/1911.08265), Dreamer (https://arxiv.org/pdf/2301.04104v1.pdf) and DreamCoder (https://arxiv.org/abs/2006.08381), and…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by sinenomine

Sun, 12 Jun 2022 00:28:46 GMT

> Most people will both lose time with false-start approaches that are scrapped To a discerning eye this pattern looks like stronger minds implementing higher quality tree search[1], either through implementing higher quality neural search heuristics for tree pruning or via greater search depth. Given relative NP-hardness of the problems[2] we are usually dealing with at our line of work and what we know about planning in machines and animals[3], it doesn't look immediately implausible. With planning capability strong enough, one doesn't need trial and error in the real world for moderately…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by ignoranceprior

Wed, 23 Dec 2020 22:02:54 GMT

This is explained in Appendix A of the paper ("Comparison to AlphaZero"): https://arxiv.org/pdf/1911.08265.pdf Basically, AlphaZero was provided with a simulator that was able to distinguish legal and illegal moves and determine which future game states would be wins or losses. This was used to generate the search tree of possible states and actions. MuZero doesn't have access to a simulator, it only has access to its direct environment. MuZero excludes actions that are immediately illegal, which solves the problem you mention in your penultimate paragraph, but it needs to learn the game's…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by bshanks

Wed, 23 Dec 2020 21:22:49 GMT

This is celebrating their Nature publication today. Here is the preprint: https://deepmind.com/research/publications/Mastering-Atari-G... https://arxiv.org/pdf/1911.08265.pdf

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by RiversHaveWings

Fri, 21 Aug 2020 19:18:46 GMT

After AlphaZero, DeepMind went on to develop MuZero (https://en.wikipedia.org/wiki/MuZero, paper: https://arxiv.org/abs/1911.08265), which is able to learn to play games without being provided an explicit model of their rules. "When evaluated on Go, chess and shogi, without any knowledge of the game rules, MuZero matched the superhuman performance of the AlphaZero algorithm that was supplied with the game rules."

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by YeGoblynQueenne

Thu, 09 Jul 2020 20:38:28 GMT

MuZero performs a "formal search". In many more ways than one, for example optimisation is still a search for an optimal search of parameters. But I guess you mean that it doesn't perform a tree search? Quoting from the abstract of the paper on arxiv [1]: In this work we present the MuZero algorithm which, by combining _a tree-based search_ with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, withoutany knowledge of their underlying dynamics. (My underlining) If I remember correctly, MuZero is model-free in the sense that it learns its…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by johnmoberg

Sat, 14 Dec 2019 20:15:31 GMT

I agree that is vague. But there are plenty of RL algorithms that learn with essentially no domain knowledge, e.g., MuZero[1] which doesn't even use a model. That doesn't make it AGI -- it only masters a single game at a time and we currently don't know how to transfer that knowledge to other domains. [1] https://arxiv.org/abs/1911.08265

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by swframe2

Wed, 27 Nov 2019 23:27:46 GMT

Check out MuZero. It learns an embedding of the game's state space potentially allowing AlphaGo like dominance in more ways. https://www.youtube.com/watch?v=We20YSAJZSE https://arxiv.org/abs/1911.08265 (Actually, ML is still not very good at causal reasoning so we have sometime. I'm more excited and worried about crispr at this point; what happens when we can make people genetically super human)

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by davidhowlett

Sat, 23 Nov 2019 21:22:44 GMT

Instead of an alpha-beta search, AlphaZero uses a general-purpose Monte Carlo tree search (MCTS) algorithm. Source: https://science.sciencemag.org/content/362/6419/1140.full I agree with your comment about System 2 like reasoning not being common right now. I am not an expert in the field but the closest thing I have seen to learned planning is: https://arxiv.org/pdf/1911.08265.pdf

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by darkmighty

Sat, 23 Nov 2019 20:30:30 GMT

> no AI that I know of has at its disposal a full blown model of the world it operates in This is a field called Model-based Reinforcement learning, and it's quite advanced already -- there are indeed models that have an internal state reflecting the world state. A good recent example: https://papers.nips.cc/paper/7512-recurrent-world-models-fac... > deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting This is also addressed by recent models, somewhat. Once you have an abstract world model, searching for a high…

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by nmca

Fri, 22 Nov 2019 19:38:24 GMT

There are certainly challenges, but "incorporating a world model" has been going well recently: "Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model" https://arxiv.org/abs/1911.08265

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — mentioned by gok

Fri, 22 Nov 2019 17:51:19 GMT

From the preprint [1]: > In Go, MuZero slightly exceeded the performance of AlphaZero, despite using less computation per node in the search tree (16 residual blocks per evaluation in MuZero compared to 20 blocks in AlphaZero). This suggests that MuZero may be caching its computation in the search tree and using each additional application of the dynamics model to gain a deeper understanding of the position. It also strikes me as possible that just not giving the system the rules to start with might have allowed it to explore more efficient strategies. [1] https://arxiv.org/pdf/1911.08265.pdf

Paper and discussion history. A mention is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — HN submission

Wed, 20 Nov 2019 23:23:11 GMT

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

161 points; 37 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — HN submission

Wed, 20 Nov 2019 12:02:46 GMT

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.