HackerPapers

arXiv:2211.00241cs.LG

Adversarial Policies Beat Superhuman Go AIs

Tony T. Wang, Adam Gleave, Tom Tseng, Kellin Pelrine, Nora Belrose, Joseph Miller, Michael D. Dennis, Yawen Duan, Viktor Pogrebniak, Sergey Levine, Stuart Russell

Abstract

We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes. Example games are available https://goattack.far.ai/.

arXiv comments
Accepted to ICML 2023, see paper for changelog
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.CRCryptography and Security stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 5 identifiable commenters in 5 comments across 5 threads, from Nov 2022 to Nov 2023. Found in 2 HN submissions from Nov 2022 to Dec 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. In two moves, AlphaGo and Lee Sedol redefined the future (2016) 1 comment

    • The commenters here saying "humans cannot win anymore", "there is no chance that humans can beat the best Go AI anymore" are apparently unaware that this is no longer true. See https://arxiv.org/abs/2211.00241 and https://goattack.far.ai/ The best Go programs have a flaw that…

      radford-nealView on HNParent context

  2. Ask HN: It is important”, a side effect of Transformers attention? 1 comment

    • Maybe we can extrapolate Godel where a complete ruleset cannot be consistent, and a consistent ruleset cannot be complete. Somewhere in there is the human ingenuity to adapt certain pathological patterns [1] that defeat (current) AI. But it's not always obvious, and the composer…

      turtleyachtView on HNParent context

  3. 60 Nations agree to regulate AI in the military before it destroys the world 1 comment

    • Notably https://arxiv.org/abs/2211.00241 > We attack the state-of-the-art Go-playing AI system, KataGo, by training adversarial policies that play against frozen KataGo victims. Our attack achieves a >99% win rate when KataGo uses no tree-search, and a >77% win rate when KataGo…

      stevenhuangView on HNParent context

  4. Man beats machine at Go in human victory over AI 1 comment

    • Congrats! I see by comparing the author list of v1 and V2 of this paper[1] that a Go player[2] was consulted before making claims about defeating a Go AI. This is indeed a nontrivial exploit that passes muster with a Go player who was skeptical last time around [3]! [1]…

      brileeView on HNParent context

  5. Adversarial Attack Beats Near-Superhuman AI Go Player 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.