HackerPapers

arXiv:2411.10053cs.AI

That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design

Anna Goldie, Azalia Mirhoseini, Jeff Dean

Abstract

In 2020, we introduced a deep reinforcement learning method capable of generating superhuman chip layouts, which we then published in Nature and open-sourced on GitHub. AlphaChip has inspired an explosion of work on AI for chip design, and has been deployed in state-of-the-art chips across Alphabet and extended by external chipmakers. Even so, a non-peer-reviewed invited paper at ISPD 2023 questioned its performance claims, despite failing to run our method as described in Nature. For example, it did not pre-train the RL method (removing its ability to learn from prior experience), used substantially fewer compute resources (20x fewer RL experience collectors and half as many GPUs), did not train to convergence (standard practice in machine learning), and evaluated on test cases that are not representative of modern chips. Recently, Igor Markov published a meta-analysis of three papers: our peer-reviewed Nature paper, the non-peer-reviewed ISPD paper, and Markov's own unpublished paper (though he does not disclose that he co-authored it). Although AlphaChip has already achieved widespread adoption and impact, we publish this response to ensure that no one is wrongly discouraged from innovating in this impactful area.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 4 identifiable commenters in 9 comments across 3 threads, from Nov 2024 to Feb 2025. Found in 2 HN submissions from Nov 2024 to Nov 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Accelerating scientific breakthroughs with an AI co-scientist 1 comment

    • Google has responded to this "controversy" already: https://x.com/JeffDean/status/1858540085794451906 https://arxiv.org/abs/2411.10053

      warbakerView on HNParent context

  2. Jeff Dean responds to EDA industry about AlphaChip 6 comments

    • The paper: https://arxiv.org/abs/2411.10053

      wholehogView on HNParent context

    • > One key argument in the rebuttal against the ISPD article is that the resources used in their comparison were significantly smaller. To me, this point alone seems sufficient to question the validity of the ISPD work's conclusions. What are your thoughts on this? I believe this…

      vighneshiyerView on HNParent context

    • You're linking to his amended complaint - his original complaint was thrown out because it alleged things like "Google's motto is don't be evil, but they were evil, thus defrauding me." According to a Google investigator's sworn statement, he admitted that he didn't have…

      wholehogView on HNParent context

    • Pre-training is just training on multiple chips. "If Cheng et al. had reached out to the corresponding authors of the Nature paper, we would have gladly helped them to correct these issues prior to publication" (https://arxiv.org/pdf/2411.10053) That's how you actually do a…

      wholehogView on HNParent context

    • > it might have a harder time with a chip designed by a third party (further from its pre-training). Then they could pre-train on chips that are in-distribution for that task. See also section 3.1 of their response paper, where they describe a comparison against commercial…

      wholehogView on HNParent context

    • So much wasted time. He even ran a study internally (with Markov), but, as the AlphaChip authors describe: In 2022, it was reviewed by an independent committee at Google, which determined that “the claims and conclusions in the draft are not scientifically backed by the…

      wholehogView on HNParent context

  3. AI Alone Isn't Ready for Chip Design 2 comments

    • Meanwhile Google has already used AI for this exact stage of chip design with AlphaChip: https://deepmind.google/discover/blog/how-alphachip-transfor... And they've responded to the absurd degree of skepticism that followed: "That Chip Has Sailed: A Critique of Unfounded…

      negativeonehalfView on HNParent context

    • The AlphaChip authors beg to differ... https://deepmind.google/discover/blog/how-alphachip-transfor... https://arxiv.org/abs/2411.10053

      wholehogView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.