HackerPapers

arXiv:2009.06489cs.CY

The Hardware Lottery

Sara Hooker

Abstract

Hardware, systems and algorithms research communities have historically had different incentive structures and fluctuating motivation to engage with each other explicitly. This historical treatment is odd given that hardware and software have frequently determined which research ideas succeed (and fail). This essay introduces the term hardware lottery to describe when a research idea wins because it is suited to the available software and hardware and not because the idea is superior to alternative research directions. Examples from early computer science history illustrate how hardware lotteries can delay research progress by casting successful ideas as failures. These lessons are particularly salient given the advent of domain specialized hardware which make it increasingly costly to stray off of the beaten path of research ideas. This essay posits that the gains from progress in computing are likely to become even more uneven, with certain research directions moving into the fast-lane while progress on others is further obstructed.

arXiv comments
Journal reference
DOI
Subjects
cs.CYComputers and Society cs.AIArtificial Intelligence cs.ARHardware Architecture cs.LGMachine Learning

On Hacker News

Linked by 10 identifiable commenters in 10 comments across 10 threads, from Nov 2020 to Jun 2026. Found in 5 HN submissions from Sep 2020 to Sep 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments10 threads

  1. How LLMs work 1 comment

    • > If you can see that these models empirically get better with scale, why would you swap the main architecture? Those events will be pretty rare c.f. hardware lotter https://arxiv.org/abs/2009.06489

      swyxView on HNParent context

  2. A Eureka machine that thinks like nature and explores what AI cannot 1 comment
  3. The Hardware Lottery 1 comment

    • It’s a well-known paper. The link at the top to the PDF works well https://arxiv.org/pdf/2009.06489

      arjieView on HNParent context

  4. Multi-Token Attention 1 comment

    • Check out The hardware lottery [1], which drove a lot of discussion a few years ago. [1]: https://arxiv.org/abs/2009.06489

      ssivarkView on HNParent context

  5. A new old kind of R&D lab 1 comment

    • Hi Jeremy & Eric, great to see your newest endeavor. I hope that Answer.AI builds on the success and impact that fast.ai has already enjoyed. Given new developments in hardware (by companies not named NVIDIA), I'm wondering if you are keen on exploring the next generation of…

      jwuphysicsView on HNParent context

  6. Nvidia and Foxconn to build 'AI factories' 1 comment

    • In the current world, deep learning with homogeneous computation graphs, tuned with backprop, has won the Hardware Lottery [1]. This is unfortunate for research outside of that area, but just looking at the momentum of development it seems like a sure bet to keep investing in…

      dkislyukView on HNParent context

  7. Ask HN: Which recent research paper blew your mind? 1 comment

    • Exactly. It's easy to see in retrospect, but hard in prospect: the original paper [1] on GPU acceleration of NNs reports a measly 20x speedup. Assuming a bit of cherry-picking on the author's side to make get the paper published, the 'real-world speedup' will have been assumed…

      GregarianChildView on HNParent context

  8. We're afraid language models aren't modeling ambiguity 1 comment

    • Related: The Hardware Lottery (https://arxiv.org/abs/2009.06489) It goes both ways: processors are built to accelerate existing applications, and applications are built to run with the constraints of the hardware. This holds back progress towards better systems where both new…

      awegioView on HNParent context

  9. Large-Scale Matrix Factorization on TPUs 1 comment

    • Because I used to work at Google on TPUs with all the teams (from ads to Physics), and have about 30 years experience in high performance computing. Neural nets and DL are just one of the reasons to use TPUs; in the future, they might not even be the preeminent approach. DL is…

      dekhnView on HNParent context

  10. Ride the Hardware Lottery 1 comment

    • Here's the 'Hardware Lottery' essay that this refers to: https://arxiv.org/pdf/2009.06489.pdf "Hardware, systems and algorithms research communities have historically had different incentive structures and fluctuating motivation to engage with each other explicitly. This…

      sdenton4View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.