HackerPapers

arXiv:2304.15004cs.AI

Are Emergent Abilities of Large Language Models a Mirage?

Rylan Schaeffer, Brando Miranda, Sanmi Koyejo

Abstract

Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due to the researcher's choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities; (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 23 identifiable commenters in 29 comments across 25 threads, from May 2023 to Sep 2026. Found in 2 HN submissions from May 2023 to Feb 2026. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments25 threads

  1. AI companies in race to demonstrate their model most threatening to humanity 1 comment

    • Hallucination rate is a highly nonlinear metric relative to other model success metrics. A similar phenomenon to what is going on here: https://arxiv.org/abs/2304.15004 . This does not mean progress has stalled.

      hodgehog11View on HNParent context

  2. Zig Creator Calls Spade a Spade, Anthropic Blows Smoke 1 comment

    • Then they are naive. I've seen this a lot first hand during my PhD. Interpreting results in a way that reinforces their beliefs and ignores alternate hypotheses. Frequently, those alternative hypotheses are demonstrated to be true. For example, "emergence" as was researched and…

      nsagentView on HNParent context

  3. Why AI Agents Cannot Change Software Systems 1 comment

    • LLMs are Markov Chains [1]. "Emergent abilities" of LLMs can be explained by decrease of perplexity in text prediction [2]. [1] https://arxiv.org/abs/2410.02724 [2] https://arxiv.org/abs/2304.15004

      theszView on HNParent context

  4. The revenge of the data scientist 1 comment

    • > You recognize that you haven't really needed strong mathematical (or coding) skills to create models for some time. And then there goes something like this [1], where researchers failed to control for p-value: "In this particular setting, emergent abilities claims are possibly…

      theszView on HNParent context

  5. Are LLM merge rates not getting better? 1 comment

    • Yes, I think this is basically an instance of the "emergent abilities mirage." https://arxiv.org/abs/2304.15004 If you measure completion rate on a task where a single mistake can cause a failure, you won't see noticeable improvements on that metric until all potential sources…

      yorwbaView on HNParent context

  6. On the slow death of scaling 1 comment

    • >the acceptance that there are emergent properties which appear out of nowhere is another way of saying our scaling laws don’t actually equip us to know what is coming. Is this actually accepted? Ever since [0], I thought people recognized that they don't appear out of nowhere.…

      charcircuitView on HNParent context

  7. What's the strongest AI model you can train on a laptop in five minutes? 1 comment

    • Which we probably should not, at least not the "sudden" emergence that those researchers claimed to see. https://arxiv.org/abs/2304.15004 Good article about why here; this helped me understand a lot: https://www.wired.com/story/how-quickly-do-large-language-mo...

      exasperaitedView on HNParent context

  8. Why do LLMs have emergent properties? 1 comment

    • Alternate view: Are Emergent Abilities of Large Language Models a Mirage? https://arxiv.org/abs/2304.15004 "Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities…

      cratermoonView on HNParent context

  9. Understanding Reasoning LLMs 1 comment

    • I don't think this makes sense and I'm not quite sure why you went to ML, but that's okay. I am a machine learning researcher, but also frustrated with the state of machine learning, in part because, well... you can probably see how "proof by empirical evidence" is dialed up to…

      godelskiView on HNParent context

  10. Learning to Reason with LLMs 1 comment

    • I'll give an argument against this with the caveat it applies only if these are pure LLMs without heuristics or helper models (I do not believe that to be the case with o1). The problem with auditing is not only are the outputs incorrect, but the "inputs" of the chained steps…

      abernard1View on HNParent context

  11. Choose your weapon: Survival strategies for depressed AI academics 2 comments

    • There was an article on HN last week discussing theoretical proof that scaling has a hard limit: https://www.quantamagazine.org/how-chain-of-thought-reasonin... Specifically: > To study how the parallel structure of transformers might limit their capabilities, the pair…

      nicklecompteView on HNParent context

    • >> To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the…

      humansareok1View on HNParent context

  12. “Emergent” abilities in LLMs actually develop gradually and predictably – study 3 comments

    • The paper: Are Emergent Abilities of Large Language Models a Mirage? https://arxiv.org/abs/2304.15004

      esafakView on HNParent context

    • It depends on how you do arithmetic. If you're a human and you do column addition, 12345+35791=58136 is just as big of a mistake as 48146 (the actual result is 48136). It's just one mistaken column in both. Binary half-adders work the same way. We don't really know how LLMs do…

      n2d4View on HNParent context

    • The original emergence paper has multiple "emergent" metrics. [1] Though, the paper this article is about [2] mentions that among the "emergent" tasks they tested, 92% were measuring accuracy on either exact-string-match or multiple choice, so probably just bad metrics. [1]…

      n2d4View on HNParent context

  13. How do neural networks learn? 1 comment

    • >Yes, we don't really understand where emergent capabilities are coming from, at least not to extent of being able to predict them ahead of time ("if we feed it this amount of data, of this type, it'll learn to do X"). New emergent capabilities arise, from time to time, as…

      nyrikkiView on HNParent context

  14. Elon Musk sues Sam Altman, Greg Brockman, and OpenAI [pdf] 1 comment
  15. Will scaling work? 2 comments

    • You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004…

      nsagentView on HNParent context

    • I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004…

      nsagentView on HNParent context

  16. Ask HN: What is the current state of "logical" AI? 1 comment

    • AI is not even close to having true logical reasoning, that's probably decades away. The issue is that cognitive scientists are clueless. Scientists have a good model for associative reasoning, which is the basis of modern neural networks, but we don't have a clue how abstract…

      nicklecompteView on HNParent context

  17. Ask HN: AI/ML papers to catch up with current state of AI? 1 comment

    • No emergence [2304.15004] Are Emergent Abilities of Large Language Models a Mirage? - arXiv https://arxiv.org/abs/2304.15004 Can't plan https://openreview.net/forum?id=X6dEqXIsEW No compositionality https://openreview.net/forum?id=Fkckkr3ya8 Apart from that it's great

      sgt101View on HNParent context

  18. Emergent Abilities of Large Language Models 1 comment

    • The paper is at https://arxiv.org/abs/2304.15004 The submitted link is to a prizegiving conference where the authors will be (probably? I'm not sure) presented with an award.

      pmontraView on HNParent context

  19. Compare how GPT-2, 3, 3.5 and 4 answer the same questions 1 comment

    • >While performance on benchmarks typically improves smoothly, sometimes specific capabilities emerge without warning (Wei et al., 2022a). The conclusions of that paper aren't very convincing (see https://arxiv.org/abs/2304.15004 ).

      alephxyzView on HNParent context

  20. Elegant and powerful new result that seriously undermines large language models 1 comment

    • Idk, one obvious reason could be that emergence is a mirage ( https://arxiv.org/abs/2304.15004 ). Given two concepts of roughly equivalent complexity and representation in training, where one is successfully generalized and another is not, that would seem to support (but not…

      photonthugView on HNParent context

  21. OpenLLaMA 13B Released 1 comment

    • The paper was fairly recent, https://arxiv.org/abs/2304.15004. It was more thoroughly confirming what was generally agreed upon while debunking other reasons. The key insight from the abstract, "Specifically, nonlinear or discontinuous metrics produce apparent emergent…

      ianbutlerView on HNParent context

  22. Cargo Cult AI 1 comment

    • Wow, this is refreshing. To your point - the underpinning technology (regression based function approximation) holds more importance than general and adaptable intelligence. It holds more importance than something that is going to, or is capable of, "escaping the box and killing…

      srslackView on HNParent context

  23. Sam Altman goes before US Congress to propose licenses for building AI 1 comment

    • "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/pdf/2304.15004.pdf our alternative suggests that existing claims of emergent abilities are creations of the researcher’s analyses, not fundamental changes in model behavior on specific tasks with scale.

      revelioView on HNParent context

  24. LLMs are not greater than the sum of their parts: researchers 1 comment

    • There was a discussion about the paper[0] this was based on 7 days ago: https://news.ycombinator.com/item?id=35768824 [0] https://arxiv.org/abs/2304.15004

      popinman322View on HNParent context

  25. Scary 'Emergent' AI Abilities Are Just a 'Mirage', Stanford Study Says 1 comment

    • The Stanford paper's title ends with a question mark. Are Emergent Abilities of Large Language Models a Mirage?, (2023) https://arxiv.org/abs/2304.15004

      eternalbanView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.