Emergent Abilities of Large Language Models · HackerPapers discussions

Emergent Abilities of Large Language Models — mentioned by chasd00

Thu, 27 Aug 2026 19:52:20 GMT

i think there's plenty of evidence of emergent behavior in llms already. This is the driving factor for just more parameters and bigger training sets, "If we just make it bigger maybe more unexpected behaviors will emerge". idk how legit this is but the summary is my point https://arxiv.org/abs/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by aspenmartin

Fri, 27 Mar 2026 14:12:49 GMT

So what do you think the difference is between humans and an agent in this respect? What makes you think this has any relevance to the problem? everything is combinatorially explosive: the combination of words that we can string into sentences and essays is also combinatorially explosive and yet LLMs and humans have no problem with it. It's just the wrong frame of thinking for what's going on. These systems are obtaining higher and higher levels of abstractions because that is the most efficient thing for them to do to gain performance. That's what reasoning looks like: compositions of higher…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Fri, 27 Feb 2026 00:58:33 GMT

Emergent Abilities of Large Language Models (2022)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by swyx

Thu, 14 Aug 2025 21:22:44 GMT

the problem is that the eval processes dont really work here if you believe in "Emergent Abilities" https://arxiv.org/abs/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by neom

Sat, 31 May 2025 18:31:09 GMT

the extraordinary claim would be that LLMs can only do things they've seen before exactly, given the compositional and emergent capabilities we observe. The evidence suggests they can generalize beyond their training in meaningful ways, even if imperfectly...if a human came out living but with a brain that had zero electrical activity, that would be extraordinary, we normally come out with a baseline of pre-programming. I sometimes think this debate happens because humans don't want to admit we're nothing more than LLMs programmed by nature and nurture, human seem to want to be especially…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by mtts

Mon, 21 Apr 2025 13:45:24 GMT

The paper in question: https://arxiv.org/pdf/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by godelski

Sat, 08 Feb 2025 01:19:33 GMT

I don't think this makes sense and I'm not quite sure why you went to ML, but that's okay. I am a machine learning researcher, but also frustrated with the state of machine learning, in part because, well... you can probably see how "proof by empirical evidence" is dialed up to 11. Sorry, long answer incoming. It is far from complete too but I think it will help build strong intuition around your questions. Will knowledge transfer? That entirely depends on the new problem. It also entirely depends on how related the problem is. But also, what information was used to solve the pre-transfer…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Thu, 03 Oct 2024 12:47:11 GMT

Emergent Abilities of Large Language Models (2022)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Mon, 20 May 2024 19:46:53 GMT

Emergent Abilities of Large Language Models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by XzAeRosho

Thu, 16 May 2024 20:40:03 GMT

It's considered an emergent phenomenon of LLMs [1]. So arithmetic reasoning seems to increase as LLMs reasoning grows too. I seem to recall a paper mentioning that LLMs that are better at numeric reasoning are better at overall conversational reasoning too, so it seems like the two come hand in hand. However we don't know the internals of ChatGPT-4, so they may be using some agents to improve performance, or fine-tuning at training. I would assume their training has been improved IMO. [1]: https://arxiv.org/pdf/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by cmcollier

Thu, 18 Apr 2024 17:57:10 GMT

In terms of "the moment", I would imagine it happened during development inside Google (Lamda) or OpenAI (GPT2/3). More technically, here's one of the key papers discussing the topic (from google): * https://arxiv.org/abs/2206.07682 Emergent Abilities of Large Language Models Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by woopsn

Mon, 25 Mar 2024 06:43:23 GMT

Not quite in retrospect -- the earlier paper on emergence addresses this > It is also important to consider the evaluation metrics used to measure emergent abilities. For instance, using exact string match as the evaluation metric for long-sequence targets may disguise compounding incremental improvements as emergence. Similar logic may apply for multi-step or arithmetic reasoning problems, where models are only scored on whether they get the final answer to a multi-step problem correct, without any credit given to partially correct solutions. - Page 7, https://arxiv.org/pdf/2206.07682.pdf…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by n2d4

Mon, 25 Mar 2024 05:31:34 GMT

The original emergence paper has multiple "emergent" metrics. [1] Though, the paper this article is about [2] mentions that among the "emergent" tasks they tested, 92% were measuring accuracy on either exact-string-match or multiple choice, so probably just bad metrics. [1] https://arxiv.org/pdf/2206.07682.pdf [2] https://arxiv.org/pdf/2304.15004.pdf

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by n2d4

Mon, 25 Mar 2024 05:27:54 GMT

When we talk about "emergence" in machine learning, we talk about those metrics with a sudden jump, as explained in the paper that introduced the term: https://arxiv.org/abs/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by ekez

Mon, 25 Mar 2024 04:05:28 GMT

The metric the authors use confuses me. Edit distance seems like a strange way to test if the model understands arithmetic ([1], Figure 3). I think `1+3=3` would be equally as correct as `1+1=9`? Why not consider how far off the model is `abs(actual-expected)`? I wonder if there is an inflection point with that metric. https://arxiv.org/abs/2206.07682

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by godelski

Mon, 25 Mar 2024 02:15:17 GMT

Emergence is a weird topic. If we go with the physicist's common definition [0,1] we'd need to differentiate into weak and strong. As for weak emergence[2] that is de facto. As for strong emergence[3] well... do we have an example of the phenomena anywhere? To be clear, ML people have a different definition which I do not think is entirely useful. Let's look at the wording used in [4] Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by lern_too_spel

Tue, 20 Feb 2024 01:49:11 GMT

The original paper on emergent behaviors in LLMs is from Google: https://arxiv.org/abs/2206.07682 The difference between OpenAI and Google isn't due to different research directions between the two firms. It is a difference in ability to execute.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by ignoramous

Sat, 04 Nov 2023 12:40:43 GMT

Depends. Models are matrices of floats and so there's little chance an umbrella-term like "stochastic parrot" will never not stick, even when they already show signs of syntactic, semantic world-building capability (https://www.arxiv-vanity.com/papers/2206.07682/). If you are like me (and them: https://archive.is/cZi83) and deem instruction following, chain-of-thought prompting, computational properties of LLMs (as researchers continue to experiment with training, memory, modality, and scaling, for example, to arrive at abstract reasoning) as emergent, then we're on the same page.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by gardnr

Sat, 28 Oct 2023 19:27:48 GMT

This is funny because everyone uses the same tests to evaluate different models. One example is the Hugging Face OpenLLM Leaderboard. [1] The problem that this "paper" is making fun of is that any model can score very well on these tests when the content of the tests themselves is used in the training of the models. Here is one example of this happening with a 13B model that scored way too high. [2] In reality, when reviewing test scores, it's difficult to discern whether a model has been pre-trained on some specific content that is related to the test, or if the model has been trained in a…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Sun, 17 Sep 2023 10:45:52 GMT

Ask HN: Transformer alternatives that could have emergent properties when scaled

I am trying to identify model architecture candidates that could, like transformers, have "emergent" properties when they are scaled (see https://arxiv.org/abs/2206.07682). Some contenders I already know about are: * Monarch Mixer (https://arxiv.org/pdf/2111.00396.pdf) * Hyena (https://hazyresearch.stanford.edu/blog/2023-03-07-hyena) Thanks for your help.

6 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by cs702

Thu, 03 Aug 2023 16:06:54 GMT

We're talking about different things. You're talking about finetuning models to tasks known in advance. I'm talking about the ability to generalize to new tasks: https://arxiv.org/pdf/2206.07682. Please don't argue against a straw-man.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by treprinum

Fri, 28 Jul 2023 11:41:25 GMT

Right, the page 28 of this article mentions "On some benchmarks, task-general models (not explicitly trained to perform a task) surpass prior state-of-the-art performance held by a task-specific model" but they don't provide data for fine-tuning of their very large models: https://arxiv.org/pdf/2206.07682.pdf

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by theptip

Tue, 18 Jul 2023 20:19:01 GMT

> GPT-4 is not a pure LLM I’ve looked a bit into the GPT architecture and haven’t seen anything suggesting it’s doing special-case experts for maths. It has MoE over 16 language models, and an image modality bolted on. If you have any evidence that there is a separate trained logic/math model I’d love to see that, as it would be interesting. (I don’t recall reading anything like that in the GPT papers for example, and this seems to claim there is no “calculator” hooked up in GPT-4 https://ai.stackexchange.com/a/40090). > To put it another way: a (pure) language model has no way of encoding…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by pgspaintbrush

Tue, 13 Jun 2023 02:51:55 GMT

Author here. First off, thank you for reading and for your thoughts. I provided examples that I thought would be intuitive for humans to help folks understand that an understanding of the underlying phenomena is useful for next token prediction (I've added this as a note). Could you share what part of the article came across as suggesting that LLMs "magically" acquire whatever ability helps them to predict? I'd like to make that section clearer, so that doesn't come across. Re: "LLMs are not particularly good at arithmetic". There are published results that show that LLMs using certain…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by celestialcheese

Thu, 18 May 2023 20:21:40 GMT

Yet there are emergent behaviours from these LLMs that are both surprising and not immediately understood. [1][2][3] Everyone has theories, of course, but still pretty "magic" considering these behaviours weren't theorised in papers prior to observation. 1 - https://www.jasonwei.net/blog/emergence 2 - https://arxiv.org/pdf/2206.07682.pdf 3 - https://www.quantamagazine.org/the-unpredictable-abilities-e...

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by kerkeslager

Thu, 18 May 2023 19:47:58 GMT

> Echoing many who now find themselves blindsided by the emergent abilities and rapid adoption of LLMs, the OP I've said this over and over again: there are no emergent abilities. Before you leap to link me this paper, I'll link it myself: https://arxiv.org/abs/2206.07682 I read that paper. Did you? Did you understand it? Because if you had, you'd have seen that early on they define what they mean when they say "emergent abilities", and it's not what almost anyone else means when they say "emergent abilities". They're not claiming that the abilities of LLMs are anything more than the sum of…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by adamsmith143

Wed, 17 May 2023 14:25:45 GMT

This is demonstrably wrong. It can clearly generate unique text not from it's training corpus and can successfully answer logic based questions that were also not in it's training corpus. Another paper not from Msft showing emergent task capabilities across a variety of LLMs as scale increases. https://arxiv.org/pdf/2206.07682.pdf You can hem and haw all you want but the reality is these models have internal representations of the world that can be probed via prompts. They are not stochastic parrots no matter how much you shout in the wind that they are.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by lhl

Mon, 08 May 2023 23:59:40 GMT

Here is Jason Wei's response "Common arguments regarding emergent abilities": https://www.jasonwei.net/blog/common-arguments-regarding-eme... Wei was the lead author of the original "Emergent Abilities" paper: Wei, Jason, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, et al. “Emergent Abilities of Large Language Models.” arXiv, October 26, 2022. https://doi.org/10.48550/arXiv.2206.07682.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Tue, 18 Apr 2023 23:06:51 GMT

Emergent Abilities of Large Language Models

4 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by BiasRegularizer

Mon, 03 Apr 2023 15:35:38 GMT

It's not impossible to see that LLMs can make finite step "reasoning" between input and output, as each block of the transformer can model probabilistic causality. Transformers are sometimes considered a fully connected graph neural network, which can be used for modeling causal graphs. One additional supporting evidence on the finite step reasoning hypothesis is that "train of thought" improves the performance of LLMs[1], meaning that letting the model explain itself explicitly reduces the amount of implicit "reasoning" steps that need to happen within the model. Additionally, while we don't…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Sun, 02 Apr 2023 13:16:17 GMT

Emergent Abilities of Large Language Models

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by computerex

Fri, 24 Mar 2023 21:38:07 GMT

This paper is pretty approachable and goes over the "scaling laws" in detail: https://arxiv.org/abs/2206.07682 In short, yes. More data, higher quality data, more epochs on the data. That is the name of the game.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by stevenhuang

Wed, 08 Mar 2023 02:59:14 GMT

The scaling appears linear in relationship, though on some performance metrics it is super linear. See Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.07682 Scaling Laws for Generative Mixed-Modal Language Models https://arxiv.org/abs/2301.03728 All this demonstrates further the "bitter lesson" http://www.incompleteideas.net/IncIdeas/BitterLesson.html that surprising ability emerges from just throwing more data and compute at the problem and is seen more fruitful an endeavor than trying to find a deeper/analytic solution to cognition (like Yann Lecun's research, who has…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by gwern

Wed, 15 Feb 2023 23:06:01 GMT

I think there is a small possibility that optimal scaling as we understand it right now (scaling compute + task/data diversity + parameter count in an optimal fashion), could just emerge (or scale) consciousness. We at present do not know what leads to emergence in the many now-documented examples of emergent capabilities in large NNs (see https://arxiv.org/abs/2206.07682#google https://gwern.net/doc/ai/scaling/emergence/index ), cannot predict either their timing or their presence, and cannot even be sure that there is not 'hidden scaling' where the capability would exhibit smooth scaling if…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — HN submission

Tue, 14 Feb 2023 05:48:21 GMT

Emergent Abilities of Large Language Models

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by dhruvdh

Thu, 09 Feb 2023 16:24:26 GMT

1. I don't understand how LLMs work. 2. I don't understand how ChatGPT works, but I have used it a few times. 3. I will use ChatGPT as the absolute measure of what LLMs are capable of. --- 1. I don't understand intelligence. 2. Humans are intelligent, humans can learn to do math. 3. LLMs are not good at math. 4. LLMs are not intelligent, they're just text compression. --- 1. I don't understand how LLMs work. 2. I have a decent grasp of how image compression works. 3. I will use my grasp of image compression to pretend LLMs are text compression. 4. I will apply all limitations of image…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by iliane5

Tue, 24 Jan 2023 18:14:10 GMT

Couldn't agree more. This paper (https://arxiv.org/abs/2206.07682) also touches on a pretty fascinating phenomenon - that when scaling up large language models they seem to "naturally" obtain new emergent abilities that do not exist on smaller models.

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by nl

Tue, 24 Jan 2023 05:29:58 GMT

This is an incomplete understanding of what very large LMs are doing. At a very large scale new behaviors emerge[1][2]. It's true that the fluency of language models is easily explained by "predict next token given context" but that doesn't preclude the fact the LLMs are functionally doing reasoning up to some limits. To quote: > However, it is unclear how these models obtain the answers and whether they rely on simple heuristics rather than the generated chain-of-thought. To enable systematic exploration of the reasoning ability of LLMs, we present a new synthetic question-answering dataset…

Paper and discussion history. A mention is not an endorsement.

Emergent Abilities of Large Language Models — mentioned by krackers

Sun, 11 Dec 2022 07:17:41 GMT

Thanks, this is a nice explanation! Also found a paper related to emergence in LLMs: https://arxiv.org/pdf/2206.07682.pdf

Paper and discussion history. A mention is not an endorsement.