Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers · HackerPapers discussions

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by sigmoid10

Sun, 07 Jul 2024 21:25:57 GMT

That's false. It has been shown that LLMs can perform e.g. gradient descent internally [1], which can explain why they are so good at few shot prompting. The universal approximation theorem already tells us that a single layer is sufficient to approximate any function, so it should come as no surprise that modern deep networks with many layers should be able to perform iterative optimisations. [1] https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by PartiallyTyped

Thu, 05 Oct 2023 11:58:46 GMT

Seems very similar to "Language Models Implicitly Perform Gradient Descent as Meta-Optimizers" https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by PartiallyTyped

Wed, 12 Jul 2023 18:21:05 GMT

I can’t provide an example as I would rather not jeopardise my employment and livelihood. The domain is actually novel, is not a subdomain in any way whatsoever unless you consider “next token prediction” to be the class of all possible problems that LLMs may be used for. As for the parameter change remark, prompts change the __effective__ parameters of the model. [0] [0] https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by famouswaffles

Sat, 13 May 2023 19:18:04 GMT

>but they can only reason using their memories and the prompt. Eh no. https://arxiv.org/abs/2212.10559 >But if you try very hard you can find "held out" data and when you test on it, GPT4 stops looking so smart: This can be done to anybody. This can be done to you. It's not a gotcha. Nobody is saying GPTs don't/can't memorize.

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Mon, 24 Apr 2023 07:09:16 GMT

> The “learning” in “In context learning” is not the same concept as learning during the network “training”. The former is loosely named and does not have any impact on what’s actually “learned” by the model. Actually (and very surprisingly!) it is related. See the "Similarity of Attention Output Updates" and "Similarity of Attention Map" metrics in https://arxiv.org/abs/2212.10559 (linked above) where they show that in-context learning actually changes the attention map in a similar way to traditional fine tuning(!!). Obviously it can't modify the weights of the model on disk, but after ICL…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Mon, 24 Apr 2023 06:16:19 GMT

For those who don't completely get the impact of this: We already know Large Language Models (LLMs) can learn at runtime (ie, separately to the training process.) This is called "In Context Learning". See [1], [2] for more details. (BTW, when anyone says "LLMs are stochastic parrots" you know they are ignorant of this) In context learning is wonderful because it means you can "train" a LLM at run time by filling the context with examples. Traditionally this "context window" has been a few thousand tokens, and GPT-4 recently extended that to 32,000 tokens. That is useful, but if you wanted to…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by FeepingCreature

Wed, 05 Apr 2023 06:04:10 GMT

The relevant keyword you want is "zero-shot learning". (EDIT: Correction; "in-context learning". Sorry for that.) LLMs can pick up patterns from the context window purely at evaluation time using dynamic reinforcement learning. (This is one of those capabilities models seem to just pick up naturally at sufficient scale.) Those patterns are ephemeral and not persisted to memory, which I agree makes LLMs less general than humans, but that seems a weak objection to hang a fundamental difference in kind on. edit: Correction: I can't find a source for my claim that the model specifically picks up…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by Herval_freire

Fri, 31 Mar 2023 16:17:44 GMT

No there is reasoning in LLMs. This is actually proven through several scientific papers. I can't say whether it's using reasoning for specific answers. I'm sure many times it just regurgitates something that is known... But saying that LLMs in general can and do reason is categorically true. LLMs do attempt to extract actual mental models of the subject of the text. See here: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Thu, 30 Mar 2023 02:54:28 GMT

I think there is an issue in vocabulary size though. If I'm correct in thinking that the OP is proposing that what Llama does ("Notably, we split all numbers into individual digits") explicitly this allows the model to treat a number as a sequence of digits so it can learn how math works. This is especially important with very rare numbers. Take a number the GPT has never (or hardly ever) seen in its training data: 832408345770928764 The GPT-3 tokenizer tokenizes[1] that into: 83,24,08,345,7,709,287,64 To some degree this is forced to occur by the use of raw BPE encoding and the vocabulary…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by famouswaffles

Wed, 29 Mar 2023 05:42:34 GMT

The idea that updating weights is particularly necessary is questionable. Memory augmented llms are computationally universal. https://arxiv.org/abs/2301.04589 and make no mistake, in context learning can be used in a meaningful way to resolve deficiencies. gpt-3 getting 98.5% accuracy (on even very large numbers) on addition arithmetic by describing the algorithm of addition to be performed on 2 numbers. https://arxiv.org/abs/2211.09066 In context learning is just overpowered. It's essentially implicit fine-tuning https://arxiv.org/abs/2212.10559v2 as for asking questions, llms can do that…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by HervalFreire

Mon, 27 Mar 2023 06:30:58 GMT

>> LLMs have no knowledge of the underlying reality > They have no common sense & they can't plan their answer If his arguments are entirely based on this, then it's not fully correct: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Fri, 24 Mar 2023 10:12:48 GMT

I don't think that's clear at all. https://arxiv.org/abs/2212.10559 shows a LLM is doing gradient descent on the context window at inference time. If it's learning relationships between concepts at runtime based on information in the context window then it seems about as useful to say it is a Markov chain as it is to say that a human is a Markov chain. Perhaps we are, but the "current state" is unmeasurably complex.

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Fri, 17 Mar 2023 03:18:19 GMT

> We know how LLMs work and they auto-complete text. This isn't true. See eg[1] It seems to be a common misconception that the training objective "predict the next word" implies something about the capabilities of a LLM. As [1[] shows, instead they perform very sophisticated calculations on their compressed knowledge representation to try to generate acceptable text. These capabilities can be accessed for other purposes too. [1] "Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers" https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by famouswaffles

Thu, 16 Mar 2023 19:28:45 GMT

In context learning is already implicit finetuning. https://arxiv.org/abs/2212.10559. It's very questionable to what extent continuous training is necessary past a threshold of intelligence. Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by HervalFreire

Thu, 16 Mar 2023 07:43:04 GMT

- GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by HervalFreire

Thu, 16 Mar 2023 06:03:11 GMT

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Tue, 14 Mar 2023 22:59:25 GMT

> I must have missed the part when it started doing anything algorithmically. Yeah. "Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers" https://arxiv.org/abs/2212.10559 @dang there's something weird about this URL in HN. It has 35 points but no discussion (I guess because the original submission is too old and never got any traction or something)

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by nl

Sun, 12 Mar 2023 02:37:52 GMT

I didn't agree with the OP but just noting this is wrong: > These nets regurgitate the most average possible output given inputs. By definition. You seem to be confusing the training objective with the capabilities. For example: > Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by cgearhart

Sat, 04 Mar 2023 17:48:52 GMT

Bingo. There’s a recent paper that posits language models are meta learners where the transformer layer is approximating stochastic gradient descent updates from the inputs—this is what allows them to perform in-context learning. [1] If that’s true, then it is going to be impossible to prevent a sufficiently large language model from being prompt-hacked. You just need to find a collection of input tokens that moves the network into the region of undesirable behavior you want to promote. This is mathematically equivalent to retraining the network to misbehave. Prompt-hacking is analogous to an…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by kilgnad

Fri, 03 Mar 2023 02:36:49 GMT

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by kilgnad

Mon, 27 Feb 2023 19:19:06 GMT

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by l33t233372

Tue, 21 Feb 2023 14:23:26 GMT

I saw this paper’s abstract and was reminded of your comment: https://arxiv.org/abs/2212.10559 > Experimentally, we comprehensively compare the behavior of ICL and explicit finetuning based on real tasks to provide empirical evidence that supports our understanding

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by spion

Tue, 14 Feb 2023 22:33:28 GMT

The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by adamnemecek

Sat, 11 Feb 2023 17:29:31 GMT

I thought you were asking about attention only transformers. This paper touches on some of it https://arxiv.org/abs/2212.10559v2.

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by dhruvdh

Thu, 09 Feb 2023 16:24:26 GMT

1. I don't understand how LLMs work. 2. I don't understand how ChatGPT works, but I have used it a few times. 3. I will use ChatGPT as the absolute measure of what LLMs are capable of. --- 1. I don't understand intelligence. 2. Humans are intelligent, humans can learn to do math. 3. LLMs are not good at math. 4. LLMs are not intelligent, they're just text compression. --- 1. I don't understand how LLMs work. 2. I have a decent grasp of how image compression works. 3. I will use my grasp of image compression to pretend LLMs are text compression. 4. I will apply all limitations of image…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — mentioned by GistNoesis

Wed, 04 Jan 2023 17:25:57 GMT

With transformer-based model, their inner-state is a deterministic function (the features encoded by the Neural Networks weights) applied to the text-generated up-until the current-time step, so it's relatively easy to know what they currently have in mind. For example if the neural network has been generating sad music, its current context which is computed from what it has already generated will light-up the the features that correspond to "sad music". And in turn the fact that the features had been lit-up will make it more likely to generate a minor chord. The dimension of this inner-state…

Paper and discussion history. A mention is not an endorsement.

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers — HN submission

Wed, 21 Dec 2022 18:46:41 GMT

Why Can GPT Learn In-Context? Language Models Perform Gradient Descent

35 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.