In-Context Learning Creates Task Vectors · HackerPapers discussions

In-Context Learning Creates Task Vectors — mentioned by potatoman22

Sun, 14 Dec 2025 07:46:36 GMT

It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled (temp > 0) and input is varied. Using the right prompt makes the model perform better on average, so it's not completely dumb. I like task vectors and soft prompts because I think they show how prompt engineering is cool and useful. https://arxiv.org/pdf/2310.15916 https://huggingface.co/docs/peft/conceptual_guides/prompting

Paper and discussion history. A mention is not an endorsement.

In-Context Learning Creates Task Vectors — mentioned by peter_d_sherman

Fri, 08 Nov 2024 00:29:23 GMT

>"In the in-context learning (ICL) paradigm, given a set of examples, the model has to learn the mapping from inputs to outputs. Prior research has demonstrated that LLMs implicitly compress this mapping into a latent activation, called the task vector" Related: In-Context Learning Creates Task Vectors: https://arxiv.org/abs/2310.15916 Function Vectors in Large Language Models: https://functions.baulab.info/ https://x.com/graceluo_/status/1852043048043319360

Paper and discussion history. A mention is not an endorsement.

In-Context Learning Creates Task Vectors — HN submission

Thu, 26 Oct 2023 16:22:39 GMT

In-Context Learning Creates Task Vectors

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

In-Context Learning Creates Task Vectors — mentioned by famouswaffles

Thu, 26 Oct 2023 01:05:20 GMT

This is speculation but i don't think this information (how it came to be) is actually irrelevant information for the LLM. Your state, which transformer LLMs notably have no way of transferring is "just" the computation that preceded the prediction. I think LLMs use previous tokens to compute the likely state vector that preceded the tokens and make predictions that continue this computed state. So the 2 board positions at time x may be identical but the state preceding it or "strategy" was not and so the next move is not identical. This way also, you can make a decision based on the…

Paper and discussion history. A mention is not an endorsement.