arXiv:2307.03172cs.CL2023-07-06

Lost in the Middle: How Language Models Use Long Contexts

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

Abstract

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

arXiv comments: 18 pages, 16 figures. Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023
Journal reference
DOI
Subjects: cs.CLComputation and Language

On Hacker News

Linked by 18 identifiable commenters in 20 comments across 20 threads, from Jul 2023 to Aug 2026. Found in 3 HN submissions from Jul 2023 to Jan 2026. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments20 threads

Serve Markdown to AI Agents with Accept Headers Aug 20261 comment
- They accept more tokens these days, but they are still more accurate with a shorter context [1]. [1] https://arxiv.org/abs/2307.03172
  
  selcukaAug 26, 2026View on HN Parent context
Handbook.md shows that long policy documents do not reliably govern agents Jul 20261 comment
- There was an article a few years ago called "Lost in the Middle: How Language Models Use Long Contexts" https://arxiv.org/abs/2307.03172 From my experience this holds true to this day. It was one of my core observations for similarity to the limitations of human working memory…
  
  supermattJul 29, 2026View on HN Parent context
OpenAI reduces Codex Model Context Size from 372k to 272k Jul 20261 comment
- Not sure if this is still up to date (2023), but https://arxiv.org/abs/2307.03172 shows that performance degrades mostly in the middle of the context. Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done…
  
  wonnageJul 20, 2026View on HN Parent context
Show HN: TraceRoot – Open-source agentic debugging for distributed services Aug 20251 comment
- Thanks for the follow-up. Let me try to clarify! When we say we "organize all logs, metrics, and traces", we mean more than just linking them together (which otel already supports). What we’re doing is: - context engineering optimization: We leverage the structure among logs…
  
  xinweiheAug 4, 2025View on HN Parent context
Codestral Mamba Jul 20241 comment
- I was under the impression that with LLMs, in order to get high-quality answers, it's always best to keep context short. Is that not the case anymore? Does Claude under this usage paradigm not struggle with very long contexts in ways as for example described in the "lost in the…
  
  funnygiraffeJul 16, 2024View on HN Parent context
LLMs and the Harry Potter problem Apr 20241 comment
- Some counterarguments: 1. If an AI company promises that their LLM has a million token context window, but in practice it only pays attention to the first and last 30k tokens, and then hallucinates, that is a bad practice. And prompt construction does not help here - the issue…
  
  araghuvanshiApr 23, 2024View on HN Parent context
Build a search engine, not a vector DB Dec 20231 comment
- I could not help but notice the Contriever curve is so much higher on y-axis Recall than the other methods (figure 11 in https://arxiv.org/pdf/2307.03172.pdf). Has anyone come across more recent experiments, results, or papers related to this? I'm acquainted with the…
  
  bayesian_limitDec 28, 2023View on HN Parent context
Benchmarking GPT-4 Turbo – A Cautionary Tale Nov 20231 comment
- Yeah, degraded performance on long contexts has been observed in plenty of other models [https://arxiv.org/abs/2307.03172] so I was cautious too. Unfortunately I don't have access to 4-32k. I would have liked to test that out too.
  
  msp26Nov 9, 2023View on HN Parent context
New models and developer products Nov 20231 comment
- > 128k context? Absolutely insane 128k context is great and all, but how effective are the middle 100,000 tokens? LLMs are known to struggle with remembering stuff that isn't at the start or end of the input. Known as the Lost Middle https://arxiv.org/abs/2307.03172
  
  SwizecNov 6, 2023View on HN Parent context
GPT-4 Update: 32K Context Window Now for All Users Nov 20231 comment
- Even full quadratic attention models seem to forget or not value information given in the middle of the prompt. Anything using any kind of context length widening tricks which cripple the attention in some way (which is usually how this is done) will make that problem worse.…
  
  Der_EinzigeNov 3, 2023View on HN Parent context
Microsoft says GPT 3.5 has 20B parameters? Oct 20231 comment
- Performance on long contexts just isn't very good right now. Relevant paper: Lost in the Middle: How Language Models Use Long Contexts https://arxiv.org/abs/2307.03172
  
  msp26Oct 30, 2023View on HN Parent context
MemGPT – LLMs with self-editing memory for unbounded context Oct 20231 comment
- For anyone who's curious, the paper in question, entitled, "Lost in the Middle: How Language Models Use Long Contexts" (https://arxiv.org/abs/2307.03172)
  
  sandkoanOct 17, 2023View on HN Parent context
A Comprehensive Guide for Building Rag-Based LLM Applications Sep 20231 comment
- Kudos to the team for a very detailed notebook going into things like pipeline evaluation wrt performance and costs etc. Even if we ignore the framework specific bits, it is a great guide to follow when building RAG systems in production. We have been building RAG systems in…
  
  ajhaiSep 14, 2023View on HN Parent context
ChatGPT Enterprise Aug 20231 comment
- does expanding the context window of LLMs still make them dumber? https://arxiv.org/pdf/2307.03172.pdf
  
  BeefinAug 29, 2023View on HN Parent context
You probably don’t need to fine-tune an LLM Aug 20231 comment
- Interesting, thanks for sharing. We also found[1] that context stuffing leads (generally) to worse quality results, as did a recent study.[2] But that's for question-answering use cases and not summarization. [1] https://www.pinecone.io/blog/why-use-retrieval-instead-of-la...…
  
  gk1Aug 18, 2023View on HN Parent context
What's new in Llama 2 and how to run it locally Aug 20231 comment
- I haven't tested the newest implementations of every large context window model, so I'm not sure how prevalent this issue still is, but generally speaking the context window tends to be U-shaped. In other words, the model seems to forget/ignore everything in the middle. So YMMV…
  
  spmurrayzzzAug 7, 2023View on HN Parent context
Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase Aug 20231 comment
- That's a good question! We tried using Anthropic 100k before (Claude 1.3 was a lot worse), and I think that it's really important to figure out how to be context efficient, at least for GPT4. My stance is with models ignoring long contexts(https://arxiv.org/pdf/2307.03172.pdf)…
  
  williamzeng0Aug 3, 2023View on HN Parent context
Llama 32K Context Released by Together AI Jul 20231 comment
- You are correct. The paper is called "Lost in the middle" [1] and it is probably one of the worst drawbacks of this technology. It makes a lot of use cases biased (think of law). [0] https://arxiv.org/pdf/2307.03172.pdf
  
  saliagatoJul 29, 2023View on HN Parent context
Llama 2 Jul 20231 comment
- Any trick which is not doing full quadratic attention cripples a models ability to reason "in the middle" more than they already are crippled. Good long context length models are currently a mirage. This is why no one is seriously using GPT-4-32k or Claude-100k in production…
  
  Der_EinzigeJul 18, 2023View on HN Parent context
Explainer Video – Lost in the Middle: How Language Models Use Long Context Jul 20231 comment
- Based on this research paper: https://arxiv.org/pdf/2307.03172.pdf
  
  bobvanluijtJul 17, 2023View on HN Parent context

Submitted to Hacker News

How language models use long contexts15 points0 commentsAug 2023
Lost in the Middle: How Language Models Use Long Contexts (2023)2 points0 commentsJan 2026
Lost in the Middle: How Language Models Use Long Contexts2 points0 commentsJul 2023

Connected through shared HN threads, not inferred similarity or a recommendation.

Abstract

On Hacker News

Discussed in comments20 threads

Submitted to Hacker News

Also discussed alongside