Lost in the Middle: How Language Models Use Long Contexts · HackerPapers discussions

Lost in the Middle: How Language Models Use Long Contexts — mentioned by selcuka

Wed, 26 Aug 2026 23:34:13 GMT

They accept more tokens these days, but they are still more accurate with a shorter context [1]. [1] https://arxiv.org/abs/2307.03172

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by supermatt

Wed, 29 Jul 2026 13:56:27 GMT

There was an article a few years ago called "Lost in the Middle: How Language Models Use Long Contexts" https://arxiv.org/abs/2307.03172 From my experience this holds true to this day. It was one of my core observations for similarity to the limitations of human working memory on "Engineering for Bounded Cognition"

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by wonnage

Mon, 20 Jul 2026 00:18:56 GMT

Not sure if this is still up to date (2023), but https://arxiv.org/abs/2307.03172 shows that performance degrades mostly in the middle of the context. Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done, and I appreciate not having to wait through a compaction. If anything, keeping the bloated context helps with accuracy at the expense of speed in these cases.

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — HN submission

Fri, 30 Jan 2026 19:44:56 GMT

Lost in the Middle: How Language Models Use Long Contexts (2023)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by xinweihe

Mon, 04 Aug 2025 19:36:15 GMT

Thanks for the follow-up. Let me try to clarify! When we say we "organize all logs, metrics, and traces", we mean more than just linking them together (which otel already supports). What we’re doing is: - context engineering optimization: We leverage the structure among logs, spans, and metadata to filter and group relevant context before passing it to the LLM. In real production issues, it's common to see 10k+ logs, traces, etc. related to a single incident — but most of it is noise. Throwing all that at agents usually leads to poor performance due to context bloat see…

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by funnygiraffe

Tue, 16 Jul 2024 18:36:50 GMT

I was under the impression that with LLMs, in order to get high-quality answers, it's always best to keep context short. Is that not the case anymore? Does Claude under this usage paradigm not struggle with very long contexts in ways as for example described in the "lost in the middle" paper (https://arxiv.org/abs/2307.03172)?

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by araghuvanshi

Tue, 23 Apr 2024 19:24:56 GMT

Some counterarguments: 1. If an AI company promises that their LLM has a million token context window, but in practice it only pays attention to the first and last 30k tokens, and then hallucinates, that is a bad practice. And prompt construction does not help here - the issue is with the fundamentals of how LLMs actually work. Proof: https://arxiv.org/abs/2307.03172 2. Regarding writing the code snippet: as I described in my post, the main issue is that the model does not understand the relationships between information in the long document. So yes, it can write a script that counts the…

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by bayesian_limit

Thu, 28 Dec 2023 18:04:46 GMT

I could not help but notice the Contriever curve is so much higher on y-axis Recall than the other methods (figure 11 in https://arxiv.org/pdf/2307.03172.pdf). Has anyone come across more recent experiments, results, or papers related to this? I'm acquainted with the: - Contriever 2021 paper https://aclanthology.org/2021.eacl-main.74.pdf - Hyde 2022 https://arxiv.org/pdf/2212.10496.pdf My suspicion is some pre-logic such as is the user's question dense enough then use Hyde with chat history. If anyone has more recent experience with Contrievers, would love to learn more about it! Feel free to…

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by msp26

Thu, 09 Nov 2023 14:34:36 GMT

Yeah, degraded performance on long contexts has been observed in plenty of other models [https://arxiv.org/abs/2307.03172] so I was cautious too. Unfortunately I don't have access to 4-32k. I would have liked to test that out too.

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by Swizec

Mon, 06 Nov 2023 18:59:53 GMT

> 128k context? Absolutely insane 128k context is great and all, but how effective are the middle 100,000 tokens? LLMs are known to struggle with remembering stuff that isn't at the start or end of the input. Known as the Lost Middle https://arxiv.org/abs/2307.03172

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by Der_Einzige

Fri, 03 Nov 2023 17:35:45 GMT

Even full quadratic attention models seem to forget or not value information given in the middle of the prompt. Anything using any kind of context length widening tricks which cripple the attention in some way (which is usually how this is done) will make that problem worse. - https://arxiv.org/abs/2307.03172 You can see this when you use Anthropic Claude which has a 100K context length today.

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by msp26

Mon, 30 Oct 2023 14:48:43 GMT

Performance on long contexts just isn't very good right now. Relevant paper: Lost in the Middle: How Language Models Use Long Contexts https://arxiv.org/abs/2307.03172

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by sandkoan

Tue, 17 Oct 2023 00:40:58 GMT

For anyone who's curious, the paper in question, entitled, "Lost in the Middle: How Language Models Use Long Contexts" (https://arxiv.org/abs/2307.03172)

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by ajhai

Thu, 14 Sep 2023 17:22:05 GMT

Kudos to the team for a very detailed notebook going into things like pipeline evaluation wrt performance and costs etc. Even if we ignore the framework specific bits, it is a great guide to follow when building RAG systems in production. We have been building RAG systems in production for a few months and have been tinkering with different strategies to get the most performance out of these pipelines. As others have pointed out, vector database may not be the right strategy for every problem. Similarly there are things like lost in the middle problems (https://arxiv.org/abs/2307.03172) that…

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by Beefin

Tue, 29 Aug 2023 02:18:41 GMT

does expanding the context window of LLMs still make them dumber? https://arxiv.org/pdf/2307.03172.pdf

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by gk1

Fri, 18 Aug 2023 16:28:23 GMT

Interesting, thanks for sharing. We also found[1] that context stuffing leads (generally) to worse quality results, as did a recent study.[2] But that's for question-answering use cases and not summarization. [1] https://www.pinecone.io/blog/why-use-retrieval-instead-of-la... [2] https://arxiv.org/pdf/2307.03172.pdf

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by spmurrayzzz

Mon, 07 Aug 2023 13:18:12 GMT

I haven't tested the newest implementations of every large context window model, so I'm not sure how prevalent this issue still is, but generally speaking the context window tends to be U-shaped. In other words, the model seems to forget/ignore everything in the middle. So YMMV if you're trying to implement RAG-esque methods with them. More reading on that problem if you're curious: https://arxiv.org/pdf/2307.03172.pdf

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by williamzeng0

Thu, 03 Aug 2023 20:54:24 GMT

That's a good question! We tried using Anthropic 100k before (Claude 1.3 was a lot worse), and I think that it's really important to figure out how to be context efficient, at least for GPT4. My stance is with models ignoring long contexts(https://arxiv.org/pdf/2307.03172.pdf), we'll have this problem for a long time. I could be wrong though. Also we did try function calling, but it doesn't allow for a chain of thought step. This made the plan/code way worse. Cool to see you found the same!

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — HN submission

Wed, 02 Aug 2023 17:22:31 GMT

How language models use long contexts

15 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by saliagato

Sat, 29 Jul 2023 09:33:50 GMT

You are correct. The paper is called "Lost in the middle" [1] and it is probably one of the worst drawbacks of this technology. It makes a lot of use cases biased (think of law). [0] https://arxiv.org/pdf/2307.03172.pdf

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by Der_Einzige

Tue, 18 Jul 2023 16:59:44 GMT

Any trick which is not doing full quadratic attention cripples a models ability to reason "in the middle" more than they already are crippled. Good long context length models are currently a mirage. This is why no one is seriously using GPT-4-32k or Claude-100k in production right now. Edit: even if it's doing full attention like the commentator says, turns out that's not good enough! https://arxiv.org/abs/2307.03172

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — mentioned by bobvanluijt

Mon, 17 Jul 2023 16:31:27 GMT

Based on this research paper: https://arxiv.org/pdf/2307.03172.pdf

Paper and discussion history. A mention is not an endorsement.

Lost in the Middle: How Language Models Use Long Contexts — HN submission

Fri, 07 Jul 2023 14:30:09 GMT

Lost in the Middle: How Language Models Use Long Contexts

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.