Where does this finding come from?
The effect was first documented rigorously by Stanford researcher Nelson Liu and colleagues in a study that tested how well language models could locate an answer buried among distractor documents, and separately how well they could retrieve a specific value from a long list of key-value pairs. Across both tasks they found the same pattern: models did best when the relevant information sat right at the start or end of the input, and noticeably worse when it sat in the middle — a result that held even for explicitly long-context models.
The paper was later published in the Transactions of the Association for Computational Linguistics and has become one of the most frequently cited findings in long-context AI research, with follow-up studies replicating the U-shaped accuracy curve on newer model families and different task types.
Is this just an old-model problem that's already been fixed?
Partly, but not entirely. Some newer models handle simple factual retrieval well across the full length of their context — one study on a recent Gemini model found no meaningful lost-in-the-middle effect on straightforward needle-in-a-haystack question answering. But researchers testing more complex, multi-step reasoning tasks — not just "find this one fact" but "reason across several facts scattered through the input" — continue to find the same positional bias showing up in current frontier models.
The honest summary: simple retrieval has gotten better; the underlying attention bias hasn't disappeared, and it reappears as soon as a task gets harder than pure lookup.
Why does this happen at all?
It comes down to how the underlying transformer architecture processes text. Every token can theoretically attend to every other token, but in practice models develop strong primacy and recency biases — over-weighting the first tokens they see and the most recent ones, while the mass of content in between gets comparatively less attention.
As the amount of surrounding, semantically-similar-but-irrelevant text grows, a single relevant fact becomes harder for the model to distinguish from the noise around it. That is why the effect gets worse as inputs get longer, rather than staying a fixed penalty.
What does this mean if you're building with AI, not just researching it?
A few practical takeaways follow directly from the research, without needing to overstate what's proven.
- > Order matters more than most people assume. If there's a fact your AI absolutely needs to use correctly, don't bury it in the middle of a long document or a sprawling conversation history — put it near the top or right before you ask the question.
- > A bigger context window is not the same as better context use. Marketing copy about million-token windows describes capacity, not reliability. Treat "how much I can fit" and "how well the model will actually use it" as two separate questions.
- > This is exactly the problem persistent memory systems are trying to solve. Rather than stuffing an ever-growing transcript into context and hoping the model finds what it needs, memory-layer approaches extract and resurface the specific facts that matter, so the model isn't relying on positional luck.
That is a different design pattern from "just have a longer context window" — our companion piece on context rot covers why the two aren't the same fix, and how AI memory actually works covers what the alternative architecture looks like.
Key takeaways
- > Language models attend most strongly to the start and end of their input, and least to the middle.
- > The effect is a U-shaped accuracy curve, replicated across model families and task types.
- > Newer models handle simple lookup across long contexts better; multi-step reasoning still shows the bias.
- > A larger context window is a capacity claim, not a reliability claim.
- > Put the fact a model must not miss at the top of the input, or immediately before the question.
Sources
- > Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. arxiv.org/abs/2307.03172
- > McKinnon, M. (2025). Retrieval Quality at Context Limit. arxiv.org/pdf/2511.05850