What did the study actually test?
Chroma Research — Kelly Hong, Anton Troynikov, and Jeff Huber — ran controlled experiments across GPT-4.1, the Claude 4 family, Gemini 2.5, Qwen3, and other current frontier models, varying input length and measuring accuracy on retrieval and reasoning tasks. The headline finding: reliability decreased significantly with longer inputs, and the decline wasn't smooth or uniform — it depended on how similar the target information was to surrounding "distractor" text, how the input was structured, and what kind of task was being asked.
Notably, this shows up even on tasks that sound trivial, like retrieving a fact or repeating back a piece of text — the kind of task where you'd expect a model to be reliable regardless of how much surrounding context there is.
How much does accuracy actually drop, and at what length?
The degradation isn't a cliff at the advertised limit — it starts much earlier. One analysis summarising the Chroma findings and related work describes a 200,000-token window showing serious accuracy loss by around 50,000 tokens of input, well under a quarter of its rated capacity.
Separate work applying similar methodology found accuracy drops of 30–50% appearing well before models reached their documented context limits, with the steepest decline typically in the higher end of the tested range.
Why does coherent, well-organised text sometimes perform worse?
This is the part of the research that surprises people. Across all 18 models tested, performance was better on shuffled, less coherent input than on logically structured documents of the same length. The likely explanation: a well-organised, plausible-sounding passage is a more convincing distractor to a model than a jumbled one — it reads as more relevant even when it isn't, which pulls attention away from the actual answer.
That finding matters practically: it means "just clean up your prompt formatting" isn't a reliable fix for context rot on its own.
What does this mean for how AI products should be built?
A few conclusions follow reasonably from the current evidence, without overstating it.
- > Bigger context windows solve a capacity problem, not a reliability problem. Being able to fit a million tokens and being able to reason well across a million tokens are separate claims, and the research is clear that current models don't yet do the second one reliably.
- > Retrieval-based approaches — pulling in only what's relevant — remain relevant even as windows get larger. The "just dump everything in and let the model sort it out" strategy is exactly the pattern this research shows breaking down.
- > This compounds with the lost-in-the-middle effect. Context rot is the broader decline in reliability as input grows; lost-in-the-middle is one specific, well-documented driver of where that decline shows up.
Our companion piece covers why AI forgets the middle of long conversations in detail, and how AI memory actually works covers the retrieval architecture that exists to avoid both problems.
Key takeaways
- > Context rot is the drop in output quality as input grows, well before the advertised context limit.
- > All 18 frontier models Chroma tested in 2025 showed the degradation.
- > It appears even on trivial tasks like retrieving a fact or repeating text back.
- > Coherent, well-structured input performed worse than shuffled input of the same length.
- > Tidier formatting is not a fix; retrieving less is.
Sources
- > Hong, K., Troynikov, A., & Huber, J. (2025). Context Rot: How Increasing Input Tokens Impacts LLM Performance. Chroma Research. trychroma.com/research/context-rot
- > Liu, N. F., et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. arxiv.org/abs/2307.03172