Context Rot: Why More Context Can Mean Worse Answers

Thirty minutes into a Claude Code session, you tell it: use snake_case for every new function, not camelCase like the rest of this legacy file. It agrees, follows the rule for the next ten minutes, and then, an hour later, writes a brand-new function in camelCase again. The instruction is still sitting right there in the transcript, nothing got deleted. The agent just stopped weighting it the way it did when you first said it.

Context rot is the documented drop in an LLM's accuracy as the amount of text in its context window grows, even when the exact fact it needs is still present and hasn't gone anywhere. It's not a bug where information disappears; it's a measured pattern where a model gets steadily less reliable at using everything in front of it, and facts buried in the middle of a long input fare worst of all. That's a different failure than the one from the last lesson: statelessness is about a session having no memory at all once it ends. Context rot happens inside a single, still-active session, while the memory is technically still there.

What Is Context Rot?

Context rot is what happens when a model's accuracy declines as its input gets longer, independent of whether the relevant information was removed. You'll sometimes see the same phrase used for a completely different problem: a CLAUDE.md or AGENTS.md file that's gone stale as a codebase evolves. That's a real issue, but it's a documentation problem, not this one. What this lesson covers is a property of how the model itself processes a long input, one that shows up whether your files are current or not.

The confusion is worth naming upfront, because it changes what you'd even try to fix. A stale rules file gets fixed by editing the file. Context rot gets fixed by managing what you put in the window in the first place, which is why it sets up the next several lessons in this module.

The Research: "Lost in the Middle"

A model uses information near the start and end of its input far more reliably than information buried in the middle, and that gap holds even for models explicitly built to handle long inputs. In 2023, researchers Nelson Liu, Kevin Lin, and colleagues at Stanford and Samaya AI ran a deceptively simple test: give a model a question, along with a document containing the answer buried among distractor documents, and move the answer's position around. Same information, same total input length, only the position changed.

The result is now called "lost in the middle", and it produces a U-shaped accuracy curve. Performance is highest when the answer sits at the very start or very end of the input, and drops noticeably, roughly 15 to 20 percentage points in some of their tests, when the same answer sits in the middle. No model the researchers tested was immune.

The finding mattered because it broke a common assumption: that a model treats every token in its context window with roughly equal attention. It doesn't. Position matters as much as presence.

The 2025 Confirmation: Chroma's 18-Model Report

Two years after the original finding, newer models still show the same degradation, and it starts earlier and runs deeper than most people assume. In July 2025, the retrieval company Chroma published a technical report testing 18 frontier models, including Claude, GPT-4.1, Gemini 2.5, and Qwen3, across tasks designed to isolate exactly this effect.

The Chroma report found accuracy degrading non-uniformly as input length grew, in some tasks by 30 to 50 percent relative to short-context performance, well before any model reached its advertised context limit. Adding even a single distractor document alongside the real answer measurably hurt retrieval, and stacking four distractors compounded the loss further. That part tracks with the 2023 findings.

The genuinely surprising part didn't: when researchers shuffled the surrounding text so it lost its logical flow, retrieval accuracy improved. Coherent, well-organized writing degraded the model's attention more than scrambled nonsense did, suggesting the model spends some of its attention tracking the narrative itself, competing with the retrieval task instead of only supporting it.

Why This Happens

Context rot traces back to how a transformer's attention mechanism spreads its focus across every token in the input, and that focus isn't distributed evenly by default. Modern models are trained with patterns and positional encodings that make the start and end of a sequence easier to attend to strongly, a side effect of how next-token prediction gets trained on typical text, where the beginning sets up what matters and the end resolves it. The middle of a long document often just holds supporting detail in ordinary writing, so the model has less training signal telling it to weight the middle heavily.

None of this requires you to understand transformer internals to act on it, and this lesson isn't the place to go deeper into the math (if you want the context window fundamentals this builds on, that's covered there). The practical upshot is the one that matters: the model isn't broken, it's behaving exactly as its architecture and training predict, which means the fix has to happen in how you manage its input, not by hoping a future model version quietly solves it.

What This Means for Claude Code and Cursor

If a fact or instruction matters, its position in your context is a lever you control, and re-stating it near the end of a long session is often cheaper than debugging why the agent "forgot" it. I've watched a naming convention I stated clearly at the start of a Claude Code session get contradicted an hour later, not because the model lost the instruction, but because ten thousand tokens of file reads and tool output had since buried it in exactly the middle-of-context spot the research predicts.

This is also why "just paste more context in," the workaround the last lesson flagged as expensive, is worse than expensive. It's actively counterproductive past a certain point, because more tokens means more middle for your important facts to get lost in. The next lesson in this module picks up exactly here: it gives you a concrete threshold, roughly 70% of the context window, for when to compact a session before this degradation sets in, plus the real commands (Cursor's /summarize, Claude Code's /compact) to do it.

Your Lab