The Attention Mechanism: Why Your LLM Forgets the Middle

LLM context rot isn't a bug. It's a mathematical property of how the attention mechanism works. Three interacting failure modes (softmax's sum-to-one constraint, RoPE's distance decay, and causal masking's primacy bias) combine to produce the U-shaped attention curve where models attend well to the beginning and end of context but lose the middle. Understanding the mechanism changes how you…