Give a model a hundred-page document and ask about something on page fifty, and it may miss the answer — not because the fact isn't in the context, but because of where it sits.
TL;DR
- LLMs use the start and end of a long context far better than the middle.
- The effect is U-shaped and shows up even in models explicitly built for long context.
- Fitting information into the window is not the same as the model using it.
The promise, and the catch
Bigger context windows sell a tempting idea: just paste everything in and let the model sort it out. No retrieval, no structure — one giant prompt. But there is a gap between having information in the window and the model actually using it, and that gap grows with length.
What the research shows
In Lost in the Middle: How Language Models Use Long Contexts, Liu and colleagues measured how well models find and use information depending on where it sits in the input. They built two deliberately controlled tests:
- Multi-document QA. The model gets a question plus a stack of documents — 10, 20, or 30 of them — where exactly one holds the answer. That gold document is slid to every position in the stack, and accuracy is measured at each spot.
- Key–value retrieval. A synthetic test stripped of linguistic clutter: a long JSON object of hundreds of random key–value pairs, where the model must return the value for one specified key — isolating pure position-based lookup from comprehension.
Across both tasks, and across open and closed models alike (GPT-3.5-Turbo, Claude, MPT-30B-Instruct, LongChat-13B), the same shape appeared: accuracy is highest when the relevant item is near the beginning or end of the context and sags when the model has to reach into the middle — a U-shaped curve.
Performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access it in the middle.
Three details make it worse than it first sounds. It holds even for models built for long context — a larger window does not mean the model reads it evenly. In the worst (middle) case, multi-document accuracy can fall below the closed-book baseline — the same model answering with no documents at all, from its parameters alone. And handing a model the extended-context version of itself gave no advantage when the input already fit the smaller window: the extra capacity bought nothing.
One mitigation moved the needle: query-aware contextualization — repeating the question both before and after the documents. It nearly fixed the synthetic key–value task, but barely helped real multi-document QA, so it is no general cure. Position, not just presence, is what the model is sensitive to.
What it means for you
If you rely on pasting a long history, a big document, or an entire project into a chat, the model's answer quality depends partly on luck — where the relevant passage happened to land. As the context grows, the "middle" that gets under-read grows with it. Two practical implications:
- Surface, don't bury. The relevant material should reach the model as a short, well-placed passage — not be hidden in the middle of a wall of text.
- Structure beats volume. Organizing and retrieving the right pieces consistently outperforms hoping a bigger window reads them all.
Where FocusLM fits
This is exactly why FocusLM keeps your material as a structured memoryrather than a growing chat log. Instead of stuffing everything into one context and hoping the middle gets read, it files what you feed it and surfaces the relevant, cited pieces when you ask — so the answer is grounded in the right material, wherever it originally came from.