A quick note to test this page — and to write down a loop I keep running into.

When we talk about the relationship between large language models and retrieval, the underlying question is almost always the same: how do we extend the effective context of a model? There are two routes.

Route one: external. Keep the window modest and bring knowledge in on demand — retrieval, RAG, agentic search. The context window acts as working memory, and the retriever decides what deserves to be in it. The model doesn’t need to hold everything; it needs to ask well.

Route two: internal. Grow the model’s native ability — longer windows, better positional handling, attention that actually uses what it is given. In the limit, you just paste the corpus in and skip retrieval entirely.

The two routes look like substitutes, but neither one escapes the other’s problems:

  • Long-window models still have to solve an attention problem, not just a capacity problem. Even with short inputs, the model needs reliable long-range attention — finding the one relevant span and not getting lost in the middle. A million-token window you can’t attend over is storage, not memory.
  • Retrieval doesn’t remove that burden either; it changes it. Retrieved passages arrive noisy, partially relevant, sometimes contradictory. The model needs robustness to distraction — the ability to ignore plausible-looking but irrelevant context — which is, again, an attention problem.

So it comes back as a loop: better retrieval demands better long-context behavior to digest what was retrieved; better long-context models raise the bar for what retrieval must contribute to be worth its latency. Progress on one side quietly redefines the evaluation target of the other.

That last word — evaluation — is where my own work sits. In our RAG evaluation surveys, the persistent difficulty was exactly this entanglement: you cannot score a retriever without assumptions about the reader, and you cannot score a long-context reader without deciding what it should have been given. And in Search-on-Graph, the loop shows up in miniature: instead of one big retrieve-then-read step, the model interleaves small retrievals with reasoning, so context is built, not fetched.

My current read: the interesting question is no longer “retrieval or long context,” but how to evaluate — and train for — the loop itself: models that know when to look things up, and retrieval that knows what the model can actually use.