Why a massive 128K context window won't save your RAG pipeline from U-shaped attention decay, and how head-tail reordering rescues your most relevant data from the dead zone.
∙ Paid
You’re in a Senior ML Engineer interview at Anthropic and the interviewer asks:
“Your retriever pulls top-10 chunks, reranks them, and you feed all 10 to the LLM to be safe. Walk me through exactly what breaks, and when you’d retrieve fewer documents on purpose.”
Don’t say: “More context gives the model more to work with, so top-10 is safer than top-3.”
He…
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.