Context reranking

1 min read
On trails:AGCTXstep 7/11AGRAGstep 8/8

LLMs have a lost in the middle problem: attention is strong at the start and the end of a long context and drops in the middle. Reranking here means ordering what goes in, so important data and instructions sit at the beginning or the end.

A U-shaped curve of how well a model uses information by its position in a long context: strong at the start, weak in the middle, strong again at the end. RECALL start middle end weakest here key facts key facts
The shape reported in the paper, drawn as a sketch, not measured data.

The paper behind the name found that performance is often highest when the relevant information is at the beginning or the end of the input, and degrades significantly when it is in the middle. That held even for models built for long contexts.

In practice

A retriever hands back chunks ranked best first. Pasting them in that order wastes the strong tail. This function deals them to both edges: best at the start, second best at the end, and the weakest end up in the middle.

Python
def reorder(docs_by_relevance):    """Best documents at the two edges, weakest in the middle."""    front, back = [], []    for i, doc in enumerate(docs_by_relevance):        (front if i % 2 == 0 else back).append(doc)    return front + back[::-1] # reorder([d1, d2, d3, d4, d5, d6]) -> [d1, d3, d5, d6, d4, d2]

Line 5 alternates the chunks between two piles, and line 6 reverses the second pile so the pair of best chunks bracket the rest. Re-scoring the chunks with a ranking model is a separate step before this one (see the query side of RAGRAG at query timeA one-way chain: embed the query, search, stuff the chunks into the prompt, generate.Opens in a new tab).

The catch

Claude’s docs make the same point for prompts: put long documents near the top and the question at the end, which in their tests improved quality by up to 30 percent on complex multi-document inputs. Order helps, but it does not rescue a bloated context. Fewer, better chunks beat a clever order (see scopingContext scopingShow the model only what it needs for this step, tools included.Opens in a new tab).

Sources