[ Context engineering decision guide ]

Context Compression vs. Conversation Summarization

A benchmark-grounded guide to choosing extractive context compilation, generative conversation summaries, or a staged combination without overstating what either preserves.

Published August 8, 2026 · Maha Strategies LLC

BM25 complete evidence

62.8%

250 annotated QASPER questions

Mean token reduction

74.4%

1,508.7 mean output tokens

Position robustness

62.9%

Complete evidence when gold evidence was in the back third

The practical difference

Context compilation selects source passages. The resulting pack can retain source IDs, passage IDs, and hashes because the evidence text is not rewritten. Conversation summarization generates a new representation. It can synthesize decisions and dialogue state, but its sentences are no longer identical to the original evidence.

Use compilation when a downstream answer must remain inspectable against supplied sources. Use summarization for conversational continuity when verbatim evidence recovery is not the primary requirement. In long-running agents, use both in stages: summarize workflow state, then compile the source documents needed for the current task.

BM25 passage selection versus LLM-generated summaries

MCRB-1 compares BM25 selection with keyword selection, front truncation, tail/recency selection, seeded random selection, and a gold-label oracle. At a similar reduction, BM25 retained a complete evidence set in 62.8% of cases versus 25.6% for front truncation and 20.4% for recency.

It deliberately does not assign a score to LLM or LangChain summaries. Exact-span scoring would punish faithful paraphrase, while an LLM judge would make the result depend on the evaluator model. The benchmark therefore supports a comparison of operating boundaries, not a claim that BM25 beats every generative summary.

Inspect all 1,500 case-method records →

Choose by failure mode

Need	Prefer	Reason
Source-linked RAG evidence	Extractive compilation	Selected text remains traceable to source passages.
Long dialogue continuity	Conversation summary	A compact generated state can preserve decisions and intent.
Agent state plus cited research	Staged combination	Summarize state; compile evidence for the current task.

A safe staged pattern

const stateSummary = await summarizeConversation(turns)
const pack = await maha.compress({
  clientRequestId: crypto.randomUUID(),
  task: currentTask + "
Agent state: " + stateSummary,
  tokenBudget: 4000,
  documents: sourceDocuments,
})

// Ground the next model call in pack.context and preserve passage IDs.

Keep generated state separate from source evidence. Do not label a summary sentence as a source quotation, and do not infer that retained evidence guarantees a correct downstream answer.