What memory costs in tokens

a few hundred tokens, or fifty thousand — the difference is retrieval

Every context-window trick is a trade: more of what you need in the window, less room for everything else. So the question about giving an agent its history is not whether it helps — it is what it costs each turn, and whether the cost is bounded.

Retrieval against replay, measured

Thirty seeded task chains, each with a fact established earlier and needed later, plus five negative controls where nothing earlier is relevant. Four ways to reach the same working context:

Same fact coverage for about 250× fewer tokens than grepping and about 70× fewer than replaying. The last column is the one people skip: on chains where no earlier fact is relevant, replay still spends nearly eighty thousand tokens saying nothing useful, and recall spends zero.

Run it yourself — the corpus generator and the relevance labels are ordinary Go in the repository, and the arms are seeded:

deja bench context    # 30 seeded chains plus five negative controls
deja bench recall     # 100 queries, half Russian; CI fails if recall drops

Audit what "relevant" means before trusting any number here, ours included. A benchmark that defines its own success is a benchmark that always passes.

The caps that keep it bounded

An average is not a guarantee, so the ceilings matter more than the median:

• Per-prompt recall: 1,536 bytes. That is the block a hook adds when what you just typed matches earlier work.
• An MCP recall: about 4 KB per call, dense text rather than whole sessions.
• The packet after a compaction: 4 KB, once per compaction, and only on the harnesses that hand deja the transcript before they shorten it.
• Nothing at all, which is the common case at any injection point. The block appears when the history answers and is absent otherwise.

That last one is a design decision rather than an optimisation. A memory that always says something teaches you to skip whatever it says, and then you are paying for a block nobody reads.

The other cost, which is not tokens

Latency. A lookup is a lexical query against a local index — a median of 0.7 to 0.8 ms in process, around 15 ms on the LongMemEval-S haystacks. A hook pays process start and a freshness check across the stores on top, tens of milliseconds on a multi-gigabyte corpus. Nothing in that path waits on a model, which is the reason the numbers are in milliseconds rather than seconds.

Why "just summarise it" is a different trade

Compaction is the built-in answer to a full window, and it keeps the wrong half: over 43 real compactions it retained 77% of the decisions and 0.2% of the commands that had been run. It is cheap and lossy in exactly the places that cost you time later — see what compaction drops. On Claude Code, Codex and the other harnesses that hand over the transcript before compacting, deja reads it as the compaction starts and hands the dropped half back once, in 4 KB. Retrieval is the opposite trade: nothing is thrown away, and you pay a few hundred tokens for the part you need at the moment you need it.

What this does not claim

Recall does not reduce your token bill on its own — it replaces a large, vague context with a small, specific one, and whether that lowers spend depends on what you were doing instead. The one task we measured end to end, in a real repository with its history seeded, cost 52,815 tokens with deja against 103,443 without, medians of eleven alternated runs an arm — about half, as in the two runs before it. That is one task, not a rate for your work. If the honest alternative was pasting nothing and re-solving the problem, the comparison is with your afternoon rather than with your invoice.

Benchmarks in full · What compaction drops · How ranking works

Found this useful? Star deja-vu on GitHub.