What memory costs in tokens

a few hundred tokens, or fifty thousand — the difference is retrieval

Every context-window trick is a trade: more of what you need in the window, less room for everything else. So the question about giving an agent its history is not whether it helps — it is what it costs each turn, and whether the cost is bounded.

Retrieval against replay, measured

Thirty seeded task chains, each with a fact established earlier and needed later, plus five negative controls where nothing earlier is relevant. Four ways to reach the same working context:

Same fact coverage for about 200× fewer tokens than grepping and about 60× fewer than replaying. The last column is the one people skip: on chains where no earlier fact is relevant, replay still spends about fifteen thousand tokens saying nothing useful, and recall spends zero.

Run it yourself — the corpus generator and the relevance labels are ordinary Go in the repository, and the arms are seeded:

deja bench context    # 30 seeded chains plus five negative controls
deja bench recall     # ranking regression floor, CI fails if recall drops

Audit what "relevant" means before trusting any number here, ours included. A benchmark that defines its own success is a benchmark that always passes.

The caps that keep it bounded

An average is not a guarantee, so the ceilings matter more than the median:

• Per-prompt recall: 1,536 bytes. That is the block a hook adds when what you just typed matches earlier work.
• An MCP recall: about 4 KB per call, dense text rather than whole sessions.
• Nothing at all, which is the common case at any injection point. The block appears when the history answers and is absent otherwise.

That last one is a design decision rather than an optimisation. A memory that always says something teaches you to skip whatever it says, and then you are paying for a block nobody reads.

The other cost, which is not tokens

Latency. A lookup is a lexical query against a local index — a median of about 0.4 ms in process, around 25 ms on the LongMemEval-S haystacks. A hook pays process start and a freshness check across the stores on top, tens of milliseconds on a multi-gigabyte corpus. Nothing in that path waits on a model, which is the reason the numbers are in milliseconds rather than seconds.

Why "just summarise it" is a different trade

Compaction is the built-in answer to a full window, and it keeps the wrong half: over 43 real compactions it retained 77% of the decisions and 0.2% of the commands that had been run. It is cheap and lossy in exactly the places that cost you time later — see what compaction drops. Retrieval is the opposite trade: nothing is thrown away, and you pay a few hundred tokens for the part you need at the moment you need it.

What this does not claim

Recall does not reduce your token bill on its own — it replaces a large, vague context with a small, specific one, and whether that lowers spend depends on what you were doing instead. If the honest alternative was pasting nothing and re-solving the problem, the comparison is with your afternoon rather than with your invoice.

Benchmarks in full · What compaction drops · How ranking works