Day zero
what each memory tool knows the minute you install it
Every memory tool is measured warm: a store it has been filling for weeks, questions about what it captured. The minute after install is different. The machine already holds months of Claude Code, Codex and Cursor sessions, and the only question that matters that minute is whether a tool can answer from them at all, how long it takes to get there, and what it had to install to do it.
This page runs that minute for five tools on one machine and one corpus. deja is the subject and I maintain it; the corpus, the drivers and the scoring rule are in the repository so the row you doubt can be re-run.
The setup
19,195 sessions from the LongMemEval-S cleaned set, written to disk in the real layouts — ~/.claude/projects/<project>/<id>.jsonl and ~/.codex/sessions/<date>/rollout-<id>.jsonl — before any tool is installed. 100 questions whose answer lives in exactly one of those sessions. A tool scores at rank k when the session holding the answer is the k-th result; hit@1 and hit@5 count ranks 0 and under 5, found@50 counts any rank in the first fifty. The control run points deja at an empty directory and scores 0/100, which is what a memory that only records forward sees on day zero.
Each tool runs with its defaults and its documented way of reading existing history: deja index, cass index --full, agentmemory import-jsonl, mempalace mine --mode convos. Latency is the wall time of one search call from a warm process; first is the very first call after the build. claude-mem has no row: it captures at session end through hooks and needs an AI provider to write memory, so on day zero it has nothing to answer from.
| deja-vu | CASS | agentmemory | MemPalace | claude-mem | |
|---|---|---|---|---|---|
| Reads the history already on disk | ✓ 22 agents, no capture step | ✓ 25+ agents | ✓ Claude Code JSONL import, ≤1000 files per call | ✓ mine per file tree | — records from install on |
| Install | 18.6 MB binary | 58.5 MB binary | 689 MB npm + 28 MB engine | 312 MB + 168 MB embedding model | npm + Claude Code plugin |
| Runs in the background | nothing | nothing | worker + engine, four ports | Chroma | worker |
| Needs a model or key | no | no | keyless mode: BM25 only | local embeddings | yes |
| Build over 19,195 sessions | 29 s | 56 min | 95 s | ≈3 h | — |
| First answer after the build | 24 ms | 0.38 s | 185 ms | 4.5 s | — |
| Search latency, p50 | 155 ms | 0.37 s | 145 ms | 2.6 s | — |
| hit@1 / 100 | 18 | 5 † | 14 | 14 | 0 |
| hit@5 / 100 | 35 | 8 † | 34 | 20 | 0 |
| found@50 / 100 | 67 | 16 † | 65 | 46 | 0 |
| Serves the agent by itself | ✓ MCP, hooks at session start and before a tool runs | robot CLI, MCP | ✓ MCP, hooks | ✓ MCP, hooks | ✓ hooks |
† CASS is a keyword search: the question as typed returns nothing (a strict AND over every word, stop words included), so its row is the question with stop words removed — the only tool given anything but the question verbatim. The released 0.7.1 fails every query on this index with Quill query fuel exhausted (their #441, fixed on main after this was measured); the row is the main build. It also indexed both layouts as separate conversations, 38,394 in all.
Versions and dates: deja main at f2cebd6 with #3017, CASS 0.7.1 and main (38f0412, built from source), agentmemory @agentmemory/agentmemory 4.x from npm on Sep 2, MemPalace 0.9 line from uv on Sep 2, measured September 2–3, 2026 on an Apple Silicon laptop. The deja row is go run ./scripts/day0bench -data longmemeval_s_cleaned.json -limit 100 -keep DIR; the other rows are the drivers under scripts/day0compare run over DIR.
Reading the numbers
The absolute hit@1 is low for every tool because these are LongMemEval questions asked against nineteen thousand sessions rather than the fifty the benchmark ships per question, and a third of them are the kinds it is built to be hard on — preferences, multi-session aggregation. What separates the rows is not the ceiling. deja and agentmemory land within a point of each other on accuracy, both plain BM25 with no model; deja gets there in half a minute with nothing else running, agentmemory in a minute and a half with a worker and an engine up. MemPalace is a semantic store and pays for it twice here: an afternoon of mining with a model, and two and a half seconds a query, for fewer hits. CASS is the fastest indexer of the three that read history at all only if you count the release that cannot answer; the main build takes as long to index as MemPalace takes to mine a tenth of the pile, and it is a keyword tool that wants the question rewritten. claude-mem never gets there from this history; it starts recording at the first session after install.
CASS and deja are the closest pair in intent — both index every agent's files, no model — and they split on what happens after the search: CASS is a search tool with a robot interface, deja hands the session to the agent on its own, at session start, before a file is edited, after a command fails. If you want a TUI over your history, CASS is the TUI. If you want the agent to already know, that is the part deja was built for.
What is not measured
Quality of memory a tool writes for itself over weeks; semantic recall with a model attached (deja has it as an option, MemPalace and agentmemory have it on by default with a model); ranking on paraphrased questions, where every lexical tool including deja loses ground and the warm benchmark puts numbers on it. This page is only the first minute.