I built a local-first memory for coding agents. It's a Rust binary and one SQLite file, using MiniLM embeddings plus BM25 keyword search, with no LLM in the retrieval path and no re-ranker.
On LongMemEval S (470 scored questions) it scores R@1 0.913 (429), R@5 0.987 (464) and R@10 0.996 (468).
I wanted more, mainly at R@1 and recall_all (every gold session in the top K). But all while preserving the strict rules of NO LLM, no re-ranker model and not tuning for the benchmark.
Claude Code did most of the legwork:
offline replays of chunk-score dumps
a DEV/TEST split by question-id hash
a "guard" suite (my coding eval plus probes against a snapshot of my real palace), so a benchmark gain couldn't quietly hurt real use
What we tried
Idea LongMemEval Real-palace guards
Fix two WordPiece tokenizer deviations 0 questions changed probes -1/200
Embed at MiniLM's trained 128 tokens, not 512 R@1 +1, recall_all@5 -7 probes 0.900 -> 0.875, fail
256 tokens +4/-4 at R@1 fail
Re-weight cosine vs BM25 (0.70 to…