For the last week I've been running LongMemEval against Palace, a local memory store I build for coding agents. It's a single Rust binary with SQLite, BM25 and a small ONNX embedding model. No LLM sits anywhere in the write or recall path.
This post isn't about the final number. It's about what I learned getting there: which changes moved recall, which didn't, and how many of the improvements turned out to be plain bugs that only a benchmark run would surface.
Short version, so you can stop here if you want:
Swapping embedding models did nothing on this dataset. Every difference was 1 to 4 questions.
Chunking conversations the way the product actually stores them helped more than anything else.
The ranking gains were mostly bug fixes. A substring match, an overeager phrase rule, and a chunker that silently dropped text.
int8 models give different vectors depending on what else is in the batch. That one surprised me.
What's being measured
LongMemEval asks questions about a long chat history.…