I built RAG over nine yearly revisions of federal acquisition regulations and asked it questions "as of" specific dates. Plain RAG answers from the wrong version 80% of the time β fluently, with citations. Date handling fixes it without hurting recall.
Scope note, first paragraph as promised: FAR/DFARS only, year-granularity revisions (2017β2025), US public-domain text, one 1.8B local generator, greedy decoding, no LLM judge (containment is string matching). NOT legal advice β a measurement study, not counsel.
The problem nobody benchmarks
Regulations change. The micro-purchase threshold moves, clauses get renumbered, paragraphs appear and vanish. A RAG system that treats its corpus as timeless will answer a 2022 question with 2024 text β correctly cited, completely wrong. I call it version leakage , and before this project I couldn't find an open benchmark that measures it over a corpus with official point-in-time history.β¦