Menu

Retrieval Is Not a Safety Boundary: Poisoned-Corpus and Unsupported-Question Evaluation of a RAG Pipeline
📰

Retrieval Is Not a Safety Boundary: Poisoned-Corpus and Unsupported-Question Evaluation of a RAG Pipeline

DEV Community: machinelearning·Tushar Kumar·13 days ago
#u3FGZyih
Reading 0:00
15s threshold

Abstract. I built a retrieval-augmented generation service with hybrid retrieval (dense + BM25), a relevance threshold that is supposed to make it refuse unanswerable questions, and an extractive generator that quotes retrieved text with citations. I tested the two failure surfaces the standard evals never touch: a poisoned corpus (a planted document carrying a false claim, a steering instruction or a canary token, next to the real policy) and unsupported questions (questions the corpus cannot answer). Across 12 attack cases, 8 unsupported questions and 10 benign controls on three retriever backends, planted claims reached the answer in 80.6% of attack runs, unsupported questions were fabricated on 83.3% of runs (100% on the hybrid backend), and benign completion stayed at 90% with zero false refusals — the system keeps all of its utility and skips the safety.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.