Menu

Close Enough to Be Wrong: An Adversarial Evaluation of a Semantic Cache
πŸ“°

Close Enough to Be Wrong: An Adversarial Evaluation of a Semantic Cache

DEV Community: machinelearningΒ·Tushar KumarΒ·8 days ago
#BUBYjBaG
#devtomachinelearning#education#devto#dev#token
Reading 0:00
15s threshold

Close Enough to Be Wrong: An Adversarial Evaluation of a Semantic Cache

Abstract. Every project in this portfolio got an adversarial eval suite; this one's system under attack is the cache itself β€” the two-tier lookup (exact match, then embedding similarity at a 0.90 threshold) that decides whether a question has already been answered. I aimed three attack families at it: cache poisoning (planted entries with wrong answers, including one planted through the service's own store-on-miss path), one-token boundary crossings (negation, number, entity and date swaps that leave a prompt almost identical but change its answer), and threshold probing (similarity ladders and a full threshold sweep), plus benign controls and a hand-written PAWS-style paraphrase slice.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.