Originally published at webofmike.com on 2026-11-07. The demo repo and every command in it were run before publishing.
A paper published on arXiv on October 27 did something I hadn't seen demonstrated end to end before: it pretrained a language model from scratch, poisoned a small slice of its training data, and came out the other side with an agent that runs an attacker's command when it sees a trigger phrase, and behaves normally the rest of the time. The HN thread picked up 103 points and 30 comments, split between "this is FUD about open weights" and "this is why you contain agents regardless of whose weights you're running." The repo for this post is github.com/themsquared/agent-backdoor-containment , and it isn't a reproduction of the paper.…