Anthropic said its Mythos 5 model didn't stop hacking, even when Anthropic told it that real people were its targets. Malte Mueller/Getty Images
In August, an Anthropic agent created fake emails and bombarded a GitHub moderator's inbox with requests to insert malware into code.
In a thorough debrief of the incident released on Friday, Anthropic found that once an agent had begun the attack, there were basically zero scenarios in which it could be called off.
Like a rabid dog ignoring the pleading cries of its owner, the agent — 98% of the time in one test — ignored Anthropic's warnings once it had, so to speak, its first taste of blood .
Anthropic called this disquieting proclivity "momentum," and added that while its Mythos 5 model, which was released in June, showed little regard for warnings, current models were more inclined to stop.
"Once it had begun performing dangerous actions, Claude Mythos 5 almost always continued doing so, ignoring information that might have suggested its trajectory was…