[Ed. note: If you listened to a podcast or read an article of ours and want to publish a response, let us know. We’re open to highlighting community views that respond to our work in ways that don’t fit in a comment section, whether you agree or disagree. Reach out at [email protected].]
In March 2025, a team at OpenAI published something you don't often get to see: the private reasoning of one of their own frontier models as it worked through a coding task during training. The model looked at the problem, decided that implementing the whole thing was too hard, and wrote down an alternative. If it called sys.exit(0) , the test harness would exit gracefully. "This is unnatural," it noted, "but tests might pass."
They passed.
The researchers called that a systemic hack—one general enough that, once a model finds it, it spreads across nearly every training environment. They caught a second of the same kind: raise an exception from outside the testing framework and skip evaluation entirely.…