I build plenty of apps due to the nature of my job. I use coding agents extensively. I decide what gets built and review what the agent writes. Tests are a big part of my development process as it ensures that whatever my agents build is up to standard.

Recently, I was working on a game that allows users to work as a bartender. In this game, you are expected to take orders, deliver them, and charge your customers. When the agent built the feature for charging the customers, it actually implemented a false-positive test: the test passed, but it shouldn’t have. In the application, orders arrive through an Amazon Simple Queue Service (Amazon SQS) queue. One of its tests simulated a crashed worker by charging an order and then forgetting to delete the message, forcing the message to be redelivered. The agent used the receipt handle as its key to prevent double-charging. Because SQS generates a new receipt handle for each delivery of the message, the check didn't recognize the order.…