An AI agent can visit 200 supplier pages, pull out product names, prices and dimensions, and hand you a clean CSV in a few minutes. The file looks perfect. The problem is that you can't tell which rows are wrong by looking at it.

That is the gap this article is about. Agents are very good at producing data that is shaped correctly. They are much weaker at knowing when the data is true . For a lot of real work, the fastest way to close that gap is still a person checking the output against the source.

Confident and wrong: the failure mode of extraction agents

When a person copies data by hand, mistakes tend to look like mistakes: a missing field, an obvious typo, a row that doesn't match the others. When a model extracts data, the errors look exactly like correct answers.

A builder we talked to on Reddit described it well. Their agent pulled product dimensions from supplier sites.…