I open-sourced a little shopping tool. It had a polite fetcher, respected robots.txt , sent a real User-Agent, and checking that the address was public before it dialled out.

It had never once fetched a product.

Artifacts Apparel FAILED HTTP 429 for https://artifactsapparel.co.uk/products.json?limit=250&page=1 You Know Who's FAILED HTTP 429 for https://youknowwhos.co.uk/products.json?limit=250&page=1 Goodhood FAILED HTTP 429 for https://goodhoodstore.com/collections/sale/products.json?limit=250&page=1 Represent FAILED HTTP 429 for https://representclo.com/products.json?limit=250&page=1 ... 0 products checked, 0 in stock in your size

Enter fullscreen mode

Exit fullscreen mode

Twenty shops, twenty-nines, zero products. And here is the part that made it interesting: the tool never failed loudly. It printed a table. It exited 0. It wrote an empty digest and posted it to Discord like a perfectly normal Monday.

The test that took two minutes

My first guess was the User-Agent.…