I had a short window this week to evaluate Bulk URL Status Checker as a data source. Here is the condensed write-up of what the data looks like, what surprised me, and the bits of infrastructure that paid off.
The source
Bulk URL Status Checker Broken Link & Redirect Audit Check the HTTP status of thousands of URLs in seconds. The relevant questions for any new source are always: is the markup stable, is pagination sensible, and how aggressively does it rate-limit. For this one, all three answers are "good enough that you can build on it" -- which is honestly more than I can say for a lot of supposedly easy targets.
The schema
What you get back per record:
url -- url
statusCode -- status code
statusMessage -- status message
isBroken -- is broken
isRedirect -- is redirect
redirectChain -- redirect chain
finalUrl -- final url
responseTime -- response time
checkedAt -- checked at
Nothing exotic, which is exactly what you want from a feed.…