Menu

Stop Using Slow APIs: The Right Way to Bulk Check HTTP Statuses (50+ URLs in Seconds)
📰

Stop Using Slow APIs: The Right Way to Bulk Check HTTP Statuses (50+ URLs in Seconds)

DEV Community: productivity·laing lii·16 days ago
#hI8fvoe7
#devtoproductivity#devto#dev#urls#requests
Reading 0:00
15s threshold

Last week, I found myself staring at a dead-links report for an old client project. Over 300 pages. I needed to know which URLs were returning 404s, which ones were silently redirecting, and which ones had just given up entirely.

My first instinct was to write a quick loop in Node.js using fetch .

I hit two massive walls immediately:

The Rate Limit / IP Ban wall: After about 15 requests, the target servers started dropping my connection.

The Timeout wall: My script hung for 5 minutes because three random URLs just never responded.

If you've ever tried to bulk check HTTP statuses at scale, you know the pain.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.