Your scraper worked fine on 50 requests. At 500, it started returning 403 Forbidden . At 5,000, you're staring at 429 Too Many Requests and CAPTCHA pages with a 200 status code. Adding more proxies didn't help, or helped for ten minutes.

The mistake most developers make here is changing several things at once: new proxies, new headers, more retries. This guide gives you a repeatable way to find out why you're being blocked so that you can fix the right thing.

What we'll cover

What 403, 429, and CAPTCHA responses actually mean

A step-by-step isolation process (and a small Python probe script)

Detecting "soft blocks" that return HTTP 200

Backoff, Retry-After , and pacing done properly

Tracking proxy health so bad IPs get benched automatically

When the problem isn't the IP at all

Examples use Python and requests , with placeholder proxies from the documentation range ( 203.0.113.x ).

What each signal usually means

Signal What the server is telling you Typical root cause

429 Too Many Requests You…