You fire 50 LLM calls at once. A batch job, a fan out over documents, a busy hour. The provider only lets 10 through at a time, so the rest come back as 429 Too Many Requests .
The usual fix is to retry 429s with backoff. That helps, but notice what it means: every retry is a request you sent after you already got rate limited. You are reacting to a limit you could have stayed under.
This post shows the difference with real numbers and how to rate limit LLM calls in TypeScript with VernLLM , before the provider says no.
Retrying 429s vs not causing them
I ran a 50 call burst against a scripted provider that accepts 10 requests in flight and rejects the rest with 429. Both runs used maxRetries: 2 .
Retries only rateLimit: { maxConcurrent: 10 }
Calls answered 20 50
Calls failed 30 0
Provider 429s 110 0
Requests sent 130 50
Retries alone sent 130 requests, collected 110 rejections, and still lost 30 calls.…