Menu

Rate limiting: fixed window’s boundary burst, token bucket, and what breaks across multiple instances

Reddit r/Backend·u/ClickOk5811·14 days ago
#y3bpTY1s
#tokens#store#window#boundary#burst
Reading 0:00
15s threshold

Limit: 100 requests per minute per client. Numbers are illustrative.

Fixed window. A counter per client per minute, reset at each boundary. A client sending 100 requests at 12:00:59 and 100 more at 12:01:00 gets 200 through in about two seconds, and no rule is broken. That’s the boundary burst: up to roughly double the limit around a reset.

Token bucket. Capacity 100, refill 100/60 ≈ 1.67 tokens per second. The same client takes all 100 tokens at 12:00:59, and one second later only one or two have refilled, so most of the second wave is rejected. Bursts are allowed up to the capacity, and the long-run rate is capped by the refill rate.

Pseudocode:

on request(key, now): tokens, last = load(key) # defaults: capacity, now tokens = min(capacity, tokens + (now - last) * refill_rate) if tokens >= 1: tokens -= 1 allowed = true else: allowed = false store(key, tokens, now) return allowed

Sliding window. A log of timestamps is precise but uses memory per request.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.