Every engineer who has scaled a high-traffic backend has lived through this nightmare:
You have an expensive endpoint (e.g., aggregating leaderboard data or rendering a personalized dashboard). To protect your database, you cache the result with a 60-second TTL.
Everything runs smoothly until the key expires while receiving 1,500 requests per second .
Within a 50-millisecond window, hundreds of concurrent requests see a cache miss. All of them immediately hammer the primary database simultaneously to recompute the same data. Your connection pool saturates, CPU spikes to 100%, latency explodes, and your server crashes.
This is the thundering herd / cache stampede problem .
The Traditional (Fragile) Workarounds
Probabilistic Early Expiration (XFetch algorithm) : Recomputes the value before it expires based on an exponential probability.β¦