Menu

How We Eliminated Cache Stampedes in Node.js & Bun with Single-Flight Coalescing
πŸ“°

How We Eliminated Cache Stampedes in Node.js & Bun with Single-Flight Coalescing

DEV Community: javascriptΒ·zisam shaikhΒ·13 days ago
#6HaVmXWL
#devtojavascript#devto#dev#kvdock#database
Reading 0:00
15s threshold

Every engineer who has scaled a high-traffic backend has lived through this nightmare:

You have an expensive endpoint (e.g., aggregating leaderboard data or rendering a personalized dashboard). To protect your database, you cache the result with a 60-second TTL.

Everything runs smoothly until the key expires while receiving 1,500 requests per second .

Within a 50-millisecond window, hundreds of concurrent requests see a cache miss. All of them immediately hammer the primary database simultaneously to recompute the same data. Your connection pool saturates, CPU spikes to 100%, latency explodes, and your server crashes.

This is the thundering herd / cache stampede problem .

The Traditional (Fragile) Workarounds

Probabilistic Early Expiration (XFetch algorithm) : Recomputes the value before it expires based on an exponential probability.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.