Overview
A cache outage is not merely a latency issue; it is a massive load-transfer event. If a system natively routes all traffic to the database when a cache (like Redis) fails, the database will instantly become overwhelmed, turning a partial outage into a complete systemic failure. This document outlines the engineering principles required to safely manage database fallback through bounded concurrency, graceful degradation, and controlled recovery.
The Core Problem: The Load-Transfer Illusion
Asymmetric Capacity: Redis is designed to absorb massive, repeated read demand. The primary database is sized for writes, ingestion, and complex queries, not cache-level throughput.
The "Retry Storm": When Redis fails or slows down, naive fallback mechanisms dump 100% of the read load onto the database, compounded by users and automated clients aggressively retrying their delayed requests.
The Objective: Fallback must be treated as a scarce emergency resource.…