At 03:02 the primary database ran out of disk and Postgres shut down. There was no runaway query. A CDC connector had been paused nine days earlier, and its replication slot had been holding every byte of write-ahead log since.
This is Part 6 of CDC in Production , the final part. In Part 5 we shipped schema changes without breaking the warehouse. This post is about keeping the pipeline, and the database under it, alive: the WAL slot trap, heartbeats, the three lag numbers that matter, the alerts and runbook you need, and what all of it costs.
Prefer to watch? Here's the 8-minute video version with animated diagrams:
Video link: Running CDC in Production: The WAL Slot Trap
The WAL slot trap
The write-ahead log (WAL) is where Postgres records every change before it counts as committed. A replication slot is a promise about that log.…