Menu

The Silent CDC Bug: Data Loss With No Errors
📰

The Silent CDC Bug: Data Loss With No Errors

DEV Community: postgres·Pipeline Whiteboard·about 2 hours ago
#YWzrau93
#devtopostgresql#devto#dev#slot#table
Reading 0:00
15s threshold

Every dashboard is green. The connector says RUNNING , capture lag is under a second, and the warehouse is missing 4,812 orders. Nobody gets paged, because nothing failed.

This is Part 3 of CDC in Production . In Part 2 we built a Debezium pipeline from Postgres to Kafka. This post covers four ways that pipeline can lose data without a single error, and the reconciliation check that catches all of them.

Prefer to watch? Here's the 8-minute video version with animated diagrams:

Video link: The Silent CDC Bug: Data Loss With No Errors

Why nothing alerts

The usual signals measure flow , not completeness . Task status tells you the connector is alive. Lag tells you how far behind the newest event is. Neither tells you whether every change made it. A pipeline that skipped a thousand rows an hour ago looks exactly like a healthy one now.

Failure 1: the slot moves on after a failover

A replication slot is Postgres's bookmark for a logical decoding reader: it remembers the last position the reader confirmed.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.