About fifteen lines of JSON. That's all it takes to stream every insert, update and delete from Postgres into Kafka with Debezium. And two of the defaults hiding in those lines will hurt you in production if you leave them alone.

This is Part 2 of CDC in Production . Part 1 explained what change data capture (CDC) is. This post opens up the most common open-source setup, Debezium in Kafka Connect reading Postgres: what to prepare in the database, what every setting does, what happens on first start, and what lands in each Kafka message.

Prefer to watch? Here's the 9-minute video version with animated diagrams:

Video link: Debezium Postgres to Kafka: Every Setting Explained

The moving parts

Read the pipeline left to right:

Postgres (WAL) -> replication slot + publication -> Debezium in Kafka Connect -> Kafka topics -> your sinks

Enter fullscreen mode

Exit fullscreen mode

Write-ahead log (WAL): Postgres writes every change here before it counts as committed.…