Menu

The transactional outbox: stop losing events between your database and Kafka
📰

The transactional outbox: stop losing events between your database and Kafka

DEV Community: microservices·Ayush Soni·14 days ago
#iqL41ulN
#devtomicroservices#devto#dev#outbox#order
Reading 0:00
15s threshold

This is one of the most common bugs in event-driven systems, and it hides in two innocent lines:

orderRepository . save ( order ); // 1. commit to the database kafkaTemplate . send ( "orders" , orderCreated ); // 2. publish the event

Enter fullscreen mode

Exit fullscreen mode

If the process dies between line 1 and line 2 — a deploy, an OOM kill, a lost connection to the broker — the order exists, but the event never goes out. Billing never bills, the warehouse never ships, and nothing errors, because from every service's point of view nothing happened.

Swapping the lines doesn't help. Publish first, and if the database commit then fails you've announced an order that doesn't exist.

The underlying problem is a dual write : two systems, no shared transaction. You can't make a Postgres commit and a Kafka publish atomic without distributed transactions, which most teams rightly avoid. But you can make two writes to the same database atomic.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.