Short answer: treat a webhook retry queue as an at-least-once transport, and make the durable business change idempotent with a stable event identity and a database transaction.
The deciding constraint is the gap between doing work and acknowledging it. A worker can commit a change, lose its process before the acknowledgement is recorded, and receive the same job again. No reasonable retry policy can infer whether that first change happened. Delayed jobs help recovery; they do not close that gap.
For an AI application, that distinction matters twice. A repeated state change can corrupt customer state, while a repeated call to a model or external API can consume latency and usage budget without producing a new result. Keep the queue simple, but make its boundaries explicit.
Why do webhook retry queues create duplicate processing with delayed jobs and at-least-once delivery?
There are usually two independent retry loops. The webhook sender retries when it does not receive a timely success response.…