You build a Patroni + etcd + HAProxy cluster, kill the primary in a test, and your write loop stops for half a minute or more. Nothing is broken. That is what the default settings are designed to do. This post explains where those seconds go and what you can change.

The three settings that matter

All three live in Patroni's dynamic configuration (stored in the DCS, changed with patronictl edit-config ). From the official documentation:

Setting What it is Default Minimum

loop_wait "the number of seconds the loop will sleep" between HA cycles 10 1

ttl the TTL of the leader lock; "think of it as the length of time before initiation of the automatic failover process" 30 20

retry_timeout "timeout for DCS and PostgreSQL operation retries"; "DCS or network issues shorter than this will not cause Patroni to demote the leader" 10 3

The documentation also sets a hard rule when you change them:

loop_wait + 2 * retry_timeout <= ttl

Enter fullscreen mode

Exit fullscreen mode

With defaults: 10 + 2 × 10 = 30 ≤…