A small calculation that changed how I look at timeouts. Take a request that has to answer the user in 3 seconds.

Step 1. Your service calls another service, and you’ve set a 2-second timeout on that call. That looks safe, since 2 is less than 3.

Step 2. The call is wrapped in a retry, with up to 2 retries after the first attempt. A timed-out attempt is retried.

Step 3. Worst case, all three attempts time out:

worst case = timeout × (retries + 1) = 2s × (2 + 1) = 6s

The user’s budget was 3 seconds. In the worst case you spend twice that waiting on one dependency, and you haven’t started counting the rest of the request yet.

Step 4. Now add a second dependency with the same settings. If both are called one after the other and both are slow, the worst case is 12 seconds.…