Menu

The Retry Storm That Delayed a Report by Six Hours
📰

The Retry Storm That Delayed a Report by Six Hours

DEV Community: ai·niuniu·2 days ago
#eq5rodBu
#devtoai#devto#dev#empty#retries
Reading 0:00
15s threshold

You open the dashboard at 08:10 and the nightly summary column is empty, yet every job in the run history is green. Nothing paged, nothing failed, and forty thousand rows sit there with "summary": "" in them. You already know the shape of the answer before you finish scrolling: the pipeline did not crash, it gave up one item at a time and then reported success.

The timeline below is a reconstructed worked example rather than one company's log, so treat the clock as a template to fill in with your own timestamps. The code is the part that reproduces, and the ranking of contributing factors is the part worth arguing about with your team.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.