Menu

Durable workflows vs. a queue and a cron job
πŸ“°

Durable workflows vs. a queue and a cron job

DEV Community: javaΒ·Ayush SoniΒ·17 days ago
#wrAXmLnc
#devtojava#devto#dev#workflow#accountid
Reading 0:00
15s threshold

Some work doesn't fit in a request. Moving a customer's data to new infrastructure. Onboarding a merchant across five systems. Generating, approving and sending a large batch of documents. These jobs have many steps, take minutes to days, call systems that fail, and must survive deploys and crashes.

Most teams start the same way: a queue, a cron job and a status column.

What queue + cron becomes

Version one is clean: a job row with status = PENDING , a worker that picks it up, does the work, sets DONE . Then reality adds requirements, one at a time:

A step fails transiently β†’ add a retries column and a RETRYING status.

A worker dies mid-job β†’ add locked_by and locked_at , and a cron that releases stale locks.

Step 4 must not run twice β†’ add per-step status columns.

A step needs to wait for a human approval or a 24-hour cool-down β†’ add resume_after and another cron.

Step 6 fails permanently after steps 1–5 succeeded β†’ write undo logic, scattered across the codebase.

Nobody can tell where a job is →…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.