Menu

Three services shared one deploy queue and the middle one never deployed
📰

Three services shared one deploy queue and the middle one never deployed

DEV Community: cicd·Sergey Shinder·14 days ago
#oqA2vC0b
#devtocicd#devto#dev#pricing#deploy
Reading 0:00
15s threshold

Our orders, pricing and returns services live in one repository, and each has its own GitHub Actions workflow that deploys it to production when a change to its directory is merged. Last spring two deploys ran into each other on the cluster, so we gave all three workflows the same concurrency block, group: production with cancel-in-progress set to false. One deploy at a time, the others wait their turn. That was the idea.

On a Monday in October the pricing team merged a fix for the VAT rate on children's clothing. On Thursday finance found three days of invoices still charging the old rate. The pricing workflow had run, waited and been cancelled. Nobody had pressed anything.

A concurrency group is not a queue. While a run in the group is in progress, a new run waits in a pending state, but GitHub keeps at most one pending run per group. When another run joins, the one already waiting is cancelled and the new one takes its place.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.