When you're migrating an application to the cloud and every deploy causes a short burst of 502s, the fix is usually two small changes. Give your Node.js service honest health check endpoints, and make it shut down gracefully when it gets SIGTERM. Together they make the load balancer stop sending traffic to an instance before that instance stops answering.

Maybe you just containerised a service that ran fine on one VM for years, and now your error graph jumps for half a minute every time you ship. If so, this is almost certainly the cause. Your business logic is fine. The service has simply never had to shut down cleanly before.

Why do rolling deploys drop requests?

A rolling deploy starts new instances and then stops the old ones. The trouble is timing. When the orchestrator (Kubernetes, ECS, Nomad) stops an old container, it sends SIGTERM and, at about the same moment, tells the load balancer to remove that instance.…