Menu

Anatomy of a multi-cloud deploy that fails
πŸ“°

Anatomy of a multi-cloud deploy that fails

DEV Community: terraformΒ·Gonzalo GjukaΒ·about 3 hours ago
#IgTfQtIk
#devtoterraform#devto#dev#cloud#terraform
Reading 0:00
15s threshold

Friday, 6:10pm. The terraform plan says "12 to add, 3 to change, 1 to destroy". You read it, nothing looks odd. Apply. Green bar. You get up for a coffee and when you come back the alerts channel has 40 new messages. The load balancer is not responding and half the pods will not start. The plan was green. What happened?

The deploy that "could not fail"

The change was small: clean up an old subnet in one cloud and move a few workloads to another region. Terraform calculated the plan against its tfstate and saw no problem β€” because from where Terraform looks, there was none. The state knows which resources it created. It does not know who else, outside its world, depends on them.

And there is the trap: the resource you destroyed was the plan's "1 to destroy". A subnet that, according to the state, had nothing hanging off it. Except it did: a load balancer created six months earlier, by hand, from the console, in a hurry. That LB referenced the subnet.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.