Introduction

There’s something very satisfying about watching a system converge. You push a change to Git. A pipeline runs. A few minutes later, the cluster reflects exactly what you defined.

The system was Clean, Predictable and Repeatable.

For a long time, I thought that was the hard part. It turns out, it isn’t.

The Setup

This was a fairly typical setup.

Kubernetes cluster on GKE

Applications deployed using Helm

GitLab CI driving deployments

Terraform managing underlying infrastructure

The deployment flow was simple:

git push origin main

Which triggered a GitLab pipeline:

deploy : stage : deploy script : - helm upgrade --install payments ./chart -f values.yaml

Enter fullscreen mode

Exit fullscreen mode

Nothing fancy.…