Our quote pricing service is a Spring Boot application on Kubernetes. In August we gave it a horizontal pod autoscaler with two targets: seventy percent CPU and seventy percent memory, between four and twenty replicas. Memory was there because the service had once run out of it, and scaling on it felt like a sensible safety net.
On the first busy Monday it scaled out to twenty, which was right. It was still at twenty on Tuesday, and on Saturday, and three weeks later, with CPU around fifteen percent and traffic at overnight levels. The cluster autoscaler had kept the nodes to match.
The autoscaler computes the replicas each metric wants as the current replicas times the current utilisation divided by the target, rounded up, and when there are several metrics it takes the largest answer. Memory utilisation is the pod's working set divided by its memory request. CPU was asking for four replicas. Memory was asking for twenty or more, every time it was asked.
The JVM sizes its heap for the work it has seen.…