14:47 UTC, Tuesday. Pods in the checkout-api deployment start OOMKilling in a loop, three or four at a time, in the middle of an ordinary afternoon. Traffic is flat. No deploy went out today. PagerDuty fires on KubePodCrashLooping , not on anything tied to a release, because as far as the deploy history shows, nothing was released.

the scramble

First theory: a memory leak shipped yesterday and took a day to build up enough pressure to matter. git log --since=48h -- apps/checkout-api shows exactly one commit, a routine lockfile refresh from the nightly dependency job. No application code changed in either direction.

Second theory: a traffic burst nobody's dashboard caught. Request rate for the service is within 3% of the same Tuesday last week. Whatever's killing these pods, it isn't load.

Third theory: a slow leak finally crossing a threshold. The service exposes a /debug/memory endpoint that dumps process.memoryUsage() on demand. heapUsed sits at a boring 140MB, same as it's sat for weeks.…