Part of the homelab build series. A football match-outcome model ships every day through Kubeflow Pipelines and KServe on RKE2. Each new model is staged next to the live one, tested, given 10% and then 50% of replayed traffic, measured revision by revision, and promoted or rolled back by the pipeline itself. The service stays highly available throughout: the serving revision keeps its traffic until the new one has earned it.

Canary Releases for an ML Model on KServe: the new revision's share of replayed requests at each stage, from staged at 0% through 58 of 600 at the 10% step and 305 of 600 at the 50% step to promoted at 100%, with any failed check rolling back to 0%

Introduction

A canary with no traffic is a coin you never flip.

The textbook canary goes like this. Send 10% of live requests to the new model, watch error rates and latency against the old one, step up, promote. Every guide says it, and every guide assumes one thing: that there are live requests.

My model has almost none.…