A basic Deployment suits a stateless web server. Model serving and vector stores add failure domains that web apps rarely have: memory-exhaustion crashes, cold-boot timeouts and broken load-balancing rings. This lesson covers the Kubernetes building blocks that handle them.
Deployments and Services for inference
An inference pod ingests context, generates tokens and discards its KV-cache. It is stateless , so a Deployment gives you horizontal scaling, rolling updates and automatic restarts. A ClusterIP Service puts one stable address in front of the pods and sends traffic only to those that pass their readiness probe, so a saturated pod can step out of rotation without being killed.
Diagram: FIG 1 · The Service sends requests only to pods whose readiness probe passes. A saturated pod is skipped until it recovers. Tune the probe so it does not remove every pod at once. See the animated version .
Headless Services for peers
A normal Service gives one virtual IP.…