Menu

Day 9: Kubernetes Core for Machine Learning — Pods, Deployments and Headless Services
📰

Day 9: Kubernetes Core for Machine Learning — Pods, Deployments and Headless Services

DEV Community: kubernetes·Suresh Kumar Pallapothu·about 10 hours ago
#DAYxi6vp
Reading 0:00
15s threshold

A basic Deployment suits a stateless web server. Model serving and vector stores add failure domains that web apps rarely have: memory-exhaustion crashes, cold-boot timeouts and broken load-balancing rings. This lesson covers the Kubernetes building blocks that handle them.

Deployments and Services for inference

An inference pod ingests context, generates tokens and discards its KV-cache. It is stateless , so a Deployment gives you horizontal scaling, rolling updates and automatic restarts. A ClusterIP Service puts one stable address in front of the pods and sends traffic only to those that pass their readiness probe, so a saturated pod can step out of rotation without being killed.

Diagram: FIG 1 · The Service sends requests only to pods whose readiness probe passes. A saturated pod is skipped until it recovers. Tune the probe so it does not remove every pod at once. See the animated version .

Headless Services for peers

A normal Service gives one virtual IP.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.