Containerising an AI workload is a different problem from containerising a web service. The dependencies are huge, the build tools are heavy, and an unoptimised image can pass 10 GB . Moving that across a registry slows every CI/CD run, and during a Kubernetes autoscale event each new pod must pull it before it can serve a single request.
The naive Dockerfile
This is the Dockerfile most teams start with.
FROM python:3.11 RUN apt-get update && apt-get install -y build-essential gcc RUN pip install torch transformers fastapi uvicorn COPY . /app WORKDIR /app CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Enter fullscreen mode
Exit fullscreen mode
It works, and it has three production problems.
Diagram: FIG 1 · A single-stage build carries compilers, caches and wheels into production. Layer sizes here are illustrative, not measured. See the animated version .
Build artifact accumulation.…