Your AI agent burns tokens to think. Most of that thinking - the tool-call planning, the YAML editing, the re-reads of the same file - does not need a frontier model behind an API. Docker Model Runner keeps a real LLM inside the Docker engine you already run, exposes it on an OpenAI-compatible endpoint, and charges you nothing per call. docker model pull is the whole setup.

This post is the wiring guide: what Model Runner actually is (and the one detail everyone gets wrong), how to size the model, and how to point Docker Agent - the CLI plugin that defines agent teams in YAML - at a local model for a fully offline agent stack.

In this post:

What is Docker Model Runner, actually?

The detail everyone gets wrong: no container starts

Sizing the brain: GGUF and the quantization table

Wiring it to an agent: the dmr.yaml contract

What a local brain can and cannot do for an agent

FAQ

Key Takeaways

Model Runner is an inference server inside Docker, not a container.…