Running a model locally sounds like a weekend project. It's actually ten minutes. This is the copy-paste version: pick a tool, pull a model, serve it over an OpenAI-compatible endpoint, and make your first tool call, with no hand-waving.

Treat this as a starting point for getting a model running, not a fine-tuning guide. The aim is a working endpoint in ten minutes; the deeper performance knobs (sampler settings, speculative decoding, GPU offload splits) are their own rabbit hole, and a separate post.

Why local, when hosted models are right there and genuinely good? Privacy is the obvious one: nothing leaves the machine. Then cost, because a long or repetitive job stops metering you by the token. And control, which matters more than it sounds. No rate limits, no refusal out of nowhere, and the version stays pinned instead of shifting under you one morning. You pay for all of it in quality and speed, and that price is real.…