Menu

Ollama in Your Dev Loop: Swapping the Base URL Is the Easy Part
📰

Ollama in Your Dev Loop: Swapping the Base URL Is the Easy Part

DEV Community: typescript·Danny Holloran·2 days ago
#uFsF5vk6
#devtotypescript#devto#dev#ollama#model
Reading 0:00
15s threshold

Running a model locally sounds like the perfect fix for a handful of annoyances: no API bill while you iterate on a prompt, no customer data leaving your laptop, and a dev loop that works on a plane. Ollama makes the first step almost suspiciously easy. It serves an OpenAI-compatible endpoint on localhost:11434 , so the official OpenAI SDK talks to it with a different baseURL and nothing else.

That one-line swap is real, and it is also where most people stop reading. The trouble shows up an hour later, when the local model "ignores" half your system prompt or a request that works against a hosted API throws a 400. Almost all of it comes from three places: the context window, the parts of the API Ollama does not implement, and how much you trust the output shape.

The swap, done so you can undo it

Ollama's docs show the whole integration: point the client at http://localhost:11434/v1/ and pass any string as the API key, because it is required by the SDK but ignored by the server.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.