Menu

5 Mistakes I Made Swapping a Hosted LLM for Local Ollama in .NET
📰

5 Mistakes I Made Swapping a Hosted LLM for Local Ollama in .NET

DEV Community: dotnet·MICHAEL-MAURICE·2 days ago
#GiX6AnJs
#devtodotnet#devto#dev#model#ollama
Reading 0:00
15s threshold

With Microsoft.Extensions.AI, switching from a hosted model to a local Ollama server really is close to a one-line change: register a different IChatClient and you're done. The code compiles, the first request works, and you move on.

What doesn't carry over are the assumptions. A hosted API is a big shared service with generous limits that tells you when something is wrong. A local Ollama server is one process, usually running one request at a time, with its own defaults, and it's quiet about its problems.

Here are the five assumptions that bit me while building a local-LLM chat API in C#, how each one showed up, and the shape of the fix.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. The complete, tested project behind this post (a .NET 10 local-LLM chat API with Clean Architecture and 187 passing tests, running offline against an Ollama simulator) plus a step-by-step PDF guide is inside.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.