Proprietary models are amazing ! But sometimes what is of importance is configurability rather than raw power. This has led to the emergence of locally hosted models. Running AI locally gives users more control over their setup. With suitable hardware and tools like Ollama and LM Studio, capable models can run entirely on-device.
But which models are worth running, and would they run on your device?
In this article, we look at five LLMs you can run locally in 2026 and help you choose one that suits your needs and hardware.
Model Start with it for
Llama 3.1 8B Instruct General chat and fine-tuning projects
Qwen2.5 7B Instruct / Coder Multilingual text and coding assistance
Mistral 7B Instruct v0.3 Everyday drafting and documentation
Gemma 4 E2B / E4B Text, image, and audio understanding
Phi-4-mini-instruct Lightweight Q&A and reasoning tasks
Chat families are roughly ordered by cumulative Ollama library pulls, which favor older releases and do not measure quality.…