TL;DR: VRAM matters more than GPU power. Most people overestimate what they need—and underestimate what actually runs well.
The confusing part about local LLMs
If you’ve tried running models locally (Ollama, llama.cpp, LM Studio, etc.), you’ve probably asked:
“Can my GPU run this model?”
“Why does it technically load but run painfully slow?”
“Do I need 24GB VRAM for everything?”
The answers online are inconsistent.
So instead of relying on benchmarks, I started tracking what actually works in real setups.
🧠 The simple rule most people miss
If it doesn’t fit comfortably in VRAM, it doesn’t really “run”.
Yes, you can offload to CPU or swap memory—but the experience quickly degrades.
📊 Practical VRAM breakdown
Here’s a simplified version of what consistently works:
🟢 Under 8GB (S-tier)
7B models (quantized)
Good for:
basic chat
light coding help
Limitations:
struggles with longer context
slower responses
🟡 8–16GB (M-tier)
7B → very smooth
13B → usable but sometimes tight
👉 This is…