Menu

Post image 1
Post image 2
Post image 3
1 / 3
0

Running a 35B MoE (Qwen3.6-35B-A3B) on 2x GTX 1080 Ti in 2026 — Real Benchmarks, and Does the Second GPU Actually Help?

DEV Community: machinelearning·byeongsoo kang·4 months ago
#Pp28NVQa
#dev#model#iq4_xs#ollama#quant#experts
Reading 0:00
15s threshold

TL;DR (Quick Answer) I actually ran Qwen3.6-35B-A3B — a 35B-parameter mixture-of-experts model (only 3B active per token) — on a pair of 8-year-old GTX 1080 Ti cards (22 GB combined). Real, measured numbers: Generation speed: ~20 tokens/sec on 2× 1080 Ti (IQ4_XS quant), stable across runs (19.4 / 21.4 / 20.0). Single GPU: ~16.8 tok/s. So the second 1080 Ti buys ~20% more throughput — not 2×. Why: the MoE expert weights stay memory-mapped in CPU RAM either way; the second GPU just lets more of the model live in fast VRAM. It only "fits" because of the MoE + CPU-mmap trick. ~13 GB of the model sits on the two GPUs; ~18 GB of expert weights are mmap'd from CPU RAM, and only the active 3B runs each token. Quant matters for 22 GB: the default qwen3.6:35b-a3b tag is 23.9 GB and spills to CPU . You want ≤ IQ4_XS (~17.7 GB) to keep it (mostly) on the GPUs. Bottom line: a 35B model is genuinely usable on used-$200 Pascal cards in 2026 — as long as it's a sparse MoE and you pick the right quant.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More