A 16 GB Mac mini gives its GPU about 12.7 GB. Google's Gemma 4 26B-A4B, in its official 4-bit file, is 14.4 GB. On paper it doesn't fit. In practice it writes TypeScript at 31 tokens a second, offline, from that one file, untouched.

The trick is that a mixture-of-experts model never uses all of itself at once. This post is about how I pick which part to keep.

What "26B-A4B" means

Gemma 4 26B-A4B has 26 billion parameters, but only about 4 billion are used for any one word. Each of its 30 layers holds 128 small "experts", and a router in each layer picks 8 of them per word. The other 120 sit idle for that word.

So the real question is never "does the model fit?" It is "do the experts this word needs fit?"

The heat map

I ran Gemma on TypeScript prompts and logged every pick the router made: which expert, in which layer, with what weight. Counted up, that log is a heat map.…