Menu

#NeoMindLabs

1 post

Feed
1 of 1 post
Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU
📰
0

Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU

Hacker News·3 months ago
#b0oN1NsE
#neomindlabs#if#else#avx2#model#build

A storage appliance that shipped before AVX2 existed now runs Google's Gemma 4 26B in my basement, no GPU. Here's how we got a modern MoE model onto a CPU older than the instructions its inference engine assumed were there.

15s
Read More