Why We Are Evaluating KTransformers
We put KTransformers on our evaluation list because “buy enough GPUs to hold every expert” is an expensive answer to a sparse-model problem. A large mixture-of-experts model activates only a subset of its experts for each token, but its inactive weights still need somewhere to live.
Published DeepSeek-V3 Memory Requirements Single socket DRAM 382G DRAM Dual socket DRAM 1T DRAM V0.3 preview DRAM 644 GB DRAM Single-socket VRAM minimum 14 GB VRAM
The published configurations trade GPU memory for substantial CPU RAM; these capacity figures are not interchangeable measured memory peaks.
KTransformers makes a different allocation: keep routed experts in CPU memory and execute their computation on the CPU, while GPU-resident components handle attention and other compute-intensive work. The SGLang integration can also keep selected hot experts on GPUs.
That is not ordinary layer offloading.…