Menu

Second-gen Optane PMem server running Kimi K3 (2.8T params) on CPU only, plus a critical thermal warning that could save your RAM
📰

Second-gen Optane PMem server running Kimi K3 (2.8T params) on CPU only, plus a critical thermal warning that could save your RAM

#homelab#reddithomelab#optane#dram#model
Reading 0:00
15s threshold

Hey everyone! Some of you may remember u/APFrisco 's post about running a 1T-parameter model (Kimi K2.5) at around 4 tok/s on a first-gen Optane server with 768GB of PMem 100 series in Memory Mode, it was even covered by Tom's Hardware. That post was a big inspiration for me, and I've now built a second-generation version using Optane Persistent Memory 200 series ("Barlow Pass") on a 3rd-gen Xeon Scalable platform with 40 core CPU.

The most important part of this post: Optane DIMMs run HOT, and the heat will kill your DRAM

This is the thing almost nobody talks about, and it's an extremely relevant fact for anyone considering a build like this: under sustained LLM inference (constant heavy reads across every module), Optane DCPMMs get extremely hot. This isn't speculation.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.