Menu

Weight-space programming needs triage first, so I built it
📰

Weight-space programming needs triage first, so I built it

DEV Community: machinelearning·Ayush Shekhar·1 day ago
#JqIw8ewI
Long article (682 words) — no AI summary yet.
Reading 0:00
15s threshold

This month MIT CSAIL researchers showed that an LLM can read and edit another model's weights directly to fix a behaviour ( Takehi, Teoh, Zhang and Khattab, "Sparks of Weight-Space Programming" ). It's a cool idea, and I've been building on it.

But the debate after the post was mostly not about weights. It was about the baseline: a critique argued much of the gain came from answers looping until the token cap with a temperature other than the model card's, and the HN thread found a cheaper fix that did as well.

That matched my own experience. Every time a local model gave me a blank reply or looped on the same line, my first guess was "the model is bad". Most times I was wrong. It was the chat template, a sampler setting, a context that got cut off, or one broken tensor in a quantised file.

So I built wsr (Weight-Space Repair): triage before surgery. You give it the failing request and reply, and it tells you why it failed and how to fix it.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.