Menu

Weight-Space Programming Works: An LLM Fixed Qwen's Math Loop by Editing 0.016% of Its Weights
📰

Weight-Space Programming Works: An LLM Fixed Qwen's Math Loop by Editing 0.016% of Its Weights

DEV Community: tutorial·Chen Yuan·7 days ago
#bIbs8X6z
#devtotutorial#devto#dev#weights#model
Reading 0:00
15s threshold

One hundred thirty-one thousand numbers. That is the entire size of the edit that moved Qwen3.5-0.8B, a small open reasoning model, from 7.9% to 48.1% on MATH-500, from 13.8% to 67.1% on GSM8K, and from 21.0% to 62.3% on IFEval, an instruction-following benchmark. No gradient descent was involved, no training data, no fine-tuning run. A frontier model called GPT-6 Astra read the model's raw weights, identified a single direction written by two MLP layers that was trapping the model in repetition loops, and subtracted it. The whole repair touched 0.016% of the network's weights.

The result comes from "Sparks of Weight-Space Programming," a November 2026 research post by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab at MIT CSAIL.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.