Language models can rewrite their own weights
A new research paper shows that a large language model can directly edit its own internal weights using reasoning alone.
2 min read, Thursday, 19 November

Language models are typically trained once and then frozen, meaning their internal values—known as weights—remain static after initial creation. Improving these models usually requires a completely new training run, careful prompt engineering, or extra computation during use that disappears as soon as the task finishes.
Researchers have now demonstrated a third approach called self-rewriting, where a model acts as its own optimizer. Instead of relying on traditional gradient descent — the mathematical process of adjusting neural network weights to reduce error — an open 27-billion-parameter model examines its own weights, analyzes its graded answers, and diagnoses its failures to rewrite the underlying parameters directly.
Key facts
- Model Size
- 27B
- Authors
- Jiajun Fan and 10 other authors
- Accuracy Improvement
- 6.6 points more accurate
How self-rewriting works
The process relies on letting the model run itself, test its edits on practice problems, and submit only the changes that work. In tests, this self-directed editing made the model 6.6 percentage points more accurate on unseen problems. Over eight consecutive sessions, its accuracy steadily increased with each edit.
Researchers also tested the method on a damaged copy of the model. Even though the damaged copy solved almost nothing, the healthy model managed to restore it to over 80 percent of its original accuracy on image tasks it had never encountered, all without reading the original weights or undergoing retraining.
Scale and implications
This self-improvement capability appears to be an emergent property of scale. Within the tested model family, smaller versions either broke themselves or made no edits at all, and only the largest model successfully improved and repaired itself.
Unlike test-time scaling, which requires additional computational effort for every query, self-rewriting bakes the performance gains directly into the model weights at no extra cost per answer. Because it relies entirely on forward passes — the standard computational step used when generating a response — it avoids the resource-heavy requirements of standard model training.
This blurring of the line between training and inference suggests that intelligent systems might continuously upgrade themselves over time, operating much like humans who learn and adapt without needing to undergo an entirely separate education phase.
Source: Original announcement
Worth a read?



Comments