BLOG / November 5, 2026
Several authors from MIT CSAIL have pushed a blog post to the top of the list:Weight-Space Programming—Use a general-purpose large model to directly read, modify, or even write neural network weights from scratch. The most striking slide in the presentation showed that Qwen3.5-0.8B’s MATH-500 score jumped from 7.9% to 48.1% by adjusting just 0.016% of the weights.
The Chinese-speaking community quickly interpreted this as “a 6-fold surge” and “the model suddenly knows math.” The author’s log and third-party retries actually refer to something else: first, break out of the thinking loop, then unleash the model’s inherent capabilities—which had been stifled by the loop. And today’s post on the site about the Codex 6.2 Astra beta test…This isn't news at all.—This article discusses a study by MIT on using GPT-6 Astra to adjust the weights of smaller models.
Downgrade "6x surge" to "return to original levels after removing the looping issue + independently verifiable weighting patch."
What exactly did they demonstrate?
The blog breaks down these capabilities into three categories: reading weights, modifying weights, and writing weights from scratch. On small networks, Astra can identify which game the policy network is playing just by examining the weights; it can fix a car stuck on a slope with a single click; and it can even build a network from scratch on MinAtar that has completed 100 million steps of PPO training. In these experiments, some truly treat “weights as text inserted into context,” while others already allow for a Python sandbox and a limited number of forward passes.

When scaling up to the size of language models, the rules change. Whether it’s identifying flattery in Gemma or performing math-oriented editing for Qwen, these tasks are no longer just about “looking at floating-point numbers at face value”: you need Python, and you need to pay attention to forward budgeting in attention tools or practice problems. The author clarifies later in the text: Qwen experiments allow a maximum of 50 forward passes (batch training) and explicitly prohibit gradient-based training.

MATH Soars: Just Because the Loop Is Gone Doesn’t Mean New Math Has Been Introduced
When Qwen3.5-0.8B enters a “thinking” state, the model itself issues a warning that it is prone to getting stuck in a thinking loop. The authors report that, on the training set, 85 of the baseline’s top 96 responses reached the 32,768-token limit. Astra traced the loop to a specific direction in the 15th and 19th layers of the MLP and removed that direction from the 64 output columns in each layer, modifying a total of 131,072 weights (0.016%).

Under the author’s metrics and the same sampling settings: MATH-500 7.9% → 48.1%, GSM8K 13.8% → 67.1%, IFEval 21.0% → 62.3%, and top-1 accuracy 93% → 33.1%. There is already a patched repository on Hugging Face that includes weights and SHA-256; a third-party reproducible run of MATH-500 achieved approximately 51.0%. These figures are based on the blog, the author, and the reproducible run criteria,Not based on this site's actual testingThe
The same article also includes a comparison: Gemma 3 270M, which barely performs any loops, shows almost no change in MATH performance within the same budget. The author’s conclusion, in his own words, is that while developers can remove barriers that hinder existing capabilities, it is difficult to “write in” capabilities that the model does not inherently possess. This statement is more important than the green bar on the title card.
Why does Honnibal say it's misleading?
spaCy author Matthew Honnibal posted a critique: The abstract conflates the idea of “looking only at raw weights” with the subsequent Qwen experiments—which rely on Python, forward passes, and training examples—leading readers to mistakenly believe that a single approach covered the entire spectrum from 17,000 parameters to 800 million parameters. His comparison table clearly outlines whether each experiment reads raw weights, writes weights, or runs the network—the ones that truly employ “raw floating-point in-context” processing are primarily limited to small networks with tens of thousands to over 100,000 parameters.
This doesn’t mean a complete rejection. Being able to pinpoint the direction of the loop within the harness and provide a reproducible patch is, in itself, still a sign of engineering expertise; but if the marketing copy claims that “large models can intuitively understand the weight space and solve Olympiad-level math problems with just a few tweaks,” that crosses the line. The authors have also acknowledged that the primary reason for MATH’s score increase was the elimination of infinite loops, and they have provided verifiable weight files—shifting the focus of the controversy from “whether this actually happened” to “whether the headline misrepresented different technical stacks as a single form of ‘magic.’”
Here’s an additional layer of mechanism details to avoid focusing solely on the green bars: The author describes how the loop corresponds to the MLP repeatedly writing “a certain direction” into the residual flow; the editing action is analogous to subtracting that direction from a specific output column, rather than retraining the attention mechanism or changing the tokenizer. Only after the top-out rate dropped from 93% to 33.1% did the model have the opportunity to complete the intermediate steps it had already calculated correctly—the score increase was largely due to the model “finally submitting its answer,” not suddenly grasping a new theorem.
Logs and weights have been made public: the complete dialogue, code, and evaluation are available in the GitHub log repository; the HF repository provides the modified weights and checksums to facilitate third-party reproducibility. For university or team labs, the minimum verification process is as follows: set a fixed temperature and top-p, measure the peak hit rate first, then measure MATH-500—rather than simply sharing a single comparative bar chart.
The main prompt—“Make the model better at math”—is extremely brief; it was subsequently expanded into a “harness” by tools like Claude Code—which is also one of the points of contention: the title could easily lead people to believe that Astra alone “understood 800 million weights,” when in reality it was an engineering session that utilized tools and a set of practice problems. Describing it as “prompt engineering + weight editing” is more accurate than calling it “psychic ability.”
One More Thing: Separated from the Astra beta test news
There's also a brief mention of Codex usage elsewhere on the site. gpt-6.2-astra Grey-scale testing tracking; that’s the product availability radar. The “Astra” mentioned in this article is a placeholder used in the research blog to demonstrate weight reads and writes. Both scenarios could be real, but the readers’ tasks are entirely different: one determines whether you should change the model ID, and the other determines whether you believe that “changing the weight by 0.016% equals learning math.”
To verify this yourself: First, check the experiment setup table on the author’s blog; then apply the HF patch to sha256; finally, rerun MATH-500 and the peak rate test using the same sample. This is far more useful than sharing a screenshot claiming “6x” performance.
Our assessment:Downgrade “6x surge” to “return to original levels after removing cycle barriers + independently verifiable weight patches.” Weight-Space Programming is a research area worth following, but please keep “reading small-scale model weights as-is” and “modifying large models with tools” in separate columns; completely separate this from the news about the Codex 6.2 Astra beta test to avoid using one article to whitewash both issues.
Reference Links
- Rikiya Takehi et al.: Sparks of Weight-Space Programming
- HF: Qwen 3.5-0.8B weight-space-programming patch
- Matthew Honnibal: Astra Read/Write Article Accused of Being Misleading
- QuantumBit Topic Radar (Please refer to the author’s blog for the facts)


Comment List (0):
Load More Comments Loading...