MIT Researchers Have GPT-6 Astra Read and Write Neural Network Weights Directly: Editing 0.016% of Qwen3.5-0.8B's Weights Lifts MATH-500 From 7.9% to 48.1%, While Critics Say a Skilled Researcher Could Do the Same

What the study did

Neural network weights are normally written by an optimizer, and it is hard for a person to look at a list of floating-point numbers and tell what a network does, let alone edit it by hand. In "Sparks of Weight-Space Programming", a blog-format study published on November 2 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL, GPT-6 Astra reads, edits and writes weights directly across eight experiments, without gradient descent or training loops, an approach the authors call "weight-space programming". The prompts, reasoning summaries and code outputs for every experiment are public on GitHub.

Some representative results:

• Reading weights: four game policies trained with PPO for a billion steps share exactly the same architecture (500→32→32→5, 17,253 weights in all). With no tools and no running of the network, Astra named the games from the weights alone: Breakout, Pong, Pac-Man and Freeway. It could also tell from diffusion model weights whether a model drew circles, triangles or squares, though it drew them all as outlines while the actual models generate filled shapes, which the authors take as a sign it was reading the weights' structure rather than computing a forward pass in its head.
• Fixing weights: a PPO car policy trained for a billion steps never got over a ramp in 256 episodes. Given the weights, the scene and one failed trajectory, Astra edited 3.5% of the weights in a single reply with no tools, reaching a 100% success rate.
• Writing weights from scratch: given only MinAtar game rules and code, networks Astra wrote in one go beat PPO trained for 100 million steps in Breakout, Asterix and Seaquest; against PPO trained for 500 million steps, it scored 20 times as much in Breakout, came close in Asterix and reached about 60% in Seaquest.

The Qwen experiment

The result that drew the most attention involves Qwen3.5-0.8B. Astra got a Python environment with the weights loaded (no GPU) and practice problems from the training sets, with a budget of 10, 25 or 50 forward passes (each letting the model answer up to 512 problems), and no training or gradients allowed. It found the model often fell into reasoning loops: 85 of its first 96 practice answers ran to the 32,768-token limit, an issue Qwen's own model card warns about. Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each, changing 131,072 weights, about 0.016%.

With a budget of 50 forward passes, MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1% and the instruction-following benchmark IFEval from 21.0% to 62.3%, while the share of answers hitting the token limit fell from 93% to 33%. With only 10 or 25 forward passes the loops barely went away, and MATH-500 reached just 12.3% and 14.1%.

Where the dispute lies

The authors argue these abilities go beyond what we'd expect from humans or conventional algorithms, and they are building a benchmark and tooling. Criticism centers on the "superhuman" framing. Engineer Sean Goedecke wrote that identifying the games only requires working out which board positions the first layer attends to, and that the Qwen fix amounts to writing contrasting looping and non-looping texts, finding the weights that drive the difference and trimming them, in line with the steering and ablation methods the community already uses; a competent ML researcher could do the same. He compares direct weight editing to an LLM tweaking compiled binaries: a neat trick, while normal training remains the main way to update a model. According to Simon Willison, spaCy author Matthew Honnibal called the paper misleading for conflating two things: reading weights without tools and editing them with code.

Willison himself reproduced the no-tools reading task: Qwen 3.8 27B running on his laptop identified the Pac-Man network after 46 minutes and Pong after about an hour and five minutes. For readers, the study's value lies less in leaps like "AI can rewrite itself" and more in a set of checkable experiments and logs for interpretability and small-model repair; Qwen's gains came mainly from removing repetitive reasoning, not from rewriting the model's mathematical ability.