GPT-6 Astra Read Another Model’s Raw Weights and Fixed It by Hand.

On 2 November 2026, 4 researchers at MIT CSAIL published a result that sounds impossible at first. They handed a frontier AI model, GPT-6 Astra, the raw numerical weights of a different neural network, as plain text, and asked it to make that network better at math by editing the numbers directly. No training. No gradient descent. Just reasoning about what the numbers do and changing them. The edited model’s score on a math benchmark went from 7.9% to 48.1%. I want to walk through exactly what happened, because the headline is true and also easy to misread, and the authors themselves found the limit that keeps it honest.

What weight space programming actually means

Start with why this is surprising at all. A neural network is, underneath, a giant list of numbers called weights. Training sets those numbers by running data through the network millions of times and nudging them with an optimizer. A human cannot look at a list of a few hundred thousand numbers and say what the network does, and until now neither could a language model. The weights were the one part of machine learning that stayed mathematical, something you optimized with algorithms rather than described in words.

The MIT team, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, call their finding weight space programming. A general purpose model reads and writes a network’s weights directly, around a goal written in plain English, without an optimizer and sometimes with no feedback from running the network at all. The model reasons about what each part of the network computes, or should compute, and sets the numbers accordingly.

Reading weights it was never trained to read

Before the math result, the simpler experiments are the ones that made me sit up. The team trained 4 small game playing networks, on Breakout, Pong, Pac-Man, and Freeway, all with identical dimensions so the only way to tell them apart is to read the weights. They gave Astra the 17,253 raw numbers of one network, with no tools and no ability to run it, and asked which game it was trained on. Astra named Breakout, correctly, and got all 4 networks right.

Its reasoning was concrete. It noticed the agent’s strongest first layer weights sat along the bottom row of the playfield, which is where a paddle lives, and that a band of reward bearing objects sat above the paddle, which is what bricks look like. It worked out the game from the geometry of the numbers, the way you might guess a machine’s purpose from the shape of its gears.

One experiment made the mechanism unusually clear. The team trained small diffusion models to generate circles, squares, and triangles, then asked Astra which shape each one draws, from the weights alone, with no forward passes allowed. It got them right by finding the shape in the output layer, where each hidden unit draws one piece of the edge. The telling part is a mistake it made. Astra drew every shape as an outline, even though the real models generate filled shapes. A model that was secretly running the network would have known they were filled. Drawing the outline shows it was reading the geometry of the weights, not computing what the network would output.

It went further in later experiments. Given only the middle layers of a small language model, with the input and output stripped off so it could not cheat by reading which character is which, Astra still named whether the network was trained on Python, Japanese romaji, or English, and correctly picked out the specific attention heads that copy earlier text. That copying head analysis matched a known interpretability method from a 2021 Anthropic paper that the researchers never mentioned to Astra. It even identified which of 2 near identical models had been trained to be sycophantic, by watching where the attention heads looked, finding that the sycophantic model’s middle layer attention pointed at the user’s suggested answer instead of the actual question.

The math result, and how it actually worked

Now the headline. The team gave Astra the weights of Qwen3.5–0.8B, a small language model, loaded in a Python notebook, plus a budget of 50 forward passes on practice math problems to see how the model behaves. It was not allowed to train the model or use gradients.

What Astra found was a specific, diagnosable flaw. Small reasoning models tend to loop, repeating the same text until they hit their length limit without ever reaching an answer. Qwen did this badly, 93% of its answers on the math test ran all the way to the 32,768 token limit without finishing. Astra traced that looping habit to a single internal signal written by the MLP parts of 2 layers, layers 15 and 19, and subtracted that one direction out of 131,072 of the model’s weights.

That edit touched 0.016% of the model’s 0.8 billion weights. With the loop removed, the share of answers that ran out of length fell from 93% to 33%, and the benchmark scores moved a lot. MATH-500 went from 7.9% to 48.1%. GSM8K went from 13.8% to 67.1%. Instruction following, measured by IFEval, went from 21.0% to 62.3%, even though Astra was only asked about math. One targeted change, no training, big jump.

The budget mattered. The paper tested 10, 25, and 50 forward passes, which is how many times Astra was allowed to run the model on practice problems to see its behavior. At 10 passes MATH-500 reached only 12.3%, and at 25 it was 14.1%. The large jump to 48.1% came only at 50 passes, which is also when the looping finally dropped off sharply. So the fix took a real diagnostic loop, looking at behavior, forming a hypothesis, editing, and checking, run enough times to land it, rather than a single clever glance at the weights.

Qwen3.5–0.8B before and after Astra’s edit, across 3 benchmarks. The edit changed 0.016% of the weights and ran no training. Source: Takehi et al, MIT CSAIL, Nov 2026

The catch the authors put right in the paper

This is where an honest reading departs from the breathless one. The edit did not make Qwen smarter at math. It removed a loop that was stopping Qwen from finishing problems it could already solve. The ability was there the whole time, buried under a behavior that wasted the model’s output budget before it reached an answer. Astra cleared the obstacle, and the latent ability showed up on the scoreboard.

The edit lifted Qwen, which loops badly, and did nothing for Gemma, which rarely loops. The gain came from clearing an obstacle that was hiding ability the model already had. Source: Takehi et al, MIT CSAIL, Nov 2026

The researchers proved this themselves with a second model. They tried the same thing on Gemma 3 270M, a model that rarely loops, only 8.5% of its answers hit the length limit. Astra could not improve it. When it read Gemma’s failures, it found ordinary reasoning mistakes, wrong arithmetic and bad algebra, with no single component behind them to switch off. Its edits moved the math score by about a point, a gain it honestly reported as not established. The paper states the lesson plainly, that editing weights could remove an obstacle hiding an ability, but it could not add an ability the model lacked.

That distinction is the whole story. The 7.9% to 48.1% jump describes an AI finding and deleting a bug in another AI’s weights, a bug that happened to be hiding a lot of capability. It did not install math skill into another AI by rewriting its brain. Impressive, real, and much more specific than the headline number suggests.

Writing networks from nothing

Reading and fixing weights is one thing. The team also showed Astra writing weights from scratch. Given the rules and source code of 3 small games but no training data, no gradients, and no ability to play the game first, Astra wrote the weights of a game playing network directly, as text, choosing every number itself. Its Breakout network scored 517.4 against 15.8 for a standard reinforcement learning network trained for 100 million steps, and 25.3 for one trained for 500 million steps.

Astra’s hand written Breakout network against reinforcement learning networks trained for millions of steps. It wrote the weights directly, with no training. Source: Takehi et al, MIT CSAIL, Nov 2026

At a larger scale, 1.5 million parameters, too big to fit in its context, Astra wrote weights for a network that plays Go. It built the network as a small Go program in Python, layer by layer, marking each side’s stones, tracing chains that were in danger, and scoring each move. Even with 0 practice games, writing all 336 lines in a single run, its network won more than half its games against 2 smaller trained networks. With a budget of 50 practice games to learn from, it beat an equally sized reinforcement learning network trained on 20 million games. The way it improved with practice reads like debugging, it played games, looked at how it lost, changed the part of the network behind the loss, and kept a change only when it won more.

What makes all of this new is narrow and specific. Interpretability researchers have read network internals with purpose built tools for years, and other work has generated weights using models trained specifically for that job, or found weights by gradient free trial and error. Here, a general purpose frontier model, used exactly as it ships, reads and writes the weights itself, from 17,253 numbers given as text up to a language model of 0.8 billion weights with the help of a Python sandbox. Nobody trained Astra to do any of this.

Why this matters, and why it is still early

The authors are careful to call these sparks, early results that are exciting but limited by context length and tooling. I think that framing is right, and the direction is the interesting part. For years we optimized the weights of neural networks only with small symbolic optimizers like Adam. This points at a different future, where a reasoning model inspects weights, explains what they do, and edits them around a goal stated in English.

If that develops, a few things change. You could audit a model for hidden backdoors or capabilities by reading its weights, without running it or building a test for the exact thing you are worried about. You could add or remove a capability by editing a few weights instead of collecting data and retraining, which touches so little of the model that it avoids the forgetting that fine tuning causes. A model could inspect and debug its own weights, or those of a newer model, which is a more direct version of self improvement than writing training code. None of that is here yet. What is here is a frontier model that can look at a list of numbers that defeats every human, tell you what it does, and change it on purpose. That was supposed to be impossible, and a month ago I would have said so.