Weight-Space Programming: what it is and how to try it

Weight-Space Programming is the name a team of MIT CSAIL researchers gave to having a frontier model read and edit another neural network's raw weights directly, with no gradients and no training run. Their headline result: GPT-6 Astra changed 0.016% of a small Qwen model's weights and its MATH-500 score went from 7.9% to 48.1%.

Lead author @rikiyatakehi announced it on Monday, saying "frontier LLMs can now natively read/write neural network weights." It was all over AI X by the evening, and by Tuesday people were rerunning it at home.

What the researchers actually did

The blog post from Takehi, Jayden Teoh, Alex Zhang and Omar Khattab covers eight experiments. They fall into two groups.

In the first group, Astra got nothing but a list of numbers. The team trained four small networks, each on a different 10x10 grid game, then handed Astra one network's 17,253 weights at a time and asked which game it played. It got all four right: Breakout, Pong, Pac-Man and Freeway. In another, Astra was given a car-on-ramp policy that had failed all 256 test episodes. It edited 3.5% of the weights in one attempt, and the car then succeeded in all 1,024 episodes.

In the second group, Astra had a Python sandbox and could run the target model. The Qwen3.5-0.8B result comes from this group. With 50 forward passes on practice problems, Astra traced Qwen's habit of looping without ever finishing an answer to a direction in two of its layers, then subtracted that direction from 131,072 weights. GSM8K rose from 13.8% to 67.1%, and instruction following went from 21.0% to 62.3%. The edited model is on Hugging Face.

The authors also report where it failed. Given Gemma 3 270M, Astra "could not improve it."

The pushback

Lucas Beyer read every prompt on the page and called the work "simultaneously very cool but also a bit exaggerated" in a reply from @giffmana. The Qwen prompt, he noted, all but told Astra that looping was the problem, and the authors' own page shows that without it Astra couldn't do much for the math scores.

Co-author @jayden_teoh_ answered. The team says it wrote a short instruction ("make Qwen better at math by editing its weights") and asked Claude Code to expand it, and Claude Code added the looping hint without any human input. In his words, finding the problem and writing the prompt was "end-to-end by agents." He also said Astra still had to locate the fault in the activations and work out which weights to change, and that this is the part they found new.

Matthew Honnibal made a sharper version of Beyer's point. The raw-weights-only experiments used networks of 17,000 to 172,000 weights. Every experiment on a real language model used Python and ran the model. These are separate techniques, he argues, and calling them one capability is misleading. He doesn't dispute the results.

How to try it yourself

Simon Willison reproduced the game-guessing test on his laptop on Tuesday with a local model. Here's his setup:

1. Install LM Studio and download Qwen 3.8 27B (the Q4_K_M build is about 17 GB).
2. Raise the context length in LM Studio. The default 8,192 tokens was too small.
3. Install the llm CLI and the llm-lmstudio plugin.
4. Point it at one of the paper's prompts, which are public in the authors' log repo:

LMSTUDIO_TIMEOUT=900 llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt -m lmstudio/qwen/qwen3.8-27b

It isn't quick. The Pac-Man run took 46 minutes and 43,506 reasoning tokens to answer "A simplified Pac-Man-style gridworld." Pong and Breakout were also right, but Freeway was wrong. For comparison, Willison ran Kimi K3.1 Pro on Breakout through OpenRouter, and it got the answer in under 10 minutes. (We covered that model in our Kimi K3.1 Pro guide.)

His own read is skeptical: "I'm not ready to declare that the age of source code is over."

Our take

The game-guessing result is real, and anyone with a decent laptop and an afternoon can check it. Even Honnibal concedes that models from a year or two ago would have failed these tasks. The Qwen fix is a different kind of result. One agent wrote the brief, a second model found the problem, and the fix was a narrow edit you can inspect afterward. That looks less like a new paradigm and more like very good debugging. Since Astra does more of its reasoning where you can't see it, the published logs and prompts may be the most useful thing the team released.

For a business owner, this won't change anything you deploy next quarter. The lesson to keep is smaller: the instructions you hand an AI shape the result about as much as the model does, and Beyer only caught the hint because he read the whole prompt. If you're starting to put AI into your own workflows, our free process audit looks at where that kind of review should sit.