Claude weight edit lifts Qwen’s math score to 58.1% in follow-up study
Edited and reviewed by AI Ping Editorial Team
3-line summary
- Claude Opus 5.5 edited Qwen3.5-0.8B’s weights and raised its MATH-500 score from 6.7% to as high as 58.1%.
- Fewer responses hit the same output limit after the edit. Claude changed how the smaller model chose its next words and when to end its reasoning.
- The same study found no improvement from editing Gemma 3, which rarely repeated itself under the test conditions.
Claude Opus 5.5 raised a small Qwen model’s score on a math test from 6.7% to as high as 58.1% by editing its weights, according to a preprint listed on arXiv on November 17. Subham Kundu of TCG Crest tested Qwen3.5-0.8B, the same Alibaba model that MIT researchers had edited with GPT-6 Astra earlier in the month. Kundu added the Qwen experiment to his public research notes on November 16.
Weights are the numbers a neural network uses to turn an input into an output. Both studies changed some of those numbers directly and then evaluated the edited models. A math score alone leaves open several possible explanations for a failed answer. The new study reported how often responses ran out of room, how often the model reached a final answer, and how those measures changed after the edit.
More answers reached the finish
The evaluation produced four responses for each of the 500 questions in MATH-500. Before editing, 93.3% of the responses hit the 32,768-token limit. Every answer that finished was correct. After the edit in the run with the largest practice allowance, 32.7% hit that limit. Under the settings used, the original score reflected how rarely the model stopped in time.
Claude could test changes on separate practice questions. Kundu ran the editing experiment once at each of three practice allowances, starting from the original weights every time. The largest allowance covered 50 batches of 64 questions. The three edits scored 51.4%, 55.2% and 58.1%. Each score evaluates one edit.
In MIT’s earlier study, Astra weakened a signal inside Qwen that was associated with repetitive reasoning. Under that study’s test, the score went from 7.9% to 48.1%. Claude changed how Qwen selected its next words, including the likelihood of ending its reasoning. The follow-up retained the earlier evaluation and sampling settings and used different random seeds.
In a separate control, changing how widely Qwen sampled its next words accounted for part of the gain. That scored 20.9% to 22.1%. Qwen’s own documentation describes how choices about sampling and output length can affect repetition and benchmark results. It lists a sampling temperature of 1.0 for reasoning on text tasks. Both studies used 0.6 in this comparison. Temperature adjusts how strongly the model favors its most likely next words. Qwen also suggests allowing up to 81,920 tokens on particularly difficult math and coding benchmarks.
In a separate experiment published November 9, Nathan Barry took the internal signal Astra had found and used mathematical optimization to choose which weights to change. In his evaluation, the original model scored 7.0%, Astra’s edit 50.9%, and the optimization edit 51.7%. Barry’s experiment addresses the same repetition problem under its own evaluation conditions.
Writing a game player and reading its rules
A separate team at the Alan Turing Institute tested whether a language model could supply the weights for a small network from scratch. In a November 16 preprint based on the institute’s earlier report, three networks written by GPT-6 Astra scored 514, 550 and 307 in a version of Breakout. The earlier MIT study reported 517 in one run. The scores came from the small networks that Astra wrote, which then played the game.
Two models also reconstructed code that matched the chosen action of one such network in all 49,189 game states checked. The code tracked the ball and moved the paddle toward its expected landing point. On handwritten digits, however, the networks Astra wrote averaged 49.6% accuracy. A comparison method built from the average image of each digit scored 82.0%.
Reading existing weights was another test. Both language models named the right game for each of twelve networks trained on familiar games. On three other game setups the Turing team had put together, Astra recognized parts of the board layout but usually missed what the controls actually did. The researchers could also see some of those positions when they plotted the weights of the first layer.
Kundu also edited Google’s Gemma 3 270M, which rarely repeated itself under the comparison conditions. Its measured score declined by 0.8 percentage points, within an uncertainty interval compatible with no change. The Claude experiment found a measurable improvement on the model whose responses often ran out before giving an answer.
Sources
- 클로드 가중치 편집 재현 논문 · Same Fix, Different Weights — Subham Kundu · arXiv
- 연구 기록과 큐원 실험 · Can Claude Read Neural Network Weights? — Subham Kundu
- 아스트라의 초기 가중치 실험 · Sparks of Weight-Space Programming — MIT CSAIL researchers
- 큐원3.5-0.8B 모델 카드 · Qwen3.5-0.8B model card — Qwen Team
- 별도 선형 최적화 실험 · Can a Linear Program Do Weight-Space Programming? — Nathan Barry
- 게임과 손글씨 가중치 실험 · Replicating and Extending Weight-Space Programming — The Alan Turing Institute researchers · arXiv
- 신경망 가중치 읽기·쓰기 연구소 공개본 · Institute working note on reading and writing weights — The Alan Turing Institute
