METR Anthropic report: what it found about AI speedup

METR's report says Anthropic is now improving its models on well-specified tasks at roughly twice the pace it managed between March 2024 and October 2025, and METR's best guess is that AI itself accounts for about 80% of that speedup. The catch: Anthropic redacted most of the evidence METR found most convincing, so the public version asks you to trust the evaluator's judgment.

The posts that started it

On Thursday, @METR_Evals announced that a team had spent months working with internal Anthropic information to test whether AI is accelerating AI development. In a follow-up post, METR said it handed Anthropic a roughly 100-page report, and that Anthropic redacted a majority of the most informative evidence before publication. Its own summary: "the public must rely on our judgment more than we consider ideal."

Forty minutes later, Anthropic co-founder @Sam_McCandlish, who leads much of the company's safety work, quote-posted the announcement. He said many of METR's conclusions match his own: AI is speeding up Anthropic's research, but it is hard to pin down by how much. On the redactions, he accepted that they make METR's work harder to check. He said that sharing what drives the speedup is commercially sensitive and "risks handing out a roadmap for others to go faster." He also said Anthropic will publish its own analysis in the coming weeks, including where it stands against its Responsible Scaling Policy.

In September I wrote that Dario Amodei's pace-the-frontier plan would only mean something once an outside group published findings the company would rather keep quiet. McCandlish says this project started before that plan and runs separately from the embedded evaluators, but it is the first real test of the idea anyway.

What the report found

METR's full write-up covers an engagement that ran from August 12 to October 29. METR worked on Anthropic devices. It saw internal benchmark scores, including scores for unreleased models, plus headcount and compute figures, internal surveys, and about 15 interviews. It did not get unmediated, employee-style access, and it did not sample from non-public models itself.

The 1.9x is higher than the 1.7x in the Claude Haiku 5.5 system card, mainly because METR added data on newer unreleased models. METR also says the extra speed did not come from scaling up training compute. Fully autonomous AI research is not a big contributor yet, though researchers described "signs of life." One more detail: the newest internal model scores above 50% on every benchmark METR used, and METR says that makes it nervous about whether it can keep measuring progress at all.

A companion review of Anthropic's August risk report pushes back on one claim. Anthropic's public report said acceleration was under a factor of two. METR says the private figures came out at 1.97, and that it is plausible Anthropic had already crossed its 2x early-warning threshold by July 15. METR agrees with Anthropic that catastrophic risk from those models was low.

How to read it without the hype

Some headlines claimed superintelligence could be six months away. The report doesn't say that. If you only check a few things, check these:

• The 1.9x figure measures benchmark scores, not real-world usefulness. METR itself warns the two may not line up cleanly.
• Add other labs' public models and the number falls to about 1.4x. METR presents arguments for both readings.
• The 12x researcher figure is self-reported Slack poll data. METR treats it as weak.
• METR says it is "extremely unlikely" Anthropic was trying to mislead it, but you cannot check the redacted parts.

My read: the redactions are a real problem, and McCandlish's reason for them is also a real reason. I don't think either side gets to wave the other away. What counts in Anthropic's favor is that an outside group said in public that the company's own risk report understated the acceleration, and that finding went out.

What it means for a business using AI

If the speedup holds, the models you build on will change faster than your yearly software review assumes. Prompts and automations tuned for this year's model may get beaten by next quarter's. Rerun your main workflows on each new release instead of locking a setup in place, and keep a short list of test cases so you can tell quickly whether a change helps.

That kind of setup is what New Face Design's free process audit looks at: which work is worth automating now, and how to build it so a better model can be swapped in.