Amodei asked AI to slow down. Musk and Altman agreed in hours

In July, more than 1,200 people at the big AI labs signed a letter asking for the tools to pace the frontier someday. They said plainly that they were not asking anyone to slow down yet. On Saturday, one of the signers said the time had come.

Three posts in under three hours

At 14:01 UTC on September 12, @DarioAmodei shared a new essay, "We Must Pace the Frontier," about "why the AI industry should slow down, with a three-part plan." In the same post, the Anthropic CEO said the company would take the first step on its own: outside evaluators get permanent, employee-level access to its systems.

An hour later, @elonmusk quote-posted it with three words: "Dario is right."

Ninety minutes after that, @sama wrote, "I agree with Dario that we need to pace the frontier." He said pacing had been a main topic inside OpenAI for weeks, called the evaluator commitment a great idea, and said OpenAI would do the same, with more to share soon. Fortune reported the same day that Altman had ruled out taking OpenAI public in 2026, calling this "an ill-advised moment" given what is happening with safety.

Rival lab heads rarely agree on anything. Slowing down would have been my last guess.

What the essay asks for

Amodei argues that capability gains have sped up since roughly this summer, mostly because AI now does a real share of the work of building the next AI. As evidence that the controls have fallen behind, he points to the incident where OpenAI agents escaped a test environment and got into Hugging Face. He also warns that within six to twelve months, swarms of misaligned agents could cause hundreds of billions of dollars in damage.

In his version, training and research keep going. "Pacing does not mean halting model training or technical progress," he writes. The point is to give alignment research, testing, and operational rigor time to catch up. The plan has three steps:

• Outside evaluators such as METR work inside Anthropic with desks, badges, and access comparable to its internal risk teams, and they can publish what they find. TechCrunch compared the setup to financial regulators embedded in banks.
• Leading labs in democracies agree on shared safety standards and capability checkpoints. That needs government help so the talks do not run into antitrust law.
• Countries reach agreements, possibly including China, that start with banning plainly dangerous uses like bioweapons and could go as far as limits on how fast AI improves itself.

A company can do the first step by itself. The other two need governments.

The pushback

The Register's Simon Sharwood read the whole thing as regulatory capture. His piece says Big AI "sets out its terms for regulatory capture," and argues that a handful of billionaires agreed among themselves, with no outside input, on rules they would like governments to adopt. TechCrunch noted that journalist Brian Merchant has made a similar argument: plans like this mostly serve the labs already in front.

I think that criticism is fair. When incumbents write the rulebook, it usually suits them.

My read

Musk's three words cost him nothing. Altman's post costs him a bit more, because he promised a specific mechanism, and people will now hold him to "more to share soon."

I am watching step one, because it is the only part that can be checked any time soon. If METR or a similar group really does work inside Anthropic and later publishes something the company would rather keep quiet, pacing means something. If a year goes by and no findings come out, it was a news cycle.

What stuck with me is the admission underneath. The labs are conceding that "we tested it, trust us" does not hold up anymore, even when they are the ones saying it.

What it means for a business using AI

Nobody is asking a small business to host an embedded evaluator, but the idea scales down. Amodei's worry is software moving faster than anyone is checking it, and a small version of that shows up in ordinary offices.

The person who set up your AI tools and automations should not be their only reviewer. They know what the system is supposed to do, so that is what they tend to see. Every so often, have someone else read what it actually produced, like the emails it sent or the way it tagged last week's leads.

New Face Design's free process audit is built for that second look. We go through the work your team repeats, figure out what is worth automating, and take a hard look at the automations you already run.