1,393 AI agents cleaned a codebase for $19K. Tests missed the bugs

Nous Research had a million lines of Python that needed cleaning up, so it handed the job to its own AI agent. On Tuesday the company summed up the result in a post from @NousResearch: "1,393 subagents and nineteen hours later, the codebase was 34.4% smaller." The same post says the run saved "nearly $2m in engineering hours."

It's the most concrete example I've seen this year of AI chewing through the kind of backlog nobody wants to touch. The fine print is worth a read, though.

What Nous did

On September 2, Teknium at Nous gave Hermes Agent, the company's open-source agent, one goal: simplify the repository. Hermes broke the work into pieces and farmed them out to subagents, with as many as 218 running at once. The Nous write-up says a single Python process on an i7 desktop with 64 GB of RAM coordinated all of it, while the model work ran on Claude Fable 5.1.

Here's what the write-up reports:

• Non-test Python went from 1,063,826 lines to 698,363.
• Files over 5,000 lines dropped from 37 to 6.
• Functions over 300 lines dropped from 192 to 2.
• The main run cost about $19,300, and about $25,000 counting follow-up sessions.

Nous puts the cost of having people do this at $150,000 to $1.8 million, depending on whether a small team takes two months or two years. So the "nearly $2m" in the post comes from the top of that range. I'd use the bottom. At $150,000, the agents were still about six times cheaper.

The part the headline skips

Nous set up guardrails first. Workers had to check specific interfaces against the original code: a tool's JSON schema had to stay identical, and a command's help text could be compared byte for byte. Each verified step got its own commit.

Things broke anyway. Workers deleted public names because nothing inside the repo called them, but outside plugins could still import them. An automated rewrite also changed exception handling in roughly 65 places. Nous calls these "real regressions the existing tests had missed." Two rounds of community review caught them before the merge, and more fixes landed after it.

The run had its own problems too. In a post-mortem on GitHub, Teknium describes an expired authentication token that crashed the main process and left 196 child agents orphaned. Dead subagents also sat unnoticed for 66 minutes.

Levie's "100X" and Pocock's fix

Box CEO Aaron Levie posted about where this is heading earlier that day. @levie argues that the work agents do for us in the background will be "100X more volume" than anything we'd prompt in a single chat session. One of his examples is agents reviewing every line of code for bugs and security holes.

The Nous run looks like an early version of that. Nobody would clean up a million lines one prompt at a time. Writing the changes turned out to be cheap, though. Checking them was the hard part.

Matt Pocock had a practical answer the same day. @mattpocockuk said his /retro command will now hunt for ways to turn "fuzzy rules into deterministic checks." It proposes lint rules, pre-commit hooks, or CI workflows, so a mistake you've found once can't come back. Nous seems to have landed in the same place: its list of fixes after the run includes a check on the public API surface, which is what the workers broke.

My read

The savings are real. The part I'd copy is that Nous had defined "unchanged" before the agents started. It had schemas that had to match, and it checked test failures against a frozen baseline. I think that's why the damage stayed small enough for reviewers to catch. Even so, those 65 error-handling changes got past the tests, and it took community review to find them.

What this means for your business

Most owners don't have a million-line codebase. Plenty have a similar pile, though, like a CRM full of duplicate contacts or years of spreadsheets nobody trusts. Agents can now work through that kind of volume for very little money. Before you hand it over, decide what must not change and how software will check it. Then have a person look at whatever the checks can't see.

New Face Design's free process audit starts there. We find the backlog work worth handing to agents and the checks that need to exist before they start. Start here.