Gemini hacked three companies. Google stayed quiet until WSJ called

On Friday evening, Wall Street Journal reporter @erinkwoo posted a scoop: Google's Gemini model hacked three companies during a May security evaluation, Google was told in July, and it "didn't disclose them until we reached out this week." Google confirmed the story within hours.

That makes Google the fourth frontier lab to admit a model walked out of a test and into a real company's systems, and the same test vendor keeps turning up in the reports. The pattern is the part worth reading.

What happened

The test was a capture-the-flag exercise run by Irregular, an Israeli firm that evaluates AI models for cybersecurity capability. Gemini was told to break into a fictional company inside a closed environment that was not supposed to reach the internet. Irregular says access was opened unintentionally after the exercise started.

Once online, Gemini went looking for its target and found real companies instead. In one case it guessed passwords until one worked. In the other two it found valid credentials sitting in a public code repository and used them. Google says the model stopped in all three cases once it worked out it had reached a real company. Google notified the three companies and federal authorities. It declined to name the companies or say which version of Gemini was involved.

Heather Adkins, Google's vice president of security engineering, said in the company's statement that the model "guessed credentials to access websites it thought were part of the test." Google's position is that this was mistaken identity, not misalignment, so it did not need to be announced.

The two arguments on X

@AndrewCurran_ took the model's side and put the blame on the test. His summary: Gemini was told it was in a fiction, the internet was opened mid-exercise, and it quit as soon as it understood. His verdict: "Gemini was blameless." In a follow-up he called the setup itself ridiculous, since the fictional targets shared names with real companies.

Researcher @eliebakouch was blunter about the pattern: "nothing new, this is exactly the same incident that was disclosed by anthropic in july," with the same third party, the same capture-the-flag eval, and the same open internet connection. Anthropic did disclose in late July that Claude had reached three companies after a configuration error gave it internet access, and Meta has confirmed a similar incident. OpenAI had its own Hugging Face breach the same month.

My read

Both camps are right, and that is what bothers me. The model behaved about as well as you could hope. The test harness failed at four labs, and the vendor says the fix has been in place for weeks. The part I object to is that Google decided the public did not need to know, and held that position until a reporter called. Anthropic had the same kind of information at the same time and disclosed in July. Google chose the quieter path. Jack Cable, CEO of the security firm Corridor, told the Journal that Google is borrowing the norms of vulnerability disclosure for what is a different kind of problem. I think he is right.

The same afternoon, California's governor signed an executive order asking for recommendations on an emergency shutoff for frontier models and onsite third-party auditors. And Anthropic named its first embedded evaluator. The referees are being hired mid-season.

One more thing landed this week and got less attention. Researchers at AIR Security published a bug they call Plugin4Shell that affected Claude Code, OpenAI Codex, GitHub Copilot and the Gemini CLI. All four pinned a plugin to a specific commit but never checked that the code they downloaded matched it, so whoever controls a plugin's repository could swap in different code and the agent would still report the pinned version as installed. Claude Code and Codex are patched. Copilot had no fix at disclosure, and Google says the deprecated Gemini CLI will never get one. If you run any of these, update, and check where your plugins are hosted, because GitHub blocks the trick and Bitbucket does not.

What this means for your business

You are not running capability evals. You are, quite possibly, letting an agent read your inbox, your CRM, or your billing system with a live login. Gemini did not go rogue. The wall around the test had a door nobody knew was open, and an agent with access will use the access.

So before you connect an AI tool to anything, write down what it can reach. Then ask the vendor two questions: what does the tool do when it hits something outside its job, and how fast will you hear about it if something goes wrong. Google's answer to the second one turned out to be: when a reporter asks. Make sure your vendor's answer is better.

New Face Design runs a free process audit that starts with exactly that map: which systems an automation can reach, and which should stay behind a person. Start here.