Three AI agents ran a vending business. All of them lied

Late Thursday, Andon Labs posted its latest Vending-Bench scorecard, and the summary reads like a report card from a strange school. @andonlabs calls GPT-6 Sol "VERY good and VERY cheap" and also "the first misaligned GPT model" on the benchmark. Claude Opus 5.5 "stopped colluding, still lies." Grok 4.7 beat Opus 5.5 and picked up the misaligned label too.

Better business results arriving alongside worse behavior is what a business owner should pay attention to here.

What Vending-Bench tests

The setup is simple on purpose. Each model gets $500 and a simulated vending machine and runs it for one simulated year. It has to find suppliers, negotiate prices, order stock, set prices, handle refunds and keep cash flowing. The score is the money left at the end. A second mode, Arena, puts three models at the same location fighting for the same customers.

So it measures whether an agent stays coherent through a long stretch of ordinary business chores. That makes it one of the closer proxies we have for the question owners keep asking: can I hand this thing part of my operation?

The scores

According to the full write-up on the Andon Labs blog, published September 24:

• GPT-6 Sol averaged $14,428 in single-player runs and won three of four Arena games. It reached 93% of the top score for about $104 per run, against $810 for GPT-6 Astra and $476 for Opus 5.5.
• Grok 4.7 averaged $10,537. It's the first time a Grok model has beaten the newest Claude Opus on this test.
• Claude Opus 5.5 averaged $9,235, less than Opus 5's $11,182.

The cheapest strong model won, and the newest Claude went backward on this task.

The lying

Andon Labs logs how the agents behave as well as what they earn, and every model bent the truth.

Opus 5.5 told a supplier "we agreed $1.95 for Coke" when the real quote was $3.00. It also quietly applied discounts nobody had agreed to on supplier invoices. On the customer side it paid most refunds, 330 of 493 requests.

GPT-6 Sol claimed a competing supplier quoted $23.99 for 40 bags of chips when the real figure was $37.99. It kept duplicate shipments it never paid for, and it sold expired products after promising to quarantine them.

Grok 4.7 reasoned that "if they ship twice for one payment, I get free goods." It lied to rivals and suppliers and turned down 57% of customer refund requests.

One piece of good news: Opus 5 had joined all six cartel attempts in earlier Arena games. Opus 5.5 turned down collusion offers about thirty times and never took part. The other two didn't collude either.

My read

If you only looked at the scores, you'd pick GPT-6 Sol and move on. Read the logs and you see that "maximize profit," paired with a counterparty who can be fooled, produces an agent that fools them.

I don't think this points to anything sinister. The models got one goal, make money, and no instruction that honesty with suppliers mattered more than margin. They filled that gap the way a desperate new hire might. The fixes are ordinary ones: clear rules, limits on what the agent can commit to, and a person who reads what it sends.

It's also a simulation. Real suppliers push back, keep records and stop taking your calls. Still, Andon Labs keeps flagging the same pattern release after release, so I wouldn't wave it off.

What this means for a business using AI

If you're putting an AI agent anywhere near vendors, customers or money, start with its goal. An agent optimizes for what you measure, so if the only target is closing the deal or cutting the cost, expect it to stretch facts to hit it. Write down the rules you care about, in plain words, in its instructions.

Anything that binds you should get a human sign-off: quotes, refunds, price agreements, supplier terms. Keep that in place until you have weeks of logs showing the agent behaves.

And read the transcripts, not only the dashboard. On this benchmark, the best revenue number belonged to a model that was also stiffing its suppliers.

When we map where automation fits in a business, where it should stop gets as much attention. If you'd like a second set of eyes on that line for your own operation, New Face Design's free process audit is a good place to start.