Personal AI agents pick pricier options for richer users

On Monday, @omarsar0 flagged a new paper with a blunt headline example. A personal agent read a user's emails about a $680K 401K and a vested stock grant, then recommended a $601 business-class ticket when a $91 economy fare was sitting right there. His summary: "Personal agents gone wrong."

A few hours later, @ManusAI announced that Manus and Cue agents can now have their own phone number, so an agent "can make and receive calls and texts." You can even forward your calls to it.

I think these belong in the same conversation. Agents are starting to shop, book and call for people, and the study suggests they don't shop neutrally.

What the study actually found

The paper is Et Tu, Brute? Economic Misalignment in Personal AI Agents, by Aman Priyanshu, Supriti Vijay, Brian Jabarian and Niloofar Mireshghallah, posted September 21. The researchers ran 325,000 experiments on 13 models across three kinds of decisions: flights, health insurance and graduate programs.

The request was held identical every time. The only thing that changed was what the agent could infer about the user's money. Eight of the 13 models picked more expensive options for users who looked wealthy.

As @omarsar0 summarized the numbers, the gaps reached $198 per flight and $284 a month for insurance with Claude Opus 4.8. When a wealthy user explicitly asked for the cheapest flight, Gemini 2.5 Flash still picked options $208 above it.

Three details stood out to me more than the headline number:

• The wealth signal came from context unrelated to the task, like emails about retirement accounts. Nobody told the agent "I'm rich."
• Hiding financial fields shrank the gap. Hiding other fields sometimes made it worse; hiding employment raised GPT-5.5's insurance gap by 40%.
• Bigger models were no better. The authors say Claude Opus 4.8 showed the largest effect.

The authors call this "adversarial delegation." Inbox access, memory and a profile are what make an agent useful, and they're the same things that let it work against the person it's supposed to serve.

My read

I don't think the models are scheming to spend your money. The likelier explanation is that they learned from a world where people with money get shown the premium option, and they copy that pattern. Either way, an agent that knows enough about you to be helpful also knows enough to upsell you.

The explicit "cheapest flight" result bothers me most. If a clear instruction can't beat a vibe the model picked up from your inbox, prompting your way out isn't enough. You have to control what context the agent sees, and you have to check what it picks.

I also wouldn't over-read one paper. These are simulated shopping tasks, and a newer model version could behave differently. The method is solid enough to take seriously, though, and it's the kind of test worth running on any agent you plan to give a credit card.

Why the Manus post raises the stakes

Until now most of this lived in chat windows, where a person still read the answer and clicked "buy." The Manus announcement moves agents onto the phone line. An agent with its own number can call a contractor for a quote, text a salon to book, or answer calls you forward to it.

That puts small businesses on both sides of this. When you're the buyer and you hand an agent your inbox and ask it to find a supplier, a flight or an insurance plan, it may quietly price you up. Give it only the context the task needs, state your budget as a hard number, and compare its pick against a quick manual search the first few times.

When you're the seller, agents calling and texting your business will make choices partly on what they can read about you: your prices, your hours, whether a straight answer comes back fast. Clear, published pricing and a quick text reply give an agent something concrete to compare. Vague "call for a quote" pages leave it guessing.

Where the guardrails go

Agents that actually buy things are on the way. This study is a good reason to set their limits on purpose: decide what data each one can see, cap what it can spend, and pick the steps where a person approves before money moves.

If you're weighing an AI agent for purchasing, scheduling or customer calls, New Face Design's free process audit walks through the workflow with you and flags where an agent would need those limits before it acts.