Humans read ChatGPT chats. Nvidia keeps its secrets from Claude

Two posts on X on Monday came at the same question from opposite ends of the market: when someone types into an AI tool, who else gets to see it?

The first came from 404 Media reporter @josephfcox, who wrote that "OpenAI has hired an army of contractors who read real ChatGPT users' chats." He said he had seen internal documents, the review system, and real user prompts. The second was a breaking-news post from @unusual_whales saying Nvidia, Palantir, and Booz Allen Hamilton would limit their use of Anthropic's models, a story that traces back to reporting by The Information.

The first story is about ordinary consumers and the second is about some of the most security-conscious companies in the country, but a small business owner should read them together.

What 404 Media found

OpenAI's internal name for the program is Project Lily. According to 404 Media's report, hundreds of contractors, recruited through Crossing Hurdles and paid through Mercor, read real prompts and rate ChatGPT's answers on a 1 to 7 scale. Part of the goal is to curb the chatbot's habit of flattering people.

Reviewers don't see usernames, and OpenAI runs a filter to strip personal details first, though the company acknowledged that sensitive details can still get through. The Next Web adds that reviewers can see a summary of the user's saved memories, which sometimes hints at where the person lives.

The setting that controls this is called "Improve the model for everyone." It's on by default for Free, Plus, and Pro accounts and off for Business, Enterprise, and Edu. Turning it off only covers new conversations, so the chats already in your history stay eligible. When 404 Media asked where OpenAI tells users that humans may read their chats, the company didn't answer at first. After the story ran, it pointed to a help page.

OpenAI isn't alone here. 404 Media says Anthropic confirmed it also uses human review to improve its models.

Why Nvidia and Palantir pulled back

The Anthropic story involves a different kind of exposure. With Fable 5 in June, Anthropic started keeping 30 days of enterprise activity so it can catch attacks that play out across many sessions and accounts. It says it has never trained on enterprise data without permission. The point of keeping the logs is misuse monitoring, which still means a month of your company's activity sitting on a vendor's servers.

For companies holding trade secrets, that's too much. As Quartz's write-up of the reporting lays out, Palantir wants a zero-retention guarantee before it offers Claude through its software. Nvidia keeps Anthropic's models on less sensitive internal work and uses its own Nemotron models for proprietary projects. Booz Allen told staff not to use Anthropic's commercial model on cybersecurity work that touches proprietary data.

Anthropic's answer is Enterprise Frontier Safeguards, announced September 1. The monitoring logs live in the customer's own Amazon, Google, or Azure storage under the customer's encryption keys. The scanning is automated, and Anthropic says no human review by its employees is required. Anthropic doesn't charge for it, though customers pay for their own storage. It rolls out in phases this fall, and eligible customers get zero retention on Fable 5 and 5.1 until it arrives.

My read

I'd take one lesson from both stories: an AI vendor's privacy terms depend on which account and which product you're using, and the default is rarely the strictest option.

Nvidia and Palantir have procurement teams arguing over retention windows. A ten-person accounting office is more likely to have a few staff with personal ChatGPT Plus accounts and good intentions. The bigger risk for that office is someone pasting a client's bank statement into a personal account on a busy Tuesday, with the training setting still on.

A few things are worth checking this week:

• Which accounts your team actually uses for work: personal Free or Plus, or a business workspace
• Whether "Improve the model for everyone" is off on any personal account that touches client information
• Which kinds of data never go into a chat window at all, like Social Security numbers, patient details, and account numbers
• For anything automated, whether your vendor offers zero retention on the model you're running

What this means for businesses using AI

Both labs use human review somewhere in their pipelines, so switching to the "safe" one won't settle this. What helps is deciding where sensitive data is allowed to go, then setting up workflows so privacy doesn't hinge on someone remembering a toggle. In practice that means a business workspace with the right settings, and automations that pass along only the fields a task needs.

New Face Design's free process audit can start with that question: where client data moves through your business today, and whether any of it lands in an AI tool nobody signed off on.