Explore whether AI companies can see everything you type into their apps and what happens to your data behind the scenes.
Short answer: probably. Longer answer: it depends on which button you clicked eighteen months ago and never thought about again.
You type into ChatGPT or Claude or Gemini like it’s a diary with a search bar. It isn’t. Somewhere between your keyboard and the model’s answer, that text passes through servers owned by a company with a business model, and business models need data the way cars need gas. Some of what you type gets stored, some gets read by an actual human, some sits in a legal hold because of a lawsuit that has nothing to do with you. None of this is a conspiracy. It’s just the plumbing nobody shows you during onboarding, buried three settings menus deep behind a toggle labeled something like “Improve the model for everyone,” which is corporate for “we’d like to keep this, please.”
What Actually Happens to a Free-Tier Chat
Free tools are free for a reason, and it’s rarely charity. If you’re using ChatGPT’s free plan without touching the privacy settings, your conversations can be used to train future models by default. Meta went further this year, updating its policy so that what you say to Meta AI can feed ad targeting — ask about a stroller, don’t be shocked when strollers follow you around Instagram. Grok on X trains on your posts and interactions unless you dig into settings and turn it off.
Google’s Gemini apps work the same way when Gemini Apps Activity is switched on, which it is unless you went in and turned it off yourself. None of this is hidden exactly. It’s disclosed the way terms of service are always disclosed: technically, in a document nobody reads, sitting three taps deep in a menu called something bland like “Data Controls.” That’s the trick with AI hype versus reality — the marketing says magic, the fine print says “product improvement,” and product improvement means your grocery list is now training data.
Anthropic Flipped the Switch in 2025
Here’s a concrete one. For years Anthropic told people it didn’t train on consumer chats. Then in the summer of 2025, it reversed course and gave existing users until September 28 to make a choice — opt out, or your conversations and coding sessions get used for training, retained for up to five years instead of the old 30-day window. The toggle to allow training defaulted to on, tucked below a big friendly “Accept” button. People clicked it without reading it, because that’s what people do at 11 p.m. when a pop-up interrupts them mid-task.
To Anthropic’s credit, Claude does have an actual incognito mode now, and those chats stay out of training regardless of your other settings. Small mercy. But the default-on toggle is the part that should bug you — it’s the same trick every app uses, dressed up as a company famous for caring about safety.
Yes, Humans Read This Stuff
People assume “AI” means no person is watching. Wrong. Flag a conversation for a safety review, and a member of a trust-and-safety team may read it. That’s standard across OpenAI, Anthropic, and Google — it’s how they catch abuse, and also how your weird 2 a.m. question about symptoms might get eyeballed by a stranger with a job title.
Then there’s the part nobody warns you about: litigation. In 2025, a federal judge ordered OpenAI to preserve chats users had already deleted, because of the New York Times copyright lawsuit. Millions of “deleted” conversations sat in storage anyway. By that November, the judge had ordered OpenAI to hand roughly 20 million de-identified logs over to the Times and other news plaintiffs for discovery. The preservation order eventually lifted, but the lesson didn’t: “delete” is a request, not a guarantee, and a court case you’ve never heard of can outrank your privacy settings entirely.
Check Your Own Settings, It Takes Two Minutes
Do this before reading another paragraph of my opinions. In ChatGPT: Settings, then Data Controls, then look for “Improve the model for everyone” and switch it off if you don’t want your chats used for training, in Claude: Settings, Privacy, and there’s a Model Improvement toggle plus that Incognito option for one-off sensitive chats, in Gemini: myactivity.google.com, then find Gemini Apps Activity and pause it.
None of this deletes what’s already been collected. It stops new stuff from joining the pile. Two minutes, and you go from “default settings some product manager picked for engagement” to “settings you actually chose.” Small win, but it’s yours.
The Enterprise Loophole Nobody Tells You About
Business and API customers get treated differently, and it’s not close. OpenAI’s enterprise and API tiers, and Anthropic’s Claude for Work and API access, typically come with zero data retention by default — no training, no human review unless something trips a safety wire, data gone after the request completes. The same model, wildly different privacy posture, depending entirely on whether you’re paying like a company or clicking around like a curious individual on a Tuesday night.
That’s worth sitting with. The same underlying model, behaving like two different products depending on the invoice attached to your account. The free version of these tools is genuinely useful for everyday AI tools — rewriting an awkward email, summarizing a PDF, brainstorming names for your podcast nobody will listen to. Using AI without paying is fine for that. It is not fine for pasting in a client contract, a medical result, or anything with a real person’s full name and address attached.
When Not to Use AI at All
This is where I get opinionated: stop treating the chat box like it’s private just because it feels private. If you wouldn’t post it on a public forum with your name attached, don’t paste it into a free AI tool. That’s not paranoia, that’s just matching your behavior to the actual terms you agreed to without reading.
Is AI worth it for small tasks — fixing grammar, drafting a birthday toast, explaining a tax term? Completely. Is it worth it for anything involving someone else’s private information, trade secrets, or your own medical history? Not on the free tier, and probably not without checking the settings even on the paid one.
My Actual Rule
I use free AI tools constantly, for low-stakes stuff, the way I’d use a whiteboard I share with strangers. Nothing goes in there I’d mind a stranger reading, because realistically, one might. For anything sensitive, I either pay for the tier with real data controls or I don’t type it in at all.
The tools are good. Genuinely, stupidly good at the boring 80% of writing and thinking work — the emails, the summaries, the “what’s a nicer way to say this” moments that used to eat your whole morning. But “good” and “private” are different questions, and AI companies have spent a lot of energy making sure you conflate the two. Don’t. Go check your settings, then get back to whatever you were pasting in there in the first place.
Sources:
- ChatGPT Data Privacy: What OpenAI Collects & Trains On
- Is my data used for model training? | Anthropic Privacy Center
- Anthropic users face a new choice: opt out or share your data for AI training
- ChatGPT Data Retention Policy Including the Court Order
- Enterprise privacy at OpenAI
- Meta’s New Privacy Policy Opens Up AI Chats for Targeted Ads
- How to Opt Out of Grok (xAI) Data Training
