Open Anthropic's privacy center and search for the word "feedback." You will find a paragraph that reads like a scandal if you arrive at it by accident: if you use the thumbs up/down button, the company may use the entire related conversation to train its models.
That is accurate. The alarm is half-right. What interests me is the design underneath it.
Two documents, one product
Anthropic publishes two articles with almost the same name.
The commercial one — Claude for Work, the API, Claude Gov — starts with a strong promise:
By default, we will not use your inputs or outputs from our commercial products to train our models.
Then it carves out an exception:
If you explicitly report feedback or bugs to us (e.g. via our thumbs up/down feedback button), or otherwise choose to allow us to use your data, then we may use your chats and coding sessions to train our models.
The consumer one — Free, Pro, Max, and Claude Code on those plans — has a different rule set. It lists three conditions: you allow model improvement, a conversation is flagged for safety review, or you opt in to something like the Trusted Tester program. Incognito chats are exempt.
Both describe feedback the same way. Pressing the button sends "the entire related conversation," including content, custom styles, conversation preferences, and model settings, stored for up to five years. It is de-linked from your user and customer IDs before use, and not combined with your other conversations. Connector data is excluded unless you pasted it in. Team and Enterprise admins can disable the whole thing under Organization settings → Data and Privacy → Rate chats.
The fair objection
The "what did you expect?" crowd has a point. If a user says "this answer was wrong," somebody has to read the answer. Feedback without context is a useless signal, and the dialog itself says the whole conversation is being sent.
So the existence of feedback-driven review is not the problem. The scope is.
Four things that don't hold up
The unit is the whole conversation. If one sentence in a forty-message exchange is bad, the other thirty-nine travel with it. Proportionate for a support ticket. Not proportionate for a debugging session that wandered through your infrastructure and your customers' names.
Five years is a legal-discovery number. Nobody improves a model by keeping one specific chat for five years. That retention period is about defending decisions, not learning from them, and it is the part users are least likely to have considered when they clicked.
An opt-out with a side door is not an opt-out. It is a preference. The people most likely to press the button are the ones who believe they have already disabled training, and nothing in that flow tells them otherwise.
And the packaging is genuinely bad. The sentence that circulates — "by default, we will not train on your data" — belongs to the commercial document. A consumer on claude.ai is governed by the other one, which makes no such promise. Two policies, one product family, URLs differing by a few digits.
What I would actually ask for
Not a ban on feedback review. A narrower design: send the rated exchange plus a few turns, redact the rest, and ask permission per report if you want to train on it. That is a product decision, not a technical limit.
Anthropic's commercial default is better than most of the industry, and "the API does not train on your data" is a real contractual commitment. I do not think this is a company quietly harvesting chats. I do think a privacy promise is only as strong as its least obvious exception, and here the exception is a button in the corner of every conversation.
The person who turned training off, then tapped thumbs-down on a bad answer, has no way to know they just turned it back on. That is the whole problem, and it fits in one sentence.
Sources: Anthropic privacy center, commercial, consumer.