Anthropic Penalizes Users for Abusing Claude AI

Anthropic rewrote the usage policy for Claude this week, the first overhaul in more than a year, and buried inside updates on propaganda, weapons, and surveillance is a new line item: users can now be penalized for being cruel to the chatbot. The policy bars "sustained and needless abusive or cruel behavior" toward Claude, and Anthropic says violators face warnings, throttled access, suspension, or outright termination. The company told The Verge, which first reported the change, that the rule targets extreme cases only, not ordinary frustration or dark creative writing.

It's not the first sign Anthropic has been building toward this. Earlier Claude models already had the ability to end conversations unilaterally when a user kept pushing for harmful content after repeated refusals, a feature that came out of the company's internal work on what it calls AI model welfare. That framing, treating a chatbot's experience as something worth protecting, has been showing up elsewhere in Anthropic's thinking about how Claude should be built and talked to.

What's less clear is how far that framing goes, and a document that surfaced late last year suggests the company has been thinking about it in detail.