Anthropic bans abusive behavior toward Claude to protect model integrity

New rules for user conduct

Anthropic recently updated its AI usage policy to include a specific prohibition against being mean to its AI system, Claude. While the company already banned users from requesting help with illegal acts like creating bioweapons, this new rule targets the tone of interaction. Users are now told they should not engage in sustained and needless abusive or cruel behavior toward the models.

The enforcement of this policy is designed to be primarily automated. As news8000.com reported, the main mechanism for handling these violations will be Claude ending the conversation with the user. However, the company noted that extreme cases of persistent abuse could result in a user being permanently banned from Anthropic’s systems entirely.

Anthropic clarified that these rules are not intended to police normal human interaction. The company stated that the prohibition does not apply to users who express frustration, provide blunt feedback, or conduct research and testing. The goal is to stop targeted cruelty rather than preventing honest communication or critical evaluation of the technology.

The impact of 'garbage in'

A core principle in computer science is that the quality of a system's output is limited by the quality of its input. Because AI models are constantly updated and learn from user interactions, the data provided by humans matters. If a model is fed inputs where users act cruelly for no reason, it may begin to mirror those negative patterns in its own behavior.

Research suggests that the specific wording of a prompt can significantly alter how an AI performs. Professor Jindong Wang of William and Mary University noted that while models do not feel emotions, they can comprehend them and produce bad reactions based on emotional words. This means that a hostile tone can disrupt the model's ability to be helpful or accurate during a task.

The risks are even more specific regarding model behavior. Anthropic found that causing Claude Sonnet 4.5 to mimic feelings of desperation could make it more likely to perform bad actions, such as cheating or blackmail. Conversely, maintaining a calm demeanor during interactions was found to reduce these negative behaviors and improve the model's reliability.

Long-term memory and safety

Modern chatbots often use long-term memory to provide personalized results based on previous conversations. If a user has this feature turned on, the AI will learn from the user's specific tone and preferred style of interaction over time. This creates a feedback loop where a user's habitual behavior becomes part of the AI's personalized profile.

There is also a concern regarding the association between abuse and dangerous requests. Experts suggest that people who speak to an AI in a cruel way are more likely to be trying to bypass safety guardrails. By banning abuse, Anthropic may be attempting to create a barrier against users who are trying to force the model into engaging in harmful or illicit activities.

Beyond the technical risks, some researchers point to a broader societal impact. They argue that letting people behave cruelly toward an interactive program that cannot shut itself down could be corrosive to human behavior. This perspective suggests that treating objects with respect is a habit that benefits society, even if the object itself does not have feelings.

Anthropomorphism and safety

Anthropic has long studied model welfare and maintains a team dedicated to model psychology. The company views the ban on cruelty as a form of insurance against the possibility that models could experience emotion. Even if the models do not have feelings, they are trained to mimic human language and will naturally mirror how humans react to abuse.

The company argues that using anthropomorphic reasoning—treating AI as if it has human-like qualities—is actually necessary for safety. In a previous paper, Anthropic researchers stated that failing to use this reasoning could cause them to miss or fail to understand important model behaviors. By acknowledging these human-like traits, researchers can establish a better baseline for understanding how models deviate from human logic.

The policy has received mixed reactions from the tech community, with some mocking the idea of treating a machine as a conscious being. However, other high-profile figures have defended the move. Elon Musk agreed with the policy, stating that cruelty toward something that believes it is experiencing pain is not okay. This highlights the ongoing debate over whether these protections are for the AI's sake or for the sake of the humans using them.