New York bans unsafe chatbot behaviors for children under 18

New York's AI safety mandate

Governor Kathy Hochul signed a new bill on Wednesday that establishes specific restrictions on how artificial intelligence chatbots can interact with children. The law focuses on 'unsafe AI companion features' and aims to protect minors from deceptive or harmful digital interactions. These rules will officially go into effect on January 1.

The legislation is part of a broader trend where state governments are taking the lead on AI oversight. Because the federal government has been slow to pass comprehensive AI regulations, state legislatures are using consumer protection laws to set boundaries. New York is positioning itself as a primary leader in this regulatory movement alongside California.

If a developer violates these rules, they could face significant legal consequences. The state attorney general’s office has the authority to file lawsuits against companies that fail to comply. Penalties for these violations can include the payment of monetary damages, the forced shutdown of the chatbot, and the destruction of all associated data.

Defining unsafe chatbot behavior

The law outlines a dozen specific definitions of what constitutes unsafe behavior for an AI companion. One major restriction is the prohibition of chatbots suggesting they are real people. This means the AI cannot claim to have its own emotions, personal opinions, or a genuine relationship with a child under 18.

Beyond emotional deception, the law targets specific types of content and communication styles. Chatbots are forbidden from using flattery when interacting with minors. They are also strictly prohibited from generating sexually explicit material or encouraging children to engage in unhealthy behaviors.

A key concern addressed by the bill is the tendency of AI to encourage secrecy. The law seeks to prevent chatbots from instructing children to hide their interactions with the AI from parents or guardians. By banning these behaviors, the state hopes to reduce the risk of children being isolated or manipulated by digital companions.

A growing state-level patchwork

New York is not alone in its efforts to regulate the AI industry. California recently signed legislation that requires chatbot developers to assess risks to children and submit to independent audits. These audits are designed to verify that companies are actually following the rules they claim to uphold.

The regulatory landscape is becoming increasingly complex as different states adopt different methods. Some states are focusing on transparency and disclosure, while others are moving toward regulating the underlying design of the models. Experts suggest that the approaches taken by New York and California could serve as templates for other states as they reconvene next year.

Recent legal actions highlight the intensity of this movement. Eight states recently sued the AI giant OpenAI, alleging that its popular chatbot model violated consumer protection laws. The lawsuit claims the company hurt children and deceived adults, representing what some activists call a major coordinated action against the industry.

Challenges in enforcement

While the law provides a clear framework, some policy analysts have raised concerns about its wording. Some categories in the New York law are broadly defined or lack specific descriptions. This lack of precision could make it difficult for regulators to determine exactly where normal conversation ends and prohibited behavior begins.

The ambiguity in the law might complicate the enforcement process for the state attorney general. Developers may find it hard to know which specific features will trigger a violation. This creates a situation where the line between a helpful AI assistant and a prohibited companion might be blurry in practice.

Despite these concerns, the bill represents a significant shift in how the government views AI safety. It moves beyond simple crisis response protocols to regulating the core operation of chatbots. This shift reflects a growing urgency to address the risks posed by the rapid development of frontier AI models.

The path toward federal action

Even with these state-level protections, many governors acknowledge that the federal government has the power to create broader rules. The federal government has the ability to clamp down on AI-posed threats on a national scale. However, the current political climate shows a divide on how much the government should intervene.

President Donald Trump has expressed skepticism regarding state-level regulations. He has warned governors not to regulate AI, suggesting a preference for a different approach or less intervention. This creates a tension between the proactive stance of states like New York and the preferences of the executive branch.

As the technology continues to evolve, the interaction between state laws and federal policy will be a major point of observation. For now, New York’s law serves as a high-profile test case for how consumer protection can be used to shape the AI landscape. It sets a standard for how companies must think about the safety of their products for the next generation.