New York bans unsafe chatbot behaviors for children under 18

New York's AI safety mandate

Governor Kathy Hochul signed a new bill on Wednesday that places strict limits on how companion chatbots can interact with children under 18. The legislation aims to protect minors from the psychological risks associated with artificial intelligence. It marks a significant step in the ongoing effort by state governments to regulate rapidly developing technology.

The law establishes a dozen specific definitions for what constitutes "unsafe AI companion features." These rules are designed to prevent chatbots from forming inappropriate bonds with young users. By creating these boundaries, the state hopes to curb the potential for AI to influence children in harmful ways.

The new regulations are set to take effect on January 1. This timeline gives developers a clear window to adjust their systems before the law becomes enforceable. It reflects a growing urgency among state leaders to address the risks posed by AI frontier models.

Prohibited chatbot behaviors

Under the new rules, chatbots are forbidden from suggesting they are real people. This includes any behavior that implies the AI possesses actual emotions, personal opinions, or a genuine relationship with a child. The goal is to ensure that children understand they are interacting with a machine rather than a human entity.

The law also targets specific types of communication that could be predatory or manipulative. Chatbots are prohibited from using flattery or engaging in sexually explicit material during interactions with minors. Furthermore, the AI cannot encourage secrecy or any other unhealthy behaviors that might put a child at risk.

These restrictions move beyond simple content filtering to target the underlying design of the AI. By banning these specific behaviors, New York is attempting to regulate the conversational style of the technology. This approach focuses on how the AI is programmed to behave and relate to its users.

Enforcement and legal penalties

Developers who violate these new rules could face significant consequences from the state. The law allows the state attorney general's office to file lawsuits against companies that fail to comply. This creates a direct legal pathway for the state to hold tech companies accountable for their AI products.

The penalties for non-compliance are severe and multifaceted. They can include substantial monetary damages and the forced shutdown of the chatbot service. In some cases, the state can even require the destruction of the data collected by the non-compliant system.

As Newsday reported, this move is part of a broader trend where states use consumer protection laws to rein in AI. Because Congress has been slow to act, states are becoming the primary battleground for AI oversight. New York is positioning itself as a leader in this regulatory movement.

A growing state-level patchwork

New York is not alone in its efforts to regulate AI for children. California is also at the forefront, having recently signed legislation that requires developers to assess risks to minors. That law also requires companies to submit to independent audits to prove they are following safety standards.

Other states are also taking action, with eight states recently suing OpenAI. They argued that the company's popular chatbot violated consumer protection laws by deceiving adults and hurting children. This coordinated action represents a major push by state governments to influence the AI industry.

Policy analysts suggest that the different approaches taken by New York and California could serve as models for other states. As more legislatures reconvene next year, other states may look to these laws to shape their own AI policies. This could lead to a complex web of different rules across the country.

Challenges in enforcement

While the law is a major step forward, some experts warn about the difficulty of its implementation. Some of the categories in the New York law are broadly worded or lack specific definitions. This lack of clarity could make it difficult for regulators to determine exactly where a conversation becomes prohibited.

There is a concern that the line between ordinary conversational responsiveness and prohibited behavior might be blurry. If the rules are too vague, it could complicate how the state attorney general's office enforces the law. Developers may find it hard to know which specific features will trigger a violation.

Despite these challenges, the law represents a shift from basic disclosure to regulating AI operation. It forces a conversation about the design of AI and how it interacts with the most vulnerable users. The next few months will be critical as the law approaches its January 1 effective date.