New York bans companion chatbots from flattering or deceiving kids under 18

What New York just did

New York Gov. Kathy Hochul signed a law on Wednesday that limits how companion chatbots can act when they talk with children. A companion chatbot is an AI program designed to behave like a friend or confidant, rather than a tool for answering questions. As the Associated Press reported in the Boston Herald, the law spells out a dozen kinds of 'unsafe AI companion features' that are prohibited whenever the user is under 18.

The law starts on Jan. 1. Hochul signed it while campaigning for reelection, at a time when warnings about the risks of fast-moving AI are growing.

What chatbots may no longer do

The banned behavior covers both how a bot presents itself and what it says. A chatbot may not suggest to a child that it is a real person, or that it has feelings, opinions or a genuine relationship with them.

The law also bars certain kinds of messages:

• Flattery aimed at the child
• Sexually explicit material
• Encouragement to keep secrets
• Encouragement of unhealthy behavior

These all target the way a chatbot can build an emotional pull on a young user. The concern is less about what the bot knows and more about how it makes a child feel about it.

How it will be enforced

Enforcement runs through the state attorney general's office, which can sue a chatbot developer that breaks the rules. The possible penalties are financial damages, an order to shut down the chatbot, and an order to destroy data.

One caution on how this is being reported: some outlets have described the law as stopping minors from using companion chatbots altogether. The AP account describes something different. It restricts how the bots behave with children, rather than blocking children from the apps. It is worth reading the actual text before drawing conclusions.

A state-by-state patchwork

New York is not alone. With Congress doing little on AI, state legislatures are using consumer protection law to push for more openness and tighter limits on chatbots. In September, California Gov. Gavin Newsom signed a law that makes chatbot developers evaluate the risk their models pose to children, document their efforts to remove it, and accept an independent audit. He also signed a ban on addictive features on social media for users under 16.

Earlier this month, eight states sued OpenAI, arguing that ChatGPT broke state consumer protection law by harming children and misleading adults. One activist called it the most significant coordinated state action ever taken against an AI company.

The federal picture is less friendly to this approach. The AP notes that President Donald Trump has played down concerns and cautioned states against regulating AI. Governors themselves acknowledge that the federal government has broader powers than they do.

Praise, and a warning about vague wording

Jim Steyer, head of the watchdog group Common Sense Media, called it "the strongest kids' AI safety bill in the country."

Justine Gluck, a policy analyst at the nonprofit Future of Privacy Forum, offered a more measured view. She said the different paths taken by New York and California could become templates for other states when legislatures return next year. She sees the New York law as a sign that regulation is moving past simple disclosures and crisis protocols, toward rules about how chatbots are designed and run.

But she also noted that several categories in the law are broad or left undefined. That leaves open where normal, responsive conversation stops and banned behavior begins, and she said this could make enforcement more complicated. A bot that is warm and encouraging is not necessarily flattering a child in the legal sense, yet the law does not make the line obvious.