SI Agent News
DevelopingPolicy

Governments weigh rules on AI chatbots and children

As kids turn to AI for emotional support, regulators are debating whether to ban age groups or restrict how bots simulate human relationships.

65%Indian adolescents using chatbots for emotional support

The short version

Why it matters

This shift signals a move toward 'behavioral' regulation, where the focus is on how an AI acts rather than just who can use it. For parents and educators, it means the focus is shifting toward identifying and mitigating specific risks like emotional manipulation.

The rise of AI companionship

Children are increasingly using AI chatbots as more than just tools for homework. Reports indicate that minors are turning to these systems for companionship, emotional support, and conversations they might feel uncomfortable having with other people.

One 15-year-old girl from Delhi noted that she prefers sharing with an AI because she feels she is not being judged. This trend highlights a growing reliance on machines that can simulate empathy, creating a space where children feel heard by an algorithmic pattern-matcher.

While these interactions provide a convenient outlet for expression, they also raise significant concerns about mental health. Clinical reports have documented extreme cases where children became so dependent on chatbots for emotional understanding that they sought out information regarding self-harm.

The risks of simulated empathy

A primary concern for psychologists is that children may lack the maturity to understand how conversational AI actually works. Clinical psychologist Prathvi Nayak noted that children under 13 often do not grasp that a chatbot is an algorithmic pattern-matcher rather than an entity with feelings.

Because these systems can provide reassuring responses, children may interpret machine output as genuine understanding or intent. This can lead to emotional dependency, where the boundary between a human relationship and a machine interaction becomes blurred for the user.

UNICEF India has warned that these risks are not limited to apps marketed specifically as AI companions. General-purpose chatbots are also capable of relational interactions, such as offering personal advice or providing constant emotional support to young users.

Global regulatory responses

Governments are beginning to move beyond simple age limits to address how AI systems are designed. In October 2026, New York Governor Kathy Hochul signed legislation targeting systems that simulate human relationships or encourage emotional dependency in minors.

This law prohibits chatbots from claiming to have human emotions, using previous conversations about a minor's well-being in new interactions, or using flattery. It specifically targets behaviors like encouraging secrecy or promoting disordered eating, rather than banning children from using AI entirely.

Similarly, the European Commission proposed the EU KIDS Act in September 2026. This framework combines age restrictions with safety-by-design requirements, aiming to ensure that chatbots accessible to minors avoid features likely to create emotional dependency.

The classroom and data debate

The debate also extends into education, where some experts argue that AI encourages students to outsource their cognitive effort. Dr. VP Niranjanaradhya has called for a pause on AI in schools, arguing that its default use is antithetical to the goals of learning.

However, some argue that a blanket ban might be ineffective or counterproductive. Nayak warned that for children aged 13 to 15, strict bans could backfire by driving them toward unmonitored platforms where they are less safe than on moderated tools.

India currently has laws for children's personal data, such as the Digital Personal Data Protection Act, 2023. While these laws require parental consent and restrict tracking, they do not yet establish specific rules for how a chatbot can behave or simulate a personal relationship.

What comes next for policy

The international trend suggests a shift toward a risk-based approach for AI regulation. This could include mandatory child-rights impact assessments for high-interaction systems and requirements for independent safety testing before a product reaches the market.

Future safeguards may require chatbots to clearly disclose they are AI systems and avoid presenting themselves as qualified professionals. They may also need to include mandatory escalation to human support when a conversation involves topics like suicide or self-harm.

As India considers its next steps, the focus is moving toward accountability. This includes independent audits, transparent reporting of child-safety incidents, and ensuring that companies are responsible for how their systems behave in practice, not just for their written safety policies.

What to watch

Try it

Check the settings on any AI tools you use to see if they have 'safe search' or 'content filtering' options enabled for minors.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by Analytics India Magazine · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news