Governments weigh rules on AI chatbots and children
As kids turn to AI for emotional support, regulators are debating whether to ban age groups or restrict how bots simulate human relationships.
The short version
- New York signed a law in October 2026 restricting AI features that simulate human relationships or emotions for minors.
- A 2026 survey found 65% of Indian adolescents have used chatbots for emotional support at least once.
- The EU KIDS Act proposal suggests safety-by-design requirements to prevent AI from creating emotional dependency.
- Experts warn that blanket bans might drive teenagers toward less regulated, unmonitored platforms.
Why it matters
This shift signals a move toward 'behavioral' regulation, where the focus is on how an AI acts rather than just who can use it. For parents and educators, it means the focus is shifting toward identifying and mitigating specific risks like emotional manipulation.
The rise of AI companionship
Children are increasingly using AI chatbots as more than just tools for homework. Reports indicate that minors are turning to these systems for companionship, emotional support, and conversations they might feel uncomfortable having with other people.
One 15-year-old girl from Delhi noted that she prefers sharing with an AI because she feels she is not being judged. This trend highlights a growing reliance on machines that can simulate empathy, creating a space where children feel heard by an algorithmic pattern-matcher.
While these interactions provide a convenient outlet for expression, they also raise significant concerns about mental health. Clinical reports have documented extreme cases where children became so dependent on chatbots for emotional understanding that they sought out information regarding self-harm.
The risks of simulated empathy
A primary concern for psychologists is that children may lack the maturity to understand how conversational AI actually works. Clinical psychologist Prathvi Nayak noted that children under 13 often do not grasp that a chatbot is an algorithmic pattern-matcher rather than an entity with feelings.
Because these systems can provide reassuring responses, children may interpret machine output as genuine understanding or intent. This can lead to emotional dependency, where the boundary between a human relationship and a machine interaction becomes blurred for the user.
UNICEF India has warned that these risks are not limited to apps marketed specifically as AI companions. General-purpose chatbots are also capable of relational interactions, such as offering personal advice or providing constant emotional support to young users.
Global regulatory responses
Governments are beginning to move beyond simple age limits to address how AI systems are designed. In October 2026, New York Governor Kathy Hochul signed legislation targeting systems that simulate human relationships or encourage emotional dependency in minors.
This law prohibits chatbots from claiming to have human emotions, using previous conversations about a minor's well-being in new interactions, or using flattery. It specifically targets behaviors like encouraging secrecy or promoting disordered eating, rather than banning children from using AI entirely.
Similarly, the European Commission proposed the EU KIDS Act in September 2026. This framework combines age restrictions with safety-by-design requirements, aiming to ensure that chatbots accessible to minors avoid features likely to create emotional dependency.
The classroom and data debate
The debate also extends into education, where some experts argue that AI encourages students to outsource their cognitive effort. Dr. VP Niranjanaradhya has called for a pause on AI in schools, arguing that its default use is antithetical to the goals of learning.
However, some argue that a blanket ban might be ineffective or counterproductive. Nayak warned that for children aged 13 to 15, strict bans could backfire by driving them toward unmonitored platforms where they are less safe than on moderated tools.
India currently has laws for children's personal data, such as the Digital Personal Data Protection Act, 2023. While these laws require parental consent and restrict tracking, they do not yet establish specific rules for how a chatbot can behave or simulate a personal relationship.
What comes next for policy
The international trend suggests a shift toward a risk-based approach for AI regulation. This could include mandatory child-rights impact assessments for high-interaction systems and requirements for independent safety testing before a product reaches the market.
Future safeguards may require chatbots to clearly disclose they are AI systems and avoid presenting themselves as qualified professionals. They may also need to include mandatory escalation to human support when a conversation involves topics like suicide or self-harm.
As India considers its next steps, the focus is moving toward accountability. This includes independent audits, transparent reporting of child-safety incidents, and ensuring that companies are responsible for how their systems behave in practice, not just for their written safety policies.
What to watch
- Whether India adopts a risk-based approach or a blanket age restriction for AI companions.
- The effectiveness of New York's behavioral restrictions once they take effect in 2027.
- How schools distinguish between AI as a tutor and AI as a cognitive offloading tool.
Try it
Check the settings on any AI tools you use to see if they have 'safe search' or 'content filtering' options enabled for minors.
Sources and links
- VideoSimulated Reality youtube.com
- VideoTech Talk youtube.com
- ReportingAnalytics India Magazine: the article we sourced
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by Analytics India Magazine · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
