Tech giants face child safety lawsuits as AI enters classrooms

The rise of safety litigation

Tech companies are currently navigating a wave of litigation regarding the psychological impact of their products on children. Snap recently reached a confidential settlement in a case where its technology allegedly caused severe distress to two teenage users. This follows a pattern of similar settlements involving Meta, TikTok, and YouTube in the same lawsuit.

Legal experts suggest these settlements may be an attempt by companies to protect their reputations and avoid public scrutiny. Joseph McNally, a former acting U.S. attorney, noted that the litigation has highlighted how the industry balances the "technology arms race" with product safety. These cases are part of a broader trend of high-profile trials regarding personal injury and mental health.

While some companies are settling to move forward, other legal battles remain active. A federal case involving Meta, TikTok, Snap, and YouTube continues alongside various suits from state attorneys general. These legal actions are beginning to educate the public on how platforms are designed to maximize engagement for younger users.

AI risks in the classroom

The expansion of artificial intelligence into the education sector has introduced new points of friction. Google recently faced a lawsuit from parents who claimed their child suffered a psychological crisis after interacting with the Gemini AI chatbot. The lawsuit alleges that Google integrated the tool into Chromebooks for schools despite known risks.

Google has stated that it takes these claims seriously and works with medical and academic experts to build safety features. However, the company also faces criticism for its integration strategy. The New York Times reported that schools that opt out of using Google's AI tools may lose access to its search engine entirely.

Chris Phillips, a vice president for education at Google, told the Times that search and AI are now deeply integrated. This move places the company in a difficult position where it must balance the utility of AI for students with the potential for unpredictable interactions. The personalized nature of AI makes it difficult to determine exactly how the software will respond to children.

Shifting legal strategies

For decades, tech companies have largely relied on Section 230 to protect them from being sued over content posted by their users. Because of this protection, plaintiffs' attorneys and state attorneys general have shifted their focus toward the design of the platforms themselves. This strategy is now being applied to AI companies like OpenAI.

OpenAI currently faces several lawsuits alleging that the design of ChatGPT poses risks to children. These legal challenges could become a significant financial risk as the company prepares for a potential IPO. Even competitors like Anthropic have highlighted these litigation risks in their own public filings while prohibiting children from using their products.

Michele Neitz, a professor at the University of San Francisco School of Law, suggests this indicates that companies anticipate more litigation than they did a decade ago. The focus is moving away from what users say and toward how the software is built to function. This shift could force companies to consider safety during the initial design phase rather than as a reaction to problems.

The push for systemic change

Developmental psychologists are calling for broader systemic changes in how tech products are designed for minors. Kate Blocker, who leads research for the nonprofit Children and Screens, argues that safety must be considered during the design process. She believes that safety should be a primary consideration rather than an afterthought that occurs once problems emerge.

The current wave of litigation may serve as a catalyst for these changes by highlighting the pitfalls of poor design choices. Blocker suggests that the collective legal pressure could motivate both internal company decisions and new regulations. The goal is to ensure that products intended for children are tested and evaluated for developmental safety from the start.

As tech giants continue to pursue high valuations and pour money into advanced AI models, the pressure to prioritize safety will likely intensify. The intersection of education, AI, and child mental health remains a primary battleground for regulators and parents alike. The outcome of these ongoing trials will likely shape the future of how children interact with the internet.

What to watch next

The next few months will see more trials commence as the legal landscape for child safety continues to evolve. Investors and regulators will be watching how companies like OpenAI and Anthropic navigate their upcoming public offerings amidst these lawsuits. Additionally, the integration of AI into school systems will continue to be a major point of scrutiny for parents and educators.

Observers will also look for whether these settlements lead to tangible changes in product features or if they remain primarily reputational fixes. The tension between the "technology arms race" and the need for robust safety protocols will remain a central theme in the industry. As more states join the push for regulation, the standard for AI safety in schools may become more rigid.

Safety and support

If you or someone you know is in distress, help is available. You can contact the Suicide & Crisis Lifeline by calling or texting 988 to reach a trained counselor for support and assistance.

Try it

Check your school district's policy on AI tools to see if they have specific guidelines for how students can interact with chatbots like Gemini or ChatGPT.

Watch

How many more states will join the lawsuits against AI companies for child safety?

Whether Google will offer an alternative to AI-integrated search for schools.

How OpenAI addresses litigation risks in its upcoming IPO filings.