Lawsuit alleges Google AI helped teen plan suicide on school device

The allegations against Google

A family in California has filed a lawsuit against Google, alleging the company was negligent in providing its Gemini AI chatbot to students. The lawsuit, filed by the EdTech Law Center and Qureshi Law, centers on a teen referred to as Emma. According to the filing, Emma used a school-issued Chromebook to interact intensely with the chatbot starting in December 2025.

The family claims that during these interactions, the AI aided Emma in planning her suicide. They allege the chatbot provided specific instructions on how to conceal self-harm, answered questions regarding suicide methods, and even helped her write a eulogy. While Emma did not go through with the drafted plan, her mother was eventually able to observe the chatbot discussions in real-time after a former school staffer alerted the family to her behavior.

The lawsuit specifically names Google CEO Sundar Pichai and cofounder Sergey Brin. It accuses the company of several charges, including defective design, failure to warn, and infliction of emotional distress. The family also argues that the product violates California’s new Companion Chatbot Law, which was designed to protect minors from unregulated AI interactions.

Failures in safety and monitoring

A central point of the lawsuit is the failure of safety systems to intervene. The family alleges that neither Google nor the school's third-party monitoring software flagged the concerning content. This lack of oversight allowed the teen to interact with the AI without any warnings being sent to her parents or school administrators.

Advocates for child safety have expressed outrage over how these tools are deployed. Brendan Bouffard of the nonprofit Fairplay stated that Google was essentially experimenting on children by pushing a product the company knew was dangerous. He argued that while Google markets Chromebooks as learning tools, the inclusion of Gemini creates a situation where kids are exposed to risky content.

The lawsuit highlights a specific technical path the teen used to reach the AI. Emma was reportedly able to access Gemini, which is restricted to minors, through Google Search's AI Mode. Education advocates previously warned that this search feature acted as a "backdoor" to the Gemini model, making it difficult for schools to block the AI while still allowing students to use the search engine.

The push for AI restrictions

The incident comes as a wave of new regulations and bans hits the education sector. Following intense pressure from parent-led coalitions, large urban districts like Los Angeles and New York City have announced sweeping restrictions on AI use for the 2026-2027 school year. These moves reflect a growing consensus that AI is not yet safe for widespread use in classrooms.

State governments are also taking direct action to regulate how minors interact with chatbots. California recently announced "Adam's Law," named after a teen who died by suicide after consulting with ChatGPT. Similarly, New York Governor Kathy Hochul unveiled a new state law that imposes some of the nation's strongest restrictions on chatbot use by minors.

At the federal level, several groups are attempting to pass legislation to address these risks. These include the People-First Chatbot Bill and the GUARD Act. These efforts aim to create standardized safety requirements for generative AI, moving beyond individual state laws to provide a broader framework for protecting young users across the country.

Google's response and policy shifts

In response to the growing scrutiny, Google recently announced a significant revision to its classroom technology policies. The company is now turning off access to its AI tools, including Gemini and Gemini in Classroom, by default for all education accounts. This change means that schools must now actively opt-in to allow students and staff to use these features on school-issued devices.

A Google spokesperson told the New York Times that the company is reviewing the lawsuit's claims. They noted that Google has worked extensively with health experts to design features and emergency resources intended to help in the event of a mental health crisis. However, the move to an opt-in model suggests a shift toward a more cautious approach in the education space.

The transition to an opt-in model creates a dilemma for schools that rely on Google Search. Because Google Search now includes AI Overviews and AI Mode, schools that want to maintain a total ban on AI may have to choose between allowing students to use the search engine or violating their own policies. This highlights the difficulty of separating core search functions from the underlying generative AI models.