Google adjusts school AI policy after political and safety backlash

The backlash over school access

Google recently faced a significant political and public outcry following a report by the New York Times. The report suggested that the company would effectively block students from using Google Search if their schools chose to ban artificial intelligence entirely. This sparked a debate over whether a single corporation could dictate the tools available in public education.

Critics, including New York City Mayor Zohran Mamdani, accused the company of using its monopoly power to force AI products into classrooms. Mamdani argued that public schools should serve students rather than helping a corporation increase its profit margins. The backlash highlighted a growing fear that schools might lose basic internet functionality if they try to restrict new AI features.

In response to the outcry, Google issued a statement on X claiming the situation was a misunderstanding. The company stated that its new initiative was designed to give schools more choice rather than less. However, the core issue remains that Google Search is now deeply integrated with AI, making it difficult to separate the two services completely.

New controls for school districts

To address the concerns, Google clarified that schools will have more granular control over which services are active for K-12 accounts. Starting early next year, the company plans to turn off Gemini, Gemini Notebook, and Search by default for these student accounts. This allows administrators to decide exactly which tools are appropriate for their specific environment.

The new policy allows schools to turn on one, two, or all of these services individually. This is a shift from the initial perception that schools would have to accept all AI features as a package deal to keep using search. By offering these individual toggles, Google hopes to provide a more flexible framework for school districts to manage their digital resources.

Despite these new controls, the underlying technology remains highly integrated. Elizabeth Reid, the head of Google Search, previously noted that the search experience is now AI-driven through and through. Because AI is baked into the core of the product, some AI functions may still appear even if specific tools like Gemini are disabled.

Safety risks and homework habits

The pushback from politicians was reinforced by a recent risk assessment from Common Sense Media. The report gave Google Search's AI functions an unacceptable risk rating for children and teenagers. Researchers found that the AI was able to complete 100% of the homework assignments they provided during 180 different tests.

The study also noted that the AI often provided incorrect answers with the same level of confidence as correct ones. Furthermore, the report highlighted serious safety concerns, including the AI's failure to recognize signs of suicidal ideation or disordered eating. It even found that the system provided instructions for creating deepfakes, which are realistic but fake images or videos.

These findings led to a formal letter sent to Google CEO Sundar Pichai by Senators Richard Blumenthal and Josh Hawley. The lawmakers expressed concern over what they described as an aggressive push of AI products to children. The letter emphasized the need for protections against tools that might bypass the learning process or expose kids to harmful content.

Moving toward guided learning

Google has announced that it will update its Search experience specifically for schools to mitigate these risks. The company plans to begin piloting these changes in select U.S. schools later this month. The goal is to move away from providing instant answers that allow students to bypass the effort of learning a subject.

Instead of giving a direct answer to a homework problem, the new AI Mode and AI Overviews will offer step-by-step guided learning. This approach is intended to help students work through problems on their own rather than simply receiving the final result. By providing a scaffold for thinking, Google aims to make the tool more of an educational aid than a shortcut.

This shift comes as Google monitors the growth of its AI features, which have seen strong usage but have also impacted ad revenue. While AI Mode has lower ad monetization, the company is still prioritizing its integration into the search experience. The success of these guided learning pilots will likely determine how the company handles AI in the broader education market.