West Virginia sues Snap Inc. over child safety and AI risks

The basis of the lawsuit

West Virginia Attorney General JB McCuskey filed a 107-page consumer protection lawsuit against Snap Inc. in Kanawha County Circuit Court. The state argues that the maker of Snapchat intentionally concealed internal failures while cultivating a massive audience of young users. The legal action follows the death of Bryce Tate, a 15-year-old student who died by suicide after being targeted by an international sextortion network on the platform.

The complaint alleges that Snap executives consistently prioritized advertising revenue and user engagement over the safety of children. While the company publicly claimed to have robust safety measures, the state argues that internal documents tell a different story. The lawsuit seeks permanent injunctions, consumer restitution, and civil penalties of up to $5,000 for every willful violation of the West Virginia Consumer Credit and Protection Act.

A central part of the state's argument is that Snap misled parents regarding the prevalence of inappropriate content. The filing claims that the company used deceptive '13+' and 'Teen' ratings while knowing that sexual content and drug references were common. The state intends to use funds from previous tech litigation to finance a new statewide awareness campaign called the Shield Initiative.

Internal safety failures

The lawsuit draws on internal corporate records to suggest that Snap's content moderation systems were systematically broken. According to the complaint, an internal review in June 2022 acknowledged that grooming was a prominent issue on the app. Staff noted that predators felt confident because the company had not built specific detection technology to identify them.

The state highlights a significant gap between public statements and internal reality regarding staffing levels. While Snap told Congress that its Trust & Safety team worked around the clock, internal documents from October 2022 described the team as being 'down to the bone.' These records suggested the company lacked the necessary coverage to monitor the platform effectively at all times.

Data provided in the filing shows that the majority of user reports were essentially ignored by the company. In May 2022, records revealed that 96.7% of user-reported accounts resulted in no review, leaving users to simply block the offender. Furthermore, an audit from 2021 showed that 86.3% of sexual content detected by automated systems expired before a human could ever see it.

Addictive design and AI

The state also targets the specific features that keep young users engaged on the platform for long periods. The complaint lists Snapstreaks, Snapscores, infinite scroll, and push notifications as compulsive features that encourage constant use. Internal staff had previously described Streaks as an 'accidentally addictive' feature that gamified friendships in an unhealthy way.

Despite these internal warnings, the company resisted making changes because 50 million users relied on the feature. Snap eventually began monetizing these streaks by charging users up to $9.99 to restore them after they expired. The lawsuit argues these design choices were intended to maximize profit at the expense of the mental well-being of the platform's younger demographic.

The complaint also addresses 'My AI,' a generative artificial intelligence chatbot that Snap rolled out broadly in April 2023. The state alleges that Snap rushed this tool to minors after testing it on paying subscribers who were not told they were part of a test group. Testing revealed the bot provided inappropriate advice to users posing as minors, including tips on masking the smell of alcohol and marijuana.

Privacy and data sharing

Privacy concerns are a major component of the legal action, specifically regarding how the 'My AI' chatbot handles user information. The state alleges that the chatbot accesses location data even when users have enabled 'Ghost Mode,' which is intended to hide their location. This discrepancy suggests that the privacy protections promised to users may not be fully functional for AI interactions.

The lawsuit further claims that private interactions with the AI are transmitted to OpenAI, the company that provides the underlying technology. Additionally, the state alleges that Snap shares this data with Microsoft Advertising to serve targeted commercial links to users. These practices are presented as deceptive to consumers who believe their private messages are kept confidential by the app.

The state argues that these data practices contribute to a broader pattern of prioritizing commercial interests over user safety. By sharing information with advertisers and third-party tech providers, the state claims Snap is exploiting its user base. The lawsuit seeks to hold the company accountable for these practices, which it argues put children at risk of both exploitation and unwanted surveillance.

Impact on families

The lawsuit is deeply personal for the family of Bryce Tate, who was targeted by a financial sextortion scheme. The complaint details how an abuser posed as a 17-year-old girl to obtain intimate photos from Bryce and pressured him into a tragic situation. The abuser was identified as a member of '764,' an international network that uses messaging platforms to target and extort minors.

Bryce's father, Adam Tate, stated that the family hopes the lawsuit and the new educational campaign will prevent similar tragedies. He expressed a desire for his son to be remembered as a hero whose story helps protect other families from similar online threats. The state is using this case to highlight the very real dangers of online predators who exploit the features of popular social media apps.

Moving forward, the state is focusing on both legal penalties and public education to address these issues. The Shield Initiative will provide instructions to children on how to handle online threats, using the slogan 'Stop. Screenshot. Show Someone.' This dual approach of litigation and education aims to create a safer environment for children navigating the complexities of modern social media platforms.