OpenAI reports 200 law enforcement referrals and new safety limits
The AI company is tightening restrictions on weapons info for teens and ending chats that show signs of violent fixation.
The short version
- OpenAI reported over 200 law enforcement referrals between July and September 2026 to disrupt violent plots.
- New limits will restrict detailed information about past mass attacks, including weapon types and equipment.
- ChatGPT will now be able to automatically end conversations that show a user fixating on harming others.
- Users predicted to be under 18 will face stricter limits on weapons and security systems by default.
- A new Community Safety Report will be published every six months starting in early 2027.
Why it matters
These changes signal a shift toward more proactive, automated intervention by AI companies to prevent real-world harm. For users, it means more conservative responses and potential account pauses if the system detects patterns of violence.
Reporting and referrals
OpenAI disclosed that it made over 200 law enforcement referrals during the third quarter of 2026. The company stated that these actions are intended to make a positive difference in disrupting violent plots. However, the disclosure did not specify which countries or agencies received the referrals or how many led to actual arrests.
The company provided two specific examples to illustrate its referral policy. One involved a man in Brazil accused of planning to kill his son and attack police, for which OpenAI shared information with the FBI. Another case involved a cyberstalking incident in Florida where the company notified authorities about a potential risk of harm.
To increase transparency, OpenAI committed to publishing a Community Safety Report every six months. The first report is scheduled for early 2027 and will cover the second half of 2026. This report aims to explain how the company trains its systems to detect and respond to violence-related harm.
Limits on mass attack data
A major policy change involves how ChatGPT discusses past mass attacks and the individuals who carried them out. The models will now provide less detailed information on these events, even if the data is publicly available on the internet. This includes restrictions on reconstructions of weapons used and comparisons of the scale of harm.
OpenAI says these limits apply to all users regardless of their age or the reason for their inquiry. The goal is to prevent the AI from providing information that could amplify an attacker's notoriety. Users will still be able to ask about how victims were affected or how to recognize warning signs of violence.
This move comes as public pressure on AI safety has increased. Recent hearings and lawsuits have alleged that chatbots could be used to help plan violent acts. By restricting specific details about past tragedies, OpenAI aims to mitigate the risk of the AI being used as a tool for planning similar events.
Automated chat interventions
The company is moving toward more active interventions within individual conversations. ChatGPT will now be able to automatically end a chat if it detects a user is fixating on hurting another person. This is designed to interrupt patterns of potential violence in the moment rather than just banning an account after the fact.
OpenAI is working with mental health and violence prevention experts to develop these automated triggers. If a user repeatedly triggers the automatic end-of-chat feature, the company may temporarily pause the account. The company has not yet disclosed how many triggers result in a pause or how long those pauses will last.
This development follows high-profile litigation involving parents who claimed the company failed to intervene in self-harm scenarios. While those specific cases involved self-harm rather than violence against others, the move toward automated termination represents a significant shift in how the AI handles high-risk content. OpenAI frames this as an ongoing effort to recognize warning signs reliably.
Stricter rules for minors
OpenAI is introducing more restrictions on weapons-related information for users it predicts are under 18. These limits will apply even when a question appears harmless or shows no signs of violent intent. The new rules cover not just firearms, but also practical information about security systems that could be misused.
The company's age prediction system analyzes behavior patterns, usage times, and account history to estimate a user's age. If the system is not confident about a user's age, it will default to the safer experience intended for minors. This means the conservative mode for weapons and security information becomes the default for teens.
While the company did not define what specific security systems are included, the change is intended to prevent the misuse of AI to facilitate violence. Teens will still be able to use the tool to seek support or learn how to de-escalate violence. Adults who are wrongly classified can use a third-party partner to confirm their age and remove these protections.
The role of AI in safety
OpenAI framed its new policies as part of a broader role for AI companies to work alongside human authorities. The company noted that it relies on mental health and behavioral experts to assess difficult cases. They acknowledge that AI companies lack the real-world reach and authority held by law enforcement and policymakers.
The company's commitment to safety is also being influenced by potential legislation. For example, a bill in Illinois would shield AI developers from liability for mass-casualty harms if they publish safety protocols and transparency reports. OpenAI has testified in favor of such legislation, which would likely cover major models from companies like Google and Meta.
Moving forward, the effectiveness of these new safeguards will be a key point of scrutiny. Because OpenAI has decided which data points are
Sources and links
- ReportingPPC Land: the article we sourced
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by PPC Land · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
