OpenAI restricts ChatGPT's information on weapons and mass attacks

New restrictions for teenagers

OpenAI announced that ChatGPT will no longer provide teenagers with practical information regarding weapons or security systems. These restrictions are designed to trigger even when a user's prompt appears to be harmless or lacks obvious violent intent. The company stated that these specific limits apply to teen accounts that do not show signs of violent intent.

The company is rolling out these new safeguards over the next few weeks to bolster its safety framework. This move follows a period of scrutiny regarding how AI might be used by minors to research dangerous activities. By limiting access to technical details on security and weaponry, OpenAI aims to reduce the risk of the tool being used for harmful planning.

To support these measures, OpenAI has improved its age prediction system to better identify users under 18 based on behavioral signals. If an adult is incorrectly identified as a minor, they can use a third-party partner called Persona to verify their actual age. Parents with linked accounts will also receive alerts if a teen's account is deactivated due to violations involving violence.

Changes for all users

The company is also implementing broader changes for all users, regardless of their age or the reason for their inquiry. ChatGPT will now provide significantly less detail about past mass attacks and the individuals who carried them out. This applies even in instances where the specific details are readily available on other parts of the internet.

The AI will no longer reconstruct the specifics of an attack, such as the types of weapons used or the methods by which the attacker obtained them. It will also avoid providing detailed comparisons regarding the amount of harm caused by different violent events. These limits are intended to prevent the model from generating content that could be misused or cause undue distress.

Users are still permitted to ask about the broader impact of these events on victims and local communities. The system will also continue to provide information on how to recognize warning signs of potential violence. OpenAI stated that these limits apply regardless of a user's reason for asking about these sensitive topics.

Automated safety interventions

OpenAI is introducing a new capability that allows ChatGPT to automatically end a conversation if it detects signs of fixation on violence. The system is designed to recognize when a user is expressing an intent to seriously harm another person. This automated intervention serves as a real-time barrier against escalating dangerous interactions.

If the AI identifies a pattern of behavior where it must end several chats for these reasons, the company may choose to pause the user's account for a period of time. This multi-layered approach combines real-time detection with account-level consequences to deter repeated attempts to generate harmful content. The goal is to create a more responsive safety net for identifying high-risk behavior.

The company acknowledges that creating these safeguards is a complex task. OpenAI noted that developing these safeguards requires solving novel problems in recognizing risk and responding appropriately. This highlights the ongoing technical challenge of distinguishing between educational interest and actual risk.

Accountability and reporting

To provide transparency on these safety efforts, OpenAI has committed to publishing a Community Safety Report in early 2027. This report will cover the second half of 2026 and will be updated every six months moving forward. It is intended to explain how the company detects and responds to various safety issues using anonymized data and examples.

The company reported making more than 200 referrals to law enforcement during the third quarter of this year. These referrals represent a tangible part of the company's effort to manage and report serious safety concerns. The upcoming reports will provide a structured way for the public to see how these internal processes function.

These updates come as the company faces increasing pressure from various stakeholders. Recently, Common Sense Media labeled the teen version of ChatGPT as unsafe for those under 18. Additionally, the company has faced legal challenges, including lawsuits from eight US states and the state of Texas regarding potential harms to children.