OpenAI cuts reasoning extraction success from 54% to 0.2%
New safeguards in GPT-6.2 models significantly hinder 'adversarial distillation,' a process where bad actors train new models on a frontier AI's internal logic.

The short version
- OpenAI reduced the success rate of extracting protected reasoning from 54.0% to 0.2% in GPT-6.2 Astra.
- Adversarial distillation allows actors to train 'student' models on a 'teacher' model's outputs without inheriting its safety rules.
- The company is using input/output classifiers and API-level protections to block leaks in assistant messages and tool-call arguments.
- OpenAI reported disrupting campaigns involving thousands of accounts and shared findings with other model providers.
Why it matters
This prevents bad actors from creating 'stripped-down' versions of powerful AI that lack safety guardrails. By protecting the internal reasoning of models, OpenAI aims to keep high-level capabilities like cybersecurity and biology research from being easily weaponized.
The risk of adversarial distillation
Adversarial distillation is a machine-learning technique where one model, known as a student, learns from the data produced by another, known as a teacher. While this is a common method for improving AI, it poses a significant safety risk when used by bad actors without authorization. An attacker can use a frontier model's outputs to train a new model that possesses similar capabilities but lacks the original model's safety protections.
This process is particularly dangerous in dual-use fields like cybersecurity and biology. For example, a model designed to help defenders find software vulnerabilities could be distilled into a version that helps an attacker exploit them. Because the student model does not inherit the teacher's safeguards, it might provide assistance that the original model would have refused or blocked.
OpenAI is focusing specifically on preventing the extraction of protected reasoning. This refers to the internal sequences a model creates while processing a prompt to reach a final result, often called its chain-of-thought. These traces are highly valuable for training other models because they often contain more detailed information than the final answer provided to the user.
Testing the new safeguards
To measure the effectiveness of their defenses, OpenAI conducted an internal adaptive evaluation using strong automated red-teaming. They tested whether attackers could recover the protected reasoning a model generated while answering an earlier prompt. The results showed a dramatic improvement in the security of their newer models compared to previous versions.
As openai.com reported, the success rate for these attacks dropped from 54.0% on the original GPT-6 Astra to just 0.2% on the updated GPT-6.2 Astra. These tests were conducted in an adaptive black-box environment, meaning the automated red-teamers adjusted their tactics based on the system's responses to find hidden weaknesses.
The company is now beginning to apply these selected safeguard improvements to the older GPT-6 Astra models as well. These updates are designed to ensure that the model's behavior remains consistent across different versions while maintaining the ability to provide legitimate explanations to users.
How the defense stack works
OpenAI uses a multi-layered approach to disrupt adversarial distillation at both the request and campaign levels. At the request level, they use input and output classifiers to detect attempts to extract reasoning or leaks in assistant messages. This includes monitoring tool-call arguments, as a model might expose protected reasoning in a tool's arguments even if it is not included in the final response.
At the campaign level, the company looks for patterns in account-level signals that reveal coordinated extraction attempts. When they identify violations, they enforce actions against the network of accounts involved. These investigations also provide data for adversarial training, which teaches the models to recognize and refuse extraction attempts while still answering legitimate requests for help.
API-level protections also play a critical role in securing the models. OpenAI has strengthened protections across user, workspace, and organization boundaries to prevent the reuse of reasoning across different interactions. For instance, if a user rewrites an earlier message, the API now drops the reasoning that followed it to prevent it from being exposed in a new context.
Industry and national security
The company views adversarial distillation as a national security risk because it allows foreign actors to reproduce advanced capabilities without the developer's oversight. OpenAI has already reported disrupting campaigns linked to individuals associated with Moonshot AI and has briefed Congress on activity involving DeepSeek. They are advocating for legislation like the AI OVERWATCH Act and the Chip Security Act to help the U.S. government sanction these actors.
To combat these threats, OpenAI is working to establish consistent standards for safeguards across the industry. They share indicators of compromise with other model providers so that everyone can investigate related abuse on their own services. This coordinated effort aims to create a high floor for distillation safeguards across all platforms offering frontier models.
OpenAI also suggests that other developers protect private reasoning across full interactions and keep logs of successful extraction attacks as regression tests. By replaying these attacks during evaluation, developers can ensure that new defenses actually work against known methods. They emphasize that combining request-level mitigations with campaign-level enforcement is the most effective way to protect model integrity.
What to watch
- How other model providers adopt OpenAI's shared indicators of compromise.
- Whether the U.S. government introduces new tools for sanctioning foreign actors who illicitly extract AI capabilities.
Try it
Check the OpenAI September disclosure to see the specific types of attacks they disrupted earlier this year.
Sources and links
- OfficialAnthropic(opens in a new window) anthropic.com
- Reportingopenai.com: the article we sourced
Read the original at openai.com
First reported by openai.com. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.