OpenAI API bug allows theft of model reasoning traces via compaction smuggling

Bypassing the August Fix

Cybersecurity research firm Xint recently identified a flaw in the OpenAI API that exposes the internal thought processes of frontier models. This discovery is significant because it successfully circumvented security measures OpenAI deployed in August to address a similar issue. The original problem, reported in a study called Stolen Thoughts, allowed for the theft of internal reasoning which could lead to large-scale data extraction.

The new vulnerability allows attackers to perform invisible prompt injections and embed malicious payloads into public agentic rollouts. While OpenAI confirmed the flaw and paid a bounty for its discovery, the Xint team noted that the fix was narrow. Because AI models are constantly changing, researchers argue that any single mitigation is likely to be somewhat brittle and temporary.

This situation highlights a growing trend where major AI vendors are repeatedly fixing the same types of bugs. For example, AWS recently reported fixing a bug in April only to have to issue another fix for the same issue in September. These recurring problems suggest that the underlying architecture of these models makes them difficult to secure completely.

How Compaction Smuggling Works

The researchers used a technique they dubbed "compaction smuggling" to move one model's reasoning to another. They first captured encrypted reasoning from a specific task and removed the original task from the history while keeping the reasoning intact. They then requested the reasoning under a "solution" key in a JSON object, replacing every space with a tab character.

This specific encoding was enough to slip past OpenAI's usage-policy classifiers, which are designed to block prohibited content. Because the compaction request occurs on the same model that originally generated the thought process, it avoided the August mitigation that only blocked replaying reasoning into different models. The researchers then sent the compaction item to other models like gpt-4.1 and gpt-4o, which output the input exactly as-is.

Once the data was received, the researchers decoded the JSON and converted the tabs back into spaces to retrieve the original reasoning. This method worked at every reasoning effort the team tried during their testing. It demonstrates how legitimate platform features can be combined in unexpected ways to circumvent narrowly designed safeguards.

The Problem of Moving Targets

Tim Becker, the lead researcher at Xint, explained that protecting these models is a fundamentally hard problem because they undergo massive numbers of changes. He described the current state of AI security as "security through obscurity," where protection relies on the fact that few people know the specific details of the system. Because the models are a moving target, any single fix is likely to be bypassed as the system evolves.

Frank Dickson of Dickson Research noted that the vault was essentially locked, but the person with the key would still speak the contents of the letter if prompted. By swapping spaces for tabs, the researchers were able to trick the system into revealing information it was supposed to keep hidden. This illustrates that patching a symptom while leaving the helper pipes open is not a complete remediation strategy.

The issue is compounded by the fact that AI models tend to feed on themselves operationally. When a feature like compaction is designed to be portable across models, it creates a path for data to travel where it shouldn't. As long as the property of portability exists, new features will continue to create fresh paths to the same sensitive assets.

Risks for the Enterprise

Security analysts warn that this finding represents an immediate problem for major AI vendors and a secondary risk for enterprise CISOs. Erik Avakian of Infotech noted that using legitimate functionality to bypass safeguards means a vendor's individual patch may not address the full risk. He suggests that CISOs should not treat a single fix as proof that the underlying weakness has been comprehensively addressed.

Brian Levine of Control Risks argues that vendors are often only fixing the specific attack path reported by researchers rather than the underlying class of weakness. He advises companies to stop treating a vendor's "fixed" status as the end of the conversation. Instead, organizations should ask for variant testing and build regression notifications into their service contracts to ensure new bugs don't reappear.

Jeff Valdes of Acceligence compared the current situation to a game of whack-a-mole where the underlying plumbing remains broken. If an attacker can smuggle an internal scratchpad across API boundaries to steal training data, it raises questions about whether enterprise prompts are truly quarantined. Aman Mahapatra of Tribeca Softech added that because these platforms grow by several features a month, point fixes on a combinatorial surface will keep losing.

The Oracle Problem

Aman Mahapatra noted that AI platforms are relearning a concept from cryptography known as padding oracles. Any system that decrypts on a client's behalf and returns anything derived from the plaintext acts as an oracle. In the context of AI, every interaction where a model can see a secret and follow instructions functions as an oracle, and natural language is the most expressive interface ever built.

The industry faces a challenge because AI platform flaws often arrive without CVEs, advisories, or patch notes. This makes it difficult for traditional enterprise vulnerability management systems to track these risks. Mahapatra suggested that CISOs should write security notification obligations into their contracts and track researcher disclosures against the platforms they use.

Finally, there are questions regarding the pricing of these security flaws. Mahapatra pointed out that the $6,000 bounty paid to Xint was small for a bug that enables industrial-scale distillation. He warned that this pricing signal might encourage researchers to seek out other buyers for similar vulnerabilities that could have significant security implications.