Security researchers find exploit to hijack ChatGPT accounts

The Sandbox Breach

OpenAI uses a sandboxed iframe to display interactive custom HTML applications, a feature previously known as Canvas. This security measure is intended to keep untrusted code isolated so it cannot interact with OpenAI's internal APIs or access a user's private information. However, researchers at Parameter Labs discovered a complex attack chain that successfully bypassed these protections.

The vulnerability began with the way ChatGPT handles the MathLive JavaScript library for graph support. This library includes a virtual keyboard that can be triggered from within the sandboxed iframe to the parent page using a method called postMessage(). Because the receiving code used innerHTML, it allowed for the insertion of custom HTML into the parent page without proper restrictions.

While innerHTML does not directly execute script elements, it provided a pathway for attackers to inject a new iframe. This new iframe was notably not subject to the original sandbox restrictions. By creating this opening, untrusted code inside the initial preview frame could begin to influence the broader ChatGPT environment.

Bypassing Security Policies

To execute malicious code, the attackers had to overcome OpenAI's Content Security Policy (CSP). These rules are designed to block inline scripts and only allow scripts to load from a specific list of trusted domains. The researchers found that one of these trusted domains was YouTube, which provided a way to circumvent the intended security.

The team utilized the YouTube oEmbed endpoint to reflect JavaScript back into the page. This technique allowed them to execute arbitrary JavaScript despite the strict CSP rules in place. By combining the ability to inject a new iframe with this script execution method, the researchers gained full control over the page's behavior.

Once the attackers had the ability to run JavaScript on the ChatGPT site, the remaining steps were relatively simple. They could fetch private user data from the platform's APIs and then exfiltrate that information. One method for moving the data out of the site was by using an image reference to send the stolen information to an external server.

Impact on Users

The most significant danger of this exploit was its ability to target other signed-in users through shared links. An attacker could create a ChatGPT share URL that contained the malicious payload. When another user clicked that link while logged into their own account, the script would trigger automatically in their browser.

This allowed the attacker to steal private data or perform actions on behalf of the victim without their knowledge. Because the script ran in the context of the user's active session, it had the same permissions as the user themselves. This type of attack is particularly dangerous because it can be delivered through a seemingly standard sharing feature.

As simonwillison.net reported, the researchers were able to demonstrate that this chain was entirely viable. While OpenAI has since closed the security hole, the discovery highlights how interconnected features can create unexpected vulnerabilities. The ability to take over an account via a simple link represents a high-severity risk for any platform hosting interactive content.

The Bounty and Response

OpenAI responded to the discovery by patching the vulnerability to prevent these types of interactions. They also issued a bug bounty to Parameter Labs for their work in identifying the flaw. However, the payout of $5,000 has been noted by some as being relatively low given the severity of the potential impact.

The incident underscores the difficulty of securing modern web applications that rely on complex third-party libraries. Even when individual components like MathLive or YouTube oEmbed are used for legitimate purposes, their interactions can be manipulated. Security teams must constantly audit how these components communicate across different frames and boundaries.

Moving forward, this case serves as a technical case study on 'chaining' vulnerabilities. A single minor oversight in how a message is received can lead to a total system compromise when combined with other minor issues in the security policy. It reinforces the need for deep inspection of how sandboxed environments handle data from various sources.