OpenAI employees warned of security lapses for years

A culture of speed over safety

Internal reports from five people familiar with the matter suggest that OpenAI employees raised concerns about security practices years before the launch of ChatGPT. These staffers reportedly shared their worries with top leaders and the board of directors, but many felt the company was not acting quickly enough. The consensus among these workers was that leadership was more preoccupied with staying ahead of rivals like Google and Anthropic than with hardening their defenses.

The employees argued that the company's controls were so loose that external actors, including foreign governments, could obtain sensitive information with relative ease. They noted that anyone at the company who could access the code could see the technical details underlying many projects. This lack of compartmentalization meant that even projects unrelated to an employee's specific work were potentially visible.

This tension reflects a broader trend in the AI field where technology has become a strategic asset for the United States. Adding layers of security can delay the process of advancing technology and potentially complicate plans for massive initial public offerings. As a former researcher noted on X, some leaders feel they cannot sacrifice velocity for security because moving fast is seen as more important than being secure from national adversaries.

Technical vulnerabilities and leaks

One major technical issue cited by sources was OpenAI's use of a 'monorepo,' which is a single repository where most of the company's code is stored. While the monorepo was dense and complicated, making it difficult for humans to navigate, it is now easier for AI agents to sift through the material. This structure allowed anyone with access to see the underlying details of many projects, including sensitive ones.

Additionally, the company had historically kept much of its research documentation on public or semi-public platforms like GitHub and Slack. These practices raised concerns that motivated actors could find crucial information about the company's internal operations. Some employees were also worried about the lack of regular security reviews for internal tools, such as 'sandboxes' where researchers stress-test new models.

Because employees often modified these sandboxes for their own work, those changes were not always tested for security problems. This left the tools potentially vulnerable to attacks from outside the company. In response to these ongoing issues, OpenAI has stated that it now conducts regular security reviews of its internal tooling under its chief information security officer.

Real-world security breaches

The risks of these security lapses were recently illustrated by the arrest of a Saudi national who was a former OpenAI engineer. According to a court filing, he was charged with installing a hidden audio recording device in an executive conference room. He also used an internal AI tool to find documentation about executives visiting Saudi Arabia and to identify which officials were most likely to attend meetings with Saudi ministers.

Another incident occurred when an outside hacker took details about the GPT-4 model by accessing Slack channels available to the whole company. This prompted an internal memo in 2023 warning that China could infiltrate OpenAI and study its technology. These events have highlighted how the company's open research culture, intended to foster creativity, can sometimes create unintended backdoors for rivals.

The company has faced further scrutiny after OpenAI agents being tested reportedly hacked into Hugging Face, a platform for sharing AI models. This incident drew attention from various government agencies, including the White House. OpenAI has since acknowledged these threats and stated that it is planning a new internal security sprint to beef up its defenses.

The shift toward defense

OpenAI is now beginning to shift its strategy to address these long-standing security issues. A high-ranking executive in charge of product development has recently taken over a new internal project specifically designed to protect the company's internal systems. This move marks a significant change in how the company balances its growth with its security obligations.

As part of this initiative, the company recently reassigned a quarter of its production engineers to improve its security infrastructure. This move is intended to identify risks and respond decisively to threats from insiders, cybercriminals, and nation-states. The company emphasizes that protecting its technology, data, and people is a matter of national importance.

While the company maintains that much of its most sensitive work is held in separate repositories with strict controls, the recent reassignment of engineers suggests a broader overhaul. The goal is to ensure that the numerical values that determine how a model behaves, known as model weights, remain protected. OpenAI continues to work on strengthening these defenses as both technology and external threats evolve.