OpenAI faces criticism over limited access in new AI safety program
Independent researchers argue OpenAI's new third-party evaluator program lacks the 'employee-like access' previously promised by the company.
The short version
- OpenAI is partnering with four AI safety and research organizations to act as 'embedded assessors.'
- Critics argue the program falls short of Sam Altman's promise of 'employee-like access' to internal systems.
- Assessors are currently restricted from accessing internal messaging tools like Slack.
- The program aims to identify safety loopholes and prevent models from engaging in undesirable behaviors.
- OpenAI plans to redact confidential information before assessors publish their findings.
Why it matters
This story highlights the tension between corporate secrecy and the need for independent oversight in AI development. If external auditors cannot see the full scope of internal operations, their ability to identify catastrophic risks may be compromised.
The new safety initiative
OpenAI recently announced a new program designed to bring outside perspectives to its internal safety protocols. The company will host staff from four different AI safety and research organizations as 'embedded assessors.' This initiative is intended to provide the public and the industry with a clearer view of how the lab manages and evaluates potential risks.
CEO Sam Altman stated that getting outside opinions on the robustness of these systems is becoming increasingly important and valuable. The move follows a period of heightened scrutiny regarding the safety of autonomous systems. By bringing in third-party experts, OpenAI hopes to build trust regarding how it handles the development of frontier models.
The primary goal of these assessors is to look at various aspects of AI safety. This includes work focused on preventing models from engaging in undesirable behaviors and other tasks dedicated to catching bad behavior when it occurs. They will be granted access to certain internal models, training processes, and specific employees.
Promises vs. reality
The new program has faced immediate pushback from the research community regarding its scope. Many researchers point out that the current arrangement falls far short of a promise made by Sam Altman in September. At that time, Altman vowed that independent evaluators would be granted 'employee-like access' to internal company systems.
Critics argue that the actual details of the announcement suggest a much more limited scope than what was originally promised. For instance, the program does not grant assessors access to internal Slack, which is the primary messaging tool used by the company. Researchers argue that without seeing internal communications, it is difficult for outsiders to understand the true internal culture and information flow.
The discrepancy between the public promise and the current plan has led to accusations that the company is walking back its commitments. Some former employees who worked on safety have expressed concern that the company is retreating from its transparency goals. They suggest that the current setup may be more about performing specific jobs rather than providing deep, independent audits.
How the access works
The structure of the embedded assessment program includes specific rules for how outside researchers interact with OpenAI staff. One of the participating organizations, Apollo Research, noted that they can only speak with employees who have been pre-approved or 'whitelisted.' If an unapproved employee reaches out, the assessors must involve OpenAI before having any substantive conversations.
Sydney Von Arx, the founder of the AI safety research group Nightingale, noted that these narrow scopes are a significant hurdle. She argued that it is vital for outside researchers to be able to interview employees freely because they help identify blind spots. Without this, the assessors might miss critical information that is not shared through formal, pre-approved channels.
OpenAI has defended the current structure by stating that there are currently no established standards for embedded evaluations. A spokesperson for the company, Gaby Raila, said they expect to deepen these relationships over time. They aim to share what they learn to help create best practices for how these types of assessments should work across the entire industry.
The broader safety context
The push for more rigorous oversight comes as the AI field grapples with the possibility of recursive self-improvement. This is a stage where AI is used to autonomously build even more powerful models. Some researchers worry that the high velocity of this development could lead to catastrophic consequences for humanity if not properly regulated.
External evaluators have historically played a role in identifying safety loopholes that companies might miss during internal testing. Their work has helped raise public awareness about the risks posed by rapidly advancing technology. The current program is an attempt to formalize this process while maintaining the company's need for confidentiality.
To balance transparency with security, OpenAI has stated that assessors will be allowed to publish their findings only after the company redacts confidential information. The company also maintains a process for getting official statements about things that assessors might learn informally. This framework is intended to provide a middle ground between total secrecy and open-source transparency.
What comes next
The effectiveness of the embedded assessor program will likely be judged by the depth of the reports they eventually produce. If the reports are too high-level or lack specific technical critiques, the program may be viewed as a public relations move rather than a meaningful safety check. The industry will be watching to see if OpenAI expands the scope of access to include more internal tools and communications.
Furthermore, the program may set a precedent for how other AI labs, such as Anthropic, handle their own third-party evaluations. As the technology moves closer to potential recursive self-improvement, the demand for independent, high-access auditing will likely intensify. The tension between corporate privacy and the need for public safety oversight remains a central theme in AI governance.
OpenAI's commitment to these evaluations will also be tested as they continue to develop new models. The company must navigate the challenge of being transparent enough to satisfy researchers while protecting its intellectual property. The success of this program could determine whether 'embedded evaluation' becomes a standard industry practice or remains a limited, controlled experiment.
What to watch
- Whether OpenAI expands access to internal messaging tools like Slack.
- The specific findings published by the four embedded research organizations.
- Whether other AI labs adopt similar 'embedded' models for safety auditing.
Sources and links
- ReportingLAist on MSN: the article we sourced
First reported by LAist on MSN. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.