SIAgentNews.com
← All news

OpenAI partners with four firms for external safety reviews

The AI company is hiring independent organizations to audit its model alignment, training data, and internal safety protocols.

· First reported by NeoTeo

fournumber of research firms hired

The short version

  • OpenAI partnered with Apollo Research, METR, Transluce, and Goodfire for specialized safety audits.
  • Evaluators receive varying levels of access to source code, training data, and internal documentation.
  • Sensitive assessments may require evaluators to work onsite using OpenAI-provided hardware.
  • The OpenAI Foundation’s Board Safety and Security Committee receives unredacted reports from the firms.
  • Some firms receive cash compensation, while others accept only API credits or compute resources.

Why it matters

This move signals a shift toward independent oversight of AI development. It provides a framework for how private companies might balance proprietary secrets with the need for public safety transparency.

The four research partners

OpenAI recently announced agreements with four different organizations to conduct its first phase of embedded safety assessments. These firms include Apollo Research, METR, Transluce, and Goodfire. Each of these entities has been assigned a distinct brief to examine specific parts of model alignment and safeguards.

The scope of these assessments is broad and varies significantly between the different teams. Some groups are tasked with testing for possible model subversion, while others focus on reviewing alignment evaluations. Other teams are assigned to look into interpretability work, which involves understanding how models reach specific internal conclusions.

The program is designed to evaluate both the training process and the internal use of the models. By involving multiple firms, OpenAI aims to cover a wide range of technical and safety concerns. Each organization operates under its own set of rules and access levels tailored to its specific goals.

Access and security protocols

The level of access granted to these evaluators depends entirely on the specific needs of each assessment. Depending on the task, a team might receive selected model checkpoints, source code, or training and evaluation data. They may also be granted access to monitoring systems and technical documentation.

OpenAI stated that the protocol does not provide a uniform access package for every team. If a team needs more information as their work progresses, they can request additional access. However, the company may consider alternatives if providing direct access could risk exposing sensitive intellectual property.

For the most sensitive types of work, evaluators are required to work onsite using OpenAI devices. These researchers must follow security standards that are at least equivalent to those of OpenAI employees. This ensures that high-level secrets remain protected while the external firms perform their audits.

Communication and reporting

A key component of the program involves how evaluators interact with OpenAI staff. Conversations with employees are authorized by the company and are held off the record. This means the information can inform the assessment, but it cannot be published or attributed to specific employees.

Evaluators have the right to publish their findings, but OpenAI reserves the right to redact confidential information. The company can also request delays for responsible disclosure to ensure safety. If an evaluator wants to include a specific statement in a report, they can request an OpenAI-attested statement.

The Safety and Security Committee of the OpenAI Foundation’s Board is the ultimate recipient of the unredacted reports. This committee has the authority to speak privately with the evaluators. If any disputes arise during the process, they are first handled by the Safety Advisory Group and OpenAI executives before reaching the committee.

Compensation and criticisms

The financial arrangements for these assessments vary among the four firms. Apollo Research and Goodfire accept compensation that is not tied to the results of their assessments. In contrast, METR and Transluce do not receive financial compensation, though they may receive API credits or compute resources.

The program has faced some criticism regarding the depth of access granted to the researchers. Some observers have noted that only two of the evaluators seem able to speak with employees. Others have pointed out that evaluators currently lack access to internal communication tools like Slack.

OpenAI has responded to these concerns by stating that evaluators can request conversations with additional staff members. They also described a process for obtaining official company statements about information learned informally. The company maintains that its published protocol defines how these authorized staff conversations are organized.

Timeline and next steps

The current phase of assessments is already underway with the four primary partners. Each workstream remains distinct to ensure that different aspects of safety are being scrutinized simultaneously. This structure allows for a multi-faceted look at the risks associated with large-scale AI models.

While the initial assessments are active, the program includes plans for future expansions. Specifically, the Transluce follow-on assessment is intended to begin in early 2027. This suggests a long-term commitment to ongoing external oversight of their technology.

The results of these audits will likely shape how OpenAI handles model alignment in the future. By creating a structured way for outside firms to look under the hood, the company is attempting to build a verifiable safety framework. The ongoing work will determine how much information the public eventually sees regarding internal safety risks.

What to watch

  • Whether the unredacted reports from the Safety and Security Committee lead to public policy changes.
  • The results of the Transluce follow-on assessment scheduled for 2027.

Try it

Look for the official OpenAI safety protocol online to see how they define 'off the record' staff conversations.

Sources and links

First reported by NeoTeo. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news