White House oversight of AI safety center sparks independence concerns

A shift in oversight

Staff at the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) are now being asked to report informally to an office within the White House. This move comes as the tiny government division, which is technically an independent branch of the Commerce Department, struggles to find a permanent leader. The center has seen a revolving door of leadership, with a director hired and fired in April and another resigning just months later.

The transition to White House oversight has sparked significant concern regarding the independence of AI safety evaluations. Critics worry that bringing these officials closer to the executive branch could lead to the politicization of safety decisions. There is a fear that findings might be altered or silenced to favor the tech industry, which the current administration has largely allowed to police itself.

The White House has not responded to specific questions about CAISSI but did announce the creation of a new 'Super Intelligence Force.' This new entity is intended to ensure U.S. leadership in the field of artificial intelligence. The move suggests a shift toward framing AI safety as a matter of national security and executive discretion rather than a standard regulatory process.

The limits of self-policing

The current approach to AI safety relies heavily on voluntary cooperation from major tech companies. Under the previous administration, companies agreed to share details and follow guidelines for transparency reports. However, the Trump administration has recently asked these same companies to undertake 'tremendous self-policing' as part of a voluntary safety pact. This shift leaves it unclear how the government will actually monitor whether these companies are following through on their promises.

This hands-off approach contrasts with how the government regulates other industries, such as automobiles. The National Highway Traffic Safety Administration uses standardized crash tests to gather data, which researchers then use to create public safety standards. In the AI world, there is currently no standard process for testing how agents act in real-world environments, such as booking a flight or interacting with police.

As AI systems evolve into 'agents' that can plan multiple steps without human prompting, the lack of standardized testing becomes more acute. Experts note that looking only at the final result of a task can hide significant errors made during the intermediate steps. Without independent government testing, the public may not see the full scope of how these systems behave in complex scenarios.

Funding and influence

Despite the growing public anxiety over out-of-control AI agents, CAISSI operates on a relatively small budget of $10 million per year. This amount is roughly one-eighth of the annual budget of its counterpart in the United Kingdom. The UK's division is often cited as a model for AI safety, even though it is less celebrated for its own frontier technology developments.

Some observers argue that the move to the White House might actually benefit the center by giving it more prominence and funding. By placing it within the orbit of the president, the division might receive the resources it currently lacks. However, this comes with the risk that the center's mission could be overshadowed by the administration's goal of maintaining U.S. dominance in the AI race.

The administration has already begun appointing figures with backgrounds in tech investments rather than technical AI evaluation to lead its AI policy. This shift toward a national security framework may prioritize secrecy and competitive advantage over consumer protection. Critics argue that decisions of such immense public consequence should be explained and rationalized in the public eye, which is not currently happening.

The role of private labs

While the government's role becomes less clear, major AI labs like OpenAI and Anthropic are increasingly promoting the use of independent evaluation firms. These third-party companies are intended to vet the safety practices of the largest models. However, the labs themselves still maintain the power to limit who has access to their systems and can choose to dispense with certain evaluations entirely.

This creates a confusing landscape for government evaluators who are tasked with oversight. If the largest companies are encouraged to set up their own internal systems or hire their own private auditors, the role of government testing becomes harder to define. This lack of clarity could weaken the independent checks necessary to prevent real-world harms caused by rapidly advancing technology.

As AI capabilities continue to grow, the government's capacity to independently check these systems has not kept pace. The current situation leaves a gap where neither the government nor the private sector has a fully transparent, standardized way to ensure safety. This leaves the public to rely on a patchwork of voluntary agreements and internal corporate policies.