US and China face hurdles in creating an AI incident reporting channel

The agreement and its obstacles

Following a summit between US President Donald Trump and Chinese President Xi Jinping, the two nations reached an agreement to establish a direct communication channel for AI safety incidents. This move is intended to provide a pathway for reporting and managing risks associated with frontier AI models. However, the agreement faces immediate hurdles because neither country has the domestic framework required to share the necessary data.

Currently, there is no federal statute in the United States that mandates AI companies to report safety incidents or dangerous capabilities to the government. While some individual states have passed such laws, federal efforts like the AI Incident Reporting Act have stalled in Congress. This leaves the industry largely responsible for its own self-regulation at the national level.

As Countercurrents reported, the White House Accord on Super Intelligence pledges to identify safety issues but does not actually require companies to notify regulators or meet specific deadlines. Without enforceable measures, the proposed communication channel between the US and China may lack the information it needs to be effective. The agreement serves as a starting point, but the underlying infrastructure for data sharing is currently missing.

US funding and regulatory gaps

The United States faces a significant shortage of funding for federal government testing and oversight of AI systems. The Commerce Department’s Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) operates with an annual budget of only $15 million. In comparison, the United Kingdom’s AI Security Institute (AISI) has a budget of almost $90 million.

This funding gap creates a situation where the government may have to rely on frontier labs to perform the bulk of their own safety testing. It remains unclear how the government would validate these private tests or if any federal body has enough compute power to conduct independent evaluations. The Trump administration's Super Intelligence Force is working on reporting mechanisms, but it lacks the authority to act as a formal regulator.

To make the reporting channel work, the US government may need to establish a transparent and mandatory policy for testing frontier models. This could involve making the current White House framework binding or passing legislation like the bipartisan FRONTIER Act. Additionally, the government would need to create rules for external audits that provide auditors with meaningful independence and clear reporting obligations.

China's dispersed oversight

China has already built a significant regulatory structure, but it differs from the US model by lacking a single, specialized AI-safety body. Oversight is currently dispersed across multiple agencies, including the Cyberspace Administration of China (CAC) and the Ministry of Industry and Information Technology. This distribution means that evaluating the risks of frontier models requires high-level coordination across several different departments.

A major challenge for China is the gap between government authority and technical expertise. While the government has the power to demand information from tech companies, many of the people with the deepest technical knowledge work in private research institutes. These experts often lack the authority to demand access to a company’s most advanced models or to inspect internal logs for safety flaws.

To improve the reporting channel, China may need to close this authority-expertise gap by defining specific safety expectations. The CAC could set reporting arrangements while technical bodies develop and validate evaluations. Other agencies, like the Ministry of Industry and Information Technology, would then weigh the effects of these rules on investment and the diffusion of technology across the country.

Technical hurdles and next steps

Both nations face technical limitations regarding the evaluation of open-weight models, which are models where the underlying weights are released to the public. Once these weights are shared, developers have much less visibility into how the systems are being used or potentially misused. This makes it difficult for any government to monitor the spread of potentially dangerous capabilities once a model is released.

One potential solution being discussed is the Z.ai-Concordia framework, which serves as a template for categorizing different types of models. It suggests sorting models into green-zone models that are safe to release, yellow-zone models that require staged access, and red-zone models that are suspended until they are reviewed. This type of categorization could help both countries manage the risks associated with different levels of AI capability.

The next Sino-American AI-safety dialogue is scheduled for November, where both countries should focus on domestic reforms. The US needs to move toward an incident-response mechanism with real teeth, while China needs to address its lack of evaluation capacity. Success depends on both powers enacting internal changes so they can effectively collaborate on the global challenge of AI safety.