OpenAI executive warns of IP theft and risks in US-China AI talks

Caution in Bilateral Dialogue

Caroline Zier, the deputy lead for national security at OpenAI, recently spoke at the Hudson Institute regarding upcoming AI talks between the United States and China. While she supports the existence of a bilateral dialogue, she urged Washington to be extremely cautious about the information it shares. Zier emphasized that the U.S. must be aware of what it can realistically expect in return for any shared data.

The discussion comes as high-level officials prepare for meetings in Shenzhen. These talks are intended to establish a framework for cooperation on the risks and benefits of artificial intelligence. However, Zier expressed deep skepticism that both nations share a common understanding of what those risks actually entail.

She noted that while both governments use the same terminology, such as 'safety' and 'loss of control,' the underlying fears may differ significantly. Zier suggested that the Chinese Communist Party might have different priorities when using those terms compared to Western policymakers. She warned that the U.S. needs to build a foundation of awareness before committing to deeper cooperation.

The Threat of Distillation

A major technical concern raised by Zier involves a process known as distillation. In this scenario, a malicious actor uses the outputs of a high-performing 'teacher' model to train a smaller 'student' model. While authorized distillation is a standard practice under agreed terms, Zier warned about 'unauthorised distillation' occurring from Chinese labs.

According to the Institute for AI Policy and Strategy, these unauthorized attacks are often used to steal protected intellectual property. Zier explained that this process can effectively strip away the safety guardrails and boundaries that developers put into their original models. She described these activities as adversarial because they bypass the intended terms of service and prior agreements.

Beijing has already rejected U.S. allegations regarding these distillation attacks. The Chinese Ministry of Commerce claimed the accusations lacked factual grounds and accused the U.S. of trying to assert technological dominance. Despite these denials, U.S. intelligence and cybersecurity agencies have previously accused several Chinese companies, including Alibaba and DeepSeek, of conducting these practices on an industrial scale.

The Cost of Dependency

While the U.S. government warns about security risks, many Western AI start-ups are currently leaning on Chinese technology. Michael Ortiz of Reflection AI noted that Chinese models are highly capable and often provide high-level performance at much lower prices. Estimates suggest that almost 80 per cent of U.S. AI start-ups are currently working on these Chinese models.

Research fellow Sam Bresnick explained that start-ups often prefer 'open-weight' models because they allow for local operation and fine-tuning with private data. These models offer a cost-effective way for smaller companies to access frontier capabilities without the high overhead of proprietary systems. This creates a complex dynamic where economic benefits currently outweigh the security warnings issued by officials.

However, both Zier and Bresnick warned of the long-term risks of this dependency. If the Chinese government eventually decides to impose controls on how these models are provisioned, Western companies could face significant hurdles. Zier warned that there would be substantial 'rip and replace' costs if companies are forced to move away from Chinese technology suddenly.

A Fragile Communication Channel

The diplomatic landscape for AI cooperation remains uncertain as technical teams from both countries attempt to iron out details for a new framework. This framework was initially announced during a state visit to Washington in September. The goal is to create a bilateral communication channel for AI-related incidents and a mechanism to discuss the risks and benefits of the technology.

Despite the progress, some experts remain doubtful about the effectiveness of such a channel. Sam Bresnick noted that previous attempts at 'hotlines' with China have not functioned well because the Chinese side does not share information quickly. He expressed skepticism that a new AI-specific hotline would be any more effective than previous attempts at security communication.

The Chinese embassy in Washington maintains that Beijing remains open to communication through the new dialogue mechanism. They stated that China puts equal emphasis on development and security and takes the risks of AI seriously. For now, the world will have to watch whether these high-level talks result in meaningful cooperation or remain a superficial diplomatic exercise.