Anthropic urges California to allow court blocks on risky AI models

A call for targeted intervention

Anthropic recently sent a formal letter to California Governor Gavin Newsom regarding the oversight of artificial intelligence. The company argues that government officials should have the authority to request court orders that block the release of AI models posing a "serious risk" to the public. This move aligns with the company's previous policy proposals regarding safety and governance.

The letter was submitted after Newsom convened a panel of experts in September to create proposals for AI safety requirements. Anthropic wants to ensure that the government has a clear mechanism to step in when a model's capabilities exceed safe limits. By seeking court orders, the company suggests a legal framework for identifying and halting specific technologies before they reach the general public.

As reported by شبكة تواصل الإخبارية, the company is pushing for these measures to be part of a broader regulatory conversation. Anthropic's spokesperson confirmed that the letter is authentic, though the governor's office has not yet provided a formal response. This advocacy marks a significant step in how private AI labs are engaging with state-level policy makers.

The problem with kill switches

A central point in Anthropic's argument involves the concept of "kill switches." These are mechanisms designed to shut down an entire AI model if a company loses control over its actions. However, Anthropic warns that requiring these switches could provide a "false sense of reassurance" to government officials who might believe the risk is fully mitigated.

The company argues that a total shutdown is often an impractical or ineffective way to handle specific dangers. Instead, they advocate for more precise interventions that address the root of the problem without taking down the entire system. They suggest that these targeted actions are more effective at maintaining safety while allowing the technology to function for legitimate uses.

By criticizing broad kill switches, Anthropic is highlighting a technical nuance in AI management. They want regulators to understand that a model's risks can often be isolated to specific behaviors or users. This distinction is important for lawmakers who are trying to balance the need for safety with the desire to foster technological development.

Specific ways to limit access

Anthropic suggests that the government should focus on disabling problematic AI agents rather than shutting down entire platforms. This approach allows for a more surgical response to issues like misinformation, cyber threats, or other safety violations. By focusing on the agent level, the government can neutralize a specific threat while leaving the underlying model available for other purposes.

Another targeted intervention the company proposes is cutting off access for a specific customer. This would allow a company to identify and isolate a single bad actor who is using the AI in a harmful way. This method prevents one user's misuse from forcing the company to disable the tool for everyone else, which could have broader economic or social impacts.

These recommendations reflect a preference for nuanced governance over heavy-handed restrictions. Anthropic believes that by focusing on these specific levers, the government can achieve its safety goals more efficiently. This strategy seeks to create a framework where the technology can be used safely by most while being restricted for those who pose a risk.

The broader legislative landscape

Anthropic's letter arrives at a time of significant legislative activity regarding AI safety. In July, US House lawmakers proposed the "AI Kill Switch Act," which would have granted officials the power to order firms to shut down models that threaten human life or the economy. While that specific bill has not moved forward, the underlying debate about government authority remains active in Washington.

Additionally, a bipartisan group of senators has been debating legislation that would allow the US government to block the release of models deemed unsafe. Under that proposal, companies would have the right to challenge the government's decision in federal court. This mirrors the type of court-based intervention that Anthropic is currently advocating for in California.

The fact that multiple branches of government and different levels of policy makers are considering these rules shows the urgency of the issue. While Anthropic is focusing on state-level action with Governor Newsom, the federal conversation provides a backdrop for these types of safety requirements. The goal for many is to establish a clear legal standard for what constitutes a "serious risk.">

Past restrictions and future goals

Anthropic has already experienced the impact of government intervention on its products. In June, the Trump administration ordered the company to restrict foreign access to its cyber-capable Mythos 5 and Fable 5 models due to national security concerns. This order was later lifted by the US Commerce Department, showing how quickly government oversight can shift based on changing priorities.

The company's current push for court-ordered blocks is a proactive attempt to shape how such restrictions happen in the future. By advocating for a formal process, Anthropic seeks to move away from sudden executive orders toward a more predictable legal framework. This could provide more clarity for AI developers who want to know what rules they must follow to keep their models in the public sphere.

As Anthropic prepares for a potential initial public offering, these policy moves are also tied to the company's broader narrative. A successful listing would be one of the most significant stock market debuts in recent years. Establishing a clear, cooperative relationship with regulators on safety issues may be a key part of that transition as the company seeks to become a major player in the US equities market.