Anthropic's AI-driven research speedup alarms safety researchers

Accelerated progress through AI

A new report by the AI safety research organization METR reveals that Anthropic is significantly speeding up its research and development. The findings show that the company's models are improving roughly twice as quickly as they were a year ago. This pace suggests that Anthropic has achieved about two years of historical progress in only twelve months.

Researchers are confident that this jump in speed is not simply a result of using more computing power to train models. Instead, the majority of the acceleration is being driven by AI tools assisting in the development process. This trend has shown no signs of hitting a bottleneck or experiencing diminishing returns as of October.

The productivity gains for human staff have been substantial during this period. Employees using Anthropic's most capable internal model reported being around 12 times as productive as they would have been without AI assistance. This figure is nearly double the productivity gains reported just a few months earlier when staff were using a different internal model called Mythos 5.

Hidden models and internal tools

Much of the progress described in the METR report is linked to advanced models that have not yet been released to the public. Anthropic researchers have finished training three new models, codenamed Model 2, Model 3, and Model 4, over the last six months. These models completed their training cycles in July, August-September, and September-October, respectively.

The report also identified an older model dating back to 2022, which is currently labeled as Unreleased 2022. While Anthropic shared these codenames with the researchers, the company maintains different internal names for the systems. These internal tools are providing the primary engine for the rapid development of the company's frontier AI capabilities.

While the current acceleration mostly involves AI assisting human researchers, METR warned that signs of fully autonomous contributions are appearing. This shift toward AI working independently could lead to a future where systems improve themselves with little to no human involvement. Such a transition would represent a significant milestone in the evolution of autonomous AI research.

Transparency and redaction concerns

Anthropic granted METR access to non-public information, including confidential research and employee interviews. METR noted that the company shared more information than was initially planned for the project. However, the company did not provide the

Commercial secrets and safety risks

Anthropic cofounder Sam McCandlish defended the company's decision to redact the most informative evidence from the METR report. He argued that revealing the specific drivers behind the research speedup is commercially sensitive and could provide a roadmap for competitors to accelerate their own development. McCandlish suggested that sharing this information would actually make the risks associated with AI progress worse for everyone.

The company's stance highlights a difficult trade-off between transparency and security. While the public wants to see the evidence of how fast AI is advancing, the company believes that opening the "how-to" of these speedups could be exploited. This tension creates a situation where the public must rely heavily on the judgment of outside researchers like METR to understand the underlying risks.

Political and expert reactions

The findings have sparked significant concern among policy experts and safety advocates who worry about the lack of government oversight. Brianna Rosen of the Institute for AI Policy and Strategy noted that it is troubling for private companies to be the ones deciding how to automate AI research. She argued that for democratic oversight to be effective, independent evaluators need the freedom to share their conclusions and the public needs to be able to trust the process.

In response to these rapid developments, some lawmakers are calling for immediate legislative action. New York state Assemblyman Alex Bores has urged Congress to ban recursive self-improvement this year. Meanwhile, other experts, including Sydney Von Arx of The Nightingale Collective, have compared these findings to established timelines like AI 2027. Apollo Research founder Marius Hobbhahn warned that the pace of change will likely be so fast that society and politics will struggle to keep up.