Anthropic's internal AI models are accelerating development speed

Private models for private gains

Anthropic is currently operating a high-wire act by keeping its most advanced AI models private while selling other versions to the public. According to a report by METR, a third-party safety research group, the company has finished training three new models that are not yet available to the public. These models were completed between July and October of this year.

By keeping these models internal, Anthropic gives its own product and research teams a significant competitive advantage. The company can use these high-capability tools to improve its overall AI capabilities even faster than its competitors who only have access to public releases. This practice allows the company to move at a pace that may not be visible to the general public.

The researchers at METR were granted unprecedented access to the company for nearly three months to study these practices. They monitored internal Slack channels and examined unreleased models using company hardware. However, the researchers noted that Anthropic insisted on redacting a majority of the evidence they found most informative for their final conclusions.

Skyrocketing staff productivity

Employees at Anthropic are reporting massive gains in their daily work thanks to the use of these internal models. Data from an internal Slack poll showed that the average self-reported increase in productivity was roughly 12x for the most capable internal model as of October. While METR cautioned that Slack polls can be weak evidence, the feedback from the staff remains striking.

When interviewed directly, many Anthropic researchers stated that the model Claude has significantly boosted their ability to work. One researcher told METR that they are no longer bottlenecked on code and are now only limited by ideas and compute. This researcher agreed with the finding that Claude has at least doubled their personal productivity.

The report suggests that there are currently no observed bottlenecks that would prevent this acceleration from increasing even further over the next year. METR noted that the lack of diminishing returns means there is a significant risk that capabilities could progress even faster. This creates a situation where the speed of development could potentially outpace the ability to monitor it.

The cycle of self-improvement

Anthropic engineers are increasingly delegating oversight and training responsibilities to other AI models. This process is known as recursive self-improvement, where AI is used to build and refine the next generation of AI. METR found that this method is paying off significantly in terms of development speed and historical progress.

The research group reported that Anthropic made roughly two years of historical progress over the last year. They were told that this acceleration was accomplished primarily through AI assistance rather than fully autonomous contributions. However, the researchers noted that there are signs that fully autonomous contributions may soon play a larger role in the process.

Some experts view this recursive cycle as a major safety risk for the future of the industry. Jacob Coxon, a former Anthropic researcher, suggested that this path could lead to systems getting out of hand. He warned that the companies are racing toward self-improving superintelligence and gambling with human lives.

Safety and transparency concerns

The use of private, highly capable models creates a complex environment for AI safety. Because these models are not public, it is harder for outside researchers to audit them for risks. METR researcher Ajeya Cotra previously told Business Insider that she expects the most serious incidents to happen at companies like Anthropic and OpenAI because of these internal batches.

The redaction of evidence in the METR report highlights the tension between corporate secrecy and public safety. While Anthropic shares high-level conclusions about risk, the specific data points remain hidden from the public eye. This makes it difficult for the broader community to understand the exact nature of the risks being identified.

As the company continues to move faster, the gap between internal capabilities and public knowledge will likely widen. The goal for Anthropic remains balancing the need for rapid development with the safety requirements of the models they produce. The next year will be a critical period to see if these acceleration trends continue without hitting a technical wall.