Anthropic AI progress is doubling in speed due to self-improvement

The acceleration of AI growth

A new assessment by the non-profit Model Evaluation and Threat Research (METR) indicates that artificial intelligence is moving at an unprecedented pace. The study found that the progress made by Anthropic’s models over the last year is equivalent to two years of development in previous periods.

This rapid increase in capability was not caused by a simple increase in the amount of computing power used to train the models. Instead, the researchers identified a specific phenomenon where the AI systems are actively contributing to the creation of the next generation of models.

As reported by The Independent on MSN, this shift suggests that the trajectory of AI development is no longer strictly linear. The speed at which these systems are evolving is creating a new environment for safety researchers to monitor.

Understanding recursive self-improvement

The primary driver behind this acceleration is a process known as recursive self-improvement. This occurs when an AI system is used to build, refine, or optimize the models that come after it, creating a feedback loop of continuous growth.

Anthropic CEO Dario Amodei recently highlighted this dynamic as a significant concern for the industry. He noted that because AI is becoming better at building the next generation of AI, the pace of advancement is increasing drastically.

Amodei warned that if this cycle remains unchecked, it could outrun the human ability to understand or control the resulting systems. He argued that the technology must be pursued with extreme care to prevent it from moving beyond human oversight.

Risks of sudden capability emergence

The METR study warns that this recursive cycle could lead to a sudden emergence of extreme and risky capabilities. Because the improvements happen so quickly, the jump from a manageable system to a highly complex one could happen in a very short window of time.

The researchers expressed concern that the speed of development might outpace the development of safety protocols. If the models begin to improve themselves significantly, the complexity of the software might exceed the limits of what human engineers can audit effectively.

The report emphasizes that while the progress is accelerating, the primary concern is the unpredictability of the outcome. A sudden leap in capability could create risks that were not present during the slower, human-led phases of training.

Human oversight and 'signs of life'

Despite the rapid progress, the researchers noted that humans are still required to oversee the most critical parts of the AI development process. They stated that AI is not yet able to make fully autonomous contributions to the most important parts of model training.

However, the study did identify what researchers described as significant 'signs of life' in this area. This suggests that while AI is not yet fully independent in high-level training, it is beginning to show the ability to contribute in more significant ways.

The maturity of these autonomous contributions varies depending on the specific area of the model being developed. This variation means that some parts of the AI lifecycle may soon be handled by machines while others remain under human control.

Industry calls for a slower pace

The findings come at a time when leaders in the AI space are calling for a deliberate slowing of development. Anthropic CEO Dario Amodei proposed a three-point plan to ensure that the frontier of AI is developed safely and responsibly.

OpenAI CEO Sam Altman expressed agreement with the need to pace the frontier of development. He specifically praised the idea of using independent evaluators to monitor the risks associated with these rapidly advancing systems.

Anthropic invited METR to review its internal data to conduct these evaluations. This collaboration aims to provide a clearer picture of how recursive self-improvement is affecting the safety and stability of modern AI models.