METR finds Anthropic is accelerating AI research but keeps evidence private

Accelerating the R&D cycle

The nonprofit evaluator METR recently concluded that Anthropic is likely increasing its AI model capabilities at roughly double the rate it achieved between March 2024 and October 2025. This acceleration is largely attributed to a shift in methodology where researchers are now using AI systems to conduct the research itself.

METR spent several weeks performing a high-access review of Anthropic's internal processes to reach this conclusion. The findings suggest that as AI contributes more to its own development, progress could happen much faster than previous human-led cycles. This creates a scenario where extreme capabilities could emerge with less direct human oversight.

Anthropic has already acknowledged this as a significant risk in its own Responsible Scaling Policy. The UK’s AI Security Institute (AISI) shares this concern, noting that faster development cycles may cause problems to arise more quickly. METR's work specifically aimed to measure this speed-up to see if it aligns with these known safety risks.

The limits of transparency

While METR was granted generous access to Anthropic's devices and data, the final public report was heavily restricted. Anthropic held unilateral redaction rights, allowing the company to decide which pieces of information were too sensitive for the public to see. As a result, METR stated that the public must rely on their judgment much more than would be ideal.

METR noted that they redacted a majority of the evidence they found most informative during the study. They characterized the remaining public information as high-level and stated it was unlikely Anthropic intended to mislead them. Instead, the redactions were likely intended to protect confidential material that could be used by rival developers.

The report also highlights a practical challenge in voluntary oversight. Because the company being assessed controls the release of the evidence, the final verdict is shaped by what the company allows the evaluator to say. METR even admitted they considered how being too critical of these access limits might affect their ability to work with companies in the future.

Timeline and scope changes

Several specific details in the timeline of the review suggest a tightening of control as the project neared completion. METR delivered its full report in late October, but Anthropic issued and then tightened redaction guidance just days later. This led to a five-day extension so METR could rework the public version of the document.

The scope of the evaluation also shifted during the process. A specific item regarding how Anthropic splits its compute between capabilities and safety was removed from the contract midway through the project. Anthropic stated it would revisit this question later, but a completed verification would have shown if this removal was simply a delay.

Furthermore, METR noted that its view of research bottlenecks changed several times during the study. The organization expressed that it was not confident in its final assessment of these bottlenecks. These shifts suggest that even with high-level access, the complexity of the AI development cycle makes it difficult to pin down specific constraints.

Technical hurdles in measurement

Measuring these rapid improvements is becoming increasingly difficult as models surpass existing testing methods. METR expressed some nervousness about the lack of hard benchmarks, noting that current tests are becoming less effective as models improve. This mirrors concerns from the AISI that benchmarks tend to saturate quickly after they are created.

The data METR did receive was sometimes described as soft or loose. For example, they cited staff Slack polls on model uplift as only very loose evidence. Additionally, the report noted that the Claude AI model had a large hand in most of the data and analysis it received, as well as in much of METR's own analysis.

The report also highlighted that the newest internal model, called Model 4, scored above 50% on every benchmark in METR's index. Because of the lack of hard benchmarks, METR warned that it is difficult to measure progress accurately now and in the future. This creates a situation where the visible evidence could weaken over time as models continue to advance.

What comes next for oversight

This strategic analysis was written by Resultsense, a UK-focused AI news and analysis publication. It highlights a preview of how voluntary, evaluation-led systems function. In this model, an expert outside verdict is produced, but the company maintains control over the publication terms and the evidence shared with the public.

The UK government may take notice of these limitations soon. PoliticsHome reports that the UK's Centre for AI Measurement is due to launch in November. This center aims to establish a trustworthy way to test and evaluate AI, and its remit will indicate whether the UK intends to set its own terms for access and publication rights.

Currently, the UK's AISI has no statutory powers to regulate or stop a firm from releasing a model without prior testing. If the UK wants to see more than what was provided in the METR report, it will have to ask for it on its own terms. The upcoming launch of the measurement center will be a key indicator of whether the government seeks to move beyond voluntary disclosures.