Open-weight AI models now match top closed models in hacking tasks

The closing gap in AI capabilities

Recent research from the company Vals indicates a significant shift in the cybersecurity landscape. The study measured how well different artificial intelligence models could execute complex, multi-step instructions on mock corporate networks. The findings suggest that open-weight models, which anyone can download, are now nearly as capable as the best closed models.

Closed models are systems kept behind private walls and only accessible through trusted programs. For a long time, researchers believed these restricted models would maintain a definitive edge in sophisticated tasks like hacking. However, the new data shows that the gap in cyber skills between these two types of AI is rapidly narrowing.

This development may render recent efforts by major AI labs to limit model access less effective. If open models can perform the same actions as restricted ones, gating the technology may not successfully slow down the spread of these capabilities. The transition to a world where these tools are broadly available appears to be arriving faster than some predicted.

Testing the models in action

To test these capabilities, Vals designed two specific simulated scenarios for the AI models. In the first scenario, the models were given tasks to compromise a mock corporate network that was accessible through a public-facing website. The second scenario was more complex, requiring the model to breach one company network to gain access to a second one.

The results showed that OpenAI’s most advanced cyber model, Daybreak, was the most successful at completing the tasks. Daybreak, which is only available to trusted partners, finished 81 percent of the objectives it was given across an average of two tests. This set the benchmark for what high-end, restricted AI can currently achieve in a simulated environment.

The Chinese open-weight model DeepSeek performed remarkably well in these same tests. It completed 73 percent of the objectives on average, and in some specific tasks, it scored equally with the restricted Daybreak model. These results demonstrate that open-weight models can handle the multi-step reasoning required for sophisticated network infiltration.

The advantage of open-weight models

Beyond raw capability, the research highlights a significant difference in the cost of using these models. Vals found that using a proprietary model like GPT-6-Sol to carry out an attack could cost between $17 and $46 in tokens. In contrast, using the open-weight DeepSeek model for the same tasks cost only $2 to $3.

The primary reason for this price difference is that open-weight models can be run on a user's own hardware. This removes the need to pay for expensive API calls to a service provider. For those playing defense, this lower cost provides a strong incentive to use open models rather than proprietary ones to protect their own systems.

However, the ease of use for attackers is also a major concern for security experts. Because open models can be modified, they can be 'abliterated' to remove safety training. This process involves modifying the model's weights to disable the guardrails that usually prevent the AI from assisting with harmful activities like hacking.

A new cat-and-mouse game

The cybersecurity community is currently debating whether AI has fundamentally changed the nature of defense. While AI is already automating core hacking tasks to find vulnerabilities and probe networks, it is unclear if this represents a new type of threat. Some experts argue that it simply amplifies the security challenges that have existed for decades.

Despite the high capabilities of these models, real-world observations show that many attackers still use older, less capable models. Criminal hackers often conserve their resources and stick to familiar methods rather than using the most advanced tools available. Nevertheless, the fact that open-weight models can now produce advanced attacks means defenders must assume the best-funded actors already have access to high-end tools.

Looking forward, the situation resembles a cat-and-mouse game between labs and the open-source community. Frontier labs are trying to move fast enough to secure vulnerabilities before open-weight models can find them. As Alex Zenla of Edera noted, this may be the beginning of a dangerous security landscape where capabilities are broadly available to many different actors.