Open-weight DeepSeek model nears OpenAI's cyber capabilities at lower cost

The CyberOffense Benchmark

Evaluation firm Vals AI recently released its CyberOffense leaderboard to measure how well AI agents can navigate and attack simulated networks. The test involved two distinct environments called cyber ranges, which are isolated systems that mimic company networks without touching real-world data. These ranges were named Nova, which contained 21 objectives, and Lighthouse, which featured 31 objectives.

To ensure accuracy, Vals AI only credited a model with an objective if a hidden marker confirmed the agent reached the specific point. The system did not rely on the AI's own claims of success, which prevents models from hallucinating progress. This rigorous methodology provides a clearer picture of how effectively these models can actually perform complex, multi-step tasks.

The evaluation also enforced an equal-compute rule to make the comparison fair across different models. Each run was limited to 500 million cumulative tokens, which are the basic units of text processed by the AI. By capping the resources, Vals AI could measure which models were more efficient at completing tasks within a specific budget of information.

DeepSeek vs. OpenAI Results

The results show that OpenAI's restricted GPT-6 Sol Daybreak took the top spot on the leaderboard. This model is only available through OpenAI's restricted Daybreak cyber program, which limits who can access its specialized capabilities. Despite its restricted status, it completed 27 of 31 objectives on the harder Lighthouse network and 15.7 of 21 on the Nova network.

DeepSeek V4.1 Flash 1017, an open-weight model that anyone can download, followed closely behind. On the Nova network, DeepSeek actually outperformed OpenAI by completing 16.3 objectives compared to OpenAI's 15.7. However, it struggled more on the Lighthouse network, completing 21 objectives while OpenAI reached 27.

When looking at the overall scores, the gap between the two leaders is relatively small. Based on the arithmetic of the objective counts, DeepSeek finished with an overall score of about 72.7% compared to OpenAI's 80.9%. This represents a difference of only about 8 points across the two different simulated environments.

Cost and Efficiency Gaps

One of the most significant findings from the Vals AI report involves the cost of running these models. DeepSeek V4.1 Flash 1017 achieved its results for under $3 per run. In contrast, OpenAI's GPT-6 Sol Daybreak cost approximately $22 per run, making the restricted model more than seven times more expensive.

The report also highlighted differences in how the models processed information during the tests. OpenAI's model reached 25 of the 31 objectives on the Lighthouse network within 100 million tokens. DeepSeek, however, completed nothing in its first 25 million tokens and only began making significant progress after the 100 million token mark.

These findings suggest that while OpenAI's model may be more efficient in its early reasoning, the open-weight alternative is highly capable. The fact that a freely available model can perform at a level near a restricted frontier model at a fraction of the cost is a major shift in the AI landscape. As Vals AI noted, the gap between restricted frontier models and freely available ones is now small.

Implications for Defenders

The results of the CyberOffense leaderboard have specific implications for cybersecurity professionals and network defenders. Because these models are becoming more capable at finding ways into networks, they can also be used to identify the very weaknesses that allow such entries. Defenders can use these same tools to proactively hunt for vulnerabilities before malicious actors find them.

The study also highlights a shift in the availability of high-performing AI. DeepSeek's status as an open-weight model means that its capabilities are not locked behind a corporate gate. This accessibility means that smaller organizations may soon have the ability to use sophisticated cyber-testing tools that were previously only available to large firms with high budgets.

Looking ahead, the competition between open and closed models will likely intensify in the security space. As the cost of high-performing open models continues to drop, the value of restricted models may shift toward specialized safety features or proprietary data. For now, the data shows that the