GPT-6 Cyber boosts security scan findings by 66 percent

Testing the new cyber model

CodeRabbit recently conducted an evaluation to see how GPT-6 Cyber impacts security scans. This model is the cybersecurity variant of OpenAI’s GPT-6 models, specifically trained for security research. It is designed to find vulnerabilities and test for exploits with fewer refusals on authorized tasks.

The researchers compared a model ensemble with and without GPT-6 Cyber using 33 benchmark cases. These cases included various programming languages such as Python, JavaScript, Go, Rust, and C#. The goal was to see if the new model could find more issues worth investigating for developers.

The evaluation focused on what reached the final report after verification. A potential issue only earned credit if the report included an explanation of how the code could fail. This ensures the results reflect what a human developer would actually see and act upon.

Quantifying the detection gains

The results showed that the ensemble including GPT-6 Cyber caught 22 of the 33 selected issues. This represents a 66.7% catch rate, compared to a 57.6% rate without the model. This means the model helped identify three more known issues than the baseline configuration.

In terms of total output, the model reported 361 high- and medium-severity findings. This is a 65.6% increase over the 218 findings reported by the ensemble without GPT-6 Cyber. The total number of findings across all severity levels also rose from 592 to 979.

While the catch rate for known issues went up by 9.1%, the volume of reported findings grew significantly. The researchers noted that while the model found more, they still need to determine how many of these additional findings are real vulnerabilities. The increase in output highlights the model's ability to surface a wider range of potential risks.

Specific vulnerability trends

The evaluation tracked several categories of security risks. Injection and server-side request forgery saw the largest percentage increases in reported findings. These types of vulnerabilities often involve attackers sending malicious data to a server to execute unintended commands.

Denial of service attacks added the most total findings to the report, with 98 new detections. Authorization bypass was another significant area of growth, contributing 77 additional findings. These categories are common targets for cyberattacks and are critical for secure software development.

The data shows that the model is particularly effective at identifying these specific risks. However, the researchers emphasized that report volume alone is not the only metric for success. The ultimate goal is to ensure that the additional findings are useful and manageable for the teams fixing them.

The challenge of alert volume

One of the primary considerations for using GPT-6 Cyber is the sheer amount of data it produces. Because the model reports 66% more high- and medium-severity issues, it could potentially overwhelm developers who already struggle with too many alerts. The team must balance finding more issues with keeping the workload practical.

CodeRabbit is currently working on integration steps to make these additional findings more useful. The criteria for this work include the preservation of existing catches and the ability to prioritize issues that deserve a fix. The goal is to turn the increased detection into actionable insights before the code is shipped.

The researchers suggest that teams should evaluate GPT-6 Cyber specifically for security-sensitive scans. They recommend keeping the issues the existing ensemble already catches in a regression set. This allows teams to see the new value without losing the reliability of their current tools.

Next steps for the research

The team plans to conduct further testing to isolate the effects of different reasoning efforts. They used high reasoning effort for the current investigation, where the model traces vulnerabilities through the code. Future tests will compare different effort settings to see if lower effort can be more practical.

Upcoming comparisons will look at complete-scan time and the cost of verification. The researchers want to know if the extra findings are worth the additional time and resources required to verify them. This will help determine where the model should be used selectively within a larger workflow.

The evaluation provides a concrete baseline for improving security coverage. By identifying exactly where the model adds value and where it loses catches, the team can refine the ensemble. This data helps ensure that the AI contributes to a safer codebase without creating unnecessary noise for developers.