Anthropic launches free AI scanner to find open-source vulnerabilities

Automated security for open source

Anthropic recently introduced OSS Scanner, a free initiative designed to identify security vulnerabilities in critical open-source projects. The tool leverages the company's most capable models, specifically Claude Mythos, to conduct periodic audits of enrolled codebases. This program is a collaborative effort between Anthropic's Frontier Red Team and their Safeguards Research team.

The initiative aims to solve a significant bottleneck in security research. During a six-month period, Anthropic's models identified over 29,000 candidate vulnerabilities, but human reviewers could only triage about 6,000 of them. OSS Scanner automates this process by sending reports directly to project maintainers without a human intermediary.

As marktechpost.com reported, this design choice allows for much faster delivery of findings. However, it also means that the responsibility for triaging and verifying the accuracy of the reports falls entirely on the project's maintenance team. The goal is to provide a high-volume stream of data for projects that have requested such assistance in the past.

How the scanner operates

To participate, core maintainers must enroll by opening a pull request on Anthropic's official repository. This process requires the project to provide a specific YAML file containing the repository link, primary contact information, and a path to a Dockerfile. The Dockerfile is used to build the project with network access during the initial setup phase.

Once the build is complete, the audit moves to a virtual machine that is completely isolated from the internet. This environment ensures that the scanning process does not pose a risk to the external network. Within this isolated space, AI agents perform root cause analysis, double-check for bugs, and attempt to propose specific patches for the issues they find.

Each report generated by the scanner is intended to be self-contained. It includes a reproducer to show the bug in action, a detailed explanation of the flaw, and a bisection to identify the specific commit that introduced the error. If the AI is able to generate a fix, a candidate patch is included in the final report sent to the maintainers.

Accuracy and performance data

Anthropic conducted an internal audit to test the precision of the scanner. Their penetration testers reviewed 97 critical and high findings across 48 different projects. The results showed that 85 of those findings, or 88%, met the company's internal standards for coordinated vulnerability disclosure.

The data suggests the model is highly effective at identifying real issues with very few errors. Out of the 97 audited findings, only one was found to be a false positive. While 11 of the findings were real, they were either duplicates of known issues or other findings already identified by the scanner.

Real-world feedback from participants has been largely positive. For example, the wolfSSL project received 74 reports from the scanner, and they found that 72 of those reports were valid. This led to five new CVEs, including a high-severity HTTP/2 upload use-after-free bug in the curl project that was subsequently patched.

Comparison to existing tools

OSS Scanner differs significantly from traditional security tools like Google's OSS-Fuzz. While OSS-Fuzz uses fuzzing techniques like libFuzzer and AFL++ to find crashes, Anthropic swaps these fuzzers for large language models. This allows the scanner to understand the logic of the code rather than just looking for memory errors or crashes.

The tool also differs from other AI-driven security projects like OpenAI's Codex Security or DeepMind's CodeMender. Unlike some of those programs, Anthropic covers the full cost of the audits for the maintainers. It also lacks the human review step that some other enterprise-grade security agents provide before a finding is shared with the public.

Because the scanner is opt-in and free, it provides a unique middle ground for the open-source community. It offers the power of frontier models without the high costs of manual penetration testing. However, because there is no human review, maintainers must be prepared to handle a high volume of reports that may include inflated severity ratings or redundant information.

The future of AI auditing

The launch of OSS Scanner highlights a growing trend in using AI to manage the scale of software security. Anthropic's results align with broader benchmarks on CyberGym, where LLMs have seen their vulnerability discovery rates jump from under 20% to over 85%. This suggests that AI is becoming a primary tool for identifying complex logic flaws in software.

For the open-source community, this means a steady stream of automated audits that can help secure the foundations of the internet. As more projects enroll, the data gathered will likely help Anthropic refine the models' ability to propose accurate patches. The scanner currently has 114 projects accepted from approximately 650 applications.

Maintainers who join the program can expect a high volume of data but must be ready to act as the final gatekeepers. The absence of a 90-day disclosure deadline for unvalidated findings means projects can take as much time as they need to verify the reports. This balance of speed and autonomy defines the new frontier of AI-assisted security research.