Anthropic launches free AI-powered vulnerability scanner for open-source projects

A new automated security pipeline

Anthropic recently introduced OSS Scanner, a free tool designed to audit critical open-source projects for security vulnerabilities. The initiative is a joint effort between the company's Frontier Red Team and its Safeguards Research team. It aims to provide high-quality security audits to the developer community at scale.

The tool is inspired by Google’s OSS-Fuzz, but it replaces traditional fuzzing methods with large language models. Specifically, it uses Anthropic’s strongest models, including Claude Mythos, to analyze enrolled codebases. The goal is to identify security flaws that might otherwise go unnoticed in large, complex projects.

As reported by MarkTechPost, the scanner is an opt-in program for projects that want to improve their security posture. It provides a way for maintainers to receive automated audits without the high costs typically associated with professional penetration testing. The program is currently free for projects that enroll.

How the scanner processes code

To participate, core maintainers enroll by opening a pull request on a specific GitHub repository. They must provide a project template, a primary contact, and a path to a Dockerfile. The Dockerfile allows the scanner to build the project with network access during the initial setup phase.

Once the build is complete, the audit moves to an isolated virtual machine that has no internet access. This environment ensures that the scanning process remains secure and contained. Agents within the pipeline then perform root cause analysis, double-check the bugs, and propose potential patches for the issues they find.

Each report delivered to a maintainer is intended to be self-contained. It includes an explanation of the bug, a reproducer to demonstrate the flaw, and a bisection to the specific commit that introduced the error. If the model can generate a fix, a candidate patch is included in the report as well.

Balancing speed and accuracy

One of the most significant design choices in OSS Scanner is the removal of human review before reports are sent to maintainers. This decision was made to eliminate a major bottleneck in the vulnerability disclosure process. By skipping the human middleman, Anthropic can deliver findings to project teams much faster than traditional methods.

However, this shift means that the burden of triaging the results falls entirely on the project maintainers. They must determine which findings are legitimate and which are not. Anthropic notes that this trade-off is necessary to handle the volume of data that large language models can produce during a scan.

The accuracy of these automated reports has been tested by Anthropic's own penetration testers. In a study of 97 critical and high findings across 48 projects, 85 of them met the company's disclosure bar. This means that 88% of the high-priority findings were considered valid for their coordinated disclosure process.

Real-world impact and results

The tool has already begun identifying significant issues in well-known software libraries. For example, the wolfSSL project received 74 reports, of which 72 were found to be valid. These reports eventually led to five new CVEs, which are standardized identifiers for publicly disclosed cybersecurity vulnerabilities.

Another example involves the curl project, which patched a high-severity HTTP/2 upload use-after-free bug. This specific flaw affected versions 8.12.0 through 8.22.0 and was fixed in version 8.23.0. Anthropic noted that the timing of this discovery aligns with recent progress in LLM performance on security benchmarks.

In the first 11 days of the program, the scanner identified 77 critical and 803 high-severity findings. While the tool is highly effective, it did produce some duplicates and some findings that maintainers felt had inflated severity levels. Despite these minor issues, the scanner is already providing a significant amount of data to the open-source community.

The future of AI security

The success of OSS Scanner highlights a growing trend in using AI to automate the discovery of software bugs. Anthropic's results on the CyberGym benchmark show that LLMs have improved from finding under 20% of vulnerabilities to over 85%. This suggests that AI is becoming a viable primary tool for security research rather than just a secondary aid.

As more projects enroll in the program, the volume of automated reports will likely increase. This will require maintainers to develop better systems for filtering and prioritizing AI-generated data. The program currently has 114 projects accepted from approximately 650 applications, showing high demand for these services.

Moving forward, the goal is to make security auditing more accessible to everyone. By providing free, high-quality audits to open-source projects, Anthropic is attempting to create a safer software ecosystem. The program continues to grow as more developers opt-in to have their codebases scanned by Claude Mythos.