Anthropic open-sources FlashProve to verify AI model outputs

The problem of model honesty

When a user interacts with an AI, they often have no way of knowing which specific model is generating the response. A service provider might use a router to switch between different models, potentially swapping a high-performing model for a cheaper or smaller version to save on costs. This lack of transparency makes it difficult for customers to verify that they are receiving the service they actually paid for.

Security risks also arise from this lack of visibility. An attacker could potentially tamper with the replies sent to a user, or a provider could engage in distillation campaigns where they use a competitor's model to train their own. Without a way to verify the source of an answer, these actions can go undetected by the end user or the regulator.

Anthropic addressed these concerns by releasing FlashProve as an open-source prototype. As reported by The Next Web, the tool allows anyone to check that an AI reply came from the specific model a provider claims to have used. It aims to provide a layer of trust in an increasingly complex ecosystem of model routing and third-party hosting.

How provable inference works

FlashProve relies on a concept called provable inference. To make this work, a provider first publishes a fingerprint of their model weights, which are the internal parameters that define how an AI functions. Each time the model generates a response, it also produces a cryptographic proof showing that those specific weights produced that exact output.

The system uses a specific type of zero-knowledge proof known as a zkSNARK. This technology is important because it allows for verification without revealing the underlying data. Anthropic stated that the proof reveals nothing about the weights themselves, which is a critical requirement for companies like Anthropic or OpenAI that keep their model weights private for competitive and safety reasons.

To ensure the security of the system, the team used a formal verification language called Lean. Because the choices made by the AI assistant Claude during the development process could be hard to audit, the team wrote the key security guarantees as machine-checked proofs. This ensures that the mathematical logic behind the security of the system is sound and verifiable by the community.

Performance and testing results

Anthropic tested the FlashProve prototype on Meta’s open Llama 4 Maverick, which is a 400bn-parameter mixture-of-experts model. A mixture-of-experts model is a type of architecture that only activates certain parts of the network for different tasks, making it more efficient. The testing was conducted using a single Nvidia H100 GPU to measure the speed and overhead of generating these proofs.

The results showed that generating a proof for a 1,024-token request took about 70 to 95 times the GPU time required to run the model normally. While this is a significant increase in processing time, the verification step itself is very fast. Checking a proof takes about 17 seconds on an ordinary CPU without needing a GPU or any access to the private weights.

This speed is a major improvement over previous methods. According to the research post, the only earlier proof of the same model took 14.3 hours for 1,000 tokens. By reducing the verification time from hours to seconds, FlashProve moves the technology closer to being a practical tool for real-time verification of AI outputs.

Use cases and limitations

There are several potential applications for this technology across different industries. Independent evaluators could use it to confirm that the models they are testing are the ones actually being served by a lab. Additionally, customers of routing services could use it to ensure they are not being served a distilled or cheaper model than the one they contracted for.

Regulators could also find the tool useful for oversight. It provides a mechanism to check that a provider's chips are serving approved models, which connects to broader discussions about pacing the frontier of AI development. These uses would help create a more transparent marketplace where the capabilities of a model can be verified independently of the provider's claims.

However, the prototype currently has several technical limits that prevent immediate wide-scale adoption. It can only handle contexts of 1,024 tokens, which is much smaller than the context windows used by most modern chatbots. Furthermore, it only works with mixture-of-experts models and does not hide the model's design, nor does it protect against timing attacks.

The path forward for FlashProve

Despite the release of FlashProve, Anthropic does not currently have plans to deploy it at their own company or any other lab. The company noted that they have already met their internal goals for verifying model outputs through a different project. That separate project involves signing traffic and rechecking it on a trusted server to ensure integrity.

FlashProve is intended as a research contribution to the broader AI community. Anthropic described the release as a way to share intermediate progress rather than a completed research agenda. By making the code available under the Apache 2.0 license, they are providing a foundation for others to build upon or integrate into their own verification systems.

As the AI industry matures, the need for verifiable outputs will likely grow. While FlashProve is a prototype with specific constraints, it introduces the concept of provable inference to the public. This could eventually lead to more robust standards for how AI providers prove the authenticity and integrity of the models they host for the public.