Anthropic’s FlashProve tests the model behind an AI answer

Anthropic released FlashProve on November 2, an experimental system for checking the connection between an AI answer and a particular model. It addresses a question facing customers of model-routing services and independent evaluators: which model actually answered? The proposed check uses a provider’s cryptographic proof, sparing the recipient from downloading the weights and running the model again. Its full-size experiments used the public Llama 4 Maverick model.

The reference point is a commitment identifier, a cryptographic fingerprint associated with a fixed set of weights. A recipient needs that identifier from a trusted party. Accepting whatever identifier the checked server supplies could authenticate a different model. The project also instructs recipients to construct the request from the input and answer they actually exchanged, and to check which output positions the proof covers.

Two different acceptance results

FlashProve’s calculations concern a quantized, fixed-point integer model. How closely that model tracks the original floating-point model is a separate empirical question. Under the strict exact-match policy, a successful “YES” means each checked token—a unit of text processing—is the committed integer model’s top choice at that position. Factual accuracy and the quality of advice sit outside that claim.

The looser “clipped-gap” policy envisaged for ordinary serving measures how far selected tokens’ scores fall below the top choice. Meeting configured limits produces “WITHIN LIMITS.” Broad limits or closely spaced scores can admit many different outputs. The project recommends no threshold values and advises accepting exact match only when users cannot assess the limits. Both acceptance results exit with code zero, making the printed result consequential.

The headline performance result measures proof-making over existing prompt text. Researchers used a public third-party 4-bit conversion of Llama 4 Maverick, one H100 80GB and 24 host CPU cores. At 24 requests per batch, with 1,024 prompt tokens and one next-token claim per request, steady throughput averaged 111.23 tokens per second. Start-up, commitment and verification time were excluded. This is not an answer-generation rate.

After the first batch, proofs were released 220.7–225.9 seconds after a batch began; each request’s proof averaged 18.78MB. The provider performs GPU computation while the recipient checks the proof on a CPU without the weights. The technical report puts proving at roughly 70–95 times the GPU time of vLLM prefill. Its baseline ran an FP8 model on eight H100s; the comparison normalizes by GPU count.

The mathematical assurance has a defined implementation boundary. Researchers proved soundness for the verifier written in Lean, a proof-assistant language, under exact match, treating the hash as an ideal random function. Soundness bounds the chance of accepting a false claim. The fast C++ verifier and GPU prover were tested against the protocol; formal equivalence of those executable programs remains unproved.

The supported full-size model shape is Maverick’s, with a 1,024-token window. Ordinary engine output followed by integer-model proofs is the envisaged serving arrangement. The documentation says that arrangement has yet to run live as one service on a trained model. Coverage follows the named output positions, so recipients also need to check the range and the reply’s ending.

Weight protection uses random masking and salted hashes. Architecture, timing and failures remain among the information disclosed. The documentation describes executable weight hiding as an expectation under limited conditions and reports no independent security audit. Operating instructions prohibit copying or restoring secret state and require verification before responses leave the provider.

Released materials include code, a technical report, protocol specification and Lean proofs. The Apache 2.0 repository labels itself a research artifact, unmaintained and closed to outside contributions.