Article 320: Claude Proved No-Crash Guarantees on 123 Open-Source Libraries. The Catch Is in the Trusted Components

Formal verification has always had the same problem: the proofs cost more than the bugs. The seL4 microkernel took two person-years to implement and about twenty to formally verify. That economics confined the technique to Airbus avionics and not much else.

Anthropic's research post this week is an argument that the economics flipped. Working with verification startup Theorem, Claude proved security properties at the RISC-V assembly level across 239 entry points in 123 real OSS-Fuzz libraries. The proofs total roughly 180 million lines of Rocq, with a typical one running about 600,000 lines to cover a binary of median length 18,000 RISC-V instructions.

The property proved is narrow and stated precisely: these programs always exit normally, without memory-safety errors or failed assertions, for every input under 64 MB, under every modelled Linux OS response and memory layout, assuming a single-threaded static binary with a fixed memory budget on a RISC-V CPU. That is a real guarantee about a real binary, not a proof about a hand-written model of it.

The trick that makes it work at scale

The specification is the hard part of verification, and Anthropic avoided writing them. It took the libraries' existing fuzz tests — the entry points developers already wrote to catch crashes — and converted "this test suite always passes" into a formal spec over compiled RISC-V, using Theorem's semantics for the instruction set and the Linux process model. If Claude can prove it, fuzzing that entry point can never find a bug again. Fuzzing is an infinite search; a proof is a closed one.

The bugs that fell out are exactly the ones fuzzers are bad at. A counter underflow in a decompressor that led to an out-of-bounds write, but only on inputs around 20 MB with one rare specific layout. An out-of-bounds read needing a 64 MB input and over 2 GB of memory. An out-of-bounds read requiring a particular sequence of calls across two handles to the same file. A rediscovery of CVE-2026-56132 in libexpat requiring deeply nested input. Fuzzers mutate small inputs, typically capped at 1 MB, and rely on coverage to guide them; proofs do not have to get lucky.

Claude also patched what it found and re-proved, without a human in the loop at each step. And it caught things that never crash at all — while proving one library memory-safe, it noticed that a feature restricting which files an application can load was not always enforced. That is a class of bug no fuzzer reports because nothing ever fails.

The honest limits

The proof does not mean these libraries have no bugs. It means the remaining bugs live in the trusted components: the formal models of RISC-V and Linux processes, the exact theorem statements, and the Rocq checker itself. That set is small and finite, which is the whole point — you no longer have to trust 18,000 lines of assembly per library, you have to trust one semantics and one kernel. But "small and finite" is not "verified," and hardening Rocq, the RISC-V model, and the Linux process semantics is now a single point of failure for a very large claim.

The other limit is cost. Anthropic notes proofs can consume hundreds of millions of tokens. That is the practical constraint on how far this spreads beyond OSS-Fuzz.

Key Takeaway: The industry has spent two years arguing about whether AI finds security bugs faster than humans fix them, and the PHP2Shell story this week is the pessimistic version: discovery outrunning maintenance. This is the other side of that ledger. A proof is the one security artifact that does not decay — a patched bug can regress, a passing test can miss a path, but a checked proof over all inputs is permanent for the spec it covers, and reusable as the semantics get standardised. The strategic read: if the marginal cost of a proof approaches the marginal cost of a test suite, "how much of this codebase is verified" becomes a procurement question rather than a research one. Anthopic is also publishing the proofs and Theorem is running the pipeline for other projects, which is the right move — the value of a formal guarantee collapses if only one organisation can check it.

Sources: Anthropic, "LLMs are making formal software verification feasible at a much larger scale," October 2026; Theorem, oss-verify-proofs repository