Anthropic and Theorem used Claude to formally verify 123 open-source libraries targeted by Google OSS-Fuzz, covering 239 fuzzing entry points at the binary level. After fixing the bugs, they expanded the verification to prove that the compiled code would not crash under specified execution environments and input-size conditions. This experiment demonstrated both the scale and reusability of LLM-based formal verification, while revealing that defining the boundaries of its guarantees remains a central issue.
Turning Fuzz Tests into Code That Never Crashes
Formal verification translates code into a mathematical model and demonstrates that it does not violate defined rules. Conventional tests check whether specific inputs pass or fail, but they do not guarantee that every code path will be explored, even when execution time is increased. This experiment transformed fuzz tests, which developers design to generate inputs that expose crashes, into a specification requiring that the code not crash with any input below 64MB.
After compiling the libraries and fuzz tests as RISC-V executables, Theorem modeled the behavior of the actual binaries using its formalizations of RISC-V instructions and Linux process semantics. Claude was used to handle large-scale proofs covering the possible behavior and interactions of tens of thousands of instructions, while the Rocq code it produced allowed the kernel to check whether the specification had been satisfied. Using AddressSanitizer to expose invalid memory accesses as crashes provided the foundation for memory-safety specifications.
Bugs Revealed at the Boundaries Missed by Fuzzing
During the proof process, Claude immediately patched the bugs it found, regenerated the specifications, and continued its work, confirming hundreds of bugs. A counter underflow in decompression code caused an out-of-bounds write in a rare arrangement of inputs of approximately 20MB, while an out-of-bounds read in an image decoder required a 64MB input and more than 2GB of memory. In another library, an out-of-bounds read occurred only when two handles to the same file were used in a specific call order.
Claude also rediscovered CVE-2026-56132 in libexpat, which required deeply nested inputs. Even in tasks that ultimately failed to produce a successful proof, the proof attempts helped identify bugs that required actual fixes. This shows that the failure of formal verification does not necessarily mean the conclusion must be discarded; instead, it can identify areas that require further investigation and patching.
The Trust Boundary of Machine Checking and Proof Reuse
The full proof consisted of approximately 180 million lines of Rocq code, far larger than the roughly 600,000 lines typically required for an ordinary proof. Although humans do not need to review every line of such a massive proof, its guarantees depend on the assumptions that the target is a single-threaded static binary and that it executes on a Linux system with a RISC-V CPU. It is still necessary to confirm that the formal models of RISC-V and Linux processes accurately reflect reality and that the statements of the proven theorems are correct.
Because proof generation requires an enormous number of tokens, reducing costs also emerged as a central challenge. Across 36 small verification tasks created from OSS-Fuzz code, Opus 5.5 proved half of the tasks using approximately 40% fewer tokens than Opus 5. A shared workspace that allowed agents to copy one another’s proofs reduced token usage by 17% and enabled some tasks to succeed that had failed when attempted independently. Recursively combining common code with smaller proofs was presented as a key strategy for improving the scalability of LLM-based formal verification.
Conditional Proofs Rather Than a Guarantee of No Bugs
The verification target was a single-threaded static RISC-V binary linked with musl libc. Since real-world deployment environments generally use binaries dynamically linked with glibc on x86 or ARM, verification would require a system model closer to the deployment environment and accurate x86 semantics. ASan also fails to detect every memory-safety bug, and verifying every entry point does not guarantee the entire library, including code that is unreachable.
Formally verified does not mean bug-free. A checker can determine only whether a formal specification has been satisfied; it cannot judge whether that specification accurately reflects reality. Problems outside the guarantees may remain if callers misuse public functions or if execution conditions not covered by the specification exist. The significance of this experiment does not lie in the claim that AI can provide complete and correct answers for all software. Instead, it lies in expanding machine-checkable guarantees to cover testable properties while making their scope and trust boundaries equally clear.
The experiment’s summary and additional background can be found in coverage of LLM-based formal verification.
Source
This article was written based on the following source: geeknews — LLMs are making much larger-scale software formal verification possible

Leave a Reply