The problem
How it works
Big Sleep is a joint Project Zero and Google DeepMind agent that grew out of the earlier Project Zero Naptime framework. The team used variant analysis: the agent received recent SQLite commit messages and diffs and was asked to look for similar problems still present in the current code. In the successful run, using Gemini 1.5 Pro, the agent browsed the source, ran test inputs under a debugger and reported a crash in the generate_series extension, where a -1 sentinel value led to a write below a stack buffer.
Results and limitations
As vendor research claims, the team reports this as the first real-world vulnerability found by the agent. They reported it to the SQLite developers in early October 2024 and it was fixed the same day, before any official release included it. Existing OSS-Fuzz harnesses had not reached the code, and 150 CPU-hours of AFL fuzzing by the team did not find it.
Before you try it
The authors call the results highly experimental and say a target-specific fuzzer would likely be at least as effective. The agent's first test case also failed because it depended on a module that was not built. Big Sleep is not a public product; the transferable idea is to seed an agent with recent fixes, give it a debugger and require a reproducible crash. Only test code you own or are authorised to test, and report findings through the project's disclosure process. Results depend on your setup and were not independently reproduced by Top 200 Repos.
Tools and repositories
No ranked repository is involved. This use case still stands on its own source.
- Tools
- A coding agent with shell access · A debugger and sanitizer build · The project's source and commit history
- Prerequisites
- Big Sleep is internal to Google; reproducing the approach needs a capable model, a local debug build and permission to test the code.
Try it yourself: Look for unfixed variants of a recent bug fix
Fill the fields to adapt the prompt. Empty fields stay as visible [placeholders].
I am authorised to test [Project you may test], which I maintain or have written permission to audit. Here is a recent security fix: [Fix commit or diff]. Explain the root cause of the bug it fixed. Then search the current code for other places with the same pattern and list each candidate with file, function and why it may be vulnerable. For each strong candidate, propose a minimal test input and the exact command to run it under [Debugger or sanitizer] in my local build, but do not run anything against systems you do not control. Label every finding as unconfirmed until a crash reproduces, and draft a short private report for the maintainers rather than a public post.Run test inputs only in a local build and follow coordinated disclosure.
Sources
- From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code (opens in a new tab)
Vendor post · projectzero.google · Published 1 Nov 2024 · Source checked 12 Nov 2026, 03:34 UTC
What this source supports
- Big Sleep, a Project Zero and Google DeepMind agent evolved from Naptime, found an exploitable stack buffer underflow in SQLite
- Used variant analysis seeded with recent commit messages and diffs; the successful run used Gemini 1.5 Pro
- The agent used a code browser and a debugger to reproduce the crash
- Reported in early October 2024 and fixed the same day, before an official release
- OSS-Fuzz did not reach the code; 150 CPU-hours of AFL did not find it; results described as highly experimental