Fuzzing has become the dominant dynamic testing approach to find bugs in
software. For complex software like Chrome, we observed that fuzzing only
reaches around 30% of coverage, i.e., 70% of code is not executed and therefore
not tested. While not all code will be reachable in practice, there's still a
substantial gap of code that remains potentially reachable by attackers but
untested.
We therefore explored variant analysis as a possible static approach to find
bugs. The idea is simple: explore existing past bugs, abstract them into
patterns, and then search for these patterns in the large code base to locate
bug candidates that then need to be validated. On one hand, the heuristics
should be broad to reduce false negatives (i.e., bugs that are missed) but also
precise to reduce false positives (i.e., bug candidates that are not true bugs).
In Grape, we explored this
trade-off. When applying variant analysis to large code bases, we observe three
main challenges: (i) code size, (ii) cross-context interactions, and (iii)
cross-domain dependencies that together influence the state of the application
and make analysis difficult. Grape therefore applies a structured variant
analysis. Instead of creating overly complex patterns, we combine patterns at
different levels of abstraction. We break the patterns into (i) context, (ii)
assumption, (iii) violation, and (iv) abuse. Together, these indicators allow us
to mix and match even complex interactions in the code. The following block
gives an example of the different indicators:
// (i) context
void CallbackLayerAnimationObserver::SetActive() {
weak_this = GetWeakPtr();
// (iii) violation
CheckAllSequencesStarted();
// (ii) assumption
if (!weak_this) { return; }
// (iv) abuse
CheckAllSequencesCompleted();
}
Our implementation leverages SemGrep and
combines several patterns to reap bug candidates and then prune them in a second
step. This combination of several patterns allows us to increase recall while
ensuring few false positives.
Our evaluation discovered 24 confirmed vulnerabilities in Chromium, a 46M LoC
code base, receiving 17,500 USD in bug bounty. We additionally discovered bugs
in Firefox and Safari due to their use of shared libraries.
Grape was published at Usenix WOOT'26 and received the best paper award. The
main student behind the work is Han Zheng who deserves most of the credit for
the work.