Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis

Joshua Bundt, Andrew Fasano, Brendan Dolan-Gavitt, William Robertson, Tim Leek

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Fuzz testing is often automated, but also frequently augmented by experts who insert themselves into the workflow in a greedy search for bugs. In this paper, we propose Homo in Machina, or HM-fuzzing, in which analyses guide the manual efforts, maximizing benefit. As one example of this paradigm, we introduce compartment analysis. Compartment analysis uses a whole-program dominator analysis to estimate the utility of reaching new code, and combines this with a dynamic analysis indicating drastically under-covered edges guarding that code. This results in a prioritized list of compartments, i.e., large, uncovered parts of the pro-gram semantically partitioned and largely unreachable given the current corpus of inputs under consideration. A human can use this categorization and ranking of compartments directly to focus manual effort, finding or fashioning inputs that make the compartments available for future fuzzing. We evaluate the effect of compartment analysis on seven projects within the OSS-Fuzz corpus where we see coverage improvements over AFL++ as high as 94%, with a median of 13%. We further observe that the determination of compartments is highly stable and thus can be done early in a fuzzing campaign, maximizing the potential for impact.

    Original languageEnglish (US)
    Title of host publicationProceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation, ICST 2023
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages117-128
    Number of pages12
    ISBN (Electronic)9781665456661
    DOIs
    StatePublished - 2023
    Event16th IEEE International Conference on Software Testing, Verification and Validation, ICST 2023 - Dublin, Ireland
    Duration: Apr 16 2023Apr 20 2023

    Publication series

    NameProceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation, ICST 2023

    Conference

    Conference16th IEEE International Conference on Software Testing, Verification and Validation, ICST 2023
    Country/TerritoryIreland
    CityDublin
    Period4/16/234/20/23

    Keywords

    • fuzz testing
    • fuzzing

    ASJC Scopus subject areas

    • Management of Technology and Innovation
    • Artificial Intelligence
    • Software
    • Safety, Risk, Reliability and Quality

    Fingerprint

    Dive into the research topics of 'Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis'. Together they form a unique fingerprint.

    Cite this