TY - GEN
T1 - Homo in Machina
T2 - 16th IEEE International Conference on Software Testing, Verification and Validation, ICST 2023
AU - Bundt, Joshua
AU - Fasano, Andrew
AU - Dolan-Gavitt, Brendan
AU - Robertson, William
AU - Leek, Tim
N1 - Funding Information:
DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the Under Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Under Secretary of Defense for Research and Engineering. ©2023 Massachusetts Institute of Technology. Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.
Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Fuzz testing is often automated, but also frequently augmented by experts who insert themselves into the workflow in a greedy search for bugs. In this paper, we propose Homo in Machina, or HM-fuzzing, in which analyses guide the manual efforts, maximizing benefit. As one example of this paradigm, we introduce compartment analysis. Compartment analysis uses a whole-program dominator analysis to estimate the utility of reaching new code, and combines this with a dynamic analysis indicating drastically under-covered edges guarding that code. This results in a prioritized list of compartments, i.e., large, uncovered parts of the pro-gram semantically partitioned and largely unreachable given the current corpus of inputs under consideration. A human can use this categorization and ranking of compartments directly to focus manual effort, finding or fashioning inputs that make the compartments available for future fuzzing. We evaluate the effect of compartment analysis on seven projects within the OSS-Fuzz corpus where we see coverage improvements over AFL++ as high as 94%, with a median of 13%. We further observe that the determination of compartments is highly stable and thus can be done early in a fuzzing campaign, maximizing the potential for impact.
AB - Fuzz testing is often automated, but also frequently augmented by experts who insert themselves into the workflow in a greedy search for bugs. In this paper, we propose Homo in Machina, or HM-fuzzing, in which analyses guide the manual efforts, maximizing benefit. As one example of this paradigm, we introduce compartment analysis. Compartment analysis uses a whole-program dominator analysis to estimate the utility of reaching new code, and combines this with a dynamic analysis indicating drastically under-covered edges guarding that code. This results in a prioritized list of compartments, i.e., large, uncovered parts of the pro-gram semantically partitioned and largely unreachable given the current corpus of inputs under consideration. A human can use this categorization and ranking of compartments directly to focus manual effort, finding or fashioning inputs that make the compartments available for future fuzzing. We evaluate the effect of compartment analysis on seven projects within the OSS-Fuzz corpus where we see coverage improvements over AFL++ as high as 94%, with a median of 13%. We further observe that the determination of compartments is highly stable and thus can be done early in a fuzzing campaign, maximizing the potential for impact.
KW - fuzz testing
KW - fuzzing
UR - http://www.scopus.com/inward/record.url?scp=85161844281&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85161844281&partnerID=8YFLogxK
U2 - 10.1109/ICST57152.2023.00020
DO - 10.1109/ICST57152.2023.00020
M3 - Conference contribution
AN - SCOPUS:85161844281
T3 - Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation, ICST 2023
SP - 117
EP - 128
BT - Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation, ICST 2023
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 16 April 2023 through 20 April 2023
ER -