TY - GEN
T1 - Bug synthesis
T2 - 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018
AU - Roy, Subhajit
AU - Pandey, Awanish
AU - Dolan-Gavitt, Brendan
AU - Hu, Yu
N1 - Publisher Copyright:
© 2018 Association for Computing Machinery.
PY - 2018/10/26
Y1 - 2018/10/26
N2 - In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques-symbolic execution, constraint-based program synthesis and model counting-to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite.We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.
AB - In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques-symbolic execution, constraint-based program synthesis and model counting-to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite.We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.
KW - Bug Injection
KW - Constraintbased Synthesis
KW - Program Synthesis
KW - Symbolic Execution
UR - http://www.scopus.com/inward/record.url?scp=85058301253&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85058301253&partnerID=8YFLogxK
U2 - 10.1145/3236024.3236084
DO - 10.1145/3236024.3236084
M3 - Conference contribution
AN - SCOPUS:85058301253
T3 - ESEC/FSE 2018 - Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
SP - 224
EP - 234
BT - ESEC/FSE 2018 - Proceedings of the 2018 26th ACM Joint Meeting on European So ftware Engineering Conference and Symposium on the Foundations of So ftware Engineering
A2 - Garci, Alessandro
A2 - Pasareanu, Corina S.
A2 - Leavens, Gary T.
PB - Association for Computing Machinery, Inc
Y2 - 4 November 2018 through 9 November 2018
ER -