An empirical comparison of two safe regression test selection techniques

Regression test selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modified program. Safe regression test selection techniques guarantee (under specific conditions) that the selected subset will not omit faults that could have been revealed by the entire suite. Many regression test selection techniques have been described in the literature. Empirical studies of some of these techniques have shown that they can be beneficial, but only a few studies have empirically compared different techniques, and fewer still have considered safe techniques. In this paper, we report the results of a comparative empirical study of implementations of two safe regression test selection techniques: DejaVu and Pytia. Our results show that, despite differences in their approaches, and despite the theoretically greater ability of DejaVu to select smaller test suites than Pythia, the two techniques often selected equivalent test suites in practice, at comparable costs. These results suggest that factors such as ease of implementation, generality, and availability of supporting tools and data may play a greater role than cost-effectiveness for practitioners choosing between these techniques.