Combining Evolutionary Mutation Testing with Random Selection

Abstract

Mutation testing is a well-known fault-based technique that has been applied to different domains as new technologies have appeared. Evolutionary Mutation Testing (EMT) finds mutants that are useful to produce new test cases. It uses evolutionary algorithms to reduce the number of mutants that are generated, keeping as many difficult to kill and stubborn mutants (strong mutants) as possible in the reduced set. Given the popularity of real-time systems, the MuEPL mutation system was developed for the Esper Event Processing Language (EPL), a query language aimed at the Internet of Things (IoT). In past work, EMT was integrated into MuEPL, and it reduced the cost of finding strong mutants in some EPL queries but not in others. This study takes a step forward by proposing and evaluating two metaheuristics for EMT that combine EMT and random selection: one which bootstraps the hall of fame with a random subset (Bootstrapped EMT), and one which falls back to random selection after a certain point (Inverse EMT). While BEMT is shown to outperform IEMT in most cases, BEMT has not managed to outperform EMT. An additional experiment studies the impact of low-quality mutation operators in the relative performance of BEMT, IEMT and plain EMT. Results suggest that the MuEPL RRO operator was the reason for the poor performance of EMT in some scenarios.