Mutation Testing

Learned in A2 of SE465.

Mutation Testing

• A method of inserting artificial defects (mutants) in the production code.
• To test whether the test cases kill the mutants (a test fails is the one that kills the mutant),
• thereby assessing the quality of the test cases. Goals:
• To find weak tests (test cases that cannot kill the mutants) and improve them.

From lecture:

Mutation Testing

• Fault-based testing: directed towards “typical” faults that could occur in a program
• Basic idea:
  • Take a program and test data generated for that program
  • Create a number of similar programs (mutants), each differing from the original in one small way, i.e., each possessing a fault (e.g., replace addition operator by multiplication operator)
  • The original test data is then run through the mutants
  • If test data detects all differences in mutants, then the mutants are said to be dead, and the test set is *adequate

Mechanism

The mechanism of mutation testing is to first introduce mutations (errors) into the source code, then run the project’s test suite, and if the test fails, the mutant will be killed, otherwise, it will survive. Therefore, the mutation testing results demonstrate the effectiveness of the existing test cases. Below is a figure showing the mechanism of mutation testing: