Part 1: Introduction to mutation testing
Welcome to my series about mutation testing. I will start with a quick general overview about mutation testing and then switch over to concrete examples using Stryker.Net and C#. The code examples can be understood by anyone with some programming experience, but if your looking for tooling examples then this is only useful for users of .Net.
Also a quick disclaimer, I’m actually quite new to mutation testing myself and could have some misconceptions about mutation testing. Should you find errors or have other feedback, please feel free to open an issue in the sample repository or write me on twitter.
Series Overview
- Part 1: Introduction to mutation testing
- Part 2: Setting up Stryker.Net
- Part 3: Limitations of mutation testing
You can find all code examples in my MutationTestingExample repository on Github.
What is mutation testing?
Mutation testing is the act of purposefully inserting faults in your production code and the running your test suite to check, whether it will find the fault.
Faults are not random, but selected from a list of so called mutators. This list depends on the chosen programming language and tooling. An example of an mutator would be switching + to - in an arithmetic operation.
What is the goal of mutation testing?
Mutation aims to close a gap in testing. We check our production code for correctness using tests, but how do we know our tests are correct?
With mutation testing we check how good the tests match our production code and know how much actually is tested (in theory, practially speaking there are some caveats).
What do I need to do mutation testing?
A prerequisite for doing mutation testing is having a test suite with decent test coverage.
It doesn’t need to be 100%, I’m not opening that can of worms, but changes in uncovered code will never be found, making mutating it a waste of time and resource.