In this part I will show you how to set up Stryker.Net and demonstrate usage with an example you can follow along.

Series Overview

You can find all code examples in my MutationTestingExample repository on Github.

Installing Stryker

Before we can install Stryker, we need to have the .Net 5 sdk on our machine. You can download the installer for your platform from dotnet.microsoft.com.

After installing .Net 5, run the command dotnet tool install -g dotnet-stryker to install stryker as a dotnet global tool.

Alternatively you can install it just for one project. You find the instructions in the Stryker.Net documentation.

Basic Usage

The simplest way to use Stryker is to call dotnet stryker while in the test directory i. e. where the .csproj of your tests lies. In our case you would run dotnet stryker in <root>/MutationTesting.Tests. This will produce an folder StrykerOutput containing all logs and reports of each run. Now select the folder with the latest timestamp and open reports/mutation-report.html in a html capable viewer like your browser.

Alternatively you can give Stryker the path to the solution and it will find all projects together with their tests. It will then mutate all of them and create the folder StrykerOutput in the working directory. In the example repository you would change to the root and call

dotnet stryker --solution-path MutationTesting.sln

Both relative and absolute paths are supported.

There are many more configuration options, but for more details about them, head over to the corresponding documentation. If you have a lot of configuration then you probably should use a config file

Take care with array options. Since Stryker.Net is originally a port of StrykerJS, it has still a similar handling of array options. To pass an array you have to write it like this:

"['element1','element2']"

There is one option I just have to mention and that is --diff. It does enable the diff feature meaning only changed files will be mutated. Especially for larger projects this will greatly reduce the execution time of the mutation tests.

Using Stryker with an example

I’ve written a small example to demonstrate Stryker with actual code. Each phase of my examples has a dedicated commit and here you’ll want to start with commit f3157dd, here the command: git checkout f3157dd.

At the core of this example is the StudentValidator class, which does check whether a student is valid (e.g. for visiting a course). I skipped the Student class, as it just contains the two properties.

To ensure our validator works correctly, I’ve written two tests, one for the happy path of having a valid student and one for the case of an invalid student.

Now run dotnet test and you’ll see both tests pass. If you have any tool for checking code coverage, you’ll also see that we have StudentValidator fully covered.

Well, let’s see what Stryker has to say about that. Run dotnet stryker --solution-path MutationTesting.sln if you haven’t already and check StudentValidator in the report. It should be like this report and show one surviving mutant.

The mutants are numbered and you can expand them by clicking (or tapping) their number. In this case it will show mutant zero as surviving. This mutator swapped && for || and the tests did not catch that.

Looking at the test we see, that for invalid students we always have both properties invalid. We never check with just one invalid property where the && actually would matter. Here mutation testing acutally showed us a part of our functionality which was not tested, even though it was exercised by tests.

This hole in our test suite is easily fixed. I add a test where the student has a valid age, but hasn’t paid. Now we’re at commit 057ffa5 and the tests look like this:

We now run dotnet stryker --solution-path MutationTesting.sln to see whether our test actually helped. Indeed, in the new report we see the mutant did not survive.

Previous Part | All | Next Part: Limitations of mutation testing