Skip to content

Update mutation-testing.md #46064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
43 changes: 43 additions & 0 deletions docs/core/testing/mutation-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,49 @@

For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation.

## Interpreting Mutation Testing Results

After running Stryker.NET, you’ll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results:

- **Killed Mutants**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors.

- **Survived Mutants**: These changes were not caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real.

- **Timeout Mutants**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed.

> **Note**: Don't chase a 100% mutation score. Focus instead on high risk or business critical areas where undetected bugs would be most costly.

## CI/CD Pipelines

Check failure on line 125 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Headings should be surrounded by blank lines

docs/core/testing/mutation-testing.md:125 MD022/blanks-around-headings Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "## CI/CD Pipelines"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md022.md
Mutation testing can be easily integrated into your CI/CD pipelines. For example, you can configure Stryker.NET to run as part of your Azure Pipelines (or GitHub Actions) workflow and enforce quality gates based on thresholds.
```

Check failure on line 127 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Fenced code blocks should be surrounded by blank lines

docs/core/testing/mutation-testing.md:127 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md031.md
"thresholds": {
"high": 85,
"low": 65,
"break": 0
}
```

Check failure on line 133 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Fenced code blocks should be surrounded by blank lines

docs/core/testing/mutation-testing.md:133 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md031.md
Comment on lines +129 to +135
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this here? There's no mention of config yet, it's just a code block with values that are not described anywhere. Let's remove this.

Copy link
Author

@cosminvladutu cosminvladutu Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove it, but we are talking about "quality thresholds". This is an example of what a threshold looks like. It's your call if you want it gone or not, but in my opinion, it should remain.

## Costumization

Check failure on line 134 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Headings should be surrounded by blank lines

docs/core/testing/mutation-testing.md:134 MD022/blanks-around-headings Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "## Costumization"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md022.md

Check failure on line 134 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Headings should be surrounded by blank lines

docs/core/testing/mutation-testing.md:134 MD022/blanks-around-headings Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Above] [Context: "## Costumization"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md022.md
Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file.
```

Check failure on line 136 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Fenced code blocks should be surrounded by blank lines

docs/core/testing/mutation-testing.md:136 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md031.md
{
"ignoreMutations": [
"ToString",
"Equals",
"GetHashCode"
],
"ignore-methods": [
"*Logs"
]
"mutate": [
"!**/Migrations/*",
"!**/*.Designer.cs"
]
}
```

Check failure on line 151 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Fenced code blocks should be surrounded by blank lines

docs/core/testing/mutation-testing.md:151 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md031.md
- **IgnoreMutations**: These are the methods or expressions that you want to ignore because they are noisy or you consider not needed based on your application logic needs. They will show up in your reports as Ignored.

Check failure on line 152 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Lists should be surrounded by blank lines

docs/core/testing/mutation-testing.md:152 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- **IgnoreMutations**: These a..."] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md032.md
- **Ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs".
- **Mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a Migrations folder and all .Designer.cs files (which are usually autogenerated)

Check failure on line 154 in docs/core/testing/mutation-testing.md

View workflow job for this annotation

GitHub Actions / lint

Trailing spaces

docs/core/testing/mutation-testing.md:154:274 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1] https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/md009.md

## Incremental improvement

If, after changing your code, the unit tests pass successfully, then they aren't sufficiently robust, and the mutant survived.
Expand Down
Loading