-
Notifications
You must be signed in to change notification settings - Fork 6k
Update mutation-testing.md #46064
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Update mutation-testing.md #46064
Changes from 5 commits
d5099dd
bb1a6e2
28300be
4754d76
37e8bdc
29c2a22
22e11cc
f1b979b
4fc3f1d
be74f01
5fa4967
5a1f869
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -110,6 +110,49 @@ | |
|
||
For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation. | ||
|
||
## Interpreting Mutation Testing Results | ||
|
||
After running Stryker.NET, you’ll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- **Killed Mutants**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- **Survived Mutants**: These changes were not caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- **Timeout Mutants**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
> **Note**: Don't chase a 100% mutation score. Focus instead on high risk or business critical areas where undetected bugs would be most costly. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## CI/CD Pipelines | ||
Check failure on line 125 in docs/core/testing/mutation-testing.md
|
||
Mutation testing can be easily integrated into your CI/CD pipelines. For example, you can configure Stryker.NET to run as part of your Azure Pipelines (or GitHub Actions) workflow and enforce quality gates based on thresholds. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
Check failure on line 127 in docs/core/testing/mutation-testing.md
|
||
"thresholds": { | ||
"high": 85, | ||
"low": 65, | ||
"break": 0 | ||
} | ||
``` | ||
Check failure on line 133 in docs/core/testing/mutation-testing.md
|
||
Comment on lines
+129
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this here? There's no mention of config yet, it's just a code block with values that are not described anywhere. Let's remove this. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can remove it, but we are talking about "quality thresholds". This is an example of what a threshold looks like. It's your call if you want it gone or not, but in my opinion, it should remain. |
||
## Costumization | ||
Check failure on line 134 in docs/core/testing/mutation-testing.md
|
||
Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
Check failure on line 136 in docs/core/testing/mutation-testing.md
|
||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{ | ||
"ignoreMutations": [ | ||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"ToString", | ||
"Equals", | ||
"GetHashCode" | ||
], | ||
"ignore-methods": [ | ||
"*Logs" | ||
] | ||
"mutate": [ | ||
"!**/Migrations/*", | ||
"!**/*.Designer.cs" | ||
] | ||
} | ||
``` | ||
Check failure on line 151 in docs/core/testing/mutation-testing.md
|
||
- **IgnoreMutations**: These are the methods or expressions that you want to ignore because they are noisy or you consider not needed based on your application logic needs. They will show up in your reports as Ignored. | ||
Check failure on line 152 in docs/core/testing/mutation-testing.md
|
||
- **Ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". | ||
- **Mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a Migrations folder and all .Designer.cs files (which are usually autogenerated) | ||
Check failure on line 154 in docs/core/testing/mutation-testing.md
|
||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
cosminvladutu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
## Incremental improvement | ||
|
||
If, after changing your code, the unit tests pass successfully, then they aren't sufficiently robust, and the mutant survived. | ||
|
Uh oh!
There was an error while loading. Please reload this page.