From 97db34c8d1c82b209f9029a7533d7e5f20b64d9a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 15:52:41 +0200 Subject: [PATCH 1/6] Cover MSTESTxxxx analyzer diagnostics in writing-mstest-tests skill Add a 'Fix MSTest analyzer diagnostics' workflow step mapping the common MSTESTxxxx rules to their idiomatic fixes, plus MSTestAnalysisMode guidance, instead of creating one skill per rule. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/writing-mstest-tests/SKILL.md | 50 +++++++++++++++++-- 1 file changed, 47 insertions(+), 3 deletions(-) diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index 1700cc94b9..67b01354cb 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -6,13 +6,12 @@ description: > USE FOR: write or create MSTest unit tests, fix or modernize MSTest assertions, better MSTest assertion than Assert.IsTrue, replace hard cast with MSTest type assertion, MSTest assertion APIs (IsInstanceOfType, Contains, ContainsSingle, HasCount, - IsEmpty, IsNotEmpty, DoesNotContain, StartsWith, EndsWith, MatchesRegex, - IsGreaterThan, IsInRange, IsNull), + IsEmpty, IsNotEmpty, DoesNotContain, IsNull), fix swapped Assert.AreEqual arguments, replace ExpectedException with Assert.Throws, data-driven tests (DataRow, DynamicData, ValueTuples), test lifecycle (sealed classes, TestInitialize, TestCleanup), async tests and cancellation tokens, test parallelization (Parallelize / DoNotParallelize), - MSTest.Sdk project setup. + MSTest.Sdk project setup, fix MSTEST analyzer diagnostics (MSTESTxxxx rules). DO NOT USE FOR: broad test quality audits (use test-anti-patterns), running tests (use run-tests), MSTest version migration (use migrate-mstest-v1v2-to-v3 or migrate-mstest-v3-to-v4), xUnit/NUnit/TUnit, or non-.NET languages. @@ -33,6 +32,7 @@ Help users write effective, modern unit tests with MSTest 3.x/4.x using current - User needs help fixing a specific MSTest test bug or failing assertion - User asks to fix swapped `Assert.AreEqual` argument order (expected first, actual second) - User asks to convert `DynamicData` from `IEnumerable` to ValueTuple-based data +- User asks to fix or understand an MSTest analyzer diagnostic (an `MSTESTxxxx` warning/error) ## When Not to Use @@ -366,3 +366,47 @@ public void LocalOnly_InteractiveTest() { } [DoNotParallelize] // Opt out specific classes public sealed class DatabaseIntegrationTests { } ``` + +### Step 8: Fix MSTest analyzer diagnostics (MSTESTxxxx) + +Starting with MSTest.TestFramework 3.7, the `MSTest.Analyzers` package ships with the framework and reports `MSTESTxxxx` diagnostics during build and in the IDE. Most rules have an automated code fix (light bulb) in Visual Studio. When fixing one by hand, apply the idiomatic change below rather than suppressing the rule. + +When asked to "fix MSTESTxxxx", look up the rule in the table, apply the fix, and rebuild to confirm the diagnostic is gone. Full reference: . + +#### Common diagnostics and their fixes + +| Rule | Problem | Fix | +|---|---|---| +| MSTEST0006 | `[ExpectedException]` used | Replace with `Assert.Throws` / `Assert.ThrowsExactly` (Step 3) | +| MSTEST0017 | `Assert.AreEqual` args swapped | Put `expected` first, `actual` second | +| MSTEST0023 | Negated boolean assertion (`Assert.IsTrue(!x)`) | Use `Assert.IsFalse(x)` | +| MSTEST0025 | Always-false condition asserted | Use `Assert.Fail("reason")` | +| MSTEST0032 | Always-true assert condition | Remove or correct the assertion | +| MSTEST0037 | Sub-optimal assert (`IsTrue(x == null)`) | Use the specific assert (`Assert.IsNull`, `HasCount`, etc.) (Step 3) | +| MSTEST0038 | `Assert.AreSame` on value types | Use `Assert.AreEqual` (value types box to distinct references) | +| MSTEST0039 | Legacy `Assert.ThrowsException` | Use `Assert.Throws` / `Assert.ThrowsExactly` (+ `Async` variants) | +| MSTEST0044 | `[DataTestMethod]` used | Replace with `[TestMethod]` (it now supports data rows) | +| MSTEST0046 | `StringAssert` used | Use the equivalent `Assert` method (`Assert.Contains`, `StartsWith`, ...) | +| MSTEST0052 | Explicit `DynamicDataSourceType` | Drop it — the source type is inferred | +| MSTEST0042 / MSTEST0060 | Duplicate `[DataRow]` / `[TestMethod]` | Remove the duplicate attribute | +| MSTEST0024 | Static `TestContext` field | Make it an instance member (Step 5) | +| MSTEST0045 / MSTEST0049 / MSTEST0054 | Timeout/token not cooperative | Flow `TestContext.CancellationToken` into the awaited call (Step 6) | +| MSTEST0036 | Member shadows a base test member | Rename or use `override` instead of `new` | +| MSTEST0061 | Runtime OS check inside a test | Use `[OSCondition(...)]` (Step 7) | +| MSTEST0002 / MSTEST0003 / MSTEST0005 / MSTEST0007–0014 | Invalid test class / method / fixture / `TestContext` / data-source layout | Fix the signature (public, correct return type, parameters, `static` where required) the rule names | + +#### Tuning which rules are enforced + +Use the `MSTestAnalysisMode` MSBuild property (MSTest 3.8+) to control the rule set globally: + +```xml + + + Recommended + +``` + +- `Recommended` escalates info-level rules to warnings and is the mode most projects should adopt. +- A handful of rules are completely opt-in (e.g. MSTEST0015, MSTEST0019–0022); enable them per project via `.editorconfig` when you want their convention enforced. +- Prefer fixing the underlying code over suppressing a diagnostic. Suppress only with a documented justification. + From 3d6b0409500c6c277b5a2578b8b0e063c39d6b35 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 16:15:43 +0200 Subject: [PATCH 2/6] Fix MD012 markdown lint (trailing blank line) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | 1 - 1 file changed, 1 deletion(-) diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index 67b01354cb..3fa5329b73 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -409,4 +409,3 @@ Use the `MSTestAnalysisMode` MSBuild property (MSTest 3.8+) to control the rule - `Recommended` escalates info-level rules to warnings and is the mode most projects should adopt. - A handful of rules are completely opt-in (e.g. MSTEST0015, MSTEST0019–0022); enable them per project via `.editorconfig` when you want their convention enforced. - Prefer fixing the underlying code over suppressing a diagnostic. Suppress only with a documented justification. - From f77c2604410c96e10eba616c75c8e7e03ed19945 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 16:17:43 +0200 Subject: [PATCH 3/6] Address review feedback: reword analyzer availability and fix table grammar - Don't tie MSTest.Analyzers availability to TestFramework 3.7; note metapackage/SDK/explicit reference. - Fix grammatically broken fix text for the MSTEST0002-0014 layout row. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index 3fa5329b73..cc6507f2f3 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -369,7 +369,7 @@ public sealed class DatabaseIntegrationTests { } ### Step 8: Fix MSTest analyzer diagnostics (MSTESTxxxx) -Starting with MSTest.TestFramework 3.7, the `MSTest.Analyzers` package ships with the framework and reports `MSTESTxxxx` diagnostics during build and in the IDE. Most rules have an automated code fix (light bulb) in Visual Studio. When fixing one by hand, apply the idiomatic change below rather than suppressing the rule. +The `MSTest.Analyzers` package reports `MSTESTxxxx` diagnostics during build and in the IDE. The analyzers come in automatically with the modern `MSTest` metapackage and `MSTest.Sdk` (and are bundled with `MSTest.TestFramework` 3.7+); for other setups, reference `MSTest.Analyzers` explicitly. Most rules have an automated code fix (light bulb) in Visual Studio. When fixing one by hand, apply the idiomatic change below rather than suppressing the rule. When asked to "fix MSTESTxxxx", look up the rule in the table, apply the fix, and rebuild to confirm the diagnostic is gone. Full reference: . @@ -393,7 +393,7 @@ When asked to "fix MSTESTxxxx", look up the rule in the table, apply the fix, an | MSTEST0045 / MSTEST0049 / MSTEST0054 | Timeout/token not cooperative | Flow `TestContext.CancellationToken` into the awaited call (Step 6) | | MSTEST0036 | Member shadows a base test member | Rename or use `override` instead of `new` | | MSTEST0061 | Runtime OS check inside a test | Use `[OSCondition(...)]` (Step 7) | -| MSTEST0002 / MSTEST0003 / MSTEST0005 / MSTEST0007–0014 | Invalid test class / method / fixture / `TestContext` / data-source layout | Fix the signature (public, correct return type, parameters, `static` where required) the rule names | +| MSTEST0002 / MSTEST0003 / MSTEST0005 / MSTEST0007–0014 | Invalid test class / method / fixture / `TestContext` / data-source layout | Correct the signature named by the rule (e.g. make it public, fix the return type and parameters, add `static` where required) | #### Tuning which rules are enforced From 15cfbbaf67a3a47e83a2b519f39d1949b4a82468 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 17:02:20 +0200 Subject: [PATCH 4/6] Clarify the MSTESTxxxx table is non-exhaustive; defer to full reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index cc6507f2f3..4dcc9acc56 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -371,7 +371,7 @@ public sealed class DatabaseIntegrationTests { } The `MSTest.Analyzers` package reports `MSTESTxxxx` diagnostics during build and in the IDE. The analyzers come in automatically with the modern `MSTest` metapackage and `MSTest.Sdk` (and are bundled with `MSTest.TestFramework` 3.7+); for other setups, reference `MSTest.Analyzers` explicitly. Most rules have an automated code fix (light bulb) in Visual Studio. When fixing one by hand, apply the idiomatic change below rather than suppressing the rule. -When asked to "fix MSTESTxxxx", look up the rule in the table, apply the fix, and rebuild to confirm the diagnostic is gone. Full reference: . +When asked to "fix MSTESTxxxx", look it up in the table of common diagnostics below, apply the fix, and rebuild to confirm the diagnostic is gone. The table is not exhaustive — for any rule it does not list, consult the full reference and apply the documented guidance: . #### Common diagnostics and their fixes From c263a61f795a2aabc2788e97629d5eb54350b824 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 17:56:34 +0200 Subject: [PATCH 5/6] Fix writing-mstest-tests activation: redirect MSTest-specific work from code-testing-agent In plugin runs, code-testing-agent (generic 'write/comprehensive unit tests') was stealing activation from writing-mstest-tests for MSTest-specific prompts. Broaden code-testing-agent's DO NOT USE carve-out to defer writing/fixing/modernizing MSTest-specific tests, assertions, attributes, and lifecycle to writing-mstest-tests, and have writing-mstest-tests claim 'comprehensive MSTest unit tests'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- plugins/dotnet-test/skills/code-testing-agent/SKILL.md | 6 +++--- plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/plugins/dotnet-test/skills/code-testing-agent/SKILL.md b/plugins/dotnet-test/skills/code-testing-agent/SKILL.md index eb85bbf6ad..f5cbcffa25 100644 --- a/plugins/dotnet-test/skills/code-testing-agent/SKILL.md +++ b/plugins/dotnet-test/skills/code-testing-agent/SKILL.md @@ -2,8 +2,7 @@ name: code-testing-agent description: >- Generates and writes new unit tests for any programming language — - scaffolds .NET test projects, pytest suites, Vitest/Jest suites, - Go test files, and JUnit suites, and configures coverage tooling + scaffolds test projects and configures coverage tooling (coverlet, pytest-cov, @vitest/coverage-v8) as part of test generation. Use when asked to generate tests, generate pytest tests, generate Vitest tests, write unit tests, add tests, improve @@ -16,7 +15,8 @@ description: >- planning, and implementation pipeline so tests compile and pass. DO NOT USE FOR: running existing tests (use run-tests); analyzing existing coverage reports (use coverage-analysis or crap-score); - MSTest modernization (use writing-mstest-tests). + writing, fixing, or modernizing MSTest-specific tests, assertions, + attributes, or lifecycle (use writing-mstest-tests). license: MIT --- diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index 4dcc9acc56..11a2aa531d 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -3,7 +3,7 @@ name: writing-mstest-tests description: > Write new MSTest unit tests and fix existing MSTest code using MSTest 3.x/4.x modern APIs and best practices. - USE FOR: write or create MSTest unit tests, fix or modernize MSTest assertions, + USE FOR: write, create, or modernize comprehensive MSTest unit tests, fix MSTest assertions, better MSTest assertion than Assert.IsTrue, replace hard cast with MSTest type assertion, MSTest assertion APIs (IsInstanceOfType, Contains, ContainsSingle, HasCount, IsEmpty, IsNotEmpty, DoesNotContain, IsNull), From 7e553b51f9796b219537fef735c7a6b5bef4b257 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Fri, 19 Jun 2026 18:38:28 +0200 Subject: [PATCH 6/6] Restore assertion keywords to writing-mstest-tests description for plugin activation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Post-fix eval (run 27836324404) showed code-testing-agent no longer steals (no sibling fires), but string/comparison/reference-assertion scenarios still don't activate in the plugin run — their trigger keywords (StartsWith, EndsWith, MatchesRegex, IsGreaterThan, IsLessThan, IsInRange, AreSame) had been trimmed for budget. Rebuild the description to restore all eval-relevant assertion APIs and lead with write/create/modernize/fix, staying at 1013 chars. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/writing-mstest-tests/SKILL.md | 27 +++++++++---------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md index 11a2aa531d..c14b8b2635 100644 --- a/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md +++ b/plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md @@ -1,20 +1,19 @@ --- name: writing-mstest-tests description: > - Write new MSTest unit tests and fix existing MSTest code using MSTest 3.x/4.x - modern APIs and best practices. - USE FOR: write, create, or modernize comprehensive MSTest unit tests, fix MSTest assertions, - better MSTest assertion than Assert.IsTrue, replace hard cast with MSTest type assertion, - MSTest assertion APIs (IsInstanceOfType, Contains, ContainsSingle, HasCount, - IsEmpty, IsNotEmpty, DoesNotContain, IsNull), - fix swapped Assert.AreEqual arguments, replace ExpectedException with Assert.Throws, - data-driven tests (DataRow, DynamicData, ValueTuples), - test lifecycle (sealed classes, TestInitialize, TestCleanup), - async tests and cancellation tokens, test parallelization (Parallelize / DoNotParallelize), - MSTest.Sdk project setup, fix MSTEST analyzer diagnostics (MSTESTxxxx rules). - DO NOT USE FOR: broad test quality audits (use test-anti-patterns), - running tests (use run-tests), MSTest version migration (use migrate-mstest-v1v2-to-v3 - or migrate-mstest-v3-to-v4), xUnit/NUnit/TUnit, or non-.NET languages. + Write, create, modernize, or fix comprehensive MSTest unit tests with MSTest 3.x/4.x APIs. + USE FOR: write or create MSTest unit tests, fix/modernize MSTest assertions, + better MSTest assertion than Assert.IsTrue, replace hard cast with type check (IsInstanceOfType), + MSTest assertion APIs (Contains, ContainsSingle, HasCount, IsEmpty, IsNotEmpty, DoesNotContain, + AreSame, IsNull, StartsWith, EndsWith, MatchesRegex, IsGreaterThan, IsLessThan, IsInRange), + swapped Assert.AreEqual args, replace ExpectedException with Assert.Throws, + data-driven (DataRow, DynamicData, ValueTuples), + lifecycle (TestInitialize, TestCleanup, TestContext), + async tests and cancellation tokens, conditional execution/retry/cleanup (OSCondition, Retry), + parallelization (Parallelize/DoNotParallelize), MSTest.Sdk setup, MSTESTxxxx analyzer fixes. + DO NOT USE FOR: test quality audits (use test-anti-patterns), + running tests (use run-tests), MSTest version migration (use the migrate-mstest skills), + xUnit/NUnit/TUnit, or non-.NET languages. license: MIT ---