Cover MSTESTxxxx analyzer diagnostics in writing-mstest-tests skill#794
Cover MSTESTxxxx analyzer diagnostics in writing-mstest-tests skill#794Evangelink wants to merge 6 commits into
Conversation
Add a 'Fix MSTest analyzer diagnostics' workflow step mapping the common MSTESTxxxx rules to their idiomatic fixes, plus MSTestAnalysisMode guidance, instead of creating one skill per rule. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Skill Coverage Report
Uncovered:
|
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Extends the writing-mstest-tests skill content to explicitly cover fixing MSTest analyzer diagnostics (MSTESTxxxx) within the existing workflow, instead of creating many separate per-rule skills.
Changes:
- Updates the skill description/triggering text to include fixing
MSTESTxxxxanalyzer diagnostics. - Adds a new “Step 8” section with a rule → problem → fix table and guidance on
MSTestAnalysisMode.
Show a summary per file
| File | Description |
|---|---|
| plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | Adds MSTest analyzer diagnostics guidance (new Step 8) and updates description/triggers to cover MSTESTxxxx rules. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 2
…rammar - Don't tie MSTest.Analyzers availability to TestFramework 3.7; note metapackage/SDK/explicit reference. - Fix grammatically broken fix text for the MSTEST0002-0014 layout row. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/evaluate |
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps
▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
|
✅ Evaluation passed for |
…om code-testing-agent In plugin runs, code-testing-agent (generic 'write/comprehensive unit tests') was stealing activation from writing-mstest-tests for MSTest-specific prompts. Broaden code-testing-agent's DO NOT USE carve-out to defer writing/fixing/modernizing MSTest-specific tests, assertions, attributes, and lifecycle to writing-mstest-tests, and have writing-mstest-tests claim 'comprehensive MSTest unit tests'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/evaluate |
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps
▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
…ugin activation Post-fix eval (run 27836324404) showed code-testing-agent no longer steals (no sibling fires), but string/comparison/reference-assertion scenarios still don't activate in the plugin run — their trigger keywords (StartsWith, EndsWith, MatchesRegex, IsGreaterThan, IsLessThan, IsInRange, AreSame) had been trimmed for budget. Rebuild the description to restore all eval-relevant assertion APIs and lead with write/create/modernize/fix, staying at 1013 chars. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/evaluate |
Skill Validation Results
[1] Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps
▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
What
Extends the existing
writing-mstest-testsskill to cover the MSTest analyzer rules (MSTESTxxxx) instead of adding a new skill per rule.Why
There are 63
MSTESTxxxxrules. They are Roslyn analyzers that already self-surface during build and in the IDE (with messages and, in most cases, automated code fixes). What an agent needs is the idiomatic fix + rationale, which is content — not 63 separate, overlapping, activation-gated skills that would cannibalize each other''s activation and require a web of "DO NOT USE" redirects. The existing skill already teaches the correct patterns these analyzers enforce, so this consolidates the remaining gaps in one place.Changes
rule → problem → fixtable covering the high-value rules not previously called out (MSTEST0023, 0025, 0032, 0038, 0044, 0052, 0024, 0036, 0061, the 0042/0060 duplicates, the 0002–0014 layout family), cross-linked to existing steps for the rules already covered (0006/0017/0037/0039/0046/0045-0049-0054).MSTestAnalysisMode(None/Default/Recommended/All) guidance and the opt-in rules note.fix MSTEST analyzer diagnostics (MSTESTxxxx rules)) and a When-to-Use trigger; trimmed the verbose assertion-API list to keep the description at 1000 chars (under the 1024 cap).Validation
skill-validator check --plugin ./plugins/dotnet-test→ all checks pass (27 skills, 11 agents). Only a soft "approaching comprehensive token range" warning on the skill size.