dotnet-test

Skills and a GitHub Copilot test-engineer agent for running, generating, repairing, analyzing, and improving tests. Originally built for .NET (MSTest, xUnit, NUnit, TUnit) and platforms (VSTest, Microsoft.Testing.Platform), the test-engineering workflows are polyglot and also work with Python (pytest/unittest), TypeScript/JavaScript (Jest/Vitest/Mocha/Jasmine/node:test), Java (JUnit 4/5/TestNG), Go (testing/testify), Ruby (RSpec/Minitest), Rust (built-in/proptest), Swift (XCTest/Swift Testing), Kotlin (JUnit/Kotest), PowerShell (Pester), and C++ (GoogleTest/Catch2/doctest/Boost.Test).

Test framework/platform migration (MSTest/xUnit upgrades, xUnit → MSTest, VSTest → Microsoft.Testing.Platform) lives in the separate dotnet-test-migration plugin.

When to use this plugin

• Run tests (.NET only) — execute SDK-style projects with dotnet test, or preserve a classic project's checked-in MSBuild + VSTest/MSTest command
• Generate tests (polyglot) — scaffold comprehensive unit tests for any language via a multi-agent pipeline
• Migrate tests (.NET only) — see the separate dotnet-test-migration plugin (MSTest v1/v2 → v3 → v4, xUnit v2 → v3, xUnit → MSTest, VSTest → Microsoft.Testing.Platform)
• Audit test quality (polyglot) — detect anti-patterns, test smells, assertion gaps, and (for .NET) coverage risks
• Validate changed behavior (polyglot) — bind regression/feature detection evidence to exact revisions, without mistaking static predictions or build failures for proof
• Improve testability (.NET only) — find static dependencies, generate wrappers, and migrate call sites to injectable abstractions
• Measure coverage (.NET only) — collect code coverage, compute CRAP scores, and surface risk hotspots

Skills

Test execution

Skill	Description
run-tests	Run .NET tests with project-system/platform/framework detection, including classic non-SDK runner commands
mtp-hot-reload	Rapid test-fix iteration using MTP hot reload (edit code → re-run without rebuilding)

Test generation

Skill	Description
code-testing	Implicit entry skill for generating, repairing, and strengthening tests; broad work delegates to test-engineer
scaffold-dotnet-test-project (.NET)	Create a missing test project or repair its project/solution/filter wiring
writing-mstest-tests	Version-compatible MSTest authoring for modern and classic projects, including MSTest 3.x/4.x APIs

Test migration

Moved to the dotnet-test-migration plugin (migrate-mstest-v1v2-to-v3, migrate-mstest-v3-to-v4, migrate-xunit-to-xunit-v3, migrate-xunit-to-mstest, migrate-vstest-to-mtp, and the test-migration orchestrator agent).

Test quality & analysis (polyglot)

These seven skills are all polyglot. The suite-analysis skills use per-language reference files from test-analysis-extensions. grade-tests additionally embeds its own decision and scoring rubric so per-test Pass / Failed / Uncertain outcomes and supporting A-F quality grades stay consistent across calls. test-value-evidence is self-contained and assesses changed contracts rather than grading tests or auditing the suite.

Skill	Description
test-anti-patterns	Quick pragmatic scan for common test quality issues with severity ranking (any language)
test-smell-detection	Deep formal audit using academic test smell taxonomy (19 smell types, any language)
assertion-quality	Measure assertion variety and depth — find shallow tests that barely verify anything (any language)
test-gap-analysis	Analyze test blind spots through pseudo-mutations, expose read-only per-test evidence for grading, and verify or close gaps only when requested (any language)
test-value-evidence	Prove tests detect the actual pre-fix defect or buildable missing/wrong feature behavior; distinguish paired execution, static predictions, survival, equivalence, and missing evidence (any language)
test-tagging	Tag tests with standardized traits (smoke, regression, boundary, critical-path, etc.); auto-edits where the framework has canonical syntax, report-only otherwise
grade-tests	Assess curated tests with Pass, Failed, or Uncertain decisions, A-F quality detail, notes, and concrete improvement actions; unresolved or empty scopes omit the grade, and a valid empty scope returns Not applicable (any language)

Grading composes test-gap-analysis in explicit per-test-read-only mode: no test runs, production edits, mutation execution, suite audit, or agent recursion. Mutation methodology stays in that skill's bundled reference; grading retains its own scoring weights and ceilings. Each test owns only its claimed behavior, not its siblings' assertions or unrelated scenarios. Static evidence uses inferred likely kills or unverified candidate survivors, not executed mutation counts. Missing production context is N/A / unverified, not a deduction. The existing result and quality fields remain independent: a focused B can Pass without improvements, and an A can Fail for actionable debug output.

Focused grading checks can run without an evaluation matrix:

python -B tests\dotnet-test\grade-tests\test_regressions.py -v
python -B tests\dotnet-test\grade-tests\test_composition.py --cli <copilot-executable> --model <model-id> --results-dir <scratch-results>

The first command replays goldens and rejects malformed actions, extra test rows, and misleading mutation evidence. The second uses the shipping Copilot CLI, a copy of the production plugin, isolated configuration, and a host-supplied path to the actual bundled Python assertion reference. It requires successful grading and gap-analysis loads plus the owned read-only reference read and their completions before the final grading report. It verifies the target row's concrete improvement, rejects standalone execution/delegation and N/A fallbacks, and checks that every fixture and plugin file is unchanged. Generation/repair dormancy cases replay golden patches and prove that the requested assertions reject wrong costs/flags while production stays byte-for-byte unchanged. The CLI integration needs Copilot access via an existing token or authenticated GitHub CLI; it does not change user settings.

test-value-evidence separates change-specific detection proof from this grading contract and from general suite blind spots. Its canonical SKILL.md has no bundled dependencies; single-file consumers can vendor exact upstream bytes, record the producer commit/blob and SHA-256, and keep repository-specific adapters outside the canonical file. It does not enforce CI or authorize publication.

Focused offline calibration:

python -B tests\dotnet-test\test-value-evidence\test_regressions.py -v

This replays ten evidence-report goldens and three routing goldens, rejects misleading alternatives, checks complete workspace scope and native discovery, and runs the same three boundary tests against the actual defective and fixed bytes in fresh owned experiment directories. The production command grader starts Python in isolated mode before authenticating its staged helper, preventing workspace module shadowing, and replays exit codes, the named assertion, and all test counts without trusting narrated or stale red results. Its captured replay proves behavioral detection, not what the model executor itself ran. Recorder conformance is tested separately from the model-facing prompts, with authenticated calls in original-to-fixed order. Each call checks the source and unchanged-test digests before executing captured bytes in an isolated child; records include those digests, the named assertion, counts, and actual exits. Temporary test edits cannot manufacture accepted red evidence. The model-facing patch case asks for a minimal correction and a static assessment without command execution; it does not prescribe this evaluator's recorder API. Calibration also rejects unrelated edits, extra files, altered helpers, and skipped or misattributed results. Only the allowed changed production file may omit its terminal LF; preserved test bytes and all other product content remain exact. It is not a measured model-vs-baseline verdict. The Vally eval adds static-edit and advisory cases plus three sibling-routing dormancy guards; run it through the repository's normal evaluation entry point. The current model-facing matrix does not certify authorized paired execution; that path is calibrated offline and still needs trusted host instrumentation before it can be graded without directing the model to evaluator-specific APIs.

Coverage & risk (.NET only)

Skill	Description
coverage-analysis	Project-wide code coverage collection with CRAP score computation and risk hotspot reporting
crap-score	Calculate CRAP (Change Risk Anti-Patterns) scores for individual methods, classes, or files

For non-.NET languages, use the native coverage tool: coverage.py/pytest-cov (Python), jest --coverage/c8/nyc/vitest --coverage (JS/TS), JaCoCo (Java), go test -coverprofile (Go), SimpleCov (Ruby), cargo-tarpaulin/cargo-llvm-cov (Rust), xcrun llvm-cov (Swift), Kover (Kotlin), Pester's built-in code coverage (PowerShell), gcov/llvm-cov (C++).

Testability improvement (.NET only)

Skill	Description
detect-static-dependencies	Scan C# code for hard-to-test statics (DateTime.Now, File.*, HttpClient, etc.)
generate-testability-wrappers	Generate wrapper interfaces or guide adoption of built-in abstractions (TimeProvider, IFileSystem)
migrate-static-to-wrapper	Bulk-replace static call sites with injected wrapper calls and add constructor injection
testability-obstacle	Resolve one concrete ambient-dependency blocker and test the behavior through fixed/in-memory dependencies

Detection and reference data

Skill	Description
code-testing-extensions	Language-specific guidance loaded by the code-testing pipeline (test generation)
test-analysis-extensions	Language-specific guidance loaded by the polyglot analysis skills (test markers, assertion APIs, sleeps, skips, mystery-guest indicators, integration markers, tag-support capability)
platform-detection (.NET)	Directly detect SDK-style vs classic, VSTest vs MTP, and the test framework from project files
filter-syntax (.NET)	Test filter syntax reference for VSTest and MTP across all frameworks

Three reference skills (code-testing-extensions, test-analysis-extensions, and filter-syntax) set disable-model-invocation: true, so the CLI keeps them out of model invocation. Consumers read their bundled files directly from supplied/runtime-listed catalog paths, resolving language files relative to that catalog rather than the project workspace. They deliberately have no direct tests/dotnet-test/<skill>/eval.yaml: the experiment's skilled arm loads a single skill, which the model could never invoke here, so such an eval would compare two identical arms and score judge noise. They are measured through consumer outcomes — the polyglot analysis skills and grade-tests for test-analysis-extensions, code-testing for code-testing-extensions, and run-tests and mtp-hot-reload for filter-syntax. The run-tests eval covers VSTest expressions, MTP argument passing, xUnit v3 native filters, and TUnit tree-node filters.

platform-detection is model-invocable because identifying a project's runner is also a direct user task; run-tests and migration skills still load it as shared detection guidance. Its command-mode rules use an on-demand reference so platform/framework-only requests do not load or echo CLI-mode detail. filter-syntax remains reference-only and is measured through the filtered-command scenarios in the run-tests eval.

Agents

The agents below are GitHub Copilot .agent.md definitions. Codex plugin installs expose this plugin's skills, but not these agents or their static handoffs.

User-facing agents

Use this single entry-point agent for end-to-end test work:

Agent	Purpose
test-engineer	Generates, repairs, runs, audits, and improves tests while coordinating the internal specialists below

Focused quality reviews use one matching skill directly. Broad audits retain per-test/behavior evidence, reconcile summary counts, and give a risk-ranked repair order without automatically grading every test. Dependency migrations include the real production construction/DI wiring; a compiling injectable class alone is not a complete migration. Mechanical clock replacements preserve read count and UTC/local semantics, with consistency corrections planned separately.

Test framework/platform migration is handled by the test-migration agent in the separate dotnet-test-migration plugin.

Internal subagents

These specialists are available to test-engineer (user-invocable: false); you do not need to call them directly. The agent owns research, planning, implementation, and review inline by default, and delegates only substantial work that benefits from separate context.

Agent	Called by	Purpose
test-quality-auditor	test-engineer	Runs multi-skill audit pipelines for comprehensive test-suite assessment
testability-migration	test-engineer	Performs explicit .NET production-code testability refactors and adds deterministic tests
code-testing-researcher	test-engineer	Analyzes codebase structure, testing patterns, and testability
code-testing-planner	test-engineer	Creates phased test implementation plans from research findings
code-testing-implementer	test-engineer	Implements one phase from the plan, runs build-test-fix cycles
code-testing-builder	code-testing-implementer	Runs build/compile commands and reports results
code-testing-tester	code-testing-implementer	Runs test commands and reports pass/fail results
code-testing-fixer	code-testing-implementer	Fixes compilation errors in source or test files
code-testing-linter	code-testing-implementer	Runs code formatting and linting

VS Code — optional nested delegation: The pipeline does not require phase-agent fan-out. When substantial work warrants a subagent invoking another available named agent, VS Code gates that nested delegation behind a setting that is off by default. To allow it, enable:

"chat.subagents.allowInvocationsFromSubagents": true

Without it, test-engineer or a delegated implementer completes the phases inline; required validation and review are unchanged. The GitHub Copilot CLI has no such gate, but delegation remains optional.

Prerequisites

For polyglot skills and agents

The test-engineer agent, code-testing skill, generation workers, internal quality auditor, and seven test-analysis skills (test-anti-patterns, test-smell-detection, assertion-quality, test-gap-analysis, test-tagging, grade-tests, test-value-evidence) work with any supported language above. Execution requires a working test runtime for the target language (for example pytest, npm test, mvn, go, cargo test, Pester, or CMake plus a C++ test runner).

test-value-evidence can also assess a change statically in read-only mode when execution is unavailable or unauthorized. This does not require a working runtime; the result must distinguish static predictions and missing evidence from executed proof.

For .NET-only skills and agents

• .NET SDK installed (dotnet on PATH)
• A project with an existing test framework (MSTest, xUnit, NUnit, or TUnit) for execution, migration, coverage, CRAP, testability, and the experimental dotnet-experimental skills.

Classic non-SDK .NET projects

The test-generation and analysis heuristics support classic projects with packages.config, explicit <Compile Include> items, older MSTest/Moq stacks, and custom base fixtures. Generation preserves those conventions and registers every new test file in the project.

Execution requires the repository's existing Windows/Visual Studio toolchain (commonly full MSBuild plus vstest.console.exe or MSTest.exe). Coverage and CRAP analysis accept existing Cobertura reports; they do not inject SDK-style coverage packages into classic projects. If the required runner or coverage workflow is absent, the skill reports the limitation rather than migrating the project.

Testability wrappers and migrations are separate, explicit opt-in workflows. Test generation and quality audits do not introduce production seams, and all testability workflows must honor repository rules that prohibit such refactors.