Add 'dotnet sln add' step to code-testing-implementer#522
Add 'dotnet sln add' step to code-testing-implementer#522drognanar wants to merge 4 commits intodotnet:mainfrom
Conversation
When a new test project is created, register it with the solution file using 'dotnet sln add' so that 'dotnet test <solution>' discovers the tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the code-testing-implementer agent guidance to ensure newly created .NET test projects are registered with the solution so solution-level test runs can discover them.
Changes:
- Adds a new “Register Test Project with Solution” step instructing use of
dotnet sln ... add ...for new test projects. - Renumbers subsequent implementation steps accordingly.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ### 3. Write Test Files | ||
| ### 3. Register Test Project with Solution | ||
|
|
||
| If the test project is new (just created by you or a previous phase), find the `.sln`/`.slnx` file and run `dotnet sln <solution> add <test-project.csproj>` so that `dotnet test <solution>` discovers the tests. Skip this if the project is already in the solution. |
There was a problem hiding this comment.
The guidance here is a bit ambiguous/possibly inaccurate for repos with multiple solutions or solution filters. Consider: (1) explicitly using the solution path identified in .testagent/research.md / the plan (rather than “find the .sln/.slnx file”), and (2) accounting for .slnf (solution filter) scenarios where adding to the underlying .sln may not be sufficient for dotnet test discovery. Also, dotnet test <solution> is not the preferred syntax on newer .NET SDKs (see skills/run-tests note); using dotnet test --solution <solution> (or avoiding a concrete command form here) would be more robust.
| If the test project is new (just created by you or a previous phase), find the `.sln`/`.slnx` file and run `dotnet sln <solution> add <test-project.csproj>` so that `dotnet test <solution>` discovers the tests. Skip this if the project is already in the solution. | |
| If the test project is new (just created by you or a previous phase), use the exact solution target already identified in `.testagent/research.md` or `.testagent/plan.md` rather than searching for any `.sln`/`.slnx` file. If that target is a `.sln` or `.slnx`, add the test project to that solution with `dotnet sln <solution> add <test-project.csproj>` so the planned test command can discover it. If the target is a `.slnf` (solution filter), make sure the new test project is included in the active test target as well as in the underlying solution if needed; adding only to the underlying `.sln` may not be enough for test discovery. Skip this if the project is already included in the solution/solution filter used for testing. When referring to test execution, prefer the researched command or `dotnet test --solution <solution>` over `dotnet test <solution>`. |
|
/evaluate |
|
⏭️ No skills to evaluate — no changed skills with tests were found in this PR. View workflow run |
- Use solution identified in research/plan rather than searching - Account for .slnf solution filter scenarios - Prefer 'dotnet test --solution' over 'dotnet test <solution>' Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The implementer is polyglot, so 'dotnet sln add' doesn't belong there. Replace with a generic 'register with build system' step that defers to extensions/, and add the .NET-specific detail (sln/slnx/slnf handling) to extensions/dotnet.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 2. If that target is a `.sln` or `.slnx`, run `dotnet sln <solution> add <test-project.csproj>`. | ||
| 3. If the target is a `.slnf` (solution filter), also ensure the new project is included in the filter; adding only to the underlying `.sln` may not be enough for test discovery. | ||
| 4. Skip this if the project is already included in the solution or solution filter used for testing. | ||
| 5. Prefer the researched test command or `dotnet test --solution <solution>` over `dotnet test <solution>`. |
There was a problem hiding this comment.
The guidance to prefer dotnet test --solution <solution> over dotnet test <solution> is only correct for the .NET SDK 10+ / MTP argument-parsing mode; in VSTest mode (and MTP-on-8/9 bridge), --solution isn’t the standard invocation and the positional <solution> form is expected. Suggest rewording to make this conditional (e.g., prefer the researched test command; use --solution only when the repo is on SDK 10+ MTP syntax).
| 5. Prefer the researched test command or `dotnet test --solution <solution>` over `dotnet test <solution>`. | |
| 5. Prefer the researched test command. If you need to run the solution directly, use `dotnet test --solution <solution>` only for repos on .NET SDK 10+ with MTP-style syntax; otherwise use the standard positional form `dotnet test <solution>`. |
|
|
||
| If a new test project was created, register it with the solution so `dotnet test` can discover it: | ||
|
|
||
| 1. Use the exact solution target identified in `.testagent/research.md` or `.testagent/plan.md` — do not search for any `.sln`/`.slnx` file. |
There was a problem hiding this comment.
Step 1 says not to search for any .sln/.slnx file, but step 3 also supports .slnf targets. Consider including .slnf in that “don’t search” guidance (or rephrasing to “don’t pick a different solution/solution filter than the one in research/plan”) to avoid a mixed message.
| 1. Use the exact solution target identified in `.testagent/research.md` or `.testagent/plan.md` — do not search for any `.sln`/`.slnx` file. | |
| 1. Use the exact solution or solution-filter target identified in `.testagent/research.md` or `.testagent/plan.md` — do not search for or substitute a different `.sln`, `.slnx`, or `.slnf` target. |
- Include .slnf in 'don't substitute' warning alongside .sln/.slnx - Qualify '--solution' flag as SDK 10+/MTP-only; fall back to positional form for older SDKs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/evaluate |
Skill Validation Results
[1] (Isolated) Quality unchanged but weighted score is -22.6% due to: judgment, quality, tool calls (88 → 126), time (465.4s → 629.2s), tokens (1777071 → 2005964)
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
When a new test project is created, register it with the solution file using 'dotnet sln add' so that 'dotnet test ' discovers the tests.