Repository navigation
Move independent MSTest APIs into the core framework - #12412
Conversation
Move message-level, run-state, private reflection and configuration definitions without changing core targets or package layout. Keep TestContext, DeploymentItem and factory/lease interfaces in Extensions for the later UI-dependent move. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🧪 Test quality review — PR #12412
Compared merge base 50b513022be7c356cb342866a3b755a4b18bf97d with head a8c7f9560bb43666655d29fe8dfc4c74a7a6798b. Evidence applies to this head, not every intermediate commit.
Test-value evidence
All entries are static predictions. No paired execution records were available, and none were run.
| Contract | Test / decisive assertion | Counterfactual | Result |
|---|---|---|---|
MessageLevel, ITestRunInfo, PlannedTest and TestRun are owned by the core MSTest.TestFramework assembly |
ExtensionsContractTests.CommonDefinitionsAreOwnedByCoreFramework: type.Assembly.Should().BeSameAs(typeof(TestMethodAttribute).Assembly) |
If a type were left in TestFramework.Extensions, the assembly identity would differ and the assertion would fail. |
Static prediction: likely detected |
The .NET Framework-only types (PrivateObject, PrivateType, DataSourceElement*, TestConfiguration*) are owned by the core assembly |
Same test, #if NETFRAMEWORK branch |
Same as above. The branch only runs on the net462 TFM. | Static prediction: likely detected, but only if the net462 unit-test run executes it |
The app.config section type resolves from MSTest.TestFramework |
DataSourceTests (acceptance): the config string now names MSTest.TestFramework |
If the type stayed in the Extensions assembly, loading the section would fail and the data-source tests would fail. | Static prediction. This is an acceptance test and was not executed. |
Public/internal API files, PACKAGE.md, the RFC and the specification |
Documentation and API-tracking files | — | Not applicable. These are not behavioral. |
Evidence gaps:
- No executed red/green records were accessible.
- The
net462AdditionalFilesconditions and theSystem.Configurationreference inTestFramework.csprojare covered only by compilation, not by a behavioral test.
🧵 Parallel safety
STATUS: PARTIAL. The read-only specialist task was not run in this session.
The changed test code gives no sign of a new hazard:
CommonDefinitionsAreOwnedByCoreFrameworkonly reads reflection metadata.- The
DataSourceTestschange is a literal string in a generated asset. - No parallelization configuration changed (
config_countis 0).
This is a manual impression, not a specialist conclusion, and it should not be read as a clean parallel-safety result.
Per-test assessment
Pseudo-mutation evidence below consists of static predictions, not executed mutation results.
| Result | Quality | Test | Notes | How to improve |
|---|---|---|---|---|
| Pass | A (90–100) | new ExtensionsContractTests.CommonDefinitionsAreOwnedByCoreFramework |
An assembly-identity assertion per type covers the ownership contract, with a per-type message. | None |
| Pass | B (80–89) | mod DataSourceTests (config section type string) |
This is a data-only change to an asset; the existing acceptance assertions exercise the section load. | None |
No verified, actionable inline findings were identified, so no inline comments were posted.
<sub>This advisory review is heuristic and non-blocking.
Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>
🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~30 AIC · ⌖ ~1 AIC · ⊞ 8.1K
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the relocation of independent MSTest APIs into the core framework.
No open issues 🎉
What changed in this PR
Moves independent MSTest APIs from Extensions into the core framework as a prerequisite to #7225’s UWP/WinUI package split. This intentionally changes assembly identity for MSTest 5; consumers must recompile and update assembly-qualified configuration names. UI packaging and TestContext ownership remain unchanged.
Changes:
- Relocates 16 source files covering message severity, run-state APIs, private reflection, and .NET Framework configuration helpers.
- Updates API tracking and adds core’s .NET Framework
System.Configurationreference. - Adds an assembly-ownership test and updates the configuration fixture and migration documentation.
Test-value evidence: For head a8c7f9560bb43666655d29fe8dfc4c74a7a6798b against baseline 50b513022be7c356cb342866a3b755a4b18bf97d, the PR reports ownership-test red/green results and preservation/package-consumer passes. Execution artifacts were retained locally and were not independently verified here; full acceptance and native UI activation remain unverified.
| File | Description |
|---|---|
| test/UnitTests/TestFramework.UnitTests/Extensions/ExtensionsContractTests.cs | Adds core assembly-ownership assertions. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/DataSourceTests.cs | Updates the configuration section’s assembly name. |
| src/TestFramework/TestFramework/TestRun.cs | Moves ambient run-state access into core. |
| src/TestFramework/TestFramework/TestFramework.csproj | Includes net462 API baselines and configuration reference. |
| src/TestFramework/TestFramework/RuntimeTypeHelper.cs | Moves reflection method-selection helpers into core. |
| src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt | Records common public API additions. |
| src/TestFramework/TestFramework/PublicAPI/net462/PublicAPI.Unshipped.txt | Records .NET Framework public API additions. |
| src/TestFramework/TestFramework/PublicAPI/net462/PublicAPI.Shipped.txt | Initializes the net462 public API baseline. |
| src/TestFramework/TestFramework/PrivateType.cs | Moves static private-member reflection into core. |
| src/TestFramework/TestFramework/PrivateObject.Properties.cs | Moves private-property access into core. |
| src/TestFramework/TestFramework/PrivateObject.Invoke.cs | Moves private-method invocation into core. |
| src/TestFramework/TestFramework/PrivateObject.Helpers.cs | Moves private-reflection lookup helpers into core. |
| src/TestFramework/TestFramework/PrivateObject.Fields.cs | Moves private-field access into core. |
| src/TestFramework/TestFramework/PrivateObject.cs | Moves the private-object wrapper into core. |
| src/TestFramework/TestFramework/PlannedTest.cs | Relocates planned-test metadata; adjusts XML documentation. |
| src/TestFramework/TestFramework/MessageLevel.cs | Relocates message severity; adjusts XML documentation. |
| src/TestFramework/TestFramework/ITestRunInfo.cs | Relocates the run-information contract; adjusts XML documentation. |
| src/TestFramework/TestFramework/InternalAPI/net462/InternalAPI.Unshipped.txt | Records .NET Framework internal API additions. |
| src/TestFramework/TestFramework/InternalAPI/net462/InternalAPI.Shipped.txt | Initializes the net462 internal API baseline. |
| src/TestFramework/TestFramework/InternalAPI/InternalAPI.Unshipped.txt | Records internal run-state API additions. |
| src/TestFramework/TestFramework/ConfigurationSettings/TestConfigurationSection.cs | Moves the configuration section into core. |
| src/TestFramework/TestFramework/ConfigurationSettings/TestConfiguration.cs | Moves configuration loading into core. |
| src/TestFramework/TestFramework/ConfigurationSettings/DataSourceElementCollection.cs | Moves the data-source configuration collection into core. |
| src/TestFramework/TestFramework/ConfigurationSettings/DataSourceElement.cs | Moves data-source configuration elements into core. |
| src/TestFramework/TestFramework/ConfigurationSettings/ConfigurationNames.cs | Moves configuration constants into core. |
| src/TestFramework/TestFramework.Extensions/PublicAPI/PublicAPI.Unshipped.txt | Records common public API removals from Extensions. |
| src/TestFramework/TestFramework.Extensions/PublicAPI/net462/PublicAPI.Unshipped.txt | Records .NET Framework public API removals. |
| src/TestFramework/TestFramework.Extensions/PACKAGE.md | Documents new ownership and migration requirements. |
| src/TestFramework/TestFramework.Extensions/InternalAPI/net462/InternalAPI.Unshipped.txt | Records .NET Framework internal API removals. |
| src/TestFramework/TestFramework.Extensions/InternalAPI/InternalAPI.Unshipped.txt | Records internal run-state API removals. |
| docs/specifications/mstest.md | Updates the TestRun source link. |
| docs/RFCs/014-TestRun-Current-PlannedTests.md | Updates ownership, target coverage, and compatibility documentation. |
Review effort: Balanced
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Final test-triage resolution — Azure Pipelines build 1676331 Triggering check: Failure signatures (Windows legs;
Assessment (confidence: medium-high that this is PR-related, not flaky)
Next step: check whether the PR registers an additional environment-variable provider, which would break the exact-count assertion in
|
Pull request
Summary
Relates to #7225.
Move the dependency-closed common definitions from
TestFramework.Extensionsinto the core framework as a small prerequisite, not the UWP/WinUI package split:MessageLevel.TestRun,ITestRunInfoandPlannedTest.PrivateObjectpartials,PrivateTypeandRuntimeTypeHelper.Scope: 16 source moves, 32 files overall. Thirteen moved files are byte-identical; three change only XML-doc references to the still-separate
TestContext. Supporting edits are API tracking, core's .NET FrameworkSystem.Configurationreference, one assembly-qualified configuration fixture, one ownership test, and documentation.Deferred:
TestContext,DeploymentItemAttribute, and the test-class factory/lease interfaces remain in Extensions because their move requires UI build/package changes. Existing core TFMs, package ownership/layout/build assets, UI source, adapter/analyzer policy and SDK selection are unchanged.Breaking change for MSTest 5: moved types change assembly identity and inherit core's existing assembly-level metadata defaults. Extensions specifies
ComVisible(false)on non-WinUI builds; core leaves that attribute unset. No core-wide COM policy or compatibility forwarders are introduced. Recompile consumers and update assembly-qualified configuration names; this is not binary-compatible.Design and decision record
The staging rationale is #7225 and this bounded relocation; no separate feature design record is needed. The existing run-state RFC now reflects its owner and the breaking identity change.
Moving the coupled types now would require UI-flavored core builds and package selection changes. Keeping those out preserves reviewability and the existing consumer composition. Existing shipped API history is retained, with explicit unshipped removals in Extensions and additions in core. Method bodies, target guards and lifecycle behavior are not rewritten.
Demonstration
This is a nonvisual assembly move, so a package-consumer transcript is more useful than screenshots. Real package consumers reported:
The first line uses the saved baseline package; the second uses the head package.
TestContextremains in Extensions in both. The package still has exactly 105 entries, and its nuspec is identical except for the source commit, including the .NET FrameworkSystem.Configurationdeclaration.Validation and regression proof
Baseline:
50b513022be7c356cb342866a3b755a4b18bf97d. Head:a8c7f9560bb43666655d29fe8dfc4c74a7a6798b. Windows, Release, repository SDK12.0.100-alpha.1.26556.115; .NET Framework assets target net462 and run on the installed .NET Framework runtime.TestRunInfoTestsUseWinUI/net10-windows; netstandard2.0 also compiled.The packed consumers exercised snapshot collection copying, duplicate properties, cross-assembly
MessageLevel/TestContextdispatch, cancellation, private instance/static reflection, and old/new assembly-qualified configuration loading. The retained WinUI consumer also checked UI attribute presence and the continued absence of deployment/directory APIs. These are managed binding/consumption checks, not native UI activation.Executed build/pack commands and test selectors
The normal repository build bootstrap was used, with Windows PowerShell and Git on the native shell PATH:
Both passed, exit 0. The first was also run before the move.
The owning package was built/packed through the same repository-pinned desktop MSBuild helper, preserving the classic UWP target:
Both passed, exit 0. The baseline was packed with the same helper. The earlier command
.\build.cmd -c Release -pack -projects src\TestFramework\TestFramework.Extensions\TestFramework.Extensions.csproj /bl:artifacts\log\Release\common-move-baseline-pack-01.binlogproduced the baseline package but exited 1 because its product-wide post-pack verifier rejected the seven other MSTest packages not requested by that partial build. No verifier or package machinery was changed.Direct test-host command shape, with the exact selections used:
Results: 11, 11 and 35 passed respectively, all exit 0. On
net10.0-windows10.0.18362.0\win-x64, the same host used the class selectorsExtensionsContractTests/*andTestContextTests/*within the same namespace: 5 and 4 passed, exit 0. Adapter hosts used/*/Microsoft.VisualStudio.TestPlatform.MSTestAdapter.UnitTests.Execution/TestRunInfoTests/*: 7 passed per target, exit 0.All nine final TRX reports were parsed: 94 executed, 94 passed, zero failed/skipped. Binlogs, TRX, the paired-run manifest and isolated consumer sources/assets/provenance are retained locally; no hosted artifact was uploaded.
MSTest.TestFramework.5.0.0-dev.nupkgSHA-256:Limitations: the full acceptance suite and real UWP/WinUI application activation were not run for this bounded move. The broader baseline WinUI extension selection executed 14 tests and had three pre-existing native activation/startup failures (
DispatcherLookupPreservesExplicitPrecedenceAndCachedFailure,NativeApplicationStartupInIsolatedProcesses,RealDispatcherCapturesDirectAndEnqueuedResultsAndFailures). Those were not repaired or counted as passing proof. No CI result is claimed.Test-value evidence
ExtensionsContractTests.CommonDefinitionsAreOwnedByCoreFrameworkMessageLevelbelonged to Extensions rather than core7693B1DC81A50FB7781B7BFD74E10180B709276D4890933B7396F9CE1E50E38D. This is an ownership assertion failure, not a build/setup failure.ExtensionsContractTests,TestContextTests,PrivateObjectTests,PrivateTypeTests,RuntimeTypeHelperTests,ConfigurationSettingsTests