Repository navigation
Extract MSTest.Uwp and MSTest.WinUI packages - #12450
Amaury Levé (Evangelink) wants to merge 11 commits into
Conversation
There was a problem hiding this comment.
🧪 Test quality review — PR #12450
Compared merge base 7b382c059ea63629db1f53d72632399af60c74ed with head 9d2bc45c1a2603ad0ff908504f8d00bbf95ceb47. Evidence applies to this head, not every intermediate commit.
Test-value evidence
All entries are static predictions or author-reported evidence. This workflow executed nothing, and no accessible paired execution records were available.
| Contract | Test and decisive assertion | Counterfactual | Evidence status |
|---|---|---|---|
UI packages pin core as [version] |
WindowsApplicationModelPackageTests.PackedUIFramework_OwnsItsPayloadWithoutAnotherAdapter (3 rows) asserts the nuspec dependency version equals [MSTestVersion] |
Dropping _AlignMSTestPlatformPackageCoreVersion yields 5.0.0-dev, so the assertion would fail |
Author-reported red/green (3 failed / 3 passed). Not independently verified. Static prediction: likely detected. |
| UI payload isolation and no other-flavor or adapter dependency | Same test, DoesNotContain on MSTest.TestFramework.dll and on the adapter/Uwp/WinUI dependency ids |
Shipping UI assets in the wrong package would fail | Static prediction: likely detected. |
| Ordinary adapter has no UI payload or dependency groups | PackedMSTestTestAdapter_ContainsOnlyOrdinaryAssets |
Leaking uap/windows entries or dependency groups would fail | Static prediction: likely detected. |
| Classic UWP runtime dependencies and bootstrap | PackedMSTestUwp_ClassicUwpDoesNotDuplicateRuntimeDependencies, PackedMSTestUwp_ClassicUwpDeclaresBootstrapDependencies, PackedMSTestUwp_PropsRegisterMSTestBuilderHook |
Wrong assemblies or dependency groups would fail | Static prediction: likely detected. |
| Core assets and sealed UI attributes | PackedMSTestTestFramework_ContainsSharedCoreAssets, PackedUIFramework_SealsConcreteUIAttributes |
Missing assets or unsealed types would fail | Static prediction: likely detected. |
| SDK automatic package selection, AppModel relocation, Bazel and verification scripts | Existing SdkTests, UnpackagedWinUITests, SdkPackageTests and updated expectations |
Separate mutations were not executed (the PR also says so) | Unverified. Package-routing mutations beyond the core pin are an evidence gap. |
Moves, solution files, csproj references, docs, .history archives |
n/a | n/a | Not applicable (non-behavioral or inert). |
Evidence gaps:
- The red/green runs are claimed in the PR body and decision record only.
- Windows-only tests are skipped on other OSes through
OSCondition, so Linux CI would not run them. - The reflection owner of AppModel is covered only by the existing
UnpackagedProcessHasNoPackageIdentityOrPath.
🧵 Parallel safety
PARALLEL-SAFETY RESULT
STATUS: CLEAN
COUNTS: A=0; B=0; C=0; D=0; Critical=0; High=0; Warning=0; Info=0
| Test assembly | Scope | Workers | Analyzer coverage |
|---|---|---|---|
| MSTest.Acceptance.IntegrationTests | changed test files | no parallelization config change | n/a (static review) |
| Microsoft.Testing.Platform.Acceptance.IntegrationTests | PackagedWinUITests.cs |
no parallelization config change | n/a (static review) |
| Bazel.SdkPackages.UnitTests, TestFramework.UnitTests | changed tests | no parallelization config change | n/a (static review) |
| Analyzers, SourceGeneration, Adapter and PlatformServices unit tests | csproj and targets only | no parallelization config change | n/a (static review) |
No [Parallelize] or [DoNotParallelize] changes and no new shared mutable state. The new package-inspection tests only read zip archives.
Per-test assessment
Pseudo-mutation evidence below consists of static predictions, not executed mutation results. This pass graded the behavior-bearing new and modified test methods. Changes to the other methods are one-line path or name updates and were not graded individually.
| Result | Quality | Test | Notes | How to improve |
|---|---|---|---|---|
| Pass | A (90–100) | new WindowsApplicationModelPackageTests.PackedUIFramework_OwnsItsPayloadWithoutAnotherAdapter |
Exact pin, payload and dependency assertions per package row. | None |
| Pass | B (80–89) | mod WindowsApplicationModelPackageTests.PackedMSTestTestAdapter_ContainsOnlyOrdinaryAssets |
Substring exclusions plus nuspec group checks protect the ordinary-asset contract. | None |
| Pass | A (90–100) | new WindowsApplicationModelPackageTests.PackedMSTestUwp_ClassicUwpDoesNotDuplicateRuntimeDependencies |
Exact sequence of the two classic UWP assemblies. | None |
| Pass | A (90–100) | new WindowsApplicationModelPackageTests.PackedMSTestUwp_ClassicUwpDeclaresBootstrapDependencies |
Asserts the version and the absence from other TFM groups. | None |
| Pass | B (80–89) | mod WindowsApplicationModelPackageTests.PackedMSTestUwp_PropsRegisterMSTestBuilderHook |
Checks the hook id and type name in the props. | None |
| Pass | B (80–89) | mod WindowsApplicationModelPackageTests.PackedMSTestTestFramework_ContainsSharedCoreAssets |
Presence-only check, but the contract is the required entry list. | None |
| Pass | B (80–89) | mod WindowsApplicationModelPackageTests.PackedUIFramework_SealsConcreteUIAttributes |
Asserts the sealed flag per package assembly. | None |
| Pass | B (80–89) | mod WindowsApplicationModelPackageTests.PackedMSTestTestFramework_DeploymentItemAvailabilityMatchesWindowsTargetFramework |
Data rows map exactly to the expected availability per TFM. | None |
| Uncertain | — | mod remaining path/name updates in SdkTests, ClassicUwpTests, UnpackagedWinUITests, WinUITests, CppCli*Tests, FrameworkTargetingTests, PackagedWinUITests, CoreFrameworkAssemblyTests, SdkPackageTests, WinUIExtensionTests |
Changes are mechanical path/name updates; not individually graded in this pass. | Grade individually if method-level scores are needed. |
No inline comments were published. No novel, verified, actionable finding passed the publication gate, and no existing review threads were found.
<sub>This advisory review is heuristic and non-blocking.
Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>
🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~59 AIC · ⌖ ~1 AIC · ⊞ 8.1K
9d2bc45 to
d46a304
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The new compatibility facade is missing from the mutation-testing inventory and Windows probe assembly expectations, causing mutation planning and UWP/WinUI validation to fail.
2 open issues
What changed in this PR
This is the package-extraction stage of #7225, building on the shared-core work in #12446. It separates UWP and WinUI support from ordinary MSTest packages while retaining a forwarding-only compatibility assembly for non-UI APIs.
Changes:
- Introduces
MSTest.UwpandMSTest.WinUI, with matching adapters, resources, and exact core-version dependencies. - Makes the core framework the package owner and updates SDK selection, consumers, samples, and build references.
- Preserves historical API records and updates package-layout and native-build validation.
Test-value evidence: The PR reports 71 passing checks at f9708f5, plus a three-case failing/passing comparison for exact core-version pinning. These executions were not independently reproduced during this review; full native Bazel execution remains explicitly unclaimed.
Confidence at a glance — 🟡 Build and validation integration
Targeted source inspection and a standalone inventory comparison support the review conclusions. No project builds or tests were run.
| File | Description |
|---|---|
| TestFx.slnx | Registers UI packages and compatibility project. |
| test/UnitTests/TestFramework.UnitTests/TestFramework.UnitTests.csproj | Selects new WinUI and services references. |
| test/UnitTests/TestFramework.UnitTests/Extensions/WinUIExtensionTests.cs | Updates AppModel reflection owner. |
| test/UnitTests/MSTestAdapter.UnitTests/MSTestAdapter.UnitTests.csproj | Removes retired Extensions reference. |
| test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/MSTestAdapter.PlatformServices.UnitTests.csproj | Uses core and conditional WinUI references. |
| test/UnitTests/MSTest.SourceGeneration.UnitTests/MSTest.SourceGeneration.UnitTests.csproj | References core directly. |
| test/UnitTests/MSTest.Analyzers.UnitTests/MSTest.Analyzers.UnitTests.csproj | References core directly. |
| test/UnitTests/Bazel.SdkPackages.UnitTests/SdkPackageTests.cs | Updates metapackage dependency fixture. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WinUITests.cs | Consumes the explicit WinUI package. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsApplicationModelTestTools.cs | Updates expected UI asset paths. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/UnpackagedWinUITests.cs | Uses the WinUI package directly. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/FrameworkTargetingTests.cs | Updates core package-layout expectations. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/CppCliVSTestTests.cs | Removes explicit Extensions reference. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/CppCliMtpTests.cs | Removes explicit Extensions reference. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/ClassicUwpTests.cs | Exercises explicit UWP package assets. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/PackagedWinUITests.cs | Updates direct-package integration and asset checks. |
| test/Directory.Build.targets | Removes retired coverage-name mapping. |
| src/TestFramework/TestFramework/TestFramework.csproj | Owns core packaging and includes compatibility facade. |
| src/TestFramework/TestFramework/PACKAGE.md | Documents shared core and UI migration. |
| src/TestFramework/TestFramework/MSTest.TestFramework.targets | Supplies shared implicit namespace import. |
| src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj | References core directly. |
| src/TestFramework/TestFramework.Extensions/buildTransitive/net10.0AndLater/MSTest.TestFramework.targets | Removes hidden UI assembly selection. |
| src/TestFramework/TestFramework.Extensions/build/uap10.0/MSTest.TestFramework.targets | Removes retired UAP import wrapper. |
| src/TestFramework/TestFramework.Extensions/build/netstandard2.0/MSTest.TestFramework.targets | Removes retired .NET Standard wrapper. |
| src/TestFramework/TestFramework.Extensions/build/net462/MSTest.TestFramework.targets | Removes retired .NET Framework wrapper. |
| src/TestFramework/TestFramework.Extensions/build/net11.0/MSTest.TestFramework.targets | Removes retired .NET 11 wrapper. |
| src/TestFramework/TestFramework.Extensions/build/net10.0/MSTest.TestFramework.targets | Removes retired .NET 10 wrapper. |
| src/TestFramework/TestFramework.Compatibility/TypeForwarders.cs | Adjusts deployment forwarding condition. |
| src/TestFramework/TestFramework.Compatibility/TestFramework.Compatibility.csproj | Defines forwarding-only compatibility assembly. |
| src/TestFramework/TestFramework.Compatibility/PublicAPI/PublicAPI.Shipped.txt | Retains common public API baseline. |
| src/TestFramework/TestFramework.Compatibility/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt | Tracks .NET Standard forwarders and removals. |
| src/TestFramework/TestFramework.Compatibility/PublicAPI/netstandard2.0/PublicAPI.Shipped.txt | Retains .NET Standard API baseline. |
| src/TestFramework/TestFramework.Compatibility/PublicAPI/net10.0/PublicAPI.Unshipped.txt | Tracks modern core forwarding. |
| src/TestFramework/TestFramework.Compatibility/PublicAPI/net10.0/PublicAPI.Shipped.txt | Retains modern public API baseline. |
| src/TestFramework/TestFramework.Compatibility/InternalAPI/InternalAPI.Unshipped.txt | Initializes internal API tracking. |
| src/TestFramework/TestFramework.Compatibility/InternalAPI/InternalAPI.Shipped.txt | Initializes internal API baseline. |
| src/TestFramework/PlatformPackage/PlatformPackage.targets | Pins core version and packages UI payloads. |
| src/TestFramework/PlatformPackage/PlatformPackage.props | Shares UI package configuration and dependencies. |
| src/TestFramework/PlatformPackage/MSTest.Platform.props | Sets package-local adapter paths and registration. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.zh-Hant.xlf | Places Traditional Chinese resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.zh-Hans.xlf | Places Simplified Chinese resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.tr.xlf | Places Turkish resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.ru.xlf | Places Russian resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.pt-BR.xlf | Places Brazilian Portuguese resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.pl.xlf | Places Polish resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.ko.xlf | Places Korean resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.ja.xlf | Places Japanese resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.it.xlf | Places Italian resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.fr.xlf | Places French resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.es.xlf | Places Spanish resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.de.xlf | Places German resources under WinUI. |
| src/TestFramework/MSTest.WinUI/Resources/xlf/FrameworkExtensionsMessages.cs.xlf | Places Czech resources under WinUI. |
| src/TestFramework/MSTest.WinUI/PublicAPI/PublicAPI.Unshipped.txt | Declares relocated WinUI APIs. |
| src/TestFramework/MSTest.WinUI/PublicAPI/PublicAPI.Shipped.txt | Initializes public API baseline. |
| src/TestFramework/MSTest.WinUI/Properties/AssemblyInfo.cs | Preserves supported Windows annotation. |
| src/TestFramework/MSTest.WinUI/PACKAGE.md | Documents WinUI ownership and migration. |
| src/TestFramework/MSTest.WinUI/MSTest.WinUI.csproj | Defines the explicit WinUI package. |
| src/TestFramework/MSTest.WinUI/InternalAPI/InternalAPI.Unshipped.txt | Tracks WinUI internal APIs. |
| src/TestFramework/MSTest.WinUI/InternalAPI/InternalAPI.Shipped.txt | Initializes internal API baseline. |
| src/TestFramework/MSTest.WinUI/Attributes/WinUITestTargetAttribute.cs | Relocates WinUI target attribute. |
| src/TestFramework/MSTest.Uwp/PublicAPI/PublicAPI.Unshipped.txt | Declares relocated UWP API. |
| src/TestFramework/MSTest.Uwp/PublicAPI/PublicAPI.Shipped.txt | Initializes public API baseline. |
| src/TestFramework/MSTest.Uwp/Properties/MSTest.Uwp.rd.xml | Updates runtime-directive library identity. |
| src/TestFramework/MSTest.Uwp/Properties/AssemblyInfo.cs | Retains UWP assembly metadata. |
| src/TestFramework/MSTest.Uwp/PACKAGE.md | Documents UWP ownership and migration. |
| src/TestFramework/MSTest.Uwp/MSTest.Uwp.csproj | Defines classic and modern UWP package. |
| src/TestFramework/MSTest.Uwp/InternalAPI/InternalAPI.Unshipped.txt | Tracks UWP internal APIs. |
| src/TestFramework/MSTest.Uwp/InternalAPI/InternalAPI.Shipped.txt | Initializes internal API baseline. |
| src/TestFramework/MSTest.Uwp/Attributes/UITestMethodAttribute.cs | Relocates UWP UI-thread test attribute. |
| src/TestFramework/APIHistory/TestFramework.Extensions/README.md | Explains inert API archives and live facade. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/uap10.0.16299/PublicAPI.Unshipped.txt.history | Archives UAP public API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/uap10.0.16299/PublicAPI.Shipped.txt.history | Archives UAP public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/PublicAPI.Shipped.txt.history | Archives common public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/netstandard2.0/PublicAPI.Unshipped.txt.history | Archives .NET Standard public API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/netstandard2.0/PublicAPI.Shipped.txt.history | Archives .NET Standard public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0/PublicAPI.Unshipped.txt.history | Archives .NET 10 public API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0/PublicAPI.Shipped.txt.history | Archives .NET 10 public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0-windows10.0.18362.0/PublicAPI.Unshipped.txt.history | Archives WinUI public API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0-windows10.0.18362.0/PublicAPI.Shipped.txt.history | Archives WinUI public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0-windows10.0.17763.0/PublicAPI.Unshipped.txt.history | Archives modern UWP public API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/PublicAPI/net10.0-windows10.0.17763.0/PublicAPI.Shipped.txt.history | Archives modern UWP public API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/uap10.0.16299/InternalAPI.Unshipped.txt.history | Archives UAP internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/uap10.0.16299/InternalAPI.Shipped.txt.history | Archives UAP internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/netstandard2.0/InternalAPI.Unshipped.txt.history | Archives .NET Standard internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/netstandard2.0/InternalAPI.Shipped.txt.history | Archives .NET Standard internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net462/InternalAPI.Unshipped.txt.history | Archives .NET Framework internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net462/InternalAPI.Shipped.txt.history | Archives .NET Framework internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0/InternalAPI.Unshipped.txt.history | Archives .NET 10 internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0/InternalAPI.Shipped.txt.history | Archives .NET 10 internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0-windows10.0.18362.0/InternalAPI.Unshipped.txt.history | Archives WinUI internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0-windows10.0.18362.0/InternalAPI.Shipped.txt.history | Archives WinUI internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0-windows10.0.17763.0/InternalAPI.Unshipped.txt.history | Archives modern UWP internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/net10.0-windows10.0.17763.0/InternalAPI.Shipped.txt.history | Archives modern UWP internal API baseline. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/InternalAPI.Unshipped.txt.history | Archives common internal API changes. |
| src/TestFramework/APIHistory/TestFramework.Extensions/InternalAPI/InternalAPI.Shipped.txt.history | Archives common internal API baseline. |
| src/Package/MSTest/PACKAGE.md | Explains explicit modern UI package selection. |
| src/Package/MSTest/MSTest.csproj | Routes classic UWP dependency to its package. |
| src/Package/MSTest.Sdk/Sdk/VSTest/VSTest.targets | Uses selected adapter package for VSTest. |
| src/Package/MSTest.Sdk/Sdk/Sdk.targets | Selects packages from existing application flags. |
| src/Package/MSTest.Sdk/Sdk/Runner/NativeAOT.targets | Uses selected adapter package for NativeAOT. |
| src/Package/MSTest.Sdk/Sdk/Runner/ClassicEngine.targets | Uses selected adapter package for classic engine. |
| src/Package/MSTest.Sdk/README.md | Documents automatic UI package selection. |
| src/Package/MSTest.Sdk/PACKAGE.md | Summarizes SDK package-selection behavior. |
| src/Adapter/MSTestAdapter.PlatformServices/Services/AppModel.cs | Relocates package-identity helper unchanged. |
| src/Adapter/MSTestAdapter.PlatformServices/MSTestAdapter.PlatformServices.csproj | References shared core directly. |
| src/Adapter/MSTestAdapter.PlatformServices/InternalAPI/InternalAPI.Unshipped.txt | Records relocated AppModel APIs. |
| src/Adapter/MSTest.TestAdapter/PackageDependencies.props | Shares adapter dependency declarations. |
| src/Adapter/MSTest.TestAdapter/PACKAGE.md | Documents ordinary adapter boundary. |
| src/Adapter/MSTest.TestAdapter/buildTransitive/uwp/MSTest.TestAdapter.targets | Uses package-local payload and resource paths. |
| src/Adapter/MSTest.TestAdapter/buildTransitive/common/MSTest.TestAdapter.targets | Replaces flavor switching with package-local paths. |
| src/Adapter/MSTest.TestAdapter/build/uap10.0/MSTest.TestAdapter.targets | Removes ordinary adapter UAP target wrapper. |
| src/Adapter/MSTest.TestAdapter/build/uap10.0/MSTest.TestAdapter.props | Removes ordinary adapter UAP props wrapper. |
| src/Adapter/MSTest.Extensions.Hosting/MSTest.Extensions.Hosting.csproj | Uses core as owning package dependency. |
| samples/public/WinUIVSTestApp/WinUIVSTestApp.csproj | Migrates sample to WinUI package. |
| samples/public/UwpVSTestApp/UwpVSTestApp.csproj | Migrates sample to UWP package. |
| samples/public/ClassicUwpMtpApp/ClassicUwpMtpApp.csproj | Updates explicitly deployed assembly paths. |
| NonWindowsTests.slnf | Replaces Extensions with compatibility project. |
| MSTest.slnf | Includes UI packages and compatibility project. |
| eng/verify-nupkgs.ps1 | Adds UI package verification. |
| eng/package-shapes/MSTest.WinUI.txt | Defines expected WinUI archive layout. |
| eng/package-shapes/MSTest.TestAdapter.txt | Removes UI payload expectations. |
| eng/mutation-testing/modules.json | Updates mutation inventory for package extraction. |
| eng/bazel/windows/validate.ps1 | Updates Windows UI assembly identities. |
| eng/bazel/windows/LibraryProbe.cs | Checks separate UI, core, and facade ownership. |
| eng/bazel/windows.bzl | Defines new UI and compatibility build references. |
| eng/bazel/validate-mstest-packages.ps1 | Checks revised core and facade archive paths. |
| eng/bazel/tests.bzl | Removes retired Extensions test dependencies. |
| eng/bazel/SdkPackageTool.cs | Updates metapackage dependency mapping. |
| eng/bazel/sdk-packages.bzl | Removes retired test dependency. |
| eng/bazel/samples.bzl | Removes retired sample dependencies. |
| eng/bazel/reference-alias-tests.bzl | Removes retired alias-test dependency. |
| eng/bazel/mstest.bzl | Replaces Extensions build with compatibility facade. |
| eng/bazel/mstest-packages.bzl | Packages core and facade as library assets. |
| eng/bazel/extensions.bzl | Updates Hosting dependency graph. |
| eng/bazel/compatibility.bzl | Renames compatibility matrix dependency. |
| docs/winui-testing.md | Documents explicit UI-package integration. |
| BUILD.bazel | Updates root build and test dependencies. |
Review effort: Balanced
| #error A Windows library probe scenario must be selected. | ||
| #endif | ||
| #if WINDOWS_DESKTOP_PROBE || WINDOWS_UWP_PROBE || WINDOWS_WINUI_PROBE | ||
| sourceAssemblies.Add(Assembly.Load("MSTest.TestFramework.Extensions")); |
There was a problem hiding this comment.
This is already covered by 3ce7c52 in the reviewed head. The validator now compares exact assembly-name multisets: packaged 2, desktop 3, UWP 5, and WinUI 5; it also rejects same-count reports with the wrong identity. Both native matrices passed on 8ea1112: https://github.com/microsoft/testfx/actions/runs/43877604147.
Auto-replied by the GitHub Copilot app
There was a problem hiding this comment.
🔵 Needs a closer look
The coordinated package-ownership and UI assembly-identity changes need maintainer validation because final-head NuGet/MSBuild compatibility and Windows activation evidence was not independently available.
1 resolved since last review
Review effort: Balanced
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep a forwarding-only facade in the core package without restoring the retired UI implementation or creating a project-reference cycle. Preserve upstream API and package-shape contracts and adapt the consumed-asset checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain the existing layout regression and include the forwarding-only assembly in the established UWP runtime allowlist. The same three real packaged consumers fail without the entry and pass after repacking with it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the existing 26/5 exact-framework expectations and include the forwarding-only assembly in both supported compatibility graphs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain the original references and UAP assets below MSTest 5, while selecting extracted Uwp and WinUI package owners for 5 and later prereleases. Verify all three actual samples against both published 4.5.3 and aligned local 5.0.0-dev packages. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Migrate the existing synthetic project reference to TestFramework.Compatibility and preserve its MSTest.TestFramework.Extensions assembly-name mapping without changing exact coverage output assertions. The focused case fails before the mapping and the existing 12-case harness passes with it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace the stale source-assembly count with exact per-scenario identities. Keep the existing signing, loaded-path, producer-hash, and Windows metadata checks unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit bf1dbe413251d994852ca11e1ae44f82c10d4ee2)
Document the verified 94711-based replay, unchanged sample and package inputs, and the twelve passing cases from the actual merged coverage/incrementality harness. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude the no-implementation compatibility assembly and account for all 43 source projects so the real mutation planner can create its 32-module matrix. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
caf47f4 to
852e558
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The cross-product package and assembly-identity migration requires coordinated compatibility review, with final-head managed CI, Windows activation, and downstream consumer evidence still pending or unavailable.
Review effort: Balanced
Keep all nine existing rows and assertions, including the Unix OS-property simulation, while gating only the six methods whose UI package is not produced on Linux. Preserve the portable console control. All ten focused rows passed against unchanged shipping packages without rebuilding product code. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Final test-triage resolution — Azure Pipelines build 1683617 Result: inconclusive (evidence incomplete), no issue warranted. The aggregate build The 8 failure/retry records in the collected CTRF reports all look intentional. They come from acceptance-test fixtures that assert expected failures:
The outer build succeeded, which fits these being expected-failure probes. I did not verify that each assertion matches its parent test's expectation. Other signals
Next step: re-run or check the failed artifact download if complete evidence is needed. No other action is required.
|

Pull request
Summary
Package extraction layer (2/4) replacing #12435, stacked on #12446. Relates to #7225 and follows #12412 / #12428. Native stack order remains #12446 → #12450 → #12452 → #12451.
MSTest.TestFramework;MSTest.UwpandMSTest.WinUIown their UI attributes and matching adapter/services. UI packages pin core exactly and carry their own per-TFM build assets/satellites; ordinary adapter assets/dependency groups remain UI-free.MSTest.TestFramework.Extensions.dllinside core. No implementation types, UI resources/selection, reverse project-reference cycle, or new facade package is introduced.AppModelmoves unchanged to PlatformServices. Fix LinuxNU5129with the explicit full package destination, WinUI unit-test owner reference/friend access, and MarkdownMD018, without suppressions.TestFramework.Compatibilitywithout weakening exact output assertions or overwriting upstream mapped-NuGet exclusions.All 28 inert API archives retain parent bytes; eight ordinary public API baseline files are preserved byte-for-byte in the facade project. Layer 3 still owns the public app-model override/helper privacy/copy-local/builder-hook policy and expanded composition matrix. Layer 4 still owns new native UI archives/range machinery. Launcher/activation/cancellation/handshake/exit/cleanup policy is not redesigned here.
Design and decision record
Maintained layer-2 contract, alternatives, compatibility decisions, failed approaches and evidence, with the shared-core prerequisite.
Published head:
c392acd7694f920d78609c4fc033e626377aaa96. Exact parent:b97623a7069f8e78794370e83aa0185293a53728. Coordinator-frozen main:94711f3b4db336920e628039fdd54437458ae441. Only this layer was replayed; base ref and native stack metadata were not changed. The final parent follow-through is test-only (retry recovery/OTel environment and processor ownership), not a new production retry or dependency policy. All ten package commits compared equal during the final parent replay; the subsequent guard commit changes only six test attributes and this decision record.Compatibility: Kept
Microsoft.VisualStudio.TestTools.UnitTesting.TestContext, MSTest.TestFramework.Extensionsand the upstream non-UI forwarders resolving to core because precompiledMicrosoft.Playwright.MSTest.v4consumers would otherwise fail assembly/type loading. This supersedes the older blanket clean-break decision. UI attributes still require package/assembly-reference migration and recompilation: one facade cannot forward the same UI identity to both UI packages.Demonstration
Package ownership is nonvisual. Actual archive, loaded-binary, activation and TRX evidence is more useful than a staged screenshot. Real classic UWP via SDK/metapackage/direct package, modern net10/net11 UWP, packaged SDK/direct WinUI and unpackaged WinUI were exercised, with selected assets, loaded identities/hashes, package identity/install location, associated result names/outcomes and owned cleanup asserted.
Facade binaries define only
<Module>; non-UI forwarder counts are 5 / 12 / 6 / 6 for netstandard2.0 / net462 / net10.0 / net11.0. UI nuspecs declare core as[5.0.0-dev], with no ordinary-adapter or other-flavor dependency. The core/adapter have no UI implementation payload/dependency groups. ZIP names, includingbuildTransitive/MSTest.TestFramework.targets, are canonical.Validation and regression proof
Evidence is deliberately revision-bound: 70 selected/executed/passed, 0 failed/skipped at managed implementation
d46a304cfa716a9657021015639011bb2477d03a. Subsequent native/sample/coverage repairs and test-only lower replays are attributed separately below. The preserved local shipping nupkg SHA-256 values were checked and remain identical; this does not assert that later CI-stamped archives have identical hashes, or silently relabel older execution as a run at this final Git head.Toolchain: repository-pinned .NET SDK
12.0.100-alpha.1.26562.108, Arcade12.0.0-beta.26565.1, desktop MSBuild18.13, Release; MTP3.0.0-dev. Producer third-party cache was reused unchanged; same-version consumers used fresh short task-owned caches. Pack preceded every compared product-consumer state.WindowsApplicationModelPackageTests; fourPackedExtensionsForwardMovedTypesToCorerows; sevenPackedConsumersUseSharedCoreAndSelectedUiPackagerows; bothPackedPackages_UseMatchingFrameworkAssetsAndRuntimerows; sixPlaywrightSdkTestsrows (net462/net10/net11 × MTP/VSTest); selected SDK helper/opt-out cases.ClassicUwp_ConsumesUapAssets_AndRunsPlainAndUiTestsThroughMtprows,ModernUwp_Net10,ModernUwp_Net11,ModernUwp_UnsupportedDeployment,SelfHostedUnpackagedWinUI_RunsTests_AndProcessExits. Classic children each validate six exact passed identities, including localization; negative modern child outcomes are intentional outer assertions.PackagedFullTrustWinUI_RegistersActivatesConnectsBackRunsUiTestsAndCleansUpandPackagedAppContainerWinUI_ExplicitIntegrationPublishesTrx(MSTest.WinUI).UITestMethodStartupWaitTestsmethods independently discovered/executed/passed on the WinUI TFM; all five declared PlatformServices unit-test TFMs built.ClassicUwpMtpApp,UwpVSTestApp, andWinUIVSTestAppeach restored/built with published 4.5.3 pins AND isolated copies of the same source each built against local 5.0.0-dev packages: six builds, exit 0. For local 5, SDK/core/UI and MTP 3.0.0-dev/VSTest 18.12.0 were aligned. All sixproject.assets.jsonfiles confirm the appropriate owners, and cached core/Uwp/WinUI/SDK package hashes match shipping.eng/test-build-incrementality.ps1passed 12/12, 0 skips, after replay onto the verified 94711-based bottom. Both production and exact expected XML retain^[\\/]_\d+[\\/].*mapped-NuGet exclusions plus the facade alias. Final parent-only OTel transfers leave these inputs byte-identical.c392acd7694f920d78609c4fc033e626377aaa96, independently discovered/executed 10/10, 0 failed/skipped: all nine existing guarded rows plusMSTestSdk_ConsoleWithExplicitPackagedAppExtension_KeepsDirectLaunch. The exactnet11.0-windows10.0.26100.0 / OS=Unix / empty-controllerrow passed. All method bodies, rows and assertions are preserved; only six package-capability attributes were added. Cached SDK/core/WinUI archives match the unchanged local shipping archives.Build commands, each exit 0, zero warnings/errors:
Filters enumerate 43 actual source projects and three actual test owners from
TestFx.slnx; no global TFM override. Main's strict named-entry package-shape/metadata validation is preserved, not replaced by counts.Acceptance groups used independently discovered selectors for execution:
All groups exited 0; counts 57 / 7 / 2. Startup host used
--treenode-filter /*/MSTestAdapter.PlatformServices.UnitTests/UITestMethodStartupWaitTests/* --report-trx, exit 0, 4/4. Fresh TRX gates reconciled every selected identity; no zero-result or skipped execution is credited. App-model preflight and final package-registration leak check exited 0. Exact selectors, discovery, TRX/gate JSON, hashes and consumed-output evidence are retained in the session artifacts.Focused follow-through commands, all exit 0:
The first two are configured-graph analysis, not compilation. Incrementality stages isolated snapshots and preserves strict task-host scenarios. Sample validation used the same pinned
dotnet restore/desktopMSBuildcommands aseng/build-samples.ps1for the two UWP solutions and pinneddotnet buildfor WinUI,Configuration=Release, warnings-as-errors and binlogs; local-v5 copies used explicit local feeds and an isolated cache. Three published and three aligned local builds passed. Initial mixed VSTest 18.10.2 setup raisedNU1605; it was aligned to the producer, never suppressed or counted as a behavioral red phase.The final SDK guard correction rebuilt only the acceptance owner, not unchanged product code:
Build, independent discovery, execution and exact fresh-TRX validation exited 0; 10 selected/executed/passed, 0 failed/skipped. The first wrapper attempt used ambiguous
-p:syntax; the corrected/p:command above produced a fresh successful binlog. That invocation error is not test evidence. The test-source patch over852e5580048a8f5aa0e74c1bd362bc3de76b2265has SHA-256D6DFB5F79D15317304ADA6142645CEF78875A7511F66B28208FDC0ACF15DB016and is exactly the six attributes in the committed head. The completed run's marked consumer cache was removed only after retaining TRX, binlogs and matching package hashes.Final-head remote CI
All relevant final-head checks are green for
c392acd7694f920d78609c4fc033e626377aaa96. The bounded monitor completed with exit 0, followed by a fresh live readback of all 35 check/status entries. Skipped advisory jobs are not counted as executed tests.NU5129suppression.Azure is bound to the actual
refs/pull/12450/mergesource815992c237ae7e222273157533a575db5045c2c0, whose second parent is exactly this head. Its first parenta389306fa9aad66dcecd37c597b2d843c03a506cis the provider's synthetic integration ofba9d282ba8cba9ac6afea36b1a9b2686aa6de883and the released bottomb97623a7069f8e78794370e83aa0185293a53728. The real source branches remain frozen at the published head and released parent; no extra main rebase was performed.Old-head greens, started jobs, advisory skips and delegated claims are not substituted for final-head execution. No local x86/ARM64 activation, full native Bazel rerun or release-branch/backport proof is claimed; CI execution is attributed above. Coordinator owns downstream rollout/servicing. Historical 71-case evidence remains separately recorded and is not added to current counts.
Test-value evidence
ClassicUwp_ConsumesUapAssets_AndRunsPlainAndUiTestsThroughMtprowsa9406f988adb7cab787bddda02431ddfc232f948: 3 executed/failed, 0 skipped, exit 2, each exact layout assertion names missingMSTest.TestFramework.Extensions.dll; build/resolution succeeded.6dcd56b32e0b6b39715f13058ec669ce30878cc1; production patch SHA-256F68B4DAB401172220AEF0FDC59C2A328B83C5295CD15C3E365CEF26DA86B71D4. Separate caches, paired consumed package snapshots and assertion records retained.CoverageSettingsPreserveExactBytesAndTimestampc070441e074aefb7aa6add325ffee8583b8f6c43; later mapped-source expected bytes are explicitly preserved, not overwritten.NU5129, reproducedCS0234/MD018, and published sampleNU1101/MSB3030are recorded.852e5580048a8f5aa0e74c1bd362bc3de76b2265records nineNU1101setup failures because Linux does not produce the UI package.