Skip to content

[tests] Add AssertCommercialBuild() to incremental build tests#11421

Open
jonathanpeppers wants to merge 5 commits into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers/fix-incremental-build-tests-commercial
Open

[tests] Add AssertCommercialBuild() to incremental build tests#11421
jonathanpeppers wants to merge 5 commits into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers/fix-incremental-build-tests-commercial

Conversation

@jonathanpeppers
Copy link
Copy Markdown
Member

These tests verify MSBuild targets are skipped on incremental builds, which only holds true when Fast Deployment is available (commercial builds). Without it, additional targets run and the assertions fail.

Fixes: #11419

jonathanpeppers and others added 4 commits May 20, 2026 10:08
These tests verify MSBuild targets are skipped on incremental builds,
which only holds true when Fast Deployment is available (commercial
builds). Without it, additional targets run and the assertions fail.

Fixes: dotnet#11419

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Additional tests found failing in build 1428373 that also need
Fast Deployment for their incremental build assertions to hold.

Tests fixed across 8 files:
- IncrementalBuildTest: BasicApplicationRepetitiveBuild,
  BasicApplicationRepetitiveReleaseBuild, JavacTaskDoesNotRunOnSecondBuild,
  AppProjectTargetsDoNotBreak, ProduceReferenceAssembly,
  LinkAssembliesNoShrink, ConvertCustomView, GenerateJavaStubsAndAssembly
- BindingBuildTest: BindingLibraryIncremental, BindingWithAndroidJavaSource
- TrimmableTypeMapBuildTests: Build_WithTrimmableTypeMap_IncrementalBuild
- AotTests: BuildAotApplicationWithNdkAndBundleAndÜmläüts
- AndroidUpdateResourcesTest: RepetiviteBuildUpdateSingleResource,
  CheckXmlResourcesFilesAreProcessed, BuildAppWithManagedResourceParser,
  CustomViewAddResourceId, InvalidFilenames
- AndroidGradleProjectTests: BuildIncremental
- BuildTest2: BuildXamarinFormsMapsApplication, SwitchBetweenDesignTimeBuild,
  CheckTimestamps
- PackagingTest: CheckAppBundle

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
For tests whose primary purpose is NOT testing incremental builds,
wrap only the target-skipping assertions with
`if (TestEnvironment.CommercialBuildAvailable)` instead of skipping
the entire test with `AssertCommercialBuild()`.

This allows the functional aspects of these tests (AOT compilation,
binding generation, asset pack packaging, XML resource processing,
timestamps, DTB switching, etc.) to continue running on public CI,
while only skipping the incremental build assertions that require
Fast Deployment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ndÜmläüts

The AssertCommercialBuild() was removed but the incremental
assertions were not wrapped with the CommercialBuildAvailable check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers jonathanpeppers marked this pull request as ready for review May 21, 2026 14:46
Copilot AI review requested due to automatic review settings May 21, 2026 14:46
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label May 21, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Xamarin.Android.Build.Tasks test suite to avoid incremental-build assertion failures on public (non-commercial) CI by recognizing that some “targets should be skipped” expectations only hold when Fast Deployment (commercial builds) is available.

Changes:

  • Added AssertCommercialBuild () to several incremental-build tests so they become inconclusive when commercial bits (Fast Deployment) aren’t available.
  • Updated other tests to conditionally run incremental target-skipping assertions based on TestEnvironment.CommercialBuildAvailable.
  • Introduced a few unintended formatting/style regressions (spacing) in multiple test files.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/TrimmableTypeMapBuildTests.cs Adds AssertCommercialBuild () gating for incremental assertions.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs Conditionally skips incremental target-skip assertions for non-commercial builds.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/ManifestTest.cs Adds AssertCommercialBuild () gating for incremental assertions.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs Adds AssertCommercialBuild () to multiple tests; also conditionally skips some key incremental assertions; includes formatting regression.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs Adds AssertCommercialBuild () to one test and conditionally skips incremental assertions in others; includes formatting regressions.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BindingBuildTest.cs Adds AssertCommercialBuild () to an incremental test and conditionally skips some incremental assertions; includes formatting regression.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssetPackTests.cs Conditionally skips incremental target-skip assertions; includes formatting regression.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs Conditionally skips incremental target-skip assertions; includes formatting regressions.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs Adds AssertCommercialBuild () in one test and conditionally skips incremental assertions in several places; includes formatting regressions.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidGradleProjectTests.cs Adds AssertCommercialBuild () gating for incremental assertions.
Comments suppressed due to low confidence (12)

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs:1005

  • Same concern here: gating all target skip/run assertions behind CommercialBuildAvailable effectively disables the incremental-build validation on OSS builds. Prefer AssertCommercialBuild () at the top of the test (or add assertions for the non-commercial behavior) so the test result is meaningful.
				if (TestEnvironment.CommercialBuildAvailable) {
					var targetsShouldSkip = new [] {
						"_BuildLibraryImportsCache",
						"_ResolveLibraryProjectImports",
						"_ConvertCustomView",

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1205

  • These incremental-build assertions are now skipped on OSS builds via CommercialBuildAvailable, which can cause the test to pass without verifying the intended behavior. If Fast Deployment is a requirement, prefer AssertCommercialBuild () at the start of the test with a brief explanation.
				if (TestEnvironment.CommercialBuildAvailable) {
					var targetsToBeSkipped = new [] {
						//TODO: We would like for this assertion to work, but the <Compile /> item group changes between DTB and regular builds
						//      $(IntermediateOutputPath)designtime\Resource.designer.cs -> Resources\Resource.designer.cs
						//      And so the built assembly changes between DTB and regular build, triggering `_LinkAssembliesNoShrink`

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1358

  • This conditional makes the test skip incremental-build assertions on OSS builds. If the assertions only hold with Fast Deployment, it would be clearer to call AssertCommercialBuild () near the start of the test (with a comment) rather than silently skipping the checks.
				if (TestEnvironment.CommercialBuildAvailable) {
					// NativeAOT always runs the linking step
					if (runtime != AndroidRuntime.NativeAOT) {
						b.Output.AssertTargetIsSkipped (isRelease ? KnownTargets.LinkAssembliesShrink : KnownTargets.LinkAssembliesNoShrink);
					}
					b.Output.AssertTargetIsSkipped ("_UpdateAndroidResgen");

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BindingBuildTest.cs:890

  • Wrapping the second-build target-skipping assertions in if (TestEnvironment.CommercialBuildAvailable) means OSS builds no longer validate the incremental behavior. Consider using AssertCommercialBuild () at the beginning of this test (with a comment about Fast Deployment) to keep the test semantics consistent with the PR goal.
				if (TestEnvironment.CommercialBuildAvailable) {
					Assert.IsTrue (libBuilder.Output.IsTargetSkipped ("_CompileBindingJava", defaultIfNotUsed: true), $"`_CompileBindingJava` should be skipped on second build!");
					Assert.IsTrue (libBuilder.Output.IsTargetSkipped ("_ClearGeneratedManagedBindings", defaultIfNotUsed: true), $"`_ClearGeneratedManagedBindings` should be skipped on second build!");
				}

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssetPackTests.cs:226

  • These incremental target-skip assertions are skipped on OSS builds via CommercialBuildAvailable, which weakens the test’s purpose. If Fast Deployment/commercial build is required for the incremental expectations, prefer calling AssertCommercialBuild () with an explanatory comment instead of silently skipping the assertions.
				if (TestEnvironment.CommercialBuildAvailable) {
					appBuilder.Output.AssertTargetIsSkipped ("_CreateAssetPackManifests");
					appBuilder.Output.AssertTargetIsSkipped ("_BuildAssetPacks");
					appBuilder.Output.AssertTargetIsSkipped ("_GenerateAndroidAssetsDir");
				}

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs:928

  • Minor formatting regression: spacing around = is inconsistent with the surrounding codebase style in this file (most assignments use var x = ...). Please add the missing spaces to keep diffs clean and avoid style-check failures.
			var path= Path.Combine ("temp", TestName);

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1306

  • Minor formatting regression: var start= ... is missing spaces around =. Please change to var start = ... to match the file’s formatting conventions.
			var start= DateTime.UtcNow.AddSeconds (-1);

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs:255

  • Minor formatting regression: this method declaration is missing the usual space before ( (e.g., BuildAotApplicationAndÜmläüts (string ...)) used throughout the tests. Please adjust to match the repo’s C# formatting conventions.
		public void BuildAotApplicationAndÜmläüts(string supportedAbis, bool enableLLVM, bool usesAssemblyBlobs)

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs:260

  • Minor formatting regression: missing spaces around = in this assignment (var abisSanitized= ...). Please change to var abisSanitized = ... for consistency.
			var abisSanitized= supportedAbis.Replace (";", "").Replace ("-", "").Replace ("_", "");

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1035

  • Minor formatting regression: var path= ... is missing spaces around =. Please change to var path = ... for consistency with the rest of the file.
			var path= Path.Combine ("temp", TestName);

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1449

  • Minor formatting regression: var proj= ... is missing spaces around =. Please change to var proj = ... to keep formatting consistent.
			var proj= new XamarinAndroidApplicationProject {

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1516

  • Minor formatting regression: local function signature is missing the usual space before ( (CreateItem (string include)) used throughout the tests. Please adjust to match the repo’s C# formatting conventions.
			BuildItem CreateItem(string include) =>

Comment on lines +616 to 620
if (TestEnvironment.CommercialBuildAvailable) {
foreach (var target in new [] { "_Sign", "_BuildApkEmbed" }) {
Assert.IsTrue (b.Output.IsTargetSkipped (target), $"`{target}` should be skipped!");
}
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).

Comment on lines +754 to +758
if (TestEnvironment.CommercialBuildAvailable) {
appBuilder.Output.AssertTargetIsSkipped ("CoreCompile");
appBuilder.Output.AssertTargetIsSkipped ("_BuildLibraryImportsCache");
appBuilder.Output.AssertTargetIsSkipped ("_ResolveLibraryProjectImports");
appBuilder.Output.AssertTargetIsSkipped ("_GenerateJavaStubs");
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).

Comment on lines +242 to +247
if (TestEnvironment.CommercialBuildAvailable) {
Assert.IsTrue (
b.Output.IsTargetSkipped ("_CompileJava"),
"the _CompileJava target should be skipped");
Assert.IsTrue (
b.Output.IsTargetSkipped ("_BuildApkEmbed"),
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).

Comment on lines +547 to +549
if (TestEnvironment.CommercialBuildAvailable) {
Assert.IsTrue (b.Output.IsTargetSkipped ("_GenerateJavaStubs"), "Target _GenerateJavaStubs should have been skipped");
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).

Comment thread src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs Outdated
Restore missing spaces before parentheses and around assignment
operators that were accidentally removed when editing out the
AssertCommercialBuild() lines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Tests] Incremental build tests need AssertCommercialBuild() for Fast Deployment

3 participants