[tests] Add AssertCommercialBuild() to incremental build tests#11421
[tests] Add AssertCommercialBuild() to incremental build tests#11421jonathanpeppers wants to merge 5 commits into
Conversation
These tests verify MSBuild targets are skipped on incremental builds, which only holds true when Fast Deployment is available (commercial builds). Without it, additional targets run and the assertions fail. Fixes: dotnet#11419 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Additional tests found failing in build 1428373 that also need Fast Deployment for their incremental build assertions to hold. Tests fixed across 8 files: - IncrementalBuildTest: BasicApplicationRepetitiveBuild, BasicApplicationRepetitiveReleaseBuild, JavacTaskDoesNotRunOnSecondBuild, AppProjectTargetsDoNotBreak, ProduceReferenceAssembly, LinkAssembliesNoShrink, ConvertCustomView, GenerateJavaStubsAndAssembly - BindingBuildTest: BindingLibraryIncremental, BindingWithAndroidJavaSource - TrimmableTypeMapBuildTests: Build_WithTrimmableTypeMap_IncrementalBuild - AotTests: BuildAotApplicationWithNdkAndBundleAndÜmläüts - AndroidUpdateResourcesTest: RepetiviteBuildUpdateSingleResource, CheckXmlResourcesFilesAreProcessed, BuildAppWithManagedResourceParser, CustomViewAddResourceId, InvalidFilenames - AndroidGradleProjectTests: BuildIncremental - BuildTest2: BuildXamarinFormsMapsApplication, SwitchBetweenDesignTimeBuild, CheckTimestamps - PackagingTest: CheckAppBundle Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
For tests whose primary purpose is NOT testing incremental builds, wrap only the target-skipping assertions with `if (TestEnvironment.CommercialBuildAvailable)` instead of skipping the entire test with `AssertCommercialBuild()`. This allows the functional aspects of these tests (AOT compilation, binding generation, asset pack packaging, XML resource processing, timestamps, DTB switching, etc.) to continue running on public CI, while only skipping the incremental build assertions that require Fast Deployment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ndÜmläüts The AssertCommercialBuild() was removed but the incremental assertions were not wrapped with the CommercialBuildAvailable check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the Xamarin.Android.Build.Tasks test suite to avoid incremental-build assertion failures on public (non-commercial) CI by recognizing that some “targets should be skipped” expectations only hold when Fast Deployment (commercial builds) is available.
Changes:
- Added
AssertCommercialBuild ()to several incremental-build tests so they become inconclusive when commercial bits (Fast Deployment) aren’t available. - Updated other tests to conditionally run incremental target-skipping assertions based on
TestEnvironment.CommercialBuildAvailable. - Introduced a few unintended formatting/style regressions (spacing) in multiple test files.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/TrimmableTypeMapBuildTests.cs | Adds AssertCommercialBuild () gating for incremental assertions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs | Conditionally skips incremental target-skip assertions for non-commercial builds. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/ManifestTest.cs | Adds AssertCommercialBuild () gating for incremental assertions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs | Adds AssertCommercialBuild () to multiple tests; also conditionally skips some key incremental assertions; includes formatting regression. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs | Adds AssertCommercialBuild () to one test and conditionally skips incremental assertions in others; includes formatting regressions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BindingBuildTest.cs | Adds AssertCommercialBuild () to an incremental test and conditionally skips some incremental assertions; includes formatting regression. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssetPackTests.cs | Conditionally skips incremental target-skip assertions; includes formatting regression. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs | Conditionally skips incremental target-skip assertions; includes formatting regressions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs | Adds AssertCommercialBuild () in one test and conditionally skips incremental assertions in several places; includes formatting regressions. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidGradleProjectTests.cs | Adds AssertCommercialBuild () gating for incremental assertions. |
Comments suppressed due to low confidence (12)
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs:1005
- Same concern here: gating all target skip/run assertions behind
CommercialBuildAvailableeffectively disables the incremental-build validation on OSS builds. PreferAssertCommercialBuild ()at the top of the test (or add assertions for the non-commercial behavior) so the test result is meaningful.
if (TestEnvironment.CommercialBuildAvailable) {
var targetsShouldSkip = new [] {
"_BuildLibraryImportsCache",
"_ResolveLibraryProjectImports",
"_ConvertCustomView",
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1205
- These incremental-build assertions are now skipped on OSS builds via
CommercialBuildAvailable, which can cause the test to pass without verifying the intended behavior. If Fast Deployment is a requirement, preferAssertCommercialBuild ()at the start of the test with a brief explanation.
if (TestEnvironment.CommercialBuildAvailable) {
var targetsToBeSkipped = new [] {
//TODO: We would like for this assertion to work, but the <Compile /> item group changes between DTB and regular builds
// $(IntermediateOutputPath)designtime\Resource.designer.cs -> Resources\Resource.designer.cs
// And so the built assembly changes between DTB and regular build, triggering `_LinkAssembliesNoShrink`
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1358
- This conditional makes the test skip incremental-build assertions on OSS builds. If the assertions only hold with Fast Deployment, it would be clearer to call
AssertCommercialBuild ()near the start of the test (with a comment) rather than silently skipping the checks.
if (TestEnvironment.CommercialBuildAvailable) {
// NativeAOT always runs the linking step
if (runtime != AndroidRuntime.NativeAOT) {
b.Output.AssertTargetIsSkipped (isRelease ? KnownTargets.LinkAssembliesShrink : KnownTargets.LinkAssembliesNoShrink);
}
b.Output.AssertTargetIsSkipped ("_UpdateAndroidResgen");
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BindingBuildTest.cs:890
- Wrapping the second-build target-skipping assertions in
if (TestEnvironment.CommercialBuildAvailable)means OSS builds no longer validate the incremental behavior. Consider usingAssertCommercialBuild ()at the beginning of this test (with a comment about Fast Deployment) to keep the test semantics consistent with the PR goal.
if (TestEnvironment.CommercialBuildAvailable) {
Assert.IsTrue (libBuilder.Output.IsTargetSkipped ("_CompileBindingJava", defaultIfNotUsed: true), $"`_CompileBindingJava` should be skipped on second build!");
Assert.IsTrue (libBuilder.Output.IsTargetSkipped ("_ClearGeneratedManagedBindings", defaultIfNotUsed: true), $"`_ClearGeneratedManagedBindings` should be skipped on second build!");
}
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AssetPackTests.cs:226
- These incremental target-skip assertions are skipped on OSS builds via
CommercialBuildAvailable, which weakens the test’s purpose. If Fast Deployment/commercial build is required for the incremental expectations, prefer callingAssertCommercialBuild ()with an explanatory comment instead of silently skipping the assertions.
if (TestEnvironment.CommercialBuildAvailable) {
appBuilder.Output.AssertTargetIsSkipped ("_CreateAssetPackManifests");
appBuilder.Output.AssertTargetIsSkipped ("_BuildAssetPacks");
appBuilder.Output.AssertTargetIsSkipped ("_GenerateAndroidAssetsDir");
}
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs:928
- Minor formatting regression: spacing around
=is inconsistent with the surrounding codebase style in this file (most assignments usevar x = ...). Please add the missing spaces to keep diffs clean and avoid style-check failures.
var path= Path.Combine ("temp", TestName);
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest2.cs:1306
- Minor formatting regression:
var start= ...is missing spaces around=. Please change tovar start = ...to match the file’s formatting conventions.
var start= DateTime.UtcNow.AddSeconds (-1);
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs:255
- Minor formatting regression: this method declaration is missing the usual space before
((e.g.,BuildAotApplicationAndÜmläüts (string ...)) used throughout the tests. Please adjust to match the repo’s C# formatting conventions.
public void BuildAotApplicationAndÜmläüts(string supportedAbis, bool enableLLVM, bool usesAssemblyBlobs)
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs:260
- Minor formatting regression: missing spaces around
=in this assignment (var abisSanitized= ...). Please change tovar abisSanitized = ...for consistency.
var abisSanitized= supportedAbis.Replace (";", "").Replace ("-", "").Replace ("_", "");
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1035
- Minor formatting regression:
var path= ...is missing spaces around=. Please change tovar path = ...for consistency with the rest of the file.
var path= Path.Combine ("temp", TestName);
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1449
- Minor formatting regression:
var proj= ...is missing spaces around=. Please change tovar proj = ...to keep formatting consistent.
var proj= new XamarinAndroidApplicationProject {
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:1516
- Minor formatting regression: local function signature is missing the usual space before
((CreateItem (string include)) used throughout the tests. Please adjust to match the repo’s C# formatting conventions.
BuildItem CreateItem(string include) =>
| if (TestEnvironment.CommercialBuildAvailable) { | ||
| foreach (var target in new [] { "_Sign", "_BuildApkEmbed" }) { | ||
| Assert.IsTrue (b.Output.IsTargetSkipped (target), $"`{target}` should be skipped!"); | ||
| } | ||
| } |
There was a problem hiding this comment.
This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).
| if (TestEnvironment.CommercialBuildAvailable) { | ||
| appBuilder.Output.AssertTargetIsSkipped ("CoreCompile"); | ||
| appBuilder.Output.AssertTargetIsSkipped ("_BuildLibraryImportsCache"); | ||
| appBuilder.Output.AssertTargetIsSkipped ("_ResolveLibraryProjectImports"); | ||
| appBuilder.Output.AssertTargetIsSkipped ("_GenerateJavaStubs"); |
There was a problem hiding this comment.
This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).
| if (TestEnvironment.CommercialBuildAvailable) { | ||
| Assert.IsTrue ( | ||
| b.Output.IsTargetSkipped ("_CompileJava"), | ||
| "the _CompileJava target should be skipped"); | ||
| Assert.IsTrue ( | ||
| b.Output.IsTargetSkipped ("_BuildApkEmbed"), |
There was a problem hiding this comment.
This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).
| if (TestEnvironment.CommercialBuildAvailable) { | ||
| Assert.IsTrue (b.Output.IsTargetSkipped ("_GenerateJavaStubs"), "Target _GenerateJavaStubs should have been skipped"); | ||
| } |
There was a problem hiding this comment.
This is intentional. These tests primarily validate non-incremental functionality (AOT compilation, asset pack packaging, binding generation, resource processing, etc.). Only the incremental target-skipping assertions are secondary checks. Skipping the entire test with AssertCommercialBuild() would lose coverage for the primary test functionality on public CI. The if (TestEnvironment.CommercialBuildAvailable) pattern is already used elsewhere in the test suite (e.g., InstallAndRunTests.cs).
Restore missing spaces before parentheses and around assignment operators that were accidentally removed when editing out the AssertCommercialBuild() lines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
These tests verify MSBuild targets are skipped on incremental builds, which only holds true when Fast Deployment is available (commercial builds). Without it, additional targets run and the assertions fail.
Fixes: #11419