Skip to content

chore(codebuild): add region constraints and fix fleet deletion timeouts in integ tests#36956

Merged
mergify[bot] merged 4 commits into
mainfrom
fix/codebuild-integ-tests-region-constraints
Mar 13, 2026
Merged

chore(codebuild): add region constraints and fix fleet deletion timeouts in integ tests#36956
mergify[bot] merged 4 commits into
mainfrom
fix/codebuild-integ-tests-region-constraints

Conversation

@aemada-aws

Copy link
Copy Markdown
Contributor

Issue # (if applicable)

Closes #.

Reason for this change

Several CodeBuild integration tests were failing across multiple error categories:

  1. Unrecognized resource types: [AWS::CodeBuild::Fleet]AWS::CodeBuild::Fleet CloudFormation resource is not available in all regions
  2. LambdaCompute feature is not supported in region — Lambda compute mode is only available in 10 regions
  3. Region X is not supported for WINDOWS_SERVER_2022_CONTAINER — Windows Server 2022 environment type is only available in 8 regions
  4. Fleet deletion timeout (NotStabilized)AWS::CodeBuild::Fleet resource deletion takes ~40 minutes, exceeding the CFN resource handler's ~21 minute stabilization timeout
  5. macOS fleet build assertion timeout — macOS dedicated host provisioning takes 20-30+ minutes, exceeding the assertion framework's timeout limits

Description of changes

Region constraints (all 11 tests):
Added regions property to IntegTest constructors, with per-test comments explaining the specific feature constraint:

  • Fleet tests: AWS::CodeBuild::Fleet not available in all regions
  • Lambda tests: ARM_LAMBDA_CONTAINER / LINUX_LAMBDA_CONTAINER not available in all regions
  • Windows fleet tests: WINDOWS_SERVER_2022_CONTAINER not available in all regions
  • macOS fleet tests: MAC_ARM only available in 5 regions
  • Windows image test: WINDOWS_SERVER_2022_CONTAINER not available in all regions

Fleet deletion workaround (8 fleet tests):
Added cdkCommandOptions.destroy.expectError = true because AWS::CodeBuild::Fleet deletion consistently takes ~40 minutes, exceeding the CFN resource handler's ~21 minute stabilization timeout (verified via CloudFormation events across multiple regions).

Assertion timeout increases (6 fleet tests):
Increased waitForAssertions totalTimeout from 5 to 15 minutes for fleet tests to account for fleet instance provisioning time before builds can execute.

Hardcoded name removal (5 fleet tests):
Removed hardcoded fleetName properties ('MyFleet', 'MacOsFleet14', 'MacOsFleet15') to avoid NAME_COLLISION on re-runs.

macOS test simplification (2 tests):
Removed startBuild/batchGetBuilds assertions from macOS fleet tests since macOS dedicated host provisioning exceeds the assertion framework's timeout limits. The listFleets assertion is retained to verify fleet creation. Build execution on fleets is already validated by the Linux fleet tests.

Describe any new or updated permissions being added

N/A — no permission changes.

Description of how you validated changes

All 11 tests validated via yarn integ with --force --update-on-failed across 5 parallel regions: us-east-1, us-east-2, us-west-2, ap-southeast-2, eu-central-1.

Tests:    11 passed, 11 total

Validation command:

yarn integ \
  test/aws-codebuild/test/integ.project-fleet.js \
  test/aws-codebuild/test/integ.project-fleet-attribute-based-compute.js \
  test/aws-codebuild/test/integ.project-fleet-custom-instance-type.js \
  test/aws-codebuild/test/integ.project-fleet-overflow-behavior.js \
  test/aws-codebuild/test/integ.project-windows-2019-fleet.js \
  test/aws-codebuild/test/integ.project-windows-2022-fleet.js \
  test/aws-codebuild/test/integ.project-macos-fleet-base14.js \
  test/aws-codebuild/test/integ.project-macos-fleet-base15.js \
  test/aws-codebuild/test/integ.project-linux-arm-lambda-image.lit.js \
  test/aws-codebuild/test/integ.project-linux-lambda-image.lit.js \
  test/aws-codebuild/test/integ.project-windows-image.js \
  --disable-update-workflow --update-on-failed --force \
  --parallel-regions us-east-1 us-east-2 us-west-2 ap-southeast-2 eu-central-1

Not fixed (external dependency):

  • integ.github-org-webhook and integ.github-webhook-batch require a pre-configured GitHub access token in the AWS account — cannot be fixed in test code.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions Bot added the p2 label Feb 11, 2026
@aws-cdk-automation aws-cdk-automation requested a review from a team February 11, 2026 19:46
@mergify mergify Bot added the contribution/core This is a PR that came from AWS. label Feb 11, 2026
@github-actions

github-actions Bot commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

⚠️ Experimental Feature: This security report is currently in experimental phase. Results may include false positives and the rules are being actively refined.
Please try merge from main to avoid findings unrelated to the PR.


TestsPassed ✅SkippedFailed
Security Guardian Results408 ran408 passed
TestResult
No test annotations available

@github-actions

github-actions Bot commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

⚠️ Experimental Feature: This security report is currently in experimental phase. Results may include false positives and the rules are being actively refined.
Please try merge from main to avoid findings unrelated to the PR.


TestsPassed ✅SkippedFailed
Security Guardian Results with resolved templates408 ran408 passed
TestResult
No test annotations available

@aemada-aws aemada-aws marked this pull request as draft February 11, 2026 19:49
@aemada-aws aemada-aws force-pushed the fix/codebuild-integ-tests-region-constraints branch from 128c260 to a7793c2 Compare February 11, 2026 22:35
…s in integ tests

Several CodeBuild integration tests were failing due to regional
availability constraints and fleet resource deletion timeouts.

Changes:
- Add region constraints to 11 integ tests for features not available
  in all regions (AWS::CodeBuild::Fleet, Lambda compute, MAC_ARM,
  WINDOWS_SERVER_2022_CONTAINER)
- Remove hardcoded fleetName properties to avoid name collisions
- Add destroy.expectError for fleet tests since AWS::CodeBuild::Fleet
  deletion takes ~40 minutes, exceeding the CFN resource handler's
  ~21 minute stabilization timeout (NotStabilized)
- Increase waitForAssertions timeouts for fleet tests (5->15 min)
  to account for fleet instance provisioning time
- Simplify macOS fleet tests to only verify fleet creation (remove
  startBuild/batchGetBuilds assertions) since macOS dedicated host
  provisioning exceeds the assertion framework's timeout limits

All 11 tests validated across us-east-1, us-east-2, us-west-2,
ap-southeast-2, and eu-central-1.
@aemada-aws aemada-aws force-pushed the fix/codebuild-integ-tests-region-constraints branch from a7793c2 to bd71853 Compare February 12, 2026 06:22
CodeBuild fleet instances have a 1-hour minimum runtime before deletion
completes. The CFN resource handler's stabilization timeout is shorter
than this, so DELETE always fails with 'Exceeded attempts to wait'
(HandlerErrorCode: NotStabilized). The fleet is still cleaned up by
CodeBuild after the 1-hour window.

Updated comment to accurately explain the root cause.
@aemada-aws aemada-aws changed the title fix(codebuild): add region constraints and fix fleet deletion timeouts in integ tests chore(codebuild): add region constraints and fix fleet deletion timeouts in integ tests Mar 3, 2026
@aemada-aws aemada-aws marked this pull request as ready for review March 3, 2026 23:45
@aemada-aws aemada-aws added the pr/needs-integration-tests-deployment Requires the PR to deploy the integration test snapshots. label Mar 3, 2026
@aemada-aws aemada-aws had a problem deploying to deployment-integ-test March 3, 2026 23:45 — with GitHub Actions Error
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Mar 4, 2026
@Abogical Abogical self-assigned this Mar 4, 2026
@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Mar 4, 2026

@Abogical Abogical left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We got the following error when trying to deploy via the workflow:

Running test /codebuild/output/src3758018834/src/actions-runner/_work/aws-cdk/aws-cdk/packages/@aws-cdk-testing/framework-integ/test/aws-codebuild/test/integ.project-fleet.js in us-west-2
  ERROR      /codebuild/output/src3758018834/src/actions-runner/_work/aws-cdk/aws-cdk/packages/@aws-cdk-testing/framework-integ/test/aws-codebuild/test/integ.project-fleet.js (undefined/us-west-2) 0.001s
      Error during integration test: ManifestLoadingError: Failed to load integ manifest 'packages/@aws-cdk-testing/framework-integ/test/aws-codebuild/test/integ.project-fleet.js.snapshot/integ.json': Cloud assembly schema version mismatch: Maximum schema version supported is 50.x.x, but found 52.0.0. You need at least CLI version 2.1107.0 to read this manifest.
      Cause: Error: Cloud assembly schema version mismatch: Maximum schema version supported is 50.x.x, but found 52.0.0. You need at least CLI version 2.1107.0 to read this manifest.

Can this be fixed?

@aemada-aws

Copy link
Copy Markdown
Contributor Author

@Abogical done, it just needed a merge from main for the latest cdk cli. The issue is now the account is suspended by Codebuild 🫠

@aemada-aws aemada-aws had a problem deploying to deployment-integ-test March 10, 2026 11:45 — with GitHub Actions Failure
@aemada-aws aemada-aws had a problem deploying to deployment-integ-test March 12, 2026 09:05 — with GitHub Actions Failure
@aemada-aws aemada-aws requested a review from Abogical March 12, 2026 12:44
@Abogical Abogical removed the pr/needs-integration-tests-deployment Requires the PR to deploy the integration test snapshots. label Mar 13, 2026
@mergify

mergify Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify

mergify Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-03-13 12:50 UTC · Rule: default-squash
  • Checks passed · in-place
  • Merged2026-03-13 13:32 UTC · at 795edc36a698210dc0c5ac99bd7c63e8408ab32a

This pull request spent 42 minutes 45 seconds in the queue, including 42 minutes 32 seconds running CI.

Required conditions to merge

@mergify

mergify Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify Bot merged commit 6903378 into main Mar 13, 2026
23 of 24 checks passed
@mergify mergify Bot deleted the fix/codebuild-integ-tests-region-constraints branch March 13, 2026 13:32
@github-actions

Copy link
Copy Markdown
Contributor

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Mar 13, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

contribution/core This is a PR that came from AWS. p2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants