Skip to content

feat: Add multi-arch build API fields and pre-flight validations#2127

Open
anchi205 wants to merge 1 commit intoshipwright-io:mainfrom
anchi205:multiarch-api
Open

feat: Add multi-arch build API fields and pre-flight validations#2127
anchi205 wants to merge 1 commit intoshipwright-io:mainfrom
anchi205:multiarch-api

Conversation

@anchi205
Copy link
Member

@anchi205 anchi205 commented Mar 5, 2026

Changes

Add ImagePlatform and MultiArch types to Build and BuildRun CRDs under
spec.output.multiArch, allowing users to declare target platforms for
multi-architecture image builds as described in SHIP-0043.

Pre-flight validation in the BuildRun controller rejects builds early
when platform fields are invalid, nodeSelector conflicts with os/arch
labels, the controller is not in PipelineRun executor mode, or the
cluster lacks a Ready schedulable node for a requested platform.

RBAC is extended with get/list/watch on nodes for the controller
ServiceAccount, required by the node availability check.

Closes #2075
Assisted-by: Cursor

Submitter Checklist

  • Includes tests if functionality changed/was added
  • Includes docs if changes are user-facing
  • Set a kind label on this PR
  • Release notes block has been filled in, or marked NONE

Release Notes

Added spec.output.multiArch.platforms field to Build and BuildRun APIs, allowing users to define target OS/architecture combinations for multi-arch image builds.

@pull-request-size pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 5, 2026
@openshift-ci openshift-ci bot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Mar 5, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 5, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign qu1queee for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added release-note Label for when a PR has specified a release note and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 5, 2026
@anchi205 anchi205 marked this pull request as draft March 6, 2026 03:36
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 6, 2026
@pull-request-size pull-request-size bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 17, 2026
@anchi205 anchi205 marked this pull request as ready for review March 17, 2026 06:41
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2026
@anchi205 anchi205 removed the request for review from apoorvajagtap March 17, 2026 06:41
@anchi205 anchi205 changed the title feat: Add API in Shipwright for multi-arch build feat: Add multi-arch build API and orchestration Mar 17, 2026
@anchi205 anchi205 force-pushed the multiarch-api branch 3 times, most recently from af45e2c to 00999d8 Compare March 17, 2026 07:42
@SaschaSchwarze0 SaschaSchwarze0 requested a review from Copilot March 23, 2026 09:34
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces first-class multi-architecture image build support to Shipwright Builds/BuildRuns, adding API fields for platform selection and implementing a PipelineRun-based fan-out/fan-in orchestration that produces an OCI image index and per-platform status reporting.

Changes:

  • Extend Build/BuildRun APIs and CRDs with spec.output.multiArch.platforms and BuildRun status fields for per-platform results and manifest list digest.
  • Generate multi-arch PipelineRuns that push a source bundle, execute per-platform build tasks on matching nodes, and assemble an OCI image index.
  • Add image-processing CLI support and supporting pkg/image utilities for source bundling and index assembly, plus tests and docs.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
pkg/validate/output_test.go Adds validation tests for multi-arch platforms, executor mode, node conflicts, and node availability checks.
pkg/validate/output.go Implements multi-arch validation helpers (platforms, nodeSelector conflicts, executor, node presence).
pkg/reconciler/buildrun/resources/results_test.go Adds tests for aggregating multi-arch PipelineRun child TaskRun results into BuildRun status.
pkg/reconciler/buildrun/resources/results.go Implements multi-arch result extraction and status population (PlatformResults + ManifestDigest).
pkg/reconciler/buildrun/resources/resource_builders.go Switches nodeSelector merge helper to exported MergeMaps; adds multi-arch workspace bindings.
pkg/reconciler/buildrun/resources/pipelinerun_generator.go Adds multi-arch detection and generates fan-out PipelineRun with per-platform tasks + assemble step.
pkg/reconciler/buildrun/resources/multiarch_test.go New tests for multi-arch PipelineRun generation behavior and wiring.
pkg/reconciler/buildrun/resources/multiarch.go New multi-arch PipelineRun building blocks (task naming, source bundle, per-platform tasks, assemble-index, TaskRunSpecs).
pkg/reconciler/buildrun/resources/image_processing.go Renames mergeMaps to exported MergeMaps and updates usage.
pkg/reconciler/buildrun/buildrun.go Adds multi-arch validation at reconcile time and multi-arch result extraction for PipelineRun execution.
pkg/image/index_test.go Adds tests for assembling an image index from per-platform images.
pkg/image/index.go Adds AssembleImageIndex helper for creating OCI image indexes (manifest lists).
pkg/image/bundle_test.go Adds tests for bundling a directory into a single-layer OCI image (including symlink behavior).
pkg/image/bundle.go Adds source directory bundling as an OCI image layer for cross-node source distribution.
pkg/apis/build/v1beta1/zz_generated.deepcopy.go Updates generated deep-copies for new API fields (MultiArch, PlatformResults, etc.).
pkg/apis/build/v1beta1/buildrun_types.go Adds BuildRun status types for multi-arch per-platform results and manifest digest.
pkg/apis/build/v1beta1/build_types.go Adds API types ImagePlatform and MultiArch plus new BuildReasons for multi-arch validation failures.
docs/build.md Documents how to configure multi-arch builds, requirements, and examples.
deploy/crds/shipwright.io_builds.yaml Updates Build CRD schema with spec.output.multiArch.platforms.
deploy/crds/shipwright.io_buildruns.yaml Updates BuildRun CRD schema with spec.output.multiArch.platforms and status fields.
deploy/200-role.yaml Grants controller RBAC permissions to list/watch nodes for platform availability validation.
cmd/image-processing/main_test.go Adds tests for source bundle push and index assembly CLI modes.
cmd/image-processing/main.go Adds CLI modes for pushing source bundles and assembling/pushing image indexes, plus platform-image parsing.
Comments suppressed due to low confidence (1)

pkg/reconciler/buildrun/resources/image_processing.go:289

  • MergeMaps mutates and returns the first input map in-place. Several new call sites pass maps from informer-cached objects (e.g., build.Spec.NodeSelector / build.Spec.Output.Annotations), so this can unintentionally mutate cached resources and cause hard-to-debug reconciliation behavior. Make MergeMaps allocate a new map, copy entries from first, then overlay second, without modifying either input map (and handle nil maps).
// MergeMaps merges two string maps. Values in second take precedence
// over first when keys overlap.
func MergeMaps(first map[string]string, second map[string]string) map[string]string {
	if len(first) == 0 {
		first = map[string]string{}
	}
	for k, v := range second {
		first[k] = v
	}
	return first

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

if in.PlatformResults != nil {
in, out := &in.PlatformResults, &out.PlatformResults
*out = make([]PlatformBuildResult, len(*in))
copy(*out, *in)
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BuildRunStatus.DeepCopyInto shallow-copies PlatformResults with copy(*out, *in). PlatformBuildResult contains a Vulnerabilities slice, so this will share underlying slice data between copies and violates Kubernetes deepcopy semantics. Regenerate deepcopy with controller-gen (or deep-copy each element via DeepCopyInto) so nested slices are fully copied.

Suggested change
copy(*out, *in)
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}

Copilot uses AI. Check for mistakes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is auto-generated code from controller-gen, so I don't think we should be changing this. I have verified this doesn't cause issues because the reconciler rebuilds PlatformResults from scratch on each reconciliation rather than mutating deep-copied structs.

Comment on lines +432 to +436
}
}
}
}
})
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test file has mismatched/missing closing braces in the nested for/if blocks (around the vulnerability scanning args assertion). As written it will not compile, which will break the test suite. Please fix the block structure and run gofmt to ensure the file builds.

Suggested change
}
}
}
}
})
}
}
}
}
})

Copilot uses AI. Check for mistakes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its compiling properly, so this is a false positive.

Comment on lines +104 to +109
defer f.Close()

if _, err := io.Copy(tarWriter, f); err != nil {
return fmt.Errorf("writing %s to tar: %w", relPath, err)
}

Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createTarGz defers f.Close() inside the filepath.Walk callback. With many files this will keep file descriptors open until the entire walk finishes and can exhaust FD limits. Close the file immediately after copying (avoid defer in the loop/callback).

Suggested change
defer f.Close()
if _, err := io.Copy(tarWriter, f); err != nil {
return fmt.Errorf("writing %s to tar: %w", relPath, err)
}
if _, err := io.Copy(tarWriter, f); err != nil {
_ = f.Close()
return fmt.Errorf("writing %s to tar: %w", relPath, err)
}
if err := f.Close(); err != nil {
return fmt.Errorf("closing %s: %w", filePath, err)
}

Copilot uses AI. Check for mistakes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a false positive

The defer f.Close() is inside the anonymous function passed to filepath.Walk

filepath.Walk calls this anonymous function once per file as a separate function invocation. f.Close() runs when each callback returns, not when the outer createTarGz returns. Each file descriptor is opened, copied, and closed before the next file is processed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to turn this anonymous function into a real function. It should fix the false positive and also make this behavior easier to unit test.

Copy link
Member

@IrvingMg IrvingMg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work here! Would it be possible to split this into smaller PRs to make it easier to review?

Also, is there an issue for this? If yes, could you link it for more context?

@anchi205
Copy link
Member Author

anchi205 commented Mar 25, 2026

Thanks for the work here! Would it be possible to split this into smaller PRs to make it easier to review?

Also, is there an issue for this? If yes, could you link it for more context?

SHIP for this : https://github.com/shipwright-io/community/blob/main/ships/0043-multi-arch-image-builds.md

I will be splitting this entire work into 3 PRs. Please review it once I am done. Thank you

@adambkaplan
Copy link
Member

We have a parent "Epic" issue in the Community repo where we are tracking all the work for this feature: shipwright-io/community#285. This is also linked on the project Roadmap.

This is most directly related to #2075, although I need to clean up that feature's description. I used an AI assistant to generate these features, and on this one it's quite clear that it hallucinated the requirements.

Add ImagePlatform and MultiArch types to Build and BuildRun CRDs under
spec.output.multiArch, allowing users to declare target platforms for
multi-architecture image builds as described in SHIP-0043.

Pre-flight validation in the BuildRun controller rejects builds early
when platform fields are invalid, nodeSelector conflicts with os/arch
labels, the controller is not in PipelineRun executor mode, or the
cluster lacks a Ready schedulable node for a requested platform.

RBAC is extended with get/list/watch on nodes for the controller
ServiceAccount, required by the node availability check.

Closes shipwright-io#2075
Assisted-by: Cursor

Signed-off-by: Anchita Borah <anborah@redhat.com>
@pull-request-size pull-request-size bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Mar 25, 2026
@anchi205 anchi205 changed the title feat: Add multi-arch build API and orchestration feat: Add multi-arch build API fields and pre-flight validations Mar 25, 2026
@anchi205 anchi205 requested a review from IrvingMg March 25, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note Label for when a PR has specified a release note size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[FEATURE] Implement multi-arch build API and validations

5 participants