Merge branch 'main' into h200-aks

Signed-off-by: Mark Chmarny <mchmarny@users.noreply.github.com>

Author: mchmarny

.claude/CLAUDE.md

@@ -247,6 +247,27 @@ slog.Error("operation failed", "error", err, "component", "gpu-collector")
 
 **Note:** A component must have either `helm` OR `kustomize` configuration, not both.
 
+**Using mixins for shared OS/platform content:**
+```yaml
+# Leaf overlay referencing mixins instead of duplicating content
+spec:
+  base: h100-eks-ubuntu-training
+  mixins:
+    - os-ubuntu          # Ubuntu constraints (defined once in recipes/mixins/)
+    - platform-kubeflow  # kubeflow-trainer component (defined once in recipes/mixins/)
+  criteria:
+    service: eks
+    accelerator: h100
+    os: ubuntu
+    intent: training
+    platform: kubeflow
+  constraints:
+    - name: K8s.server.version
+      value: ">= 1.32.4"
+```
+
+Mixins carry only `constraints` and `componentRefs` — no `criteria`, `base`, `mixins`, or `validation`. They live in `recipes/mixins/` with `kind: RecipeMixin`.
+
 ## Error Wrapping Rules
 
 **Never return bare errors.** Every `return err` must wrap with context:
@@ -457,6 +478,35 @@ ${AICR_BIN} validate -r recipe.yaml -s snapshot.yaml --no-cluster
 | Ignore `Close()` error on writable file handles | Capture and check `closeErr := f.Close()` |
 | Hardcode resource names from templates | Extract to named constants to keep code and templates in sync |
 
+## Pull Request Requirements
+
+**Pre-push checklist:** Always run `make qualify` before pushing. This is the CI-equivalent gate that covers tests, linting (golangci-lint + yamllint), e2e, vulnerability scan, and repo-specific checks (docs sidebar, agents sync). Do not substitute a subset of commands — if `make qualify` passes locally, CI will pass.
+
+**Branch hygiene:**
+- Always rebase onto the target branch before pushing: `git fetch origin main && git rebase origin/main`
+- Squash commits into a single commit before push
+- Cryptographically sign commits (`git commit -S`)
+
+**PR description:** Use the template from `.github/PULL_REQUEST_TEMPLATE.md` exactly as defined there. Do not inline a modified copy — read and fill in the canonical template. The template covers: Summary, Motivation/Context (with Fixes/Related), Type of Change, Components Affected, Implementation Notes, Testing, Risk Assessment, and Checklist.
+
+**Test coverage gate (Go packages only):**
+Before pushing a PR that changes Go source files, check test coverage on affected packages. Set `pkg` to the narrowest directory root you want to measure — `$pkg/...` intentionally includes descendant packages. Prefer the narrowest changed root (e.g., if only `pkg/collector/topology` changed, use `pkg=pkg/collector/topology`, not `pkg=pkg/collector`). Use a broader root only when you intentionally want one combined delta across related subpackages.
+1. Run `GOFLAGS="-mod=vendor" go test -coverprofile=cover.out ./$pkg/...` on each changed package
+2. Get the baseline using a clean worktree (changes must be committed first): `(git worktree add $TMPDIR/baseline origin/main && (cd $TMPDIR/baseline && GOFLAGS="-mod=vendor" go test -coverprofile=$TMPDIR/base.out ./$pkg/...); rc=$?; git worktree remove --force $TMPDIR/baseline; return $rc 2>/dev/null || (exit $rc))`. This preserves the test exit status through cleanup. Write the profile to `$TMPDIR/base.out` (outside the worktree) so it survives cleanup. Compare with `go tool cover -func` on both profiles. Skip this step for entirely new packages.
+3. **Block** if `make test-coverage` fails — this enforces the project-wide 70% floor (from `.settings.yaml`). Do not use per-package profiles for this check.
+4. **Flag** any package with per-package coverage decrease > 0.5% (comparing step 1 vs step 2)
+5. **Block** if any new exported function or method (identified via `git diff origin/main -- $pkg/` — look for added `func` lines with uppercase names) has 0% coverage — add tests before pushing
+6. Report the delta in the PR description's Testing section (e.g., `pkg/recipe: 90.4% → 90.3% (-0.1%)`)
+This rule does not apply to non-Go changes (YAML, docs, CI workflows). Note: CI also posts per-package coverage deltas post-push via `go-coverage-report` in `on-push-comment.yaml`; this gate catches regressions before push.
+
+**PR policy:**
+- Do NOT add `Co-Authored-By` lines (organization policy)
+- Do NOT add "Generated with Claude Code", "Created by Codex", or similar attribution
+- Add appropriate type labels: `enhancement`, `bug`, `documentation`
+- Area labels are auto-assigned by `.github/labeler.yml` based on changed file paths (e.g., `area/recipes`, `area/ci`, `area/api`, `area/cli`, `area/bundler`, `area/collector`, `area/validator`, `area/docs`, `area/infra`, `area/tests`). You may also add them manually when the auto-labeler wouldn't match (e.g., issue-only PRs or cross-cutting changes).
+- Do NOT add `size/*` labels (auto-assigned by bot)
+- Keep the PR title under 70 characters; use the description for details
+
 ## Key Files
 
 | File | Purpose |
@@ -467,6 +517,7 @@ ${AICR_BIN} validate -r recipe.yaml -s snapshot.yaml --no-cluster
 | `.settings.yaml` | Project settings: tool versions, quality thresholds, build/test config (single source of truth) |
 | `recipes/registry.yaml` | Declarative component configuration |
 | `recipes/overlays/*.yaml` | Recipe overlay definitions |
+| `recipes/mixins/*.yaml` | Composable mixin fragments (OS constraints, platform components) |
 | `recipes/components/*/values.yaml` | Component Helm values |
 | `api/aicr/v1/server.yaml` | OpenAPI spec |
 | `.goreleaser.yaml` | Release configuration |

.github/actions/aicr-build/action.yml

@@ -15,6 +15,16 @@
 name: 'AICR Build'
 description: 'Builds the aicr validator image (via Dockerfile) and CLI binary, and loads the image into kind.'
 
+inputs:
+  build_validators:
+    description: 'Deprecated: use validator_phases instead. Ignored when validator_phases is set.'
+    required: false
+    default: 'true'
+  validator_phases:
+    description: 'Comma-separated validator phases to build (e.g., "conformance,deployment"), or "none" to skip all. Takes precedence over build_validators.'
+    required: false
+    default: ''
+
 runs:
   using: 'composite'
   steps:
@@ -27,28 +37,54 @@ runs:
 
     - name: Build snapshot agent image and load into kind
       shell: bash
+      env:
+        GOFLAGS: -mod=vendor
       run: |
-        # Build snapshot agent image with CUDA runtime (provides nvidia-smi for GPU detection).
+        # Build snapshot agent image with CUDA base (provides nvidia-smi for GPU detection).
+        # Uses cuda:base (~250MB) instead of cuda:runtime (~1.8GB) — only nvidia-smi is needed.
         # GPU test workflows use --image=ko.local:smoke-test for aicr snapshot.
         CGO_ENABLED=0 go build -trimpath -o dist/aicr ./cmd/aicr
         docker build -t ko.local:smoke-test -f - . <<'DOCKERFILE'
-        FROM nvcr.io/nvidia/cuda:13.1.0-runtime-ubuntu24.04
+        FROM nvcr.io/nvidia/cuda:13.1.0-base-ubuntu24.04
         COPY dist/aicr /usr/local/bin/aicr
         ENTRYPOINT ["/usr/local/bin/aicr"]
         DOCKERFILE
-        kind load docker-image ko.local:smoke-test --name "${KIND_CLUSTER_NAME}"
+
+        # Load onto all nodes. The snapshot agent requests nvidia.com/gpu but
+        # does not set a node selector, so it can land on any GPU-capable node
+        # including the control-plane (e.g., T4 smoke test).
+        timeout 600 kind load docker-image ko.local:smoke-test --name "${KIND_CLUSTER_NAME}" || {
+          echo "::warning::kind load attempt 1 failed for ko.local:smoke-test, retrying..."
+          timeout 600 kind load docker-image ko.local:smoke-test --name "${KIND_CLUSTER_NAME}"
+        }
 
     - name: Build validator images and load into kind
+      if: "!(inputs.validator_phases == 'none' || (inputs.validator_phases == '' && inputs.build_validators == 'false'))"
       shell: bash
       env:
         GOFLAGS: -mod=vendor
       run: |
-        # Compile validator binaries on host (Go build cache) then COPY-only images.
+        # Determine which validator phases to build.
+        # validator_phases takes precedence; build_validators is a deprecated fallback.
+        if [[ -n "${{ inputs.validator_phases }}" ]]; then
+          if [[ "${{ inputs.validator_phases }}" == "none" ]]; then
+            echo "Skipping validator builds (validator_phases=none)"
+            exit 0
+          fi
+          PHASES="${{ inputs.validator_phases }}"
+        else
+          # Default: build all phases (backwards compatible)
+          PHASES="deployment,performance,conformance"
+        fi
+
+        # Compile only the requested validator binaries.
         mkdir -p dist/validator
-        CGO_ENABLED=0 go build -trimpath -o dist/validator/deployment ./validators/deployment
-        CGO_ENABLED=0 go build -trimpath -o dist/validator/performance ./validators/performance
-        CGO_ENABLED=0 go build -trimpath -o dist/validator/conformance ./validators/conformance
-        for phase in deployment performance conformance; do
+        for phase in ${PHASES//,/ }; do
+          echo "Building validator binary: ${phase}"
+          CGO_ENABLED=0 go build -trimpath -o "dist/validator/${phase}" "./validators/${phase}"
+        done
+
+        for phase in ${PHASES//,/ }; do
           mkdir -p "validators/${phase}/testdata"
           docker build -t "ko.local/aicr-validators/${phase}:latest" -f - . <<DOCKERFILE
         FROM gcr.io/distroless/static-debian12:nonroot
@@ -58,7 +94,10 @@ runs:
         USER nonroot
         ENTRYPOINT ["/${phase}"]
         DOCKERFILE
-          kind load docker-image "ko.local/aicr-validators/${phase}:latest" --name "${KIND_CLUSTER_NAME}"
+          timeout 300 kind load docker-image "ko.local/aicr-validators/${phase}:latest" --name "${KIND_CLUSTER_NAME}" || {
+            echo "::warning::kind load attempt 1 failed for ko.local/aicr-validators/${phase}:latest, retrying..."
+            timeout 300 kind load docker-image "ko.local/aicr-validators/${phase}:latest" --name "${KIND_CLUSTER_NAME}"
+          }
         done
 
     - name: Build aicr binary

.github/actions/gpu-snapshot-validate/action.yml

@@ -45,8 +45,9 @@ runs:
     - name: Validate snapshot detected GPU
       shell: bash
       run: |
-        GPU_MODEL=$(yq eval '.measurements[] | select(.type == "GPU") | .subtypes[0].data["gpu.model"]' snapshot.yaml)
-        GPU_COUNT=$(yq eval '.measurements[] | select(.type == "GPU") | .subtypes[0].data["gpu-count"]' snapshot.yaml)
+        # Query by subtype field (not index) — #502 added a "hardware" subtype before "smi".
+        GPU_MODEL=$(yq eval '.measurements[] | select(.type == "GPU") | .subtypes[] | select(.subtype == "smi") | .data["gpu.model"]' snapshot.yaml)
+        GPU_COUNT=$(yq eval '.measurements[] | select(.type == "GPU") | .subtypes[] | select(.subtype == "smi") | .data["gpu-count"]' snapshot.yaml)
         echo "GPU model: ${GPU_MODEL}"
         echo "GPU count: ${GPU_COUNT}"
         if [[ "${GPU_MODEL}" != *"${{ inputs.gpu_model }}"* ]]; then

.github/workflows/build-attested.yaml

@@ -92,7 +92,7 @@ jobs:
           done
 
       - name: Upload archives
-        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f  # v7.0.0
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a  # v7.0.1
         with:
           name: aicr-attested-binaries
           path: dist/*.tar.gz

.github/workflows/conflict-check.yaml

@@ -37,7 +37,7 @@ jobs:
     timeout-minutes: 10
     steps:
       - name: Check Mergeable State
-        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd  # v8.0.0
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3  # v9.0.0
         with:
           script: |
             const label = 'needs-rebase';

.github/workflows/fern-docs-ci.yaml

@@ -0,0 +1,54 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Validates Fern docs configuration on pull requests that touch docs or fern/.
+
+name: Fern Docs CI
+
+on:
+  pull_request:
+    paths:
+      - 'docs/**'
+      - 'fern/**'
+      - '.github/workflows/fern-docs-ci.yaml'
+  workflow_dispatch: {}
+
+permissions:
+  contents: read
+
+jobs:
+  fern-check:
+    name: Fern Check
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Setup Node.js
+        uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
+        with:
+          node-version: '20'
+
+      - name: Install Fern CLI
+        run: npm install -g fern-api@$(jq -r .version fern/fern.config.json)
+
+      - name: Fern check
+        run: fern check
+
+      - name: Check links
+        uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 # v2.8.0
+        with:
+          args: --offline --no-progress 'docs/**/*.md'
+          fail: true

.github/workflows/fern-docs-preview-build.yml

@@ -0,0 +1,64 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Workflow 1 of 2 for Fern doc previews.
+#
+# Collects the fern/ sources and PR metadata from the (possibly untrusted) PR
+# branch and uploads them as an artifact. No secrets are used here, so this is
+# safe to run on fork PRs via the regular pull_request trigger.
+#
+# The companion workflow (fern-docs-preview-comment.yml) picks up the artifact,
+# builds the preview with DOCS_FERN_TOKEN, and posts the PR comment.
+
+name: "Preview Fern Docs: Build"
+
+on:
+  pull_request:
+    paths:
+      - 'docs/**'
+      - 'fern/**'
+      - '.github/workflows/fern-docs-preview-build.yml'
+
+permissions:
+  contents: read
+
+jobs:
+  collect:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout PR
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+
+      - name: Save PR metadata
+        env:
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          HEAD_REF: ${{ github.head_ref }}
+          BASE_REF: ${{ github.base_ref }}
+        run: |
+          mkdir -p preview-metadata
+          echo "$PR_NUMBER" > preview-metadata/pr_number
+          echo "$HEAD_REF" > preview-metadata/head_ref
+          git diff --name-only "origin/${BASE_REF}...HEAD" -- '*.md' > preview-metadata/changed_md_files 2>/dev/null || true
+
+      - name: Upload fern sources and metadata
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: fern-preview
+          path: |
+            fern/
+            docs/
+            preview-metadata/
+          retention-days: 1