Skip to content

[v3-4-test] Run dependency explanations separately from main AMD canaries (#74311) - #76182

Merged
potiuk merged 1 commit into
v3-4-testfrom
backport-ca68614-v3-4-test
Oct 31, 2026
Merged

potiuk merged 1 commit into
v3-4-testfrom
backport-ca68614-v3-4-test

Conversation

@github-actions

Copy link
Copy Markdown
Contributor
  • Keep dependency upgrade explanations off the canary completion path

Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics.

  • Keep dependency reports from writing to the main branch cache

Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary.

  • Keep dependency reports aligned with supported Python versions

  • Keep dependency reports going when one input is gone

  • Skip entries whose saved constraints or CI image are missing or expired,
    with a warning and a summary line, instead of failing every report.
  • Name report inputs by canary attempt and explain each input once, so a
    re-run canary no longer queues a second report from the same inputs.
  • Post the same notice on every attempt when the canary saved no inputs.
  • Free up disk space and move Docker to /mnt before loading the image.
  • Save the checked constraints through breeze's --save-constraints-to,
    pinned to the resolved constraints commit; failing to save only warns,
    so the canary no longer depends on raw gh api calls.
  • Say that no cache step, not workflow_run, keeps the cache untouched, and
    skip the plan tests when node is unavailable.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

  • Give dependency explanations room for the slowest constraints mode

Explaining constraints-source-providers took 105-120 minutes per job in
the last four main canaries, past the report job's 90-minute timeout.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

  • Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints

CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the
constraints-* branches, apache/airflow by default, and the Dockerfiles
read it. Breeze's download_constraints_file and get_branch_commit always
used apache/airflow instead, so setting it changed nothing for
constraints-version-check, the SBOM and provider dependency downloads,
or the constraints commit saved for the dependency report.

Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to
apache/airflow when it is unset or empty, so existing callers and CI on
apache/airflow behave as before, and a saved commit always comes from
the repository its constraints were downloaded from. Other files fetched
through download_file_from_github still come from apache/airflow.

  • Explain dependencies from the image a RunsOn canary keeps in S3

Since #74889 scheduled canaries run on RunsOn, where the CI image stash
goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so
every report entry was skipped as "CI image missing". The plan now also
accepts that digest artifact; the explain job then runs on a RunsOn
runner, downloads exactly that tarball by its digest and checks it like
stash-restore does. It prepares the disk with the workflow version's
runner scripts, as the image build job now does.

  • Use the current RunsOn runner label for dependency reports from S3

main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families
and spot strategy); the report's copy now matches it again, as the plan
test that compares the two requires.

  • Describe re-used report inputs as saved and refuse parent segments in the S3 stash

An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check.

Generated-by: Claude Opus 5


(cherry picked from commit ca68614)

Co-authored-by: Yossi Eliaz zozo123@users.noreply.github.com
Co-authored-by: Claude Opus 5.5 noreply@anthropic.com
Co-authored-by: Shahar Epstein 60007259+shahar1@users.noreply.github.com
Co-authored-by: Jarek Potiuk jarek@potiuk.com

…ries (#74311)

* Keep dependency upgrade explanations off the canary completion path

Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics.

* Keep dependency reports from writing to the main branch cache

Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary.

* Keep dependency reports aligned with supported Python versions

* Keep dependency reports going when one input is gone

- Skip entries whose saved constraints or CI image are missing or expired,
  with a warning and a summary line, instead of failing every report.
- Name report inputs by canary attempt and explain each input once, so a
  re-run canary no longer queues a second report from the same inputs.
- Post the same notice on every attempt when the canary saved no inputs.
- Free up disk space and move Docker to /mnt before loading the image.
- Save the checked constraints through breeze's --save-constraints-to,
  pinned to the resolved constraints commit; failing to save only warns,
  so the canary no longer depends on raw `gh api` calls.
- Say that no cache step, not workflow_run, keeps the cache untouched, and
  skip the plan tests when node is unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Give dependency explanations room for the slowest constraints mode

Explaining constraints-source-providers took 105-120 minutes per job in
the last four main canaries, past the report job's 90-minute timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints

CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the
constraints-* branches, apache/airflow by default, and the Dockerfiles
read it. Breeze's download_constraints_file and get_branch_commit always
used apache/airflow instead, so setting it changed nothing for
constraints-version-check, the SBOM and provider dependency downloads,
or the constraints commit saved for the dependency report.

Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to
apache/airflow when it is unset or empty, so existing callers and CI on
apache/airflow behave as before, and a saved commit always comes from
the repository its constraints were downloaded from. Other files fetched
through download_file_from_github still come from apache/airflow.

* Explain dependencies from the image a RunsOn canary keeps in S3

Since #74889 scheduled canaries run on RunsOn, where the CI image stash
goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so
every report entry was skipped as "CI image missing". The plan now also
accepts that digest artifact; the explain job then runs on a RunsOn
runner, downloads exactly that tarball by its digest and checks it like
stash-restore does. It prepares the disk with the workflow version's
runner scripts, as the image build job now does.

* Use the current RunsOn runner label for dependency reports from S3

main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families
and spot strategy); the report's copy now matches it again, as the plan
test that compares the two requires.

* Describe re-used report inputs as saved and refuse parent segments in the S3 stash

An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check.

Generated-by: Claude Opus 5

---------
(cherry picked from commit ca68614)

Co-authored-by: Yossi Eliaz <zozo123@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@shahar1
shahar1 marked this pull request as ready for review October 31, 2026 21:01
@potiuk
potiuk merged commit a187d8a into v3-4-test Oct 31, 2026
5 of 7 checks passed
@potiuk
potiuk deleted the backport-ca68614-v3-4-test branch October 31, 2026 21:02
vatsrahul1001 pushed a commit that referenced this pull request Nov 4, 2026
…ries (#74311) (#76182)

* Keep dependency upgrade explanations off the canary completion path

Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics.

* Keep dependency reports from writing to the main branch cache

Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary.

* Keep dependency reports aligned with supported Python versions

* Keep dependency reports going when one input is gone

- Skip entries whose saved constraints or CI image are missing or expired,
  with a warning and a summary line, instead of failing every report.
- Name report inputs by canary attempt and explain each input once, so a
  re-run canary no longer queues a second report from the same inputs.
- Post the same notice on every attempt when the canary saved no inputs.
- Free up disk space and move Docker to /mnt before loading the image.
- Save the checked constraints through breeze's --save-constraints-to,
  pinned to the resolved constraints commit; failing to save only warns,
  so the canary no longer depends on raw `gh api` calls.
- Say that no cache step, not workflow_run, keeps the cache untouched, and
  skip the plan tests when node is unavailable.



* Give dependency explanations room for the slowest constraints mode

Explaining constraints-source-providers took 105-120 minutes per job in
the last four main canaries, past the report job's 90-minute timeout.



* Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints

CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the
constraints-* branches, apache/airflow by default, and the Dockerfiles
read it. Breeze's download_constraints_file and get_branch_commit always
used apache/airflow instead, so setting it changed nothing for
constraints-version-check, the SBOM and provider dependency downloads,
or the constraints commit saved for the dependency report.

Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to
apache/airflow when it is unset or empty, so existing callers and CI on
apache/airflow behave as before, and a saved commit always comes from
the repository its constraints were downloaded from. Other files fetched
through download_file_from_github still come from apache/airflow.

* Explain dependencies from the image a RunsOn canary keeps in S3

Since #74889 scheduled canaries run on RunsOn, where the CI image stash
goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so
every report entry was skipped as "CI image missing". The plan now also
accepts that digest artifact; the explain job then runs on a RunsOn
runner, downloads exactly that tarball by its digest and checks it like
stash-restore does. It prepares the disk with the workflow version's
runner scripts, as the image build job now does.

* Use the current RunsOn runner label for dependency reports from S3

main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families
and spot strategy); the report's copy now matches it again, as the plan
test that compares the two requires.

* Describe re-used report inputs as saved and refuse parent segments in the S3 stash

An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check.

Generated-by: Claude Opus 5

---------
(cherry picked from commit ca68614)

Co-authored-by: Yossi Eliaz <zozo123@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants