Repository navigation
[v3-4-test] Run dependency explanations separately from main AMD canaries (#74311) - #76182
Merged
Merged
Conversation
…ries (#74311) * Keep dependency upgrade explanations off the canary completion path Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics. * Keep dependency reports from writing to the main branch cache Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary. * Keep dependency reports aligned with supported Python versions * Keep dependency reports going when one input is gone - Skip entries whose saved constraints or CI image are missing or expired, with a warning and a summary line, instead of failing every report. - Name report inputs by canary attempt and explain each input once, so a re-run canary no longer queues a second report from the same inputs. - Post the same notice on every attempt when the canary saved no inputs. - Free up disk space and move Docker to /mnt before loading the image. - Save the checked constraints through breeze's --save-constraints-to, pinned to the resolved constraints commit; failing to save only warns, so the canary no longer depends on raw `gh api` calls. - Say that no cache step, not workflow_run, keeps the cache untouched, and skip the plan tests when node is unavailable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Give dependency explanations room for the slowest constraints mode Explaining constraints-source-providers took 105-120 minutes per job in the last four main canaries, past the report job's 90-minute timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the constraints-* branches, apache/airflow by default, and the Dockerfiles read it. Breeze's download_constraints_file and get_branch_commit always used apache/airflow instead, so setting it changed nothing for constraints-version-check, the SBOM and provider dependency downloads, or the constraints commit saved for the dependency report. Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to apache/airflow when it is unset or empty, so existing callers and CI on apache/airflow behave as before, and a saved commit always comes from the repository its constraints were downloaded from. Other files fetched through download_file_from_github still come from apache/airflow. * Explain dependencies from the image a RunsOn canary keeps in S3 Since #74889 scheduled canaries run on RunsOn, where the CI image stash goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so every report entry was skipped as "CI image missing". The plan now also accepts that digest artifact; the explain job then runs on a RunsOn runner, downloads exactly that tarball by its digest and checks it like stash-restore does. It prepares the disk with the workflow version's runner scripts, as the image build job now does. * Use the current RunsOn runner label for dependency reports from S3 main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families and spot strategy); the report's copy now matches it again, as the plan test that compares the two requires. * Describe re-used report inputs as saved and refuse parent segments in the S3 stash An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check. Generated-by: Claude Opus 5 --------- (cherry picked from commit ca68614) Co-authored-by: Yossi Eliaz <zozo123@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
1 task done
shahar1
marked this pull request as ready for review
October 31, 2026 21:01
shahar1
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
gopidesupavan,
jason810496,
jedcunningham,
jscheffl,
potiuk and
vatsrahul1001
as code owners
October 31, 2026 21:01
vatsrahul1001
pushed a commit
that referenced
this pull request
Nov 4, 2026
…ries (#74311) (#76182) * Keep dependency upgrade explanations off the canary completion path Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics. * Keep dependency reports from writing to the main branch cache Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary. * Keep dependency reports aligned with supported Python versions * Keep dependency reports going when one input is gone - Skip entries whose saved constraints or CI image are missing or expired, with a warning and a summary line, instead of failing every report. - Name report inputs by canary attempt and explain each input once, so a re-run canary no longer queues a second report from the same inputs. - Post the same notice on every attempt when the canary saved no inputs. - Free up disk space and move Docker to /mnt before loading the image. - Save the checked constraints through breeze's --save-constraints-to, pinned to the resolved constraints commit; failing to save only warns, so the canary no longer depends on raw `gh api` calls. - Say that no cache step, not workflow_run, keeps the cache untouched, and skip the plan tests when node is unavailable. * Give dependency explanations room for the slowest constraints mode Explaining constraints-source-providers took 105-120 minutes per job in the last four main canaries, past the report job's 90-minute timeout. * Honor CONSTRAINTS_GITHUB_REPOSITORY when breeze downloads constraints CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the constraints-* branches, apache/airflow by default, and the Dockerfiles read it. Breeze's download_constraints_file and get_branch_commit always used apache/airflow instead, so setting it changed nothing for constraints-version-check, the SBOM and provider dependency downloads, or the constraints commit saved for the dependency report. Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to apache/airflow when it is unset or empty, so existing callers and CI on apache/airflow behave as before, and a saved commit always comes from the repository its constraints were downloaded from. Other files fetched through download_file_from_github still come from apache/airflow. * Explain dependencies from the image a RunsOn canary keeps in S3 Since #74889 scheduled canaries run on RunsOn, where the CI image stash goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so every report entry was skipped as "CI image missing". The plan now also accepts that digest artifact; the explain job then runs on a RunsOn runner, downloads exactly that tarball by its digest and checks it like stash-restore does. It prepares the disk with the workflow version's runner scripts, as the image build job now does. * Use the current RunsOn runner label for dependency reports from S3 main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families and spot strategy); the report's copy now matches it again, as the plan test that compares the two requires. * Describe re-used report inputs as saved and refuse parent segments in the S3 stash An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check. Generated-by: Claude Opus 5 --------- (cherry picked from commit ca68614) Co-authored-by: Yossi Eliaz <zozo123@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Detailed upgrade investigations grow with the number of outdated packages and delay canary completion even when test preparation gets faster. Maintainers need prompt test results while retaining reproducible upgrade diagnostics.
Diagnostic reruns must retain their original inputs without gaining cache write access on the default branch. The workflow-run event already provides that boundary and the standard rerun action preserves the source canary.
Keep dependency reports aligned with supported Python versions
Keep dependency reports going when one input is gone
with a warning and a summary line, instead of failing every report.
re-run canary no longer queues a second report from the same inputs.
pinned to the resolved constraints commit; failing to save only warns,
so the canary no longer depends on raw
gh apicalls.skip the plan tests when node is unavailable.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
Explaining constraints-source-providers took 105-120 minutes per job in
the last four main canaries, past the report job's 90-minute timeout.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
CONSTRAINTS_GITHUB_REPOSITORY names the repository holding the
constraints-* branches, apache/airflow by default, and the Dockerfiles
read it. Breeze's download_constraints_file and get_branch_commit always
used apache/airflow instead, so setting it changed nothing for
constraints-version-check, the SBOM and provider dependency downloads,
or the constraints commit saved for the dependency report.
Both functions now read CONSTRAINTS_GITHUB_REPOSITORY and fall back to
apache/airflow when it is unset or empty, so existing callers and CI on
apache/airflow behave as before, and a saved commit always comes from
the repository its constraints were downloaded from. Other files fetched
through download_file_from_github still come from apache/airflow.
Since #74889 scheduled canaries run on RunsOn, where the CI image stash
goes to RunsOn's S3 cache and only its digest is a GitHub artifact, so
every report entry was skipped as "CI image missing". The plan now also
accepts that digest artifact; the explain job then runs on a RunsOn
runner, downloads exactly that tarball by its digest and checks it like
stash-restore does. It prepares the disk with the workflow version's
runner scripts, as the image build job now does.
main changed breeze's RUNS_ON_AMD_RUNNER_TEMPLATE (instance families
and spot strategy); the report's copy now matches it again, as the plan
test that compares the two requires.
An earlier canary attempt only saved the inputs; its report may have failed, so the summary should not call them explained. The stash tarball is extracted with tar -P, so an entry with a .. segment could leave /mnt even under the /mnt prefix check.
Generated-by: Claude Opus 5
(cherry picked from commit ca68614)
Co-authored-by: Yossi Eliaz zozo123@users.noreply.github.com
Co-authored-by: Claude Opus 5.5 noreply@anthropic.com
Co-authored-by: Shahar Epstein 60007259+shahar1@users.noreply.github.com
Co-authored-by: Jarek Potiuk jarek@potiuk.com