fix(ci): retry dockhand build-skill on transient GHCR 5xx#517
Merged
Conversation
When a push to main touches N skill specs, the `build-skill-artifacts` matrix schedules N concurrent first-time pushes to GHCR, each creating a brand-new package. GHCR intermittently fails the manifest PUT with `response status code 500: unknown: unknown error`, and the failure rate scales with matrix width (observed ~16/17 failures on an 18-skill batch, ~2/3 on smaller ones). Re-running the same jobs later succeeds because by then the packages exist. Wrap the dockhand invocation in a bounded retry loop (3 attempts, 10s/20s linear backoff). The backoff is short on purpose: a successful retry typically lands on the first extra attempt, and matrix jobs have a 30-minute timeout so the total ceiling is well under budget. Real (non-transient) build or validation failures still fail the step — they just cost ~30s extra before surfacing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
🛡️ Skill Security Scan Results |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Concurrent first-time pushes to brand-new GHCR packages regularly fail with
response status code 500: unknown: unknown erroron the manifest PUT. We observed this clearly once PR #516 unmasked the dockhand error:workflow_dispatchonce the package shells existed.Wrap the
dockhand build-skillinvocation in a bounded retry loop (3 attempts, 10s then 20s backoff). A successful retry typically lands on the first extra attempt, and matrix jobs have a 30-minute timeout — worst-case added latency is well under budget. Genuine build/validation errors still fail; they just cost ~30s before surfacing.Behavior
teeto the job log (same visibility as PR fix(ci): surface dockhand errors in Build Skill Artifacts step #516).|| true; downstream steps already gate ondigest != ''.Test plan
🤖 Generated with Claude Code