Skip to content

Stop gstack-pr-wait, the land gate and the release cut from calling a head green while a check on it failed (GST-2252) - #4257

Merged
capy-ai[bot] merged 5 commits into
mainfrom
pr-wait-and-ci-gate-read-every-check-attempt
Nov 17, 2026
Merged

capy-ai[bot] merged 5 commits into
mainfrom
pr-wait-and-ci-gate-read-every-check-attempt

Conversation

@garrytan

@garrytan garrytan commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

gstack-pr-wait, the land gate (gstack-ci-gate) and the stable release cut no longer call a commit green while a check on it failed, because they now see a failed run hidden behind a jobless rerun the way the merge queue does.
This serves the merge-check goal: no gate in gstack passes a head the merge queue would refuse.

What changed and why

Summary

gstack-evals#303 went ready with its checks job red, because GitHub's rollup (gh pr checks) and its default check-runs list only show each check suite's latest attempt. A rerun that creates no jobs hides the failed run before it, and a workflow run that fails before creating a job never appears as a check run. #4235 fixed this for the merge queue (ciAtHead). This PR fixes the same blind spot in gstack-pr-wait (lib/pr-wait-github.ts fetchChecks), gstack-ci-gate (lib/ci-gate.ts) and the release cut (crew/scripts/release-cut.ts gateVerdict). All three now also read CI at the commit through ciAtHead, via a small adapter, lib/ci-head-checks.ts, and keep the worse answer per check name. A rerun that passed still reads green, and an unreadable read is an error, never a pass. The release cut sets its own "Cut the release" run and the never-gating workflows aside by run id, so it no longer waits on itself.

Blast radius

The change touches the three readers above and their tests. It is safe because each reader only adds rows and keeps the worst bucket per name, so no check that gated before stops gating. Three probes run against b86d26d confirm that an unreadable read is never green (script ran). pr-wait and the cut make more GitHub REST reads per poll, which is noted below as deferred.

Tests, review and docs

Test Coverage

Coverage: 1 of 5 changed runtime files have no changed test naming them (docs/ship-policy.md). Step 7's test-writing reader was not run, because this head was shipped read-only.
Tests ran: free 1672 (files), carried from CI run 44048079219 at tree a562f654d396.

Pre-Landing Review

No blocking findings. One read-only full review ran at b86d26d (specialists, red-team, adversarial, a second-model review on openai/gpt-6.1-sol and the targeted gate reader).
Second-model review: CLEAN, no findings.
Gate edits: 1 listed, 1 read, 0 findings — 0 kept, 0 restored, 0 open

Deferred findings

All below the blocking floor. Each fails closed and none gives a false green. They stay on this PR.

  • crew/scripts/release-cut.ts:166 (testing): gateVerdict's branch that fails closed on an unreadable every-attempt read has no test. A probe confirms it returns "cannot read its check runs".
  • lib/pr-wait-github.ts:195 (testing): fetchChecks' transient GhError on an unknown read has no test. A probe confirms transient is true.
  • lib/pr-wait-github.ts:195 (red-team, and adversarial Fix project-local /browse installs and harden setup #4 separately): permanent errors such as more than 1000 entries are retried as transient, so pr-wait spends about 10 minutes before it stops. A probe reproduces it.
  • lib/ci-gate.ts:191 (adversarial): ci-gate returns ERROR on a short read inside its --wait window instead of retrying, and its FIX line points at gh auth.
  • lib/pr-wait-github.ts:193 (adversarial): about 6 list reads per poll instead of 2.
  • crew/scripts/release-cut.ts:164 (adversarial): the cut spends more of the workflow token's hourly limit per commit, which could delay stable.
  • crew/scripts/release-cut.ts:169 (adversarial): a third-party commit status stuck at pending holds the cut. The changelog says this is intended.

Exploratory QA

Three probes run against b86d26d (scratch, not committed), 3 passed: the release cut fails closed on an unreadable read; pr-wait throws a transient GhError on an incomplete read; an over-cap workflow-run list also throws a transient GhError.

Documentation

Status: blocked. Scope: release diff be73938..b86d26d: docs/releases.md, docs/ship-policy.md, docs/troubleshooting.md and the changelog fragment (docs roots docs/, README.md, AGENTS.md, CLAUDE.md, CONTRIBUTING.md)

Documentation exception: accepted by GStack Minder (GSTA-36), Captain 2026-11-17 via Captain reply relayed by the GSTA-487 lane to the /ship subagent, answering ask A (A1): I accept the named docs gap on #4257 at b86d26d. docs/releases.md:44-50's Green definition doesn't yet say that the release cut also treats a failed or pending commit status as red or still running. The code and the changelog fragment are correct, and a docs-only follow-up fixes it after merge.

Status: blocked. One factual correction is needed in docs/releases.md, and this audit was read-only.

Scope: Release diff be73938..b86d26d on pr-wait-and-ci-gate-read-every-check-attempt (15 files: lib/ci-gate.ts, lib/ci-head-checks.ts, lib/pr-wait-github.ts, crew/scripts/release-cut.ts, tests, a changelog fragment and 3 docs). Nothing was staged, unstaged or untracked.

Documentation health:

  • docs/releases.md: needs a fix. The every-attempt paragraph is accurate, but the Green definition doesn't say that a failed commit status now also blocks a cut.
  • docs/ship-policy.md: current. It names lib/ci-head-checks.ts as the shared helper for gstack-pr-wait, gstack-ci-gate and the release cut.
  • docs/troubleshooting.md: current. It documents the every-attempt read and the check failed: <workflow> (run <id>, no jobs) line.
  • changelog.d/pr-wait-and-ci-gate-read-every-check-attempt.md: current. It was reviewed only, since the changelog isn't edited here.

Coverage debt: Each changed behavior is described in a reference doc (ship-policy, troubleshooting, releases). There are no how-to or tutorial pages, which is low priority for internal gate tools.

Diagram drift: none. No diagrams cover these modules.

Agent notes

Refs GST-2252

Issues

Refs #3488
Refs #3489
Refs #3890
Refs #4145
Refs #4155
Refs #4157
Refs #4159
Refs #4162
Refs #4167
Refs #4170
Refs #4171
Refs #4173
Refs #4174
Refs #4176
Refs #4177
Refs #4178
Refs #4181
Refs #4184
Refs #4185
Refs #4186
Refs #4187
Refs #4188
Refs #4192
Refs #4195
Refs #4198
Refs #4199
Refs #4200
Refs #4201
Refs #4202
Refs #4203
Refs #4204
Refs #4205
Refs #4208
Refs #4209
Refs #4210
Refs #4212
Refs #4213
Refs #4218
Refs #4219
Refs #4220
Refs #4223
Refs #4224
Refs #4225
Refs #4226
Refs #4228
Refs #4229
Refs #4235
Refs #4237
Refs #4240
Refs #4242
Refs #4244
Refs #4245
Refs #4246
Refs #4297

Approvals

Approved: yes (record r-e5cd0e101e10). The full approval record stays private.
Plan: exempt: bug fix

REVIEW_LOOP: passes=1 blocking_found=0 deferred=8 noted=0 fixed=0 scope=frozen ledgers_read=1 dispatches=4
Gate edits: 1 listed, 1 read, 0 findings — 0 kept, 0 restored, 0 open

{
  "schema_version": 1,
  "pr": 4257,
  "head": "b86d26d2003f14eed24b6226b7323aebc854e4ab",
  "version": "1.99.5.0",
  "base": "main",
  "gate": {
    "free": {
      "pass": 1672,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "free",
      "run": 44048079219,
      "attempt": 1,
      "tree": "a562f654d3965f427b9b32a9d83ed04fc1f1eb9e"
    }
  ],
  "session_kind": "unattended",
  "head_tree": "a562f654d3965f427b9b32a9d83ed04fc1f1eb9e",
  "product_patch_id": "f50dd4ec73bc8838910999626fd1055c0e462eba",
  "gstack_version": "1.99.5.0",
  "review_depth": {
    "depth": "full",
    "class": "gate",
    "runtime_lines": 134,
    "rules": "66e5174161eecc2c",
    "base": "be7393888f91c2820857cdd49a43e249ca019882",
    "registry": "be7393888f91c2820857cdd49a43e249ca019882",
    "gate_scan": {
      "tagged": 1,
      "head": "b86d26d2003f14eed24b6226b7323aebc854e4ab"
    },
    "reasons": [
      "gate-class",
      "runtime=134"
    ],
    "risk": [],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @a562f654d396 (8 checks, current for b86d26d2003f)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/44048079219/attempts/1",
    "windows-smoke.yml/windows-smoke: https://github.com/garrytan/gstack/actions/runs/44048079142/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=a562f654d396 tests=37345 skipped=73 runs=44048079219/1 profiles=ci-reuse/free-tests heads=b86d26d2003f verified=2026-11-17T09:55:54.565Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [
      3488,
      3489,
      3890,
      4145,
      4155,
      4157,
      4159,
      4162,
      4167,
      4170,
      4171,
      4173,
      4174,
      4176,
      4177,
      4178,
      4181,
      4184,
      4185,
      4186,
      4187,
      4188,
      4192,
      4195,
      4198,
      4199,
      4200,
      4201,
      4202,
      4203,
      4204,
      4205,
      4208,
      4209,
      4210,
      4212,
      4213,
      4218,
      4219,
      4220,
      4223,
      4224,
      4225,
      4226,
      4228,
      4229,
      4235,
      4237,
      4240,
      4242,
      4244,
      4245,
      4246,
      4297
    ],
    "flakes": []
  },
  "docs": {
    "status": "blocked",
    "scope": "release diff be73938..b86d26d: docs/releases.md, docs/ship-policy.md, docs/troubleshooting.md and the changelog fragment (docs roots docs/, README.md, AGENTS.md, CLAUDE.md, CONTRIBUTING.md)",
    "exception": {
      "accepted_by": "GStack Minder (GSTA-36), Captain",
      "date": "2026-11-17",
      "via": "Captain reply relayed by the GSTA-487 lane to the /ship subagent, answering ask A (A1)",
      "reason": "I accept the named docs gap on #4257 at b86d26d2003f14eed24b6226b7323aebc854e4ab. docs/releases.md:44-50's Green definition doesn't yet say that the release cut also treats a failed or pending commit status as red or still running. The code and the changelog fragment are correct, and a docs-only follow-up fixes it after merge."
    }
  },
  "review_loop": {
    "passes": 1,
    "blocking_found": 0,
    "deferred": 8,
    "noted": 0,
    "fixed": 0,
    "scope": "frozen",
    "rounds": 1,
    "ledgers_read": 1,
    "dispatches": {
      "specialist": 3,
      "red-team": 1
    }
  },
  "gate_dispositions": [],
  "review_targeted": {
    "classes": [
      "gate"
    ],
    "rules": "8811fd300bf305f0",
    "brief_sha": "7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5",
    "product_patch_id": "f50dd4ec73bc8838910999626fd1055c0e462eba",
    "status": "done",
    "findings": 0,
    "raised_by": "targeted"
  }
}

…d check hidden behind an empty rerun (GST-2252)

GitHub rollup and default check-runs list hold only each check suite latest attempt, so a rerun attempt with no jobs hid the failed run before it, and a workflow run that failed before any job never appeared. All three readers now add CI at the commit as the merge queue reads it (ciAtHead via lib/ci-head-checks.ts) and keep the worse bucket per check. Tests reuse the gstack-evals#303 recording from #4235.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 2 commits November 9, 2026 22:58
…ass its named checks (GST-2252)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…atuses (GST-2252)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
garrytan and others added 2 commits November 17, 2026 08:35
…ead-every-check-attempt

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… read too (GST-2252)

After #4297 the cut sets aside check runs from its own workflow and the Windows nightly by job id, but the rows from lib/ci-head-checks.ts skipped that filter, so the cut waited on its own run. headCheckRows now takes the set-aside check run and workflow run ids and removes them from every list before ciAtHead reads it.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai
capy-ai Bot merged commit 83cc10a into main Nov 17, 2026
59 checks passed
capy-ai Bot added a commit that referenced this pull request Nov 17, 2026
…uses (GST-2252) (#4436)

* Say in the release docs that the stable cut also waits on commit statuses (GST-2252)

The release cut skips a commit whose commit status failed and waits on one still pending, since #4257. The Green definition in docs/releases.md only named check runs.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Add the changelog entry for the release docs' commit-status sentence (GST-2252)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
capy-ai Bot added a commit that referenced this pull request Nov 17, 2026
- /autoplan's approval page no longer hides a reviewer's product objection behind "keep your direction". (#4431)
- Every skill that loads the shared preamble, on every host, now says to ask a question once and to check a file before handing it over. (#4383)
- The Captain manuals now name the `Failed or skipped:` and `Also changed:` lines that READY and lane reports carry. (#4432)
- gstack's shell locks hold again on Ubuntu 26.04, where `mkdir` reports success when another process won the race (GST-2247). (#4262)
- `test/gstack-lock-race.test.ts` runs the old pattern and the helper against a PATH shim that behaves like uutils 0.8.0 `mkdir`, so it runs on any runner image: the old pattern lets two of four racers win, the helper admits exactly one in each of 60 races and never overlaps across take, hold and release loops, and parallel `gstack-config` writers keep every key. (#4262)
- Windows Setup E2E now runs when `bin/gstack-lock.sh` changes. (#4433)
- Claude Code's skill list now says when to use each gstack skill, so Opus loads /review and /plan-eng-review when a request fits. (#4387)
- A pull request's merge head can no longer keep its `/ship` receipt without a full review. (#4365)
- `/ship` now counts review rounds per pull request, and the head that merges always gets a full review. (#4360)
- /ship's documentation audit no longer accepts a passing result when an earlier line from the same audit said "blocked". (#4434)
- /ship's documentation audit no longer spends an attempt when the audit child writes text after its result. (#4285)
- The README lists the two new guides, and the install doc counts its hooks right. (#4403)
- docs/releases.md now says the stable release cut treats a failed commit status as red and a pending one as still going, as it has since #4257. (#4436)
- /ship's `finish --docs-requested` now accepts a docs fix when the audit named the file with a line number. (#4437)
- Document the `.gstack/approval-classes.json` schema, `gstack-approval-classes` CLI commands, and owner approval citation format in `docs/ship-policy.md#approval-classes`. (#4355)
- `/eval-plan` now runs four result checks instead of leaving them to judgment. (#4384)
- The /review coverage-audit eval no longer fails a run that really read the source file when zsh stops a command line at an `echo ====` separator. (#4364)
- Sixteen in-repo documentation links reach their section again. (#4394)
- gstack agents now open every status report with what failed, what they didn't verify and what else they changed. (#4273)
- Two new short guides. (#4380)
- A skill you wrote whose contract still lists `check_after_last_edit` keeps the rest of its contract. (#4439)
- The ship policy guide now explains how to merge any pull request with `gstack-ship-queue land`, even in a repository with one maintainer. (#4377)
- /ship's test step runs a large unit suite on Linux. (#4426)
- Every skill that loads the shared preamble, on every host, now says never to print a whole environment, git config or process table. (#4382)
- Pull requests from /ship now open with at most three plain lines, and every merge lands as one commit titled with the PR title. (#4438)
- `gstack-pr-wait`, `gstack-ci-gate` and the release cut no longer call a commit green while a check on it failed. (#4257)
- `gstack-runaway` now says what a thread's cold and empty wakes cost, not just how many there were (GST-2770). (#4385)
- The optional Stop hook no longer checks for a test run after the last edit, and /verify asks for it in plain words instead. (#4428)
- /ship's test step says when it ran out of time, and you can set its budget. (#4429)
- The remote test command no longer puts every file on one command line. (#4429)
- The stable release no longer waits on the nightly Windows tests (GST-2815). (#4400)
- Ubicloud fan-outs can no longer overshoot the vCPU limit, and owners no longer delete each other's VMs. (#4430)
- hono is pinned to 4.13.16, which fixes five advisories the weekly OSV scan found in 4.13.13 (gstack#4406). (#4408)
- `package.json` overrides (#4408)
- `test/dependency-security.test.ts`: the lockfile floor check now covers `hono >= 4.13.16`. (#4408)
- The nightly Windows test run finishes again. (#4409)
- A short write-up checklist in ETHOS.md. (#4381)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant