Repository navigation
v1.94.9.0 feat(pregate): owner-granted flaky exception for named cases in a red run (GST-1096) - #3911
Merged
Merged
Conversation
… run (GST-1096) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…he grant command, recheck red runs (review fixes) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
…-b-flaky-exception Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> # Conflicts: # lib/pregate/index.ts
This was referenced Nov 3, 2026
garrytan
added a commit
that referenced
this pull request
Nov 3, 2026
lib/ship-receipt.ts KEY_ORDER keeps flaky_exception (#3911) and no_stop. bin/gstack-ship-receipt: main's review_exclude help and digest refusal text plus --no-stop help; the J1 re-check block before one planCarry call carrying both stackParent (#3928) and recheck; the recheck risk-class check after the digest check. test/review-packets-removed.test.ts keeps both NO_LOSS_REPLAYS keys (GST-940 and GST-508+GST-464+GST-967+GSTA-258-L4). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
garrytan
added a commit
that referenced
this pull request
Nov 3, 2026
docs/pregate.md keeps the ci-reuse-pr subsection and main's Owner-granted flaky exception section (#3911), in that order. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
gstack-pregate clear now takes --flaky-exception with --pr and --grant, so an owner can let named flaky cases in an untouched test file clear the pregate. (The image render failed in this environment, so the before/after is text only.)
Before/after as text: gstack-pregate clear now takes --flaky-exception with --pr and --grant, so an owner can let named flaky cases in an untouched test file clear the pregate
Why
GSTA-258 design piece 6 (GST-1096): "a timing flake in a test file the PR never touched keeps the pregate red, with no way through but another paid run." The owner can now accept up to three named failing cases from one red Free Tests or Windows Free Tests run for one PR head. Every required CI check still needs its own green run at the head, so the exception never waives CI.
Blast radius
The change adds a new path to
gstack-pregate clear, a newpass-with-exceptionverdict thatverify,readyand the ship receipt read, and aflaky_exceptionreceipt field. Every existing clear path is untouched, because the exception code runs only when--flaky-exceptionis passed.lib/ship-release.ts:373matches/^(pass|warn|requires-remote cleared)/, so finish accepts the new verdict (file:line read). At this head, CI carry accepted Free Tests run 41579867879 at tree 1f243ef4 (34,598 tests), which cleared the remote gate (script ran).Approvals
No approvals recorded for branch gsta-258-b-flaky-exception.
Summary
A red run whose only failures are named flaky cases can now clear a requires-remote pregate obligation once the owner grants it. The grant is a gstack-approvals row whose scope hashes the repository, PR, head, tree, lane, run attempt and case list, so it cannot move to another head or PR. The run's own shard results and logs must name every failure (B8: a red run uploads no tree.json, so the evidence comes from the strict shard artifacts). A case in the PR's diff or import closure, a paid test file or a security test refuses. In the review round, the printed grant command started quoting case names for the shell;
readyand the receipt refuse an exception granted for another PR or head; red runs are read again after the inventory and grant checks; the security-test match is wider; and the clear-only code loads scripts/ and test/ modules lazily, so a bin+lib install still loads the receipt bin.This change never touches the Ubicloud
--receiptgate. That gate's output is text the lane produces itself, so a grant has nothing independent to bind to. Build A's CI carry handles flakes in the remote pregate.Test Coverage
Coverage audit (Step 7, audit only, at the first review tree fe9fc759): 30 new code paths, 17 covered (57% any test, 57% value-weighted), 13 gaps. The fix round then added tests for the biggest gaps: the run provenance refusals (other_repository, fork_run, workflow_mismatch, event_mismatch, run_in_progress, run_stale), a red run that moves after the inventory runs, shell quoting of case names, two more security-test names, and ready refusing an exception bound to another PR.
Tests: 1667 test files before, 1667 after (the new file came in the first commit; the fixes extended it).
Gaps still open: the real
ghArtifactDepsandapprovalGrantsboundaries are always replaced by fakes in tests, so the real GitHub artifact format is unproven; the receipt's other-head refusal has no test of its own.Tests ran: focused 408
Pre-Landing Review
Review depth: full (gate class, security risk). Pass 1 (tree fe9fc759) ran a specialist batch (testing, security, state machines), a red team, the native adversarial reviewer and an outside voice (GPT-6.1 Sol). Together they returned 13 findings. Three were marked blocking by the outside voice: shell injection in the printed grant command, a grant reused across heads with the same tree, and red runs not read again after the inventory. All three are fixed in a96e76d, along with the security-test regex gap, an honest note in the docs on the owner check, and the import-closure limit. Pass 2 (range 549613e..a96e76d, specialist plus red team) found no blocking issues and one INFORMATIONAL testing note.
REVIEW_LOOP: passes=1 blocking_found=3 deferred=0 noted=1 fixed=0 scope=frozen unrecorded=3 ledgers_read=2 fix_checks=2 dispatches=4
Review-loop exception: accepted by GStack Minder 2026-11-03 via standing grant, GSTA-258 builds, 2026-11-03: rule 19 merge of main 5f705f3 into #3911; one scoped fix-check on the merge hunks only (fix-check at 4340ed4, 0 findings)
Gate edits: none detected (GATE_SCAN tagged=0)
Outside review: GPT-6.1 Sol, request changes on pass 1; every finding it raised is fixed.
Deferred findings
readychecks both, and ready is required before the train.newer_runcan lock out both of two runs on one head when an older run is re-run (adversarial Feature request: saving all the reports, or generating some kind of artifact/hand-off file to persist the information between the sessions #6). This is conservative.verifydoes not re-read the grant row, so a row later edited to rejected does not un-clear the obligation.grant_not_owneronly catches rows honestly labeledagent:. The docs now say so; real owner proof needs a source an agent cannot write.Review notes
After the rule 19 merge of main at 5f705f3 (one conflict, the imports in lib/pregate/index.ts, kept both; #3901's carryGate taken as main wrote it), one scoped fix-check read the merge hunks and returned NO FINDINGS. Its review-log row was written twice by a repeated command, so the loop line shows fix_checks=2 for one fix-check.
The REVIEW_LOOP line shows passes=1 and fixed=0 because pass 1 was run without first taking a review-log start token, so its row could not be written honestly. Its briefs, outputs and judgment are in that pass's dispatch ledger. Pass 2's row is bound to tree 42cea903.
Exploratory QA
No browser or UI surface. The behavior is covered by the fixture tests and by the remote pregate at this tree.
Design Review
No frontend files changed — design review skipped.
Eval Results
No prompt-related files changed — evals skipped.
Review bot comments
No review bot comments.
Plan Completion
Plan completion audit: not run (no plan is bound to this branch and no docs/designs/ file matches). The design is GSTA-258's ship-90-90 design, piece 6, with fixes B8 (red runs have no tree.json; evidence comes from the shard artifacts) and B9 (the exception is pregate only, and required CI still needs green). B10 is covered by the head-and-PR binding.
Verification Results
No Step 8.1 obligations were bound.
TODOS
No TODO items completed in this PR.
Documentation
Status: current. No documentation edits were needed.
Scope: base 14d7414 through head a96e76d plus the unstaged changelog.d fragment edit. The 12 selected changed files were bin/gstack-pregate, bin/gstack-ship-receipt, the changelog fragment, docs/pregate.md, lib/headless-artifacts.ts, lib/pregate/flaky-exception.ts, lib/pregate/index.ts, lib/pregate/run-check.ts, lib/run-evidence.ts, lib/ship-queue.ts, lib/ship-receipt.ts and test/pregate-flaky-exception.test.ts.
Documentation health:
pass-with-exceptionverdict, and thereadyand receipt output).Coverage debt:
--flaky-exception,--prand--grantongstack-pregate clear: reference and how-to are in docs/pregate.md (plus the --help text); explanation is the "Why not base_red" subsection. There is no tutorial. Not listed in docs/troubleshooting.md or docs/ship-policy.md, which is a minor gap.flaky_exceptionfield in the ship receipt and theFLAKY_EXCEPTION:READY line: documented in docs/pregate.md only.Test plan