Repository navigation
v1.91.302.0 fix: repair three flaky gate cases from the 2026-10-28 flaky pass (index flags, exploratory QA, cookie auth) - #3553
Conversation
… flaky, gate 40619718717) The failing assume-unchanged trial ran git update-index --really-refresh while gathering evidence, which cleared the flag the checker reads. Step 1 now says the read is read-only and names that command; nearby prose tightened so the section stays byte-identical in size (no pin or ledger move). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…s Step 11 voices (late-input 40664336141, plan-checks 40707142223) Gate edit: the caller interface admits the gstack-review-plan forms Step 9 names at the exact installed path; the ship prompt says the plan's adversarial-native and outside-voice lines are Step 11's, outside the Steps 8-9 excerpt. Free replays of both 40707142223 captures and the real plan for all four ship cases. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…ection cases use the product default timeout Gate edit approved by GStack Minder. The shared 400 ms came from a84b0b5 (#2964) as a fixture value no product path uses; --verify-auth callers run under MAX_TIMEOUT_MS 15000. Identity mismatch/missing/ambiguous cases resolve only at the deadline, so they keep the 400 ms bound; explicit timeout tests are unchanged. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… host (test-of-test ratchet) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Its fixture launches bin/gstack-review-log, a bash script, directly; windows-latest reports it not found in PATH (Windows Free Tests run 40727825859, shard 5). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…s own entry" The swap changed an input the s6-docs-2 documentation audit hashed, and /ship allows no third audit in one invocation. The misplaced comment is listed as a review note instead. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
E2E Evals: ✅ PASS (flaky: 2 case(s) passed on their one rerun)125 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $32.48 rule/judge cost | reconcile exit: 0 Profile: pr / mode Derived dependents (scripts/pr-dependencies.ts):
Rule and judge shards
Fail-closed reconciliationSliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry. Flaky: failed once, passed on its one rerun (rule 16)Each failed and passed on its rerun in run 40749421675 at the attempts named above. Logged for the daily flaky-fix pass in this run's Dispatched narrow screen (gstack-ship-queue ready --screen) of merge commit |
…merge base (screen 40737741454 ship-exploratory-unavailable) The review plan dispatches the state-machines specialist, and review-army tells the parent to write the merge-base sha into its diff command. Both screen trials failed on that child's git diff <merge-base sha>. Also: cite the deadline loop beside the identity_* cookie-auth cases (GStack Minder accepted the 400 ms narrowing). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…c detector gaps (2026-10-29 flaky pass, GSTA-162) (#3606) * fix(flaky): qa-callers parents hand children literal commands and paths; Step 9 gate scan joins diff discovery Nine paid ship qa-callers reds after #3553 (late-input, plan-checks, small-cli) had two causes, replayed from their retained captures (2026-10-29 flaky pass, GSTA-162): - six trials: the parent paraphrased its specialist prompt and the child ran the one refused call (git -C <dir> diff, or a cd into the checklist directory); - three late-input trials hit error_max_turns at 25/25 after running Step 9's gate scan only after the receipt, or never. The session prompt now tells the parent the child starts in the product directory, to write each command exactly as it must run and each Read as one literal path; the ship scheduling contract lists Step 9's review-plan command and gate-scan block with the first diff discovery. The detector is unchanged. Seven new replay tests fail with the fixture prompt stashed. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(flaky): docsync detectors accept a read given --help, the cap gate's wrong-path 127 and a full read before a failed separator; only the accepted docs attempt is parsed strictly Replayed from the 2026-10-29 window's ship-docsync reds (GSTA-162): - current 40822033640, completion 40815828320, store 40809051510: the parent joined `cat <candidate> --help 2>&1 | head -N` onto a line of reads; GNU cat printed its usage and read nothing. A declared read command given --help is neutral; its operands are still checked. - stale-after 40749421675: the cap gate typed at the shard root exited 127, then ran at the declared path. The wrong-path 127 excuse now covers the cap gate as well as the candidate. - failure 40828580338: phase.md printed in full, then zsh failed the `echo ======` separator. The read counts when the line's first unpiped cat printed the whole file first. - completion 40805543956: the parent rejected attempt 1 (no documentation_section) and re-audited; the case now parses only the accepted attempt strictly. Four public fixtures, registered as free fixtures. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * CHANGELOG: [Unreleased] entry for the 2026-10-29 flaky pass Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: re-pin the ship-docsync completion/current detector-inventory row to its moved lines Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
What changed
Ship's exploratory-QA fixture now admits Step 9's
gstack-review-plancall at the installed path and nothing else, so both refused captures from paid screen 40707142223 replay clean.Before/after as text: Ship's exploratory-QA fixture now admits Step 9's gstack-review-plan call at the installed path and nothing else; both refused paid-screen captures replay clean
Shared-code reuse step 1 in /review and /ship now says evidence gathering is read-only and names
git update-index --really-refresh, the command that cleared the assume-unchanged flag in gate 40619718717.Before/after as text: Evidence gathering is now read-only and names git update-index --really-refresh, the command that cleared the assume-unchanged flag in gate 40619718717
Why
The 2026-10-28 daily flaky pass (Garry: "we should have a flaky test fix pass every day that tries to fix flaky tests") found three gate cases that failed for reasons we could see in their captures. A reviewer refreshed the index and erased the flag it was checking, ship's exploratory-QA fixture refused the review-plan command that #3527 added to Step 9, and the real-browser cookie-auth checks used a 400 ms limit that no product path uses. Each case now fails only when the product is wrong.
Blast radius
The prompt edit changes the rendered
shared-code-reuse.mdin review/ and ship/ and nothing else. Both renders keep their exact byte size (2342), so no size pin or context ledger moves; the generated sections andgen:skill-docsfreshness check were run. The transport and cookie-auth edits are test-only:reviewPlanCommandAllowedis a non-exported helper intest/helpers/qa-callers-fixture.ts, andbrowse/test/cookie-auth-verification.test.tschanges only the options its real-Chromium describe passes. The Windows curation entry removes a test that is new on this branch, so no test that ran on Windows before stops running there (file:line read,scripts/test-free-shards.ts --windows-only --listran).Approvals
Source: gstack-approvals record for flaky-2026-10-28-s6 (display only; not an approval citation).
Summary
Three flaky gate cases are repaired, each diagnosed from its captured run first:
git update-index --really-refreshwhile gathering evidence, which cleared the assume-unchanged (h) flag the checker reads, so its prerequisites went stale. The passing rerun never ran it. Step 1 of the shared-code reuse section, rendered fromscripts/resolvers/review-scope.ts, now says evidence gathering is read-only and names the command.gstack-review-plan --base main. The qa-callers fixture refused it (paid screen 40707142223, both plan-checks trials) and never said that the plan's adversarial-native and outside-voice lines belong to Step 11, outside the fixture's Steps 8–9 excerpt. In gate 40664336141, late-input reported pass while listing that Step 11 review pass as remaining work. Both cases share this root cause. Once the plan ran, it dispatched the state-machines specialist, and review-army tells the parent to write the merge-base sha into that specialist's diff command. The fixture refused that child'sgit diff <merge-base sha>, which failed both ship-exploratory-unavailable trials in this PR's first screen. The transport now admits the fixture's own merge base, bare or in review-army'sDIFF_BASE=<sha> && git diff "$DIFF_BASE"form, and the prompt declares it.browse/src/cookie-auth-verification.ts.In the first screen, shared-libs-review-revalidation failed once and passed its rerun. That was a flake, not this edit. In the filtered scenario, the actor ran
echo ===ROUTEunder zsh, where=ROUTEis command-path expansion. The call exited 1 beforeretry-worker.tswas read, and the rerun quoted the separator. Both runs read every evidence path as step 1 says, and neither touched the index.A Windows curation entry follows from the first fix:
test/shared-libs-index-flags-readonly.test.tslaunchesbin/gstack-review-logthrough its fixture, which windows-latest cannot run (Windows Free Tests run 40727825859, shard 5).Test Coverage
Two new free test files cover the repairs, and each fails without its fix:
test/qa-caller-review-plan.test.ts: 12 tests. It replays both captures from that paid screen and both ship-exploratory-unavailable captures from screen 40737741454, checks the admitted helper forms with negative controls (pipes, other arguments, another binary, no runtime), and runs the real plan for all four ship cases. Regression proof: 7 tests fail without the review-plan admission, and 3 more fail without the merge-base admission. With both, all 12 tests pass.test/shared-libs-index-flags-readonly.test.ts: 26 tests. It replays the captured refresh against the assume-unchanged fixture with a read-only control, and checks the rendered rule on every host. Regression proof: 2 tests pass and 24 tests fail against main's resolver, 26 tests pass on this branch.Coverage percentages were not recomputed for this body.
Pre-Landing Review
Pre-Landing Review: no blocking issues. Informational findings are listed under Review notes. One specialist pass, a native adversarial pass on the gate and fixture code, and a cross-family outside voice. Three fix-checks covered the later commits. 49f780c was clean, and 60f0780 had one informational item, listed under Review notes. The screen repair (ad7e2ef) went to a reviewer with an adversarial brief, who found no over-admission. That reviewer flagged one informational prompt inconsistency, and dd76b43 fixes it.
REVIEW_LOOP: passes=1 blocking_found=0 deferred=0 noted=0 fixed=0 scope=frozen dispatches=1
Gate edits: 18 listed, 9 read, 3 findings — 3 kept (reasons below), 0 restored, 0 open
test/helpers/qa-callers-fixture.ts. The fixture must admit the review-plan helper that ship Step 9 runs (feat(ship): /ship and /autoplan size their review to the diff (GSTA-131) #3527); admission is limited to the installed path and literal argument forms, with negative controls.test/helpers/qa-callers-fixture.ts. review-army's specialist dispatch writes the merge-base sha into the child's diff command. Only a prefix of the fixture's owngit merge-base origin/main HEADof at least 7 characters is admitted, with negative controls.browse/test/cookie-auth-verification.test.ts. The sharedtimeoutMs: 400came from a84b0b5 (v1.90.0.0 feat: make browser cookie imports explicit and safe #2964) as a fixture value no product path uses (--verify-authcallers run underMAX_TIMEOUT_MS15000). Approved by GStack Minder. The identity_* cases keep 400 ms under GStack Minder's second ruling (see Approvals).Outside review: openai/gpt-6.1-sol (Capy subagent), verdict clean, no findings.
Review notes
--fix-check-promptoperands accept any literal path, including absolute and..paths. The fixture only checks the command shape, and the helper reads the files.--pass $PASS(a shell variable) is refused; only literal numbers are admitted.scripts/lib/windows-curation.ts, the new entry sits between the credential-rejection comment and its entry. A swap was reverted because it would have changed an input the documentation audit had hashed.Exploratory QA
Not run: the diff has no user-facing surface. The repaired cases are themselves exploratory-QA eval fixtures, and their paid runs are under Eval Results.
Design Review
No frontend files changed — design review skipped.
Eval Results
Focused paid runs at 24bf7e5, one trial each, all pass, $5.50 of the $10 lane cap: shared-libs-review-index-flags $1.00, ship-exploratory-late-input $1.05, ship-exploratory-plan-checks $0.86, review-exploratory-small-cli $0.76, ship-exploratory-small-cli $0.90, ship-exploratory-unavailable $0.93. Both the late-input and plan-checks transcripts run
gstack-review-plan --base main. Commits after 24bf7e5 change tests, a curation list and docs only, so the prompts these cases read are byte-identical. These results are reused evidence for the three cases this PR's later commits do not touch.First screen (evals run 40737741454, at 4cd0777): RED. ship-exploratory-unavailable failed both trials on the specialist diff that this PR now admits. shared-libs-review-revalidation was flaky ($1.76 rerun), and the cause is described under Summary. After the repair, a focused run of ship-exploratory-unavailable at dd76b43 passed ($0.82), and its specialist child ran
DIFF_BASE=<sha> && git diff "$DIFF_BASE". A run at ad7e2ef was cancelled at about 65 s for the prompt fix, and its cost was not reported. Lane focused spend: $6.32 plus that partial run, against the $10 cap.Greptile Review
No Greptile comments.
Issues
Refs #2964
Verification Results
Each repair was checked by its failing-first test and its focused paid case: all three repairs PASS; none failed, blocked or not run. Windows Free Tests on the branch are listed under Test plan.
TODOS
No TODO items completed in this PR.
Documentation
Status: current. No documentation edits were needed for ad7e2ef and dd76b43.
Scope: the selected diff from 61e201e to dd76b43, focused on the qa-callers changes since 80955d9 (
test/helpers/qa-callers-fixture.ts, the new fixture, the new tests, a cookie test comment and CHANGELOG). Nothing was staged, unstaged or untracked.Documentation health:
Coverage debt: the caller command interface has no reference doc; the fixture prompt is the only description of its Git forms. This is an existing gap. Diagram drift: none.
Test plan
gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commandsat dd76b43: 19180 tests, 0 failed, $0.07; macOS named regressions and Windows lanes cleared from this PR's Free Tests and Windows Free Tests runs at dd76b43, both greenbrowse/test/cookie-auth-verification.test.tspassing therebun run typecheck0 errors,bun run typecheck:testno new diagnostics; qa-caller-review-plan, qa-exploratory-callers, qa-caller-authority, qa-caller-report-observer, qa-checkpoint-evidence, qa-evidence-producer, free-fixtures, spawnsync tripwire, test-of-test and module-size ratchets, harness-version, cookie-auth-verification all green after the fixture was trackedbun test test/qa-caller-review-plan.test.ts3 tests fail without the merge-base admission and 12 tests pass with it; 7 tests fail without the review-plan admission;bun test test/shared-libs-index-flags-readonly.test.ts24 tests fail against main's resolver, 26 tests pass on the branch🤖 Generated with Claude Code