Repository navigation
v1.91.322.0 feat: narrow PR screens leave hub files to the train gate - #3587
Conversation
…elines and the careful mask awk Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…hubs, READY and summary previews) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
E2E Evals: ✅ PASS125 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $32.32 rule/judge cost | reconcile exit: 0 Profile: pr / mode Full gate restored by:
Derived dependents (scripts/pr-dependencies.ts):
Rule and judge shards
Fail-closed reconciliationSliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry. Dispatched narrow screen (gstack-ship-queue ready --screen) of merge commit |
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
What changed
A PR that edits only
lib/restamp.ts: its narrow screen no longer runs all 113 gate cases and 24 judges; the train gate's full census runs them.Before/after as text
Why
Garry asked for a cost and latency pass on paid evals without losing coverage (GSTA-150). 19 of the 41 full PR screens between Oct 20 and 29 ran the whole $33 gate only because a changed file such as
lib/restamp.tssits on every case's import path. Every case still runs before merge in the train gate's full census, so a narrow screen now leaves those hub files to it.Replay on those 41 screens with today's maps (merge-base diffs, current selection): 23 screens get cheaper, total $1,338.72 → $809.45, about $457 a week saved. The screens that caught real PR-caused reds (#3434, #3453) still run the full gate, and #3527's red judge is still selected.
Blast radius
Only narrow PR screens change:
ready --screendispatches (inputs.screen_policydefaulting to narrow forworkflow_dispatch) andbun run eval:bg:pr/test:pr.pull_requestruns,ready --screen --broad, the train gate,eval:bg:gate, periodic and release runs and the full profile keep their selection; unit tests intest/pr-dependencies.test.tspin each of those (script ran). Full-gate files, shared runtime and build inputs, paid-runner inputs, unknown dependencies and deleted files still restore the full gate.Approvals
Source: gstack-approvals record for garrytan/gsta150-narrow-screen (display only; not an approval citation).
Summary
scripts/test-pr-profile.ts,scripts/lib/paid-select.ts): a changed file whose derived references reach every gate case and that nothing narrower selects is a hub file; on a narrow screen it selects no case or judge, and the paid plan's reason and the coverage summary name it and say the train gate runs those cases (approved exception D1).EVALS_SCREEN_POLICY(unset means broad);evals.ymlplans narrow only for dispatches;test:prand the READY screen preview (lib/ship-queue-train-ready.ts) use narrow.test/size-ledger/**is a touchfile of the 20 cases that readtest/helpers/size-ledger.ts, all periodic or marathon, so a size-ledger baseline change no longer restores the full gate and runs no PR-time case.careful/bin/mask-commit-message.awkis free-only (deviation from the plan'scareful/bin/**registration: only the free-testedcheck-careful.shloads it, andtest/helpers/workflow-boundaries-fixture.ts:64copies onlyhook-extract.shandcheck-freeze.shinto paid fixtures).scripts/test-pr-profile.tsis recorded non-behavioral at HARNESS_VERSION 6.docs/ship-policy.md,docs/TESTING_INTERNALS.md; CHANGELOG under [Unreleased] (stamped at merge).Test Coverage
test/pr-dependencies.test.ts(new describe "narrow screens leave hub files to the train gate"): hub-only diff selects nothing narrow and every gate case broad or unset; hub beside shared runtime, an unknown file or a full-gate file stays full; hub beside a direct edit keeps that edit's cases; deleted files are never hubs; the workflow plans narrow only for dispatches; size-ledger baselines select no gate case. Focused: 188 pass, 0 fail.Coverage audit: 70% value-weighted; proposed, not written: a hub beside an ordinary derived file; the narrow reason omitted when the diff already runs the full gate.
Tests ran: remote free gate (Ubicloud) 13486 tests, 0 failed on tree 07b0646d7365. macOS named regressions and Windows free shard clear from this PR's CI.
Pre-Landing Review
Pass 1 (9 dispatches incl. red team, adversarial and outside voice openai/gpt-6.1-sol): 11 findings, 7 fixed (pull_request runs stay broad, deleted files never hubs, READY and summary previews narrow, gate-reuse integration, tests), 2 noted (eval-bg cap sized from broad; no hub-set ratchet), 1 irrelevant (outside voice: "hub + paid-runner input drops 113→57" is main's existing PR-profile intersection), 1 kept. Pass 2 fix-check: no findings.
Gate edits: 1 listed — kept: RH-gate
scripts/test-pr-profile.ts(Garry approved D1 on 2026-10-29, "Do it"; the replay kept every PR-caused red).Remote pregate caught
scripts/test-pr-profile.tsmissing its HARNESS_VERSION decision; recorded non-behavioral.Eval Results
No paid eval run for this PR: it changes selection only. Evidence is the 41-screen replay above (
handoffs/GSTA-150/replay.jsonlin the Captain drive).Plan Completion
PR 1 items: size-ledger and careful registration DONE (careful CHANGED to free-only, above); narrow-screen edge policy DONE (structured
hubFiles, not the reason string); tests DONE; replay DONE ($457/week); gate-integrity edit listed here.