Skip to content

v1.91.322.0 feat: narrow PR screens leave hub files to the train gate - #3587

Merged
capy-ai[bot] merged 6 commits into
mainfrom
garrytan/gsta150-narrow-screen
Oct 29, 2026
Merged

capy-ai[bot] merged 6 commits into
mainfrom
garrytan/gsta150-narrow-screen

Conversation

@garrytan

@garrytan garrytan commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner
{
  "schema_version": 1,
  "pr": 3587,
  "head": "b1dd0dc396d7af508eedcf840c4a8ec9bcd3f69f",
  "version": "1.91.304.0",
  "base": "main",
  "session_kind": "unattended",
  "thread": "GSTA-150",
  "queue_mode": "stamp-at-merge",
  "pregate": "requires-remote cleared @07b0646d7365 (1 checks, current for b1dd0dc396d7)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/40819897397",
    "windows-free-tests.yml/windows-free-shard: https://github.com/garrytan/gstack/actions/runs/40819901625",
    "pregate.json/remote_gate: remote-gate tree=07b0646d7365 tests=13486"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  }
}

What changed

A PR that edits only lib/restamp.ts: its narrow screen no longer runs all 113 gate cases and 24 judges; the train gate's full census runs them

A PR that edits only lib/restamp.ts: its narrow screen no longer runs all 113 gate cases and 24 judges; the train gate's full census runs them.

Before/after as text
-$ EVALS_SCREEN_POLICY=broad paid plan for a PR that changes only lib/restamp.ts
-selected: 113 e2e case(s), 24 judge(s)
-reason: Derived references select every dependent gate case: lib/restamp.ts (255 cases); 142 selected behaviors remain
-        scheduled/release coverage, not PR passes
+$ EVALS_SCREEN_POLICY=narrow paid plan for a PR that changes only lib/restamp.ts
+selected: 0 e2e case(s), 0 judge(s)
+reason: Narrow screen: derived references reach every gate case only through the static closure, so these files select
+        no case here and the train gate's full census runs them: lib/restamp.ts (EVALS_SCREEN_POLICY=broad restores
+        them); Changed-input selection intersected with the fast PR profile; selected judges retained

Why

Garry asked for a cost and latency pass on paid evals without losing coverage (GSTA-150). 19 of the 41 full PR screens between Oct 20 and 29 ran the whole $33 gate only because a changed file such as lib/restamp.ts sits on every case's import path. Every case still runs before merge in the train gate's full census, so a narrow screen now leaves those hub files to it.

Replay on those 41 screens with today's maps (merge-base diffs, current selection): 23 screens get cheaper, total $1,338.72 → $809.45, about $457 a week saved. The screens that caught real PR-caused reds (#3434, #3453) still run the full gate, and #3527's red judge is still selected.

Blast radius

Only narrow PR screens change: ready --screen dispatches (inputs.screen_policy defaulting to narrow for workflow_dispatch) and bun run eval:bg:pr / test:pr. pull_request runs, ready --screen --broad, the train gate, eval:bg:gate, periodic and release runs and the full profile keep their selection; unit tests in test/pr-dependencies.test.ts pin each of those (script ran). Full-gate files, shared runtime and build inputs, paid-runner inputs, unknown dependencies and deleted files still restore the full gate.

Approvals

Date Asked Exact words Outcome Covers Still pending
2026-10-29 Approve the plan "Paid evals: cut duplicate screen spend and price every gate case (GSTA-150)" (autoplan run GSTA-150-20261029033743, 4 decisions) “Garry: Do it” (Garry, relayed by agent:gstack-minder, relayed by agent:gstack-minder) approved the reviewed plan (sha256 b65a8610ce75), every recommendation taken none
2026-10-29 Approve the plan "Paid evals: cut duplicate screen spend and price every gate case (GSTA-150)" (autoplan run GSTA-150-20261029033743, 3 decisions) “Garry: Do it” (Garry, relayed by agent:gstack-minder, relayed by agent:gstack-minder) approved the reviewed plan (sha256 b65a8610ce75), every recommendation taken none

Source: gstack-approvals record for garrytan/gsta150-narrow-screen (display only; not an approval citation).

Summary

  • Narrow screens (scripts/test-pr-profile.ts, scripts/lib/paid-select.ts): a changed file whose derived references reach every gate case and that nothing narrower selects is a hub file; on a narrow screen it selects no case or judge, and the paid plan's reason and the coverage summary name it and say the train gate runs those cases (approved exception D1).
  • Plumbing: EVALS_SCREEN_POLICY (unset means broad); evals.yml plans narrow only for dispatches; test:pr and the READY screen preview (lib/ship-queue-train-ready.ts) use narrow.
  • Mapping gaps: test/size-ledger/** is a touchfile of the 20 cases that read test/helpers/size-ledger.ts, all periodic or marathon, so a size-ledger baseline change no longer restores the full gate and runs no PR-time case. careful/bin/mask-commit-message.awk is free-only (deviation from the plan's careful/bin/** registration: only the free-tested check-careful.sh loads it, and test/helpers/workflow-boundaries-fixture.ts:64 copies only hook-extract.sh and check-freeze.sh into paid fixtures).
  • Gate-integrity edit (screen selection only): no threshold, trial count, sample, judge, case kind or quarantine changes. scripts/test-pr-profile.ts is recorded non-behavioral at HARNESS_VERSION 6.
  • Docs: docs/ship-policy.md, docs/TESTING_INTERNALS.md; CHANGELOG under [Unreleased] (stamped at merge).

Test Coverage

test/pr-dependencies.test.ts (new describe "narrow screens leave hub files to the train gate"): hub-only diff selects nothing narrow and every gate case broad or unset; hub beside shared runtime, an unknown file or a full-gate file stays full; hub beside a direct edit keeps that edit's cases; deleted files are never hubs; the workflow plans narrow only for dispatches; size-ledger baselines select no gate case. Focused: 188 pass, 0 fail.
Coverage audit: 70% value-weighted; proposed, not written: a hub beside an ordinary derived file; the narrow reason omitted when the diff already runs the full gate.
Tests ran: remote free gate (Ubicloud) 13486 tests, 0 failed on tree 07b0646d7365. macOS named regressions and Windows free shard clear from this PR's CI.

Pre-Landing Review

Pass 1 (9 dispatches incl. red team, adversarial and outside voice openai/gpt-6.1-sol): 11 findings, 7 fixed (pull_request runs stay broad, deleted files never hubs, READY and summary previews narrow, gate-reuse integration, tests), 2 noted (eval-bg cap sized from broad; no hub-set ratchet), 1 irrelevant (outside voice: "hub + paid-runner input drops 113→57" is main's existing PR-profile intersection), 1 kept. Pass 2 fix-check: no findings.
Gate edits: 1 listed — kept: RH-gate scripts/test-pr-profile.ts (Garry approved D1 on 2026-10-29, "Do it"; the replay kept every PR-caused red).
Remote pregate caught scripts/test-pr-profile.ts missing its HARNESS_VERSION decision; recorded non-behavioral.

Eval Results

No paid eval run for this PR: it changes selection only. Evidence is the 41-screen replay above (handoffs/GSTA-150/replay.jsonl in the Captain drive).

Plan Completion

PR 1 items: size-ledger and careful registration DONE (careful CHANGED to free-only, above); narrow-screen edge policy DONE (structured hubFiles, not the reason string); tests DONE; replay DONE ($457/week); gate-integrity edit listed here.

garrytan and others added 5 commits October 29, 2026 04:05
…elines and the careful mask awk

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…hubs, READY and summary previews)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Evals: ✅ PASS

[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 112/112 · behavior 1/1 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 6m13s · cost $32.96 known + 32 of 139 trial(s) cost unknown

125 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $32.32 rule/judge cost | reconcile exit: 0

Profile: pr / mode full-fallback; selected behaviors: 113, judges: 24. Deferred to scheduled/release coverage: 143 behaviors and 0 changed prompt files. Deferred checks did not run and receive no PR-pass credit.

Full gate restored by:

  • .github/workflows/evals.yml (full-gate input): .github/workflows/evals.yml is in PR_FULL_GATE_FILES (scripts/test-pr-profile.ts): every paid case consumes it, so the full gate is intended and no mapping narrows it

Derived dependents (scripts/pr-dependencies.ts):

  • lib/ship-queue-train-ready.ts: 255 case(s), 24 judge(s)
  • scripts/harness-version.json: 255 case(s), 24 judge(s)
Rule and judge shards
Shard Automated result Manual/unscored Executed Reused Status Cost
llm-judge/skill-llm-eval 24/24 0 24 0 ✅ $1.44
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-current 1/1 0 1 0 ✅ $0.9
e2e/skill-e2e-coverage-audit 2/2 0 2 0 ✅ $0.29
e2e-deploy/skill-e2e-deploy--setup-deploy-workflow 1/1 0 1 0 ✅ $0.13
e2e-qa-functional-fix/skill-e2e-qa-functional-fix 2/2 0 2 0 ✅ $1.06
e2e/skill-e2e-plan--plan-review-report 1/1 0 1 0 ✅ $1.03
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-unavailable 1/1 0 1 0 ✅ $0.92
e2e-retro/skill-e2e-retro 1/1 0 1 0 ✅ $0.33
e2e/skill-e2e-shared-libs-paths--shared-libs-review-prior-coverage 1/1 0 1 0 ✅ $1
e2e-qa-workflow/skill-e2e-qa-workflow--qa-quick 1/1 0 1 0 ✅ $0.51
e2e-review-attribution/skill-e2e-review-attribution 3/3 0 3 0 ✅ $0.28
e2e/skill-e2e-ship-skip 1/1 0 1 0 ✅ $0.23
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-before 1/1 0 1 0 ✅ $0.43
e2e/skill-e2e-shared-libs--shared-libs-review-lifecycle 1/1 0 1 0 ✅ $0.84
e2e-review-army/skill-e2e-review-army--review-army-migration-safety 1/1 0 1 0 ✅ $0.27
e2e-deploy/skill-e2e-deploy--land-and-deploy-first-run 1/1 0 1 0 ✅ $0.23
e2e/skill-e2e-shared-libs--shared-libs-read-only 1/1 0 1 0 ✅ $0.24
e2e-deploy/skill-e2e-deploy--land-and-deploy-review-gate 1/1 0 1 0 ✅ $0.28
e2e-docsync-spawned/skill-e2e-docsync-spawned 1/1 0 1 0 ✅ $0.44
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-after 1/1 0 1 0 ✅ $0.4
e2e-safety-codex-consult/skill-e2e-safety-codex-consult 1/1 0 1 0 ✅ $0.27
e2e-deploy/skill-e2e-deploy--land-and-deploy-workflow 1/1 0 1 0 ✅ $0.53
e2e/skill-e2e-workflow 4/4 0 4 0 ✅ $0.82
e2e-review-army/skill-e2e-review-army--review-army-perf-n-plus-one 1/1 0 1 0 ✅ $0.2
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-late-result 1/1 0 1 0 ✅ $0.35
e2e/skill-e2e-shared-libs-paths--shared-libs-review-index-flags 1/1 0 1 0 ✅ $0.97
e2e-deploy/skill-e2e-deploy--benchmark-workflow 1/1 0 1 0 ✅ $0.26
e2e/skill-e2e-investigate-owned-completion 1/1 0 1 0 ✅ $0.07
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-small-cli 1/1 0 1 0 ✅ $0.96
e2e-review-army/skill-e2e-review-army--review-army-json-findings 1/1 0 1 0 ✅ $0.07
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-store 1/1 0 1 0 ✅ $0.84
e2e-triage/skill-e2e-triage 1/1 0 1 0 ✅ $0.1
e2e-review-army/skill-e2e-review-army--review-army-delivery-audit 1/1 0 1 0 ✅ $0.14
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-timeout-unsettled 1/1 0 1 0 ✅ $0.2
e2e-design/skill-e2e-design--design-review-detector-shim 1/1 0 1 0 ✅ $0.12
e2e-session-intelligence/skill-e2e-session-intelligence 4/4 0 4 0 ✅ $0.29
e2e-safety-pair-agent/skill-e2e-safety-pair-agent 1/1 0 1 0 ✅ $0.1
e2e-plan-tune/skill-e2e-plan-tune 1/1 0 1 0 ✅ $0.14
e2e-qa-workflow/skill-e2e-qa-workflow--qa-bootstrap 1/1 0 1 0 ✅ $0.08
e2e-deploy/skill-e2e-deploy--canary-workflow 1/1 0 1 0 ✅ $0.11
e2e/skill-e2e-investigate-owned-termination 2/2 0 2 0 ✅ $0.13
e2e/skill-e2e-ship-hook-refresh 1/1 0 1 0 ✅ $0.1
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-launch-failure 1/1 0 1 0 ✅ $0.16
e2e-design/skill-e2e-design--plan-design-review-no-ui-scope 1/1 0 1 0 ✅ $0.08
e2e-third-party-actions/skill-e2e-third-party-actions 5/5 0 5 0 ✅ $0.41
e2e/skill-e2e-ship-hook-consent 2/2 0 2 0 ✅ $0.21
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-recovery 1/1 0 1 0 ✅ $0.38
e2e/skill-e2e-shared-libs--shared-libs-review-revalidation 1/1 0 1 0 ✅ $1.88
e2e-review/skill-e2e-review 2/2 0 2 0 ✅ $0.34
e2e-cso/skill-e2e-cso 1/1 0 1 0 ✅ $0.44
e2e-qa-functional/skill-e2e-qa-functional 2/2 0 2 0 ✅ $0.77
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-late-input 1/1 0 1 0 ✅ $1
e2e-learnings/skill-e2e-learnings 1/1 0 1 0 ✅ $0.07
e2e-failure-routing/skill-e2e-failure-routing 4/4 0 4 0 ✅ $0.36
e2e-qa-workflow/skill-e2e-qa-workflow--qa-only-no-fix 1/1 0 1 0 ✅ $0.52
e2e-qa-callers/skill-e2e-qa-callers--review-exploratory-small-cli 1/1 0 1 0 ✅ $0.78
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-plan-checks 1/1 0 1 0 ✅ $0.92
e2e/skill-e2e-test-value 3/3 0 3 0 ✅ $0.69
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-failure 1/1 0 1 0 ✅ $0.53
e2e-skillify/skill-e2e-skillify 3/3 0 3 0 ✅ $0.78
e2e/skill-e2e-shared-libs-paths--shared-libs-review-path-eligibility 1/1 0 1 0 ✅ $1.5
e2e-hermetic/skill-e2e-hermetic-canary 2/2 0 2 0 ✅ $0.02
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-completion 1/1 0 1 0 ✅ $0.9
e2e-design/skill-e2e-design--design-review-plugin-handoff 1/1 0 1 0 ✅ $0.12
e2e-review-army/skill-e2e-review-army--review-army-quality-score 1/1 0 1 0 ✅ $0.11
e2e-design/skill-e2e-design--design-review-detector-shim-dom 1/1 0 1 0 ✅ $0.22
e2e-safety-ship-evidence/skill-e2e-safety-ship-evidence 1/1 0 1 0 ✅ $0.13
e2e-browse/skill-e2e-bws 6/6 0 6 0 ✅ $0.29
e2e/skill-e2e-shared-libs--shared-libs-unsupported-git 1/1 0 1 0 ✅ $0.17
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-asset 1/1 0 1 0 ✅ $0.27
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-marker 1/1 0 1 0 ✅ $0.17
Fail-closed reconciliation
timeout-unsettled
  slice 16  passed             32s  test/skill-e2e-ask-user-question-format-compliance.test.ts
  slice 16  passed             25s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-launch-failure
  slice 16  passed             29s  test/skill-e2e-session-intelligence.test.ts
  slice 16  passed             34s  test/skill-e2e-ship-hook-consent.test.ts
  slice 16  passed             23s  test/skill-e2e-investigate-owned-termination.test.ts
  slice 16  passed             19s  test/skill-e2e-design.test.ts#design-review-detector-shim
  slice 16  passed             25s  test/skill-e2e-deploy.test.ts#canary-workflow
  slice 16  passed             29s  test/skill-e2e-ship-hook-refresh.test.ts
  slice 16  passed             15s  test/skill-e2e-design.test.ts#plan-design-review-no-ui-scope
  slice 16  passed             27s  test/skill-e2e-review-army.test.ts#review-army-delivery-audit
  slice 16  passed             23s  test/skill-e2e-safety-pair-agent.test.ts
  slice 16  passed             20s  test/skill-e2e-qa-workflow.test.ts#qa-bootstrap
  slice 17  passed             10s  test/skill-e2e-learnings.test.ts
  slice 17  passed             12s  test/skill-e2e-diagram.test.ts
[test:paid] collector final outcomes: 125 executed, 0 reused; 125 passed, 0 failed, 0 manual accepted (unscored; no score-cache credit) (125 attempt records from 71 collectors; every record counts)
[test:paid] deselected testcases, zero credit (316):
[test:paid]    302  sibling case of a case shard (its own shard runs it)
[test:paid]      8  periodic-tier case (this lane runs gate)
[test:paid]      6  Bun hook placeholder of a skipped describe block (not a test)
[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 112/112 · behavior 1/1 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 6m13s · cost $32.96 known + 32 of 139 trial(s) cost unknown
[test:paid] exercised-path map (shadow, non-blocking): 11 MAP_GAP line(s) over 106 executed case(s) with recorded commands
  MAP_GAP document-release bin/gstack-banned-terms: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP plan-review-report bin/gstack-plan-reality: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP plan-review-report browse/dist/browse: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP plan-review-report design/dist/design: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP shared-libs-review-revalidation bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP ship-exploratory-late-input bin/gstack-review-plan: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP ship-exploratory-plan-checks bin/gstack-review-plan: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP ship-exploratory-small-cli bin/gstack-review-plan: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP ship-exploratory-unavailable bin/gstack-review-plan: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP test-audit-report-only design/dist/design: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP timeline-event-flow bin/gstack-skill-end: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
[test:paid] report: every planned shard accounted and passed

Sliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry.

Dispatched narrow screen (gstack-ship-queue ready --screen) of merge commit 62cdf5aaad4a379b9756993f34633dbb7a2aa23d.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@github-actions github-actions Bot changed the title feat: narrow PR screens leave hub files to the train gate v1.91.322.0 feat: narrow PR screens leave hub files to the train gate Oct 29, 2026
@capy-ai
capy-ai Bot merged commit a8d60be into main Oct 29, 2026
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant