Repository navigation
v1.91.255.0 fix(ship-queue): fill trains at 3 READY PRs or one hour, and price every case shard's trial - #3478
Merged
Merged
Conversation
…ery case shard's trial
The 5-PR / 3-hour fill left READY PRs waiting most of a day; .gstack/ship-policy.json
now fills at max_size 3 or fill_hours 1, whichever comes first (fillDue already
dispatches at either bound). train verdict printed cost=$24.41 for a train whose slice
report said cost $32.92 known + 54 of 136 trial(s) cost unknown: E2E records named by a
display label ("/benchmark workflow") were never attributed to their case id, so the
per-case sum was short. A case shard's records now belong to its one case; a multi-case
shard with display-labelled records keeps those trials unknown. The ATTEMPT: line names
how many cases have no cost instead of printing a partial sum as the total.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
…ue-train-cost.test.ts off the Windows subset scripts/lib/paid-report.ts is a pinned shared-harness file, so test/harness-version.test.ts wants a HARNESS_VERSION decision for its edit: non-behavioral, because a case shard's records are attributed to its one case for the trial's cost, class and error fields while no case's outcome, panel verdict or selection changes. test/ship-queue-train-cost.test.ts drives bin/gstack-ship-queue against the #!/bin/sh gh shim, which windows-latest cannot run (windows-free-shard 2 on #3478 at d5e6a7d), so it joins test/ship-queue-train.test.ts in the Windows exclusion list. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… line; say so in the CHANGELOG The widened record filter also feeds failedRecord, so a failed display-labelled trial in a case shard now takes failure_class, exit_reason and error from its record (train-gate classes a red infra record INFRA instead of VERDICT). The CHANGELOG entry said only the cost fields changed; it now says both. A train priced at zero prints cost=$0.00. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
The non-behavioral note now names the one verdict-side effect of attributing a case shard's records to its case: a red whose own record says infra or timeout is classed INFRA instead of VERDICT, which changes rerun and bisect eligibility and not a pass rate. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…cost line; put the undercounted train costs on record /document-release (ship-owned audit) added the attribution rule and the ATTEMPT: cost-line form to docs/TESTING_INTERNALS.md, docs/troubleshooting.md#cost-unknown and docs/evals/census-red.md. The CHANGELOG headline now says every train cost the coordinator reported until this fix was the undercounted verdict figure (about $24 per train against about $33 real). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…some but not every case is priced The second /document-release audit tightened docs/TESTING_INTERNALS.md and docs/troubleshooting.md#cost-unknown: a train record with no priced case prints no cost at all (classifyGate's cost is null). The CHANGELOG bullet says the same. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This was referenced Oct 26, 2026
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
garrytan
added a commit
that referenced
this pull request
Oct 26, 2026
Rule 19 grant (train 15 QUEUE_TRAIN_CONFLICT). CHANGELOG additive. test/fixtures/context-budget.json by hand: canary keeps the hoisted ceiling (main did not change it; the conflict was main's adjacent new captain row); captain (new on main, now hoisted) and five loaded-skill ceilings lowered to a fresh capture (captain 12011 -> 6551, autoplan, office-hours, plan-ceo-review, plan-devex-review, plan-eng-review by 2-22). No ceiling raised. Generated files and ship goldens regenerated. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Trains fill at 3 READY PRs or one hour (was 5 PRs or three hours), and the train verdict's ATTEMPT: line names the cases it could not price instead of printing a partial sum as the run's cost.
Before/after as text: Trains fill at 3 READY PRs or one hour (was 5 PRs or three hours), and the train verdict's ATTEMPT: line names the cases it could not price instead of printing a partial sum as the run's cost
Trains fill at 3 READY PRs or one hour after the first READY, whichever comes first.
.gstack/ship-policy.jsonmovestrain.max_size5 → 3 andtrain.fill_hours3 → 1.fillDuealready dispatches at either bound, so this is the live block plus a pin:test/gstack-ship-policy.test.tsasserts the values and exercisesfillDuewith the live block at 59 and 60 minutes and at 3 PRs. A fourth READY PR isHELD:for the next train.The train verdict's cost agrees with the slice report, and says what it could not price.
train verdictprintedcost=$24.41for train76db1d1f10c4while the same run's slice report saidcost $32.92 known + 54 of 136 trial(s) cost unknown. The slice report was right.scripts/lib/paid-report.tsmatched result records to a case only byname === idorcase_id; E2E records are named by a display label (/benchmark workflow) while the JUnit case is the id (benchmark-workflow), so 54 trials gotcost_usd: 0, cost_known: falseand the per-case sum was short by $8.51. Now a shard that ran exactly one case owns its records whatever they are named (a failed record'sfailure_class,exit_reasonanderrorcome from that record too); a multi-case shard with display-labelled records keeps those trials cost-unknown rather than guessing.lib/ship-queue-train-verdict.ts'sATTEMPT:line printscost=$<known> known + <n> of <m> case(s) cost unknownwhen some but not every case is priced, the barecost=$<sum>only when every case is priced, and no cost at all when none is./ship additions on top of the fix: a non-behavioral
HARNESS_VERSIONdecision forscripts/lib/paid-report.ts(no case's pass or fail changes, only a trial record's cost and failure fields), andtest/ship-queue-train-cost.test.tsjoinstest/ship-queue-train.test.tsinscripts/lib/windows-curation.ts's exclusion list because windows-latest cannot run the#!/bin/shgh shim it drives.Why
The merge-train coordinator reads
train verdictto report spend and left READY PRs waiting on a 5-PR fill that rarely completed inside three hours; on 2026-10-26 both numbers it read were misleading in operation. Every train cost the coordinator reported until this fix was that undercounted verdict figure (about $24 per train against about $33 real). After this change the coordinator sees a train dispatch within an hour of the first READY and a cost line that names what it could not price.Approvals
Source: gstack-approvals record for train-fill-and-cost (display only; not an approval citation).
Summary
.gstack/ship-policy.jsontrain block 3 PRs / 1 hour, pinned bytest/gstack-ship-policy.test.ts; docs/ship-policy.md's fill paragraph and example block updated.scripts/lib/paid-report.tsattributes a single-case shard's records to that case regardless of their display name;lib/ship-queue-train-verdict.tsprints the known + unknown cost form. Pinned bytest/paid-report-census.test.tsandtest/ship-queue-train-cost.test.ts.scripts/harness-version.jsonnon-behavioral decision for paid-report.ts; Windows exclusion fortest/ship-queue-train-cost.test.ts.## [Unreleased]entry (no version bump:queue_mode stamp-at-merge, the train stamps at the slot).Test Coverage
15 code paths in the diff; 13 have a test (87% value-weighted), 12 at ★★ or better (80%), authoring gate PASS. Base control: the three claimed-red tests (
test/gstack-ship-policy.test.tsfill rule,test/paid-report-census.test.tsdisplay-labelled single-case shard,test/ship-queue-train-cost.test.tsknown+unknown line) fail at origin/main and pass at HEAD.Tests: 1455 test files before → 1455 after (0 new files; 3 existing test files extended)
Coverage: 87% value-weighted (80% at ★★ or better, 1 weakly covered path)
Test value: 0 tests written, 0 rejected by the authoring gate, 3 existing tests extended, 1 path weakly covered (weak = ★, gate-failing or unrated).
test/gstack-ship-policy.test.tslive fill rule: fails at base ([5, 3]) · passes after fix.test/paid-report-census.test.tsdisplay-labelled record in a one-case shard: fails at base (cost_usd: 0, cost_known: false) · passes after fix; a failed display-labelled trial keeps the record'sfailure_class/exit_reason/error(added in this /ship, ★★★).test/ship-queue-train-cost.test.tsknown+unknown line: fails at base (barecost=$0.70) · passes after fix; the all-unknown record printscost=$0.00 known + 3 of 3 case(s) cost unknownonly from a priced-zero record (added in this /ship, ★★).lib/ship-queue-train-verdict.tscostTextwitha.cost_usd === nullprints no cost text; covered only bytest/ship-queue-train.test.ts:686. Proposed test: a train record whose panels are all cost-unknown asserts theATTEMPT:line carries nocost=token (value: pins the "no priced case prints no cost" rule the docs now state).Tests ran: focused 341
Tests ran: guards 75
Tests ran: windows-curation 122
Tests ran: docs-anchors 249
Tests ran: typecheck count unavailable
Pre-Landing Review
Three review passes (cap, crew rule 23), plus two adversarial passes. Pass 1 (whole diff at d5e6a7d): 3 INFORMATIONAL, 0 CRITICAL, quality 9. The coverage audit found an undeclared behavior change (the widened record filter also feeds
failedRecord, so a failed trial'sfailure_class/exit_reason/errorcome from the record) → fixed in the CHANGELOG and pinned by a test. Pass 2 (delta, de5d385): clean. Pass 3 (delta, bd9c934): clean. Adversarial pass at de5d385: no blocker; led to the rewordednonBehavioralnote inscripts/harness-version.json(bd9c934). Adversarial pass at bd9c934: no blocker; one wording finding below.Advisory and informational items (listed, not fixed, per the owner's blocking floor):
scripts/lib/paid-report.ts:280missing-edge-case-coverage (INFO, conf 5): a single-case shard accepts any record withoutcase_id, including one named for a different case id. Proposed test in test/paid-report-census.test.ts; not added.docs/ship-policy.md:444stale-comments (INFO, conf 6): a dated aside ("it was 5 PRs or three hours until 2026-10-26…") will age. Pre-existing prose style, deferred.lib/ship-queue-train-verdict.ts:86completeness-gaps (INFO):costTexthas no explicit branch for the all-unknown record (it prints nothing, which the docs now state).nonBehavioralnote says a red whose record saysinfraortimeoutis classed by that record;scripts/train-gate.tsclassesinfraas INFRA but nottimeout. The note's claim about pass/fail is unaffected.Gate edits: 4 listed, 0 recognised gate edits (gstack-gate-diff 5755601+6a22f8dad079, coverage RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16). Self-declared gate-adjacent edits, both kept:
harness-versionguard, scripts/harness-version.json: non-behavioral decision for scripts/lib/paid-report.ts, kept. Reason: no case's pass or fail changes; only a trial record's cost and failure fields do (actor: ship, parent-directed).windows-curationsubset, scripts/lib/windows-curation.ts: test/ship-queue-train-cost.test.ts excluded from the Windows subset, kept. Reason: it drives bin/gstack-ship-queue against the#!/bin/shgh shim, which windows-latest resolves to the runner's gh.exe (windows-free-shard 2 at d5e6a7d), same reason as the existing test/ship-queue-train.test.ts entry (actor: ship, parent-directed).Outside review: Codex unavailable in this harness (
outside_status: unavailable); the adversarial passes ran in-host. Listed as missing outside coverage.Exploratory QA
Ship smoke, report-only, 3 of 12 probes used (6.9 s guarded), verdict pass:
gstack-ship-policy validate --policy-from head→POLICY_VALID;gstack-ship-policy show-effective --policy-from head --base main→max_size 3, fill_hours 1;bun test test/ship-queue-train-cost.test.ts test/paid-report-census.test.ts→ 13 pass 0 fail. No findings reproduced. Not probed: a single-case shard with a foreign-named record (advisory above). Report under.gstack/qa-reports/ship-3478/(git-excluded).Design Review
No frontend files changed — design review skipped.
Eval Results
No paid eval dispatched: no skill prompt or template changed and no paid case reads the changed files (
scripts/lib/paid-report.tsis the harness that reads results; its fixture tests are the proof). The free tier runs on the pregate's remote lane; the merge train runs the full gate at the slot.Greptile Review
No Greptile comments (GREPTILE_EARLY: off; fetch succeeded, only trunk-io[bot] had commented).
Plan Completion
No saved plan file for this branch; Step 8 plan completion not applicable.
Issues
Refs #3478
reference #3478 ← commit : test/ship-queue-train-cost.test.ts drives bin/gstack-ship-queue against the #!/bin/sh gh shim, which windows-latest cannot run (windows-free-shard 2 on #3478 at d5e6a7d), so it joins test/ship-queue-train.test.ts in the Windows exclusion list. (promote: fix #3478)
Verification Results
Step 8.1 obligations: none (no plan file). Contracts verified at Step 9 by QA smoke: 3 PASS, 0 FAIL, 0 BLOCKED, 0 NOT RUN.
TODOS
No TODO items completed in this PR.
Documentation
Status: updated — second audit (ship-3478-docs-reaudit-153813) verified the first audit's three doc edits against
junitCensus,trialCostKnown,train-gate.ts,costTextandfillDue, found the new CHANGELOG sentence consistent with every doc, and tightened one rule in two docs: a train verdict'sATTEMPT:line printscost=$<known> known + <n> of <m> case(s) cost unknownwhen some but not every case is priced, and no cost at all when no case is.Audited scope. Base 5755601 → head 1b3aab9 on
train-fill-and-cost(5 commits, 13 files), clean tree:.gstack/ship-policy.json(train 3 PRs / 1 hour),lib/ship-queue-train-verdict.ts(costText),scripts/lib/paid-report.ts(one-case-shard record attribution),scripts/lib/windows-curation.ts,scripts/harness-version.json, three tests, CHANGELOG## [Unreleased], and the docs below. 123 documentation files inventoried; the four touched by the diff plus README, CONTRIBUTING, CLAUDE.md and AGENTS.md read against the diff. Banned terms: none.Documentation health:
docs/TESTING_INTERNALS.md Updated (ATTEMPT: cost line prints the known+unknown form when some but not every case is priced; a record with no priced case prints no cost)
docs/troubleshooting.md Updated (#cost-unknown: bare
cost=$<x>only when every case is priced, no cost at all when none is)docs/evals/census-red.md Current (cost unknown row matches junitCensus and trialCostKnown)
docs/ship-policy.md Current (fill rule, 3 PRs / one hour, example block and QUEUE_TRAIN_WAITING match fillDue and .gstack/ship-policy.json)
CHANGELOG.md Current (not mutated; its Fixed bullet says the known+unknown line prints "when any panel has no cost", which glosses the all-unknown record that prints nothing)
README.md / CONTRIBUTING.md / CLAUDE.md / AGENTS.md Current (no train fill or cost attribution content; none needed)
Documentation coverage:
train.max_size 3 / fill_hours 1 ✅ reference (ship-policy.md table, example block) ✅ how-to (run-a-merge-train) ❌ tutorial ✅ explanation (fill paragraph)
one-case-shard cost attribution ✅ reference (TESTING_INTERNALS, census-red) ✅ how-to (troubleshooting) ❌ tutorial ✅ explanation (TESTING_INTERNALS)
ATTEMPT: cost=$ known + … ✅ reference ✅ how-to (troubleshooting) ❌ tutorial ✅ explanation
Coverage debt: no tutorial covers merge trains or paid cost reporting (pre-existing, common gap, not critical). Diagram drift: none; no diagram entity was renamed, split, removed or moved in this diff. The audit's two doc edits and the matching CHANGELOG wording are committed in 2724f30.
Test plan
bun test test/paid-report-census.test.ts test/paid-report-fail-open.test.ts test/train-gate.test.ts test/gstack-ship-policy.test.ts test/ship-queue-train-cost.test.ts test/ship-queue-train-logic.test.ts test/module-size-ratchet.test.ts test/troubleshooting-anchors.test.ts test/harness-version.test.ts(focused): 341 pass, 0 failbun test test/parity-suite.test.ts test/bun-entry-isolation.test.ts test/spawnsync-timeout-tripwire.test.ts test/ship-queue-train.test.ts test/evals-train-workflow.test.ts test/changelog-merge-damage.test.ts(guards): 75 pass, 0 failbun test test/test-free-shards.test.ts test/windows-native-workflows.test.ts test/setup-hook-syntax-gate.test.ts test/ubicloud-pregate-remote.test.ts(windows-curation): 122 pass, 0 failbun test test/cost-known.test.ts test/census-red-docs.test.ts test/testing-internals-anchors.test.ts test/troubleshooting-anchors.test.ts(docs-anchors): 249 pass, 0 failbun run typecheck: exit 0gstack-pregate --base main --allow-repo-commands: preflight 7 checks pass; tests tier requires-remote on this 4-core machine, cleared by the remote Ubicloud free-tier lane (run linked in the READY report)bun run testlocally: not run (4-core machine; the remote lane runs the full free tier)🤖 Generated with gstack /ship under Capy