Skip to content

/ship's test step says when it hits its time limit, and long test file lists no longer overflow the remote command (GST-2958) - #4429

Merged
capy-ai[bot] merged 5 commits into
mainfrom
ship-step-time-limit-and-remote-file-lists
Nov 17, 2026
Merged

capy-ai[bot] merged 5 commits into
mainfrom
ship-step-time-limit-and-remote-file-lists

Conversation

@garrytan

@garrytan garrytan commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

/ship's test step now says when it runs out of time and lets you set its budget, and the remote test command no longer packs every test file into one command-line argument, which Linux refuses past 128 KiB.
You don't need to do anything: the default budget is still one hour, and --step-timeout <s> on gstack-ship-preflight raises it.

Details

What changed

  • gstack-ship-preflight --step-timeout <s> (1 to 86400, default 3600) sets the budget for the whole test step. When a run hits it, the output says so and names the way out:
PREGATE: time limit hit (Step 5's whole pregate ran past its 2 s budget, --step-timeout, and was stopped with exit 143; no verdict)
STOP: the tests are longer than this step's 2 s budget on this machine: push, let CI's full run at this tree carry them (gstack-ci-carry wait --base main, then rerun --tests-only; gbrain docs/ci-event-parity.md#ship-lanes), or rerun --tests-only with --step-timeout <s> above 2

(Captured on this branch with a 2-second budget in a throwaway clone. Before this change the same stop printed PREGATE: unreadable (exit 143).)

  • On a machine with fewer than 8 cores and a remote test backend, the command printed for gstack's Ubicloud runner now sends the selected test files in a small setup script instead of one long argument. The script is named by its content, so the same selection prints the same command every run.
  • A selected test path that contains a newline keeps the old quoted command, so it can never end the file list early and run as shell on the test machine.
  • docs/troubleshooting.md has an arg-limit entry, and docs/pregate.md describes both changes.

Why

"PREGATE: unreadable (exit 143)" told nobody that the suite simply ran past the hour /ship gave it. Now the stop says it hit the time limit and how to get past it. Separately, a large selection made the remote test command too long for Linux to start, and now the command stays short however many files are selected.

Blast radius

The changes stay inside gstack-ship-preflight's test step and the command pregate prints for a remote test backend. Pregate never runs that command itself. The default budget is the same 3600 s as the old fixed limit, and a run that finishes in time takes the same path as before (lib/ship-preflight.ts, read). Bun's spawnSync sets ETIMEDOUT when the limit hits, and pregate's SIGTERM handler kills the test processes it supervises (walked failure path; QA probe 005 found no test process left behind).

Summary

Long suites and big selections both failed in ways that hid the cause. The test step now has a budget you can set and a clear message when it runs out. The remote command passes its file list through a setup script. After review, odd file names fall back to the old quoted form.

Test Coverage

All new code paths have test coverage: each of the 4 changed runtime files has a changed test that names it.

  • test/ship-preflight.test.ts: a run past --step-timeout prints the time-limit line and the way out; bad values are usage errors.
  • test/pregate-lanes-arg-limit.test.ts: 700 long file names give a command under 2 KB that still reaches all 700 files.
  • test/pregate-remote.test.ts: the new command shape and its setup file, and a path with a newline that keeps the quoted inline form.
  • The new tests fail on main, and the newline test fails without its fix.

Tests ran: test 1678 (files)

Pre-Landing Review

Two rounds:

  • Round 1. A batched reader for testing, security and state machines, a red-team reader, a native adversarial reader, and a second-model review (gpt-6.1-sol).
  • One blocker, fixed. The second-model review found that a test path containing a newline could end the setup script's file list early and run the rest as shell on the test machine. It is fixed, with a test, in f043430.
  • Round 2. A full re-read of the fix found nothing blocking.
  • Skipped. A round-1 test-gap note: the scratch fallback for an unwritable state root has no test. The fallback was read and is correct.

REVIEW_LOOP: passes=2 blocking_found=1 deferred=0 noted=3 fixed=1 scope=frozen unbound=1 ledgers_read=3 dispatches=8
Gate edits: none detected (6 listed, 3 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)
Second-model review (openai/gpt-6.1-sol): completed, 1 blocking finding, fixed.

Review notes

These are below the blocking floor (advisory, informational, test-gap or wording items). They are recorded here and not fixed in this PR.

  • INFORMATIONAL lib/pregate/checks/lanes.ts:521 Two pregate runs with the same selection rewrite the same content-named setup file without an atomic write. If the runner reads it mid-rewrite, it could see an empty list.
  • INFORMATIONAL lib/pregate/checks/lanes.ts:528 The printed command now names a file on this machine, so it fails elsewhere or after the state root is cleaned. In the scratch fallback its path changes every run, so a later clear never matches.
  • INFORMATIONAL lib/pregate/checks/lanes.ts:519 Old setup scripts are never pruned. Each is a few KB, one per distinct selection.
  • INFORMATIONAL lib/pregate/checks/lanes.ts:518 The end-marker branch of the new guard has no test of its own.
  • INFORMATIONAL lib/pregate/checks/lanes.ts:578 The declared remote_gate route (scripts/ubicloud/pregate-remote.sh) has the same newline problem. That code predates this PR.
  • INFORMATIONAL lib/pregate/checks/lanes.ts:518 On a very large selection that also holds an odd path, the quoted fallback can exceed the argument limit and fail loudly.

Exploratory QA

Pass, 5 captures in a throwaway clone with its own state root:

  • --help documents the option.
  • --step-timeout 0 and 86401 exit 2 with a usage message.
  • A 2-second budget stops with the time-limit line and exit 3.
  • No test process is left running afterwards.

Limits: the budget ran out during pregate's own checks, before the slow test started. The stop also took about 19 s of wall time for a 2-second budget, because pregate cleans up after SIGTERM.

TODOS

The 3600 s preflight and finish wrapper deadline (P3) is narrowed to finish's half. The preflight half is done in this PR.

Documentation

Scope: This audit covered the committed changes in base 2af3822..84c9e0d at reviewed tree 4486a9a9b0ba9de0c74ebdb2669580f7ff9d6fc9. Those changes add the --step-timeout option with its PREGATE: time limit hit output, move the remote file list into a setup script, and describe the Windows free tests as eight shards instead of six. The working tree had no staged, unstaged or new files.

Result: The docs were already current, so I edited nothing. docs/pregate.md covers the time limit in the timeout_s row, and TODOS.md has a matching note. The move of the remote file list into a setup script is covered in the remote gate section and the arg-limit row of docs/pregate.md, and docs/troubleshooting.md has a new arg-limit entry. Both anchors resolve. The eight-shard wording matches .github/workflows/windows-free-tests.yml and agrees with docs/support-tiers.md. No ship template or README mentions the preflight time budget or the old PREGATE: unreadable (exit 143) output, so no template needs to be regenerated. The banned-terms check found nothing because main has no .gstack/banned-terms.json.

Debt: None. The changelog fragment and the TODOS.md edit belong to the parent ship run, so I read them but did not change them.

Test plan

  • bun run test (the full free suite, in CI): Free Tests run 44087393983 at 84c9e0d passed, 1678 files.
  • Focused tests on the merged tree: ship-preflight, pregate-lanes-arg-limit, pregate-remote and pregate-hidden-folder-paths passed (59 tests); after the fix, the three pregate files passed (25 tests).
Agent notes

Lane GSTA-576 for GST-2958. Refs GST-2958

Approvals

No approvals recorded for branch ship-step-time-limit-and-remote-file-lists.
Plan: exempt: bug fix (failing-first tests on main, no plan review)
Plan check: exempt bug fix

REVIEW_LOOP: passes=2 blocking_found=1 deferred=0 noted=3 fixed=1 scope=frozen unbound=1 ledgers_read=3 dispatches=8
Gate edits: none detected (6 listed, 3 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)
CI_CARRY: accepted ci-carry tree=4486a9a9b0ba tests=37412 runs=44087393983/1 profiles=ci-reuse/free-tests heads=84c9e0da3f66
PREGATE: requires-remote cleared (final tree)

{
  "schema_version": 1,
  "pr": 4429,
  "head": "84c9e0da3f660e4007a8e9f0bf34d90f12852b2f",
  "version": "1.99.5.0",
  "base": "main",
  "gate": {
    "test": {
      "pass": 1678,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "test",
      "run": 44087393983,
      "attempt": 1,
      "tree": "4486a9a9b0ba9de0c74ebdb2669580f7ff9d6fc9"
    }
  ],
  "session_kind": "unattended",
  "run": "ship-4429",
  "head_tree": "4486a9a9b0ba9de0c74ebdb2669580f7ff9d6fc9",
  "product_patch_id": "dee790c04650deb3a864daf331d93527e9bd82e5",
  "gstack_version": "1.99.5.0",
  "review_depth": {
    "depth": "full",
    "class": "gate",
    "runtime_lines": 66,
    "rules": "66e5174161eecc2c",
    "base": "e8c8625bd65fdcc0f0650f982830649d326c4bf4",
    "registry": "e8c8625bd65fdcc0f0650f982830649d326c4bf4",
    "gate_scan": {
      "tagged": 0,
      "head": "84c9e0da3f660e4007a8e9f0bf34d90f12852b2f"
    },
    "reasons": [
      "gate-class",
      "runtime=66"
    ],
    "risk": [],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @4486a9a9b0ba (8 checks, current for 84c9e0da3f66)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/44087393983/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=4486a9a9b0ba tests=37412 skipped=90 runs=44087393983/1 profiles=ci-reuse/free-tests heads=84c9e0da3f66 verified=2026-11-17T13:05:21.026Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "current",
    "scope": "docs/ and README.md, base 2af3822f..84c9e0da"
  },
  "review_loop": {
    "passes": 2,
    "blocking_found": 1,
    "deferred": 0,
    "noted": 3,
    "fixed": 1,
    "scope": "frozen",
    "rounds": 2,
    "ledgers_read": 3,
    "unbound": 1,
    "dispatches": {
      "specialist": 6,
      "red-team": 2
    }
  },
  "gate_dispositions": []
}

…ts off the remote test command line

gstack-ship-preflight stopped gstack-pregate after a fixed hour and printed
'PREGATE: unreadable (exit 143)'. The whole step's budget is now
--step-timeout <s> (default 3600), and a stop at it prints
'PREGATE: time limit hit' with the way out: push and let CI's full run
carry the tests, or a larger budget.

The test_backend command pregate prints for a small machine put every
selected file into one argument, which Linux refuses past 128 KiB. The
files now reach the VM in a setup script named by its content.

Also adds an arg-limit troubleshooting entry and says the Windows job has
eight shards.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 4 commits November 17, 2026 08:50
…folder path test

The remote command now names a setup script instead of the files, so the
test checks that the script lists the ./-prefixed path.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
A selected test path containing a newline, or equal to the list's end marker, could end the setup script's file list early and run the rest as shell on the test VM. Such paths now keep the quoted inline command.

Review-Finding: p1/outside-voice#1 reader=outside-voice
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai
capy-ai Bot merged commit aeb74eb into main Nov 17, 2026
82 checks passed
@garrytan garrytan mentioned this pull request Nov 17, 2026
capy-ai Bot added a commit that referenced this pull request Nov 17, 2026
- /autoplan's approval page no longer hides a reviewer's product objection behind "keep your direction". (#4431)
- Every skill that loads the shared preamble, on every host, now says to ask a question once and to check a file before handing it over. (#4383)
- The Captain manuals now name the `Failed or skipped:` and `Also changed:` lines that READY and lane reports carry. (#4432)
- gstack's shell locks hold again on Ubuntu 26.04, where `mkdir` reports success when another process won the race (GST-2247). (#4262)
- `test/gstack-lock-race.test.ts` runs the old pattern and the helper against a PATH shim that behaves like uutils 0.8.0 `mkdir`, so it runs on any runner image: the old pattern lets two of four racers win, the helper admits exactly one in each of 60 races and never overlaps across take, hold and release loops, and parallel `gstack-config` writers keep every key. (#4262)
- Windows Setup E2E now runs when `bin/gstack-lock.sh` changes. (#4433)
- Claude Code's skill list now says when to use each gstack skill, so Opus loads /review and /plan-eng-review when a request fits. (#4387)
- A pull request's merge head can no longer keep its `/ship` receipt without a full review. (#4365)
- `/ship` now counts review rounds per pull request, and the head that merges always gets a full review. (#4360)
- /ship's documentation audit no longer accepts a passing result when an earlier line from the same audit said "blocked". (#4434)
- /ship's documentation audit no longer spends an attempt when the audit child writes text after its result. (#4285)
- The README lists the two new guides, and the install doc counts its hooks right. (#4403)
- docs/releases.md now says the stable release cut treats a failed commit status as red and a pending one as still going, as it has since #4257. (#4436)
- /ship's `finish --docs-requested` now accepts a docs fix when the audit named the file with a line number. (#4437)
- Document the `.gstack/approval-classes.json` schema, `gstack-approval-classes` CLI commands, and owner approval citation format in `docs/ship-policy.md#approval-classes`. (#4355)
- `/eval-plan` now runs four result checks instead of leaving them to judgment. (#4384)
- The /review coverage-audit eval no longer fails a run that really read the source file when zsh stops a command line at an `echo ====` separator. (#4364)
- Sixteen in-repo documentation links reach their section again. (#4394)
- gstack agents now open every status report with what failed, what they didn't verify and what else they changed. (#4273)
- Two new short guides. (#4380)
- A skill you wrote whose contract still lists `check_after_last_edit` keeps the rest of its contract. (#4439)
- The ship policy guide now explains how to merge any pull request with `gstack-ship-queue land`, even in a repository with one maintainer. (#4377)
- /ship's test step runs a large unit suite on Linux. (#4426)
- Every skill that loads the shared preamble, on every host, now says never to print a whole environment, git config or process table. (#4382)
- Pull requests from /ship now open with at most three plain lines, and every merge lands as one commit titled with the PR title. (#4438)
- `gstack-pr-wait`, `gstack-ci-gate` and the release cut no longer call a commit green while a check on it failed. (#4257)
- `gstack-runaway` now says what a thread's cold and empty wakes cost, not just how many there were (GST-2770). (#4385)
- The optional Stop hook no longer checks for a test run after the last edit, and /verify asks for it in plain words instead. (#4428)
- /ship's test step says when it ran out of time, and you can set its budget. (#4429)
- The remote test command no longer puts every file on one command line. (#4429)
- The stable release no longer waits on the nightly Windows tests (GST-2815). (#4400)
- Ubicloud fan-outs can no longer overshoot the vCPU limit, and owners no longer delete each other's VMs. (#4430)
- hono is pinned to 4.13.16, which fixes five advisories the weekly OSV scan found in 4.13.13 (gstack#4406). (#4408)
- `package.json` overrides (#4408)
- `test/dependency-security.test.ts`: the lockfile floor check now covers `hono >= 4.13.16`. (#4408)
- The nightly Windows test run finishes again. (#4409)
- A short write-up checklist in ETHOS.md. (#4381)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant