Skip to content

Second-model plan reviews in autoplan stop being reported unavailable; --help stops printing a fake refusal (GST-3051) - #4471

Merged
capy-ai[bot] merged 1 commit into
mainfrom
capy/autoplan-second-model-review-not-unavailable
Nov 18, 2026
Merged

capy-ai[bot] merged 1 commit into
mainfrom
capy/autoplan-second-model-review-not-unavailable

Conversation

@garrytan

@garrytan garrytan commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

A second-model plan review in /autoplan is no longer reported as unavailable when it actually ran, and gstack-outside-voice --help no longer prints a line that reads like a real consent refusal.
Why: sessions believed the false label and paid to re-run the review, or closed the plan phase without its second-model review.
Nothing to do: the command autoplan already runs now picks the right check by itself.

What changed

gstack-outside-voice --help: the two status lines are now indented examples, so no help line reads like a real consent refusal, and --gate lists autoplan

The help text now shows the two status lines as indented examples, so no help line reads like a real refusal, and --gate lists autoplan.

Before and after as text:

--- Before · before.txt · captured output
+++ After · after.txt · captured output
 gstack-outside-voice — one outside voice, three runners (docs/unattended.md)
 
   run --runner <r> --prompt <file> --out <file> [--model <id>] [--result <file>] [--native-model <id>]
+      [--fallback <r>] [--gate review|structured|spec|proposal|autoplan] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
-      [--fallback <r>] [--gate review|structured|spec|proposal] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
   runners [--model <id>] [--json]        the runner table: id, capabilities, available here
   family <model-id>                      print the model family (anthropic, openai, google, ...)
 
@@ ⋯ 2 unchanged ⋯ @@
 cross-family subagent the host ran; needs --model). --max-tokens caps the api runner's output
 (Anthropic default 16384; its default time limit grows with it); a response cut off at the limit
 is unavailable (truncated). --native-model enforces the same-family
+rule. GSTACK_OUTSIDE_RUNNER names the fallback when --fallback is absent. --gate defaults to
+autoplan (its INPUT receipt and gstack-findings fence) when the prompt carries gstack-autoplan's
+RESULT FORMAT, else review. Writes <out>.meta.json and prints a status line, for example
+    OUTSIDE_STATUS: <status> provider=<runner> model=<id> family=<f>
+codex_reviews=disabled (or an unreadable config) refuses every runner before it starts, exit 3,
+with the status line
+    OUTSIDE_STATUS: not_run reason=consent_missing
+Enable: gstack-config set codex_reviews enabled.
-rule. GSTACK_OUTSIDE_RUNNER names the fallback when --fallback is absent.
-Prints OUTSIDE_STATUS: <status> provider=<runner> model=<id> family=<f> and writes <out>.meta.json.
-codex_reviews=disabled (or an unreadable config) refuses every runner before it starts:
-OUTSIDE_STATUS: not_run reason=consent_missing, exit 3. Enable: gstack-config set codex_reviews enabled.
 Exit codes: 0 ok · 1 fail · 2 usage · 3 refused or needs a flag
 

A plan review that followed autoplan's own result format was judged by /review's rule, which wants a Recommendation: ... because ... line that autoplan never asks for, so a finished review came back VERDICT: unavailable. Now a prompt that carries autoplan's result format is checked against that format, which needs its INPUT: <phase> <sha256> line and its gstack-findings block. Critical or High rows read as findings, Medium, Low or no rows read as clean, and a reply missing the line or the block, or an empty reply, is still unavailable. --gate autoplan asks for this check on any prompt, and every other prompt keeps the old check. The text autoplan sends to reviewers is byte-for-byte unchanged.

Why

Sessions saw "unavailable" on reviews that had run and either paid to run them again or dropped them (the day-one eval at e671a53 showed it). After this change the label matches what happened.

Blast radius

Only autoplan calls gstack-outside-voice run. /review and /ship call lib/outside-review-result.ts with an explicit check name (scripts/resolvers/outside-voice.ts:146, read), so the new default never reaches them. The shared parser's other callers (lib/autoplan-run.ts, lib/autoplan-carry.ts) still pass the expected phase and hash, and the new runtime file lib/autoplan-reconcile.ts imports only types, so the Claude Code copy list stays complete (file read, and the targeted tests ran).

Test Coverage

Tests: 1809 → 1809 (+0 new in this review; the PR itself adds test/outside-voice-autoplan-gate.test.ts)
Coverage: 82% value-weighted (82% including 0 weakly covered paths)
Test value: 0 tests written, 0 rejected by the authoring gate, 0 existing tests extended, 0 paths weakly covered.
Two small gaps are left as proposals: a Critical row mapping to P0 and an Informational row mapping to nothing (only High and Medium rows are tested), and the outside-review-result.ts --verdict autoplan command form has no test. Both were checked by hand during review and behave correctly.
Tests ran: targeted 339
Tests ran: free 1691 (files, from CI run 44219378952)

Pre-Landing Review

No blocking issues found. One review round; three low-severity notes, listed below, are recorded and not fixed here.

Review notes

Below the blocking floor; recorded here and not fixed in this PR.

  • bin/gstack-outside-voice:84: the check is picked by a plain text match on the whole prompt, so a prompt that only quotes autoplan's format header would get autoplan's check. Only autoplan calls this command today, and --gate review overrides it.
  • lib/outside-review-result.ts:271: the autoplan check accepts any phase and hash in the INPUT: line, so a stale result file reads as completed until autoplan's own submit step refuses it.
  • docs/troubleshooting.md:390: the sentence could also say that an explicit --gate review on an autoplan prompt still gives the old message.
  • One parsed item was a line from the reviewer's own summary of the tests, not a defect.
Exploratory QA

Smoke checks on the built command: --help prints no line starting with OUTSIDE_STATUS: or VERDICT:; a reply with a Critical row under --gate autoplan reads VERDICT: findings, FINDINGS: P0, status completed; outside-review-result.ts --verdict autoplan gives the same verdict. QA's method files were not loaded for this small CLI change.

Eval Results

The paid-case selector picked 106 behavioral cases, all through files many cases depend on; no prompt text changed and no quality judge was selected, so no paid cases ran before review.

TODOS

No TODO items completed in this PR.

Documentation

Scope: I audited the diff from merge base cbd21f9 to head ef26db9 on capy/autoplan-second-model-review-not-unavailable, including committed, staged and unstaged changes. The change: gstack-outside-voice run now checks a prompt that carries autoplan's result format against that format (--gate autoplan), and --help no longer prints a line that starts with OUTSIDE_STATUS:.

Result: The docs are current, so I made no edits. docs/troubleshooting.md already describes the autoplan gate and what to do about the old missing_markers result. The changelog.d fragment covers both user-visible changes, and the help text in bin/gstack-outside-voice lists --gate autoplan. I searched docs/unattended.md, docs/skills.md and the other docs that mention --gate and found no stale lists of gate values.

Debt: None.

Tests
  • bun --no-env-file test --timeout=30000 on 8 targeted files: 339 tests passed
  • CI free suite at this head (run 44219378952): 1691 files passed
Agent notes

Refs GST-3051 (thread GSTA-603).

Approvals

No approvals recorded for branch capy/autoplan-second-model-review-not-unavailable.
Plan: exempt: bug fix
Plan check: exempt bug fix

REVIEW_LOOP: passes=1 blocking_found=0 deferred=0 noted=0 fixed=0 scope=frozen dispatches=4
Gate edits: none detected (2 listed, 1 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)

🤖 Generated with Claude Code

{
  "schema_version": 1,
  "pr": 4471,
  "head": "ef26db960e802b6831dfc1f5b4a19597943cfc34",
  "version": "1.100.0.0",
  "base": "main",
  "gate": {
    "targeted": {
      "pass": 339,
      "fail": 0,
      "skip": 0
    },
    "free": {
      "pass": 1691,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "free",
      "run": 44219378952,
      "attempt": 1,
      "tree": "eafc489503c4f70dd7fbd5b594f3e9bc470ec7bf"
    }
  ],
  "calls": 19,
  "session_kind": "unattended",
  "run": "ship-4471-ef26db9",
  "head_tree": "eafc489503c4f70dd7fbd5b594f3e9bc470ec7bf",
  "product_patch_id": "b5d142304f7b89705a551636c6976ff712bfc509",
  "gstack_version": "1.100.0.0",
  "review_depth": {
    "depth": "light",
    "class": "code",
    "runtime_lines": 50,
    "rules": "85446ac5884d68a3",
    "base": "29e950227b2d6b87a0529492885f42e522483969",
    "registry": "29e950227b2d6b87a0529492885f42e522483969",
    "gate_scan": {
      "tagged": 0,
      "head": "ef26db960e802b6831dfc1f5b4a19597943cfc34"
    },
    "reasons": [
      "runtime=50"
    ],
    "risk": [],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @eafc489503c4 (8 checks, current for ef26db960e80)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/44217036837/attempts/1",
    "windows-smoke.yml/windows-smoke: https://github.com/garrytan/gstack/actions/runs/44217035359/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=eafc489503c4 tests=37668 skipped=76 runs=44219378952/1 profiles=ci-reuse/free-tests heads=ef26db960e80 verified=2026-11-18T02:02:08.292Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "current",
    "scope": "docs/, README.md, CONTRIBUTING.md, CLAUDE.md, AGENTS.md, autoplan/DRIVER.md.tmpl against merge base cbd21f94"
  },
  "review_loop": {
    "passes": 1,
    "blocking_found": 0,
    "deferred": 0,
    "noted": 0,
    "fixed": 0,
    "scope": "frozen",
    "rounds": 1,
    "ledgers_read": 3,
    "dispatches": {
      "specialist": 3,
      "red-team": 1
    }
  },
  "gate_dispositions": []
}

…; --help stops printing a fake refusal (GST-3051)

gstack-outside-voice judged autoplan's outside reviews by /review's Recommendation-line rule, so a review that ran printed VERDICT: unavailable (missing_markers). A prompt carrying autoplan's RESULT FORMAT now defaults to an autoplan gate that checks its INPUT line and gstack-findings fence with autoplan-reconcile's parser; a reply without them, or an empty one, is still unavailable. --help shows status lines indented as examples.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

@capy-ai
capy-ai Bot merged commit c9b6fe0 into main Nov 18, 2026
83 of 84 checks passed
This was referenced Nov 18, 2026
capy-ai Bot added a commit that referenced this pull request Nov 18, 2026
- A test, or a scratch repository in the temp directory, can no longer write approval rows into your real Capy Drive (GST-3027). (#4466)
- An unattended /autoplan run on Claude Code no longer says the publication guard is off while it is on. (#4481)
- A phase report written only in reasoning now gets a denial that says so and names the fix. (#4481)
- `gstack-autoplan revise` no longer refuses a plan that the first `next` accepted. (#4470)
- `revise --plan` with a different file now says the run follows that file. (#4470)
- A second-model plan review in `/autoplan` is no longer reported as unavailable when it actually ran (GST-3051). (#4471)
- The /ship conductor trial plan now lets case reverts that conflict with later code be resolved by hand in docs, tests and the planted file, so large pull requests can be cases (GST-3003). (#4460)
- A locked plan for a trial of /ship without its separate conductor subagent (GST-3003). (#4456)
- A driven /ship now runs each phase as one command, counts its calls, and stops itself at 100 minutes. (#4451)
- /ship stops asking owners to clear two kinds of bookkeeping noise: base-side merge text and a reviewer's confirmation prose. (#4458)
- A base merge that only touched base-side text no longer stops `/ship` finish for an owner's grant. (#4458)
- A reviewer's confirmation prose no longer becomes a Critical finding that only an owner can clear. (#4458)
- A CI run that passed only after its failed jobs were re-run is now marked wherever gstack reuses it. (#4473)
- /ship reuses CI's passing test run when merging the base changed no file, and stops instead of quietly starting hour-long test runs or reviewing an empty diff (GST-3019). (#4461)
- /ship on Codex no longer stops at the test coverage audit. (#4489)
- A cold first-use check before an app build reaches its owner (GST-3025). (#4465)
- `skill_prefix=true` now survives updates when gbrain is installed. (#4454)
- The OpenClaw docs no longer send you to ClawHub, which held only April 2026 builds of office-hours, ceo-review and retro and no longer has gstack skills (the docs also said four skills while listing three). (#4454)
- `gstack-issue-links` no longer reads other repositories' issues as gstack's. (#4454)
- /ship's pre-gate finds the tests of a monorepo app that imports through its own `@/` alias. (#4464)
- /office-hours explains why it recommends a mode. (#4450)
- /office-hours keeps its 34 founder videos and essays out of the skill file. (#4463)
- Two `gstack-owner-answers add` runs for the same answer no longer collide. (#4490)
- A Captain can check an ask against the owner's earlier answers before sending it. (#4467)
- `gstack-post` stops reading /ship's own receipt identifiers as phone numbers, so posting a /ship PR body no longer needs a confirm. (#4488)
- GitHub release notes now list every change in plain words, with a link to its pull request. (#4390)
- gstack's router no longer gives bug reports a special rule, so the model fixes a bug the way it would without gstack. (#4442)
- /ship's review plan now sorts Swift and Xcode files by risk instead of sending every iOS PR to the full review panel (GST-3041). (#4469)
- /ship's review plan now sends Swift code that touches private user data to the full review panel (GST-3052). (#4472)
- A round grant at /ship's review round cap now buys the fix round it names, in the same run or the next (GST-2968). (#4479)
- The secrets check stops flagging three kinds of ordinary code as a secret, and catches two misses. (#4457)
- A security finding's location and reason no longer go into a PR body. (#4486)
- The token-capped /ship child on Claude Code now starts its prompt with the typed `/ship` command, so the skill actually loads. (#4487)
- A /ship receipt now counts the second-model review's blocking findings, and refuses to be written while one is still open. (#4485)
- A `/ship` with a token cap on Claude Code no longer stops at its first handoff. (#4452)
- /ship's push step now always ends with one line that says whether it pushed (GST-3053). (#4474)
- A PR's review-round count can no longer be overwritten by the body `/ship` posts. (#4440)
- One queued record with a malformed path no longer stops artifacts sync for every project. (#4455)
- A failed artifacts push stays reported until a push succeeds. (#4455)
- The /sync-gbrain memory row no longer says OK when pages failed to import. (#4455)
- ### Tests (#4495)
- A policy-floor test no longer fails once gstack's own version reaches the minor after the newest policy key; it now checks a floor above both, so the v1.101.0.0 release can pass CI. (#4495)
- CI now uploads free-suite test results to Trunk with analytics-uploader v2.1.8 and analytics-cli 0.17.0, up from v2.1.6 and 0.16.2. (#4453)
- Packing a checkout for a Ubicloud run no longer fails when git beside it is rewriting a ref. (#4447)
- The /plan-ceo-review HOLD SCOPE eval no longer fails a review because its follow-up question didn't repeat the mode name. (#4447)
- Unattended /autoplan now records the owner's relayed gate reply instead of refusing it. (#4484)
- Unattended /autoplan now runs the second-model review you turned on, and waits for it instead of ending the session. (#4459)
- Two /autoplan runs started in the same second no longer share a run id. (#4459)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant