Skip to content

Codex /review with gpt-6.1-sol counts the second-model pass it already ran (GEVA-562) - #4526

Merged
capy-ai[bot] merged 3 commits into
mainfrom
codex-sol-spawn-dispatch
Nov 19, 2026
Merged

capy-ai[bot] merged 3 commits into
mainfrom
codex-sol-spawn-dispatch

Conversation

@garrytan

@garrytan garrytan commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

On Codex with gpt-6.1-sol, /review is no longer blocked at session end for a missing second-model pass that it ran.
Why: with Sol, Codex starts subagents through its collaboration tools and names a started agent by task_name, not agent_id, so the checklist reader missed it; this blocks gstack-evals' Codex pin.
Nothing to do: the fix ships with the next gstack release.

What changed
  • lib/codex-rollout.ts counts a started call to Codex's own spawn_agent: an agent_id result from the multi_agent_v1 namespace, or a task_name result from the collaboration namespace (multi_agents_v2). A call with no namespace may hold either. A failed spawn still doesn't count, and a same-named tool from any other namespace never counts.
  • New fixture test/fixtures/skill-contract/codex/exec-sol-spawn-met.jsonl: a keyless Codex 0.170 run with -m gpt-6.1-sol against a local stub endpoint, trimmed like the other fixtures.
  • Docs: the dispatch rule in docs/skill-routing-and-contracts.md, and a changelog fragment.
Tests
  • test/skill-contract-codex.test.ts: the Sol run's adversarial pass is met at the first stop (fails at b0f865c); a collaboration spawn whose result has no task_name (empty, error text, nickname only) is not a dispatch, and neither is a spawn_agent from another namespace.
  • Full free suite: CI run 44530390627 attempt 2 at the reviewed head e006027 (1709 files, green), and CI run 44535570812 at the merged head 2c3ee55, green.
Why

The end-of-session contract check reads the Codex rollout to see whether /review ran its adversarial pass. Its fixtures came from gpt-5.4-mini, where a started agent answers with agent_id. Sol answers with task_name, so a session that did the work was held once for a pass it had already run.

Blast radius

Only the Codex rollout reader behind the skill contract check changes. Claude Code sessions and other hosts are untouched. A Codex release that renames the namespace would make the check hold a session once, which is the same failure this fixes, not data loss.

Documentation

Status: current. The docs already match the change, so nothing was edited.

Scope: the diff from base b0f865c to head e006027 on codex-sol-spawn-dispatch, which counts a Codex spawn_agent call in the collaboration namespace as started when its result names a task_name. The diff touches lib/codex-rollout.ts, a new Sol fixture and its test, the codex fixture README, docs/skill-routing-and-contracts.md and a changelog.d fragment. There were no staged, unstaged or untracked changes.

Documentation health:

  • docs/skill-routing-and-contracts.md: Current. Its dispatch definition already covers agent_id and task_name, failed spawns and tools of the same name from other namespaces.
  • test/fixtures/skill-contract/codex/README.md: Current. The new fixture's row describes how it was captured and what it shows.
  • changelog.d/codex-sol-spawn-dispatch.md: Current (read only).
  • docs/hosts.md: Current. Its Codex subagents paragraph doesn't depend on the result shape.
  • CHANGELOG.md: Current. It has no stale claim.

Coverage debt: none. Diagram drift: none.

Agent notes

Bug fix (no plan review). Reported by GEVA-562 via GEVA-2 on 2026-11-19, reproduced on b0f865c with gstack-skill-contract check on the run's rollout (missing adversarial-pass). Decision trail: reports/GSTA-651-decisions.tsv in the controller drive.

Approvals

No approvals recorded for branch codex-sol-spawn-dispatch.
Plan: exempt: bug fix (failing-first test in test/skill-contract-codex.test.ts fails at b0f865c)

Pre-Landing Review

Pre-Landing Review: 2 issues — 0 auto-fixed, 0 asked, 0 deferred, 2 noted
REVIEW_LOOP: passes=1 blocking_found=0 deferred=0 noted=2 fixed=0 scope=frozen dispatches=1
Gate edits: none detected (GATE_SCAN tagged=0 at e006027)

  • Plan: light depth, one pass: a batched specialist (testing, security, state machines, red team) found nothing; the second-model review (openai/gpt-6.1-sol) was clean.
  • Noted, below the floor (adversarial pass, both Low, Blocks: no): (1) an unknown or empty-string namespace now fails closed, where any agent_id used to count; (2) task_name is chosen by the model, so a structured error that echoed it back would count; unconfirmed, since the fixture came from a stub.
  • Exploratory QA: replaying GEVA-562's real Sol rollout counts 1 dispatch at this head and 0 at b0f865c; the same rollout with another namespace or a failed spawn counts 0.
  • Paid evals: none run. The selector picked 106 paid cases through the derived reference lib/codex-rollout.ts, over this PR's $10 paid cap; they ride the scheduled gate on main.
  • Base merge: main (45e6d54) merged at 2c3ee55; the PR's own added and removed lines are byte-identical to e006027. Captain GStack Minder (GSTA-36): one scoped carry reader for the spliced hunks, per CREW.md standing grant of 2026-11-17. Reader openai/gpt-6.2-sol on docs/skill-routing-and-contracts.md: clean.
{
  "schema_version": 1,
  "pr": 4526,
  "head": "2c3ee550f4480a53a48cf46f57e00ad7a9695e83",
  "version": "1.101.0.0",
  "base": "main",
  "gate": {
    "test": {
      "pass": 1709,
      "fail": 0,
      "skip": 0
    },
    "focused": {
      "pass": 31,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "test",
      "run": 44530390627,
      "attempt": 2,
      "tree": "e88e964e44a6ce4f1ddc34c274451c5ac1026978"
    }
  ],
  "calls": 15,
  "session_kind": "unattended",
  "run": "ship-4526-e006027",
  "head_tree": "42be0e98bca98a69ddca4898645169deac348569",
  "product_patch_id": "dbcbfffdff6e005c9784f9d4d3b7822eff13f341",
  "gstack_version": "1.101.0.0",
  "review_depth": {
    "depth": "light",
    "class": "code",
    "runtime_lines": 21,
    "rules": "1afa38a4be3b5eef",
    "base": "45e6d549e34ee3a4f891f52ae952b3905c9ab4b5",
    "registry": "45e6d549e34ee3a4f891f52ae952b3905c9ab4b5",
    "gate_scan": {
      "tagged": 0,
      "head": "2c3ee550f4480a53a48cf46f57e00ad7a9695e83"
    },
    "reasons": [
      "runtime=21"
    ],
    "risk": [],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @42be0e98bca9 (8 checks, current for 2c3ee550f448)",
  "requires_remote": [
    "free-tests.yml/cso-macos-launcher: https://github.com/garrytan/gstack/actions/runs/44535570812/attempts/1",
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/44535570812/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=42be0e98bca9 tests=38234 skipped=67 runs=44535570812/1 profiles=ci-reuse/free-tests heads=2c3ee550f448 verified=2026-11-19T13:51:09.427Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "current",
    "scope": "docs/ and README.md against the PR diff from b0f865c0 (6 files)"
  },
  "review_loop": {
    "passes": 1,
    "blocking_found": 0,
    "deferred": 0,
    "noted": 2,
    "fixed": 0,
    "scope": "frozen",
    "rounds": 1,
    "dispatches": {
      "specialist": 1
    }
  },
  "gate_dispositions": [],
  "carry": {
    "from": "e0060272c10600d28ef3ee99a902de36354a3cb8",
    "reason": "base merge + base context moved (scoped review by openai/gpt-6.2-sol: docs/skill-routing-and-contracts.md) + release metadata",
    "product_patch_id": "dbcbfffdff6e005c9784f9d4d3b7822eff13f341",
    "changed_paths": [
      "product:README.md",
      "product:autoplan/SKILL.md.tmpl",
      "product:bin/gstack-review-dispatch",
      "product:bin/gstack-review-plan",
      "metadata:changelog.d/codex-skill-mentions-use-listed-names.md",
      "metadata:changelog.d/round-2-review-runs-after-a-gate-fix.md",
      "product:docs/hosts.md",
      "product:docs/review-policy.md",
      "product:docs/skill-routing-and-contracts.md",
      "product:docs/skills.md",
      "product:docs/troubleshooting.md",
      "product:land-and-deploy/SKILL.md.tmpl",
      "product:lib/gate-outcomes.ts",
      "product:lib/review-depth.ts",
      "product:lib/review-targeted-delta.ts",
      "product:modernize/SKILL.md.tmpl",
      "product:office-hours/SKILL.md.tmpl",
      "product:qa/SKILL.md.tmpl",
      "product:review/SKILL.md.tmpl",
      "product:runaway-stopper/SKILL.md.tmpl",
      "product:ship/SKILL.md.tmpl",
      "test:test/fixtures/review-pins/files/lib/review-depth.ts.json",
      "test:test/fixtures/review-pins/replays/GST-3187.json",
      "test:test/gen-skill-docs.test.ts",
      "test:test/modernize-skill.test.ts",
      "test:test/review-depth.test.ts",
      "test:test/review-risk-token-paths.test.ts",
      "test:test/review-round2-plan.test.ts",
      "product:write-skill/SKILL.md.tmpl"
    ],
    "merge_commits": 1,
    "range_diff": "1:  0894423 = 1:  0894423 Codex /review with gpt-6.1-sol counts the second-model pass it already ran (GEVA-562)\n2:  e006027 = 2:  e006027 Codex /review counts only Codex's own spawn_agent as a dispatch (GEVA-562)",
    "covers": [
      "pre-landing review",
      "adversarial review",
      "gate integrity",
      "documentation audit"
    ],
    "reruns": [
      "CI at head (gstack-ship-queue ready requires it); test and helper code changed since the reviewed head runs there unreviewed"
    ],
    "ancestry": "linear",
    "trees": [
      "e88e964e44a6ce4f1ddc34c274451c5ac1026978"
    ]
  }
}
{"head":"e0060272c10600d28ef3ee99a902de36354a3cb8","version":"1.101.0.0","gstack_version":"1.101.0.0","run":"ship-4526-e006027","review_loop":{"passes":1,"blocking_found":0,"deferred":0,"noted":2,"fixed":0,"scope":"frozen","rounds":1,"dispatches":{"specialist":1}},"depth":"light"}

…y ran (GEVA-562)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 2 commits November 19, 2026 13:07
…-562)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

# Conflicts:
#	docs/skill-routing-and-contracts.md
@capy-ai
capy-ai Bot merged commit 18e24a2 into main Nov 19, 2026
53 checks passed
@garrytan garrytan mentioned this pull request Nov 19, 2026
capy-ai Bot added a commit that referenced this pull request Nov 19, 2026
- `gstack-artifact validate` no longer passes a run.json that gstack-autoplan never wrote. (#4494)
- /ship's changed-number check no longer stops on tests that only share a word with the change (#4478). (#4496)
- On Capy, /ship no longer edits or comments on pull requests as you. (#4518)
- The destructive-command check stops asking about scratch deletes and catches the real damage it used to miss. (#4513)
- The destructive-command check reads commands more carefully, closing gaps found after the last release. (#4525)
- The Codex readiness check and the second-model review now pick the same model for a custom Codex provider (GST-3089). (#4491)
- gstack's Codex hooks work when your Codex folder's path has a `(`, `&` or other shell character in it. (#4528)
- Uninstalling gstack keeps your own Codex hooks trusted. (#4528)
- Codex hook setup matches Codex more closely. (#4528)
- On Windows, setup copies the Claude Code hook code that gstack's Codex hooks import (#4528)
- /review's checklist on Codex reads shell commands more carefully. (#4528)
- On Codex, gstack now suggests the right skill for a prompt and stops destructive commands before they run. (#4512)
- On Codex, /review is now held once when it tries to finish without its adversarial pass or its report, as it already is on Claude Code. (#4522)
- On Codex, gstack's skill suggestion now names the skill the way Codex lists it. (#4523)
- On Codex, gstack's suggested prompts and docs now name skills the way Codex lists them. (#4524)
- /review on Codex with gpt-6.1-sol is no longer held back for a second-model pass it already ran. (#4526)
- On Codex, office-hours, plan-design-review and gstack-upgrade work again when you name them. (#4510)
- gstack knows when Codex is running unattended. (#4510)
- Codex reviewers start with a clean context. (#4510)
- Docs no longer say Codex has no hook support. (#4510)
- A command whose output was cut off no longer counts as a finished read. (#4492)
- A folder marked as published data can no longer hide source code or agent files from /ship's review. (#4516)
- Correction to the on-by-default helper hooks entry: the hook still routes only to /review, /qa and /verify, so skills you've installed yourself aren't suggested yet. With gstack's skills it routes 14 of 60 real bug reports (was 17), and long prompts are no longer skipped. (#4505)
- /sync-gbrain's read check now works when the first page in your code source is over 64 KB. (#4497)
- Free-suite CI no longer fails before any test runs when Ubuntu's package mirror is slow. (#4530)
- Syncing a second repo no longer makes the first one ask for /sync-gbrain again. (#4532)
- gstack's helper hooks are now on by default in Claude Code, OpenClaw and Hermes, and the prompt-routing hook fires on real bug reports. (#4503)
- `gstack-pregate clear --receipt` accepts a remote gate run from before a merged release when the release only regenerated a declared release output. (#4483)
- The second-model review now keeps the reviewer's own result, and reports a used-up budget as a budget stop. (#4515)
- When the reviewer wrote its review to the result file it was asked for, Codex then replaced that file with its short closing message ("Done, I wrote the review"), so the review was thrown away and reported as missing its markers. (#4515)
- When the provider refuses the call with HTTP 402 Payment Required (usually a metering proxy that hit its allowance, or an account out of credits or quota), gstack used to record only "exit 1". (#4515)
- The pre-commit /verify check now fails a check that times out, and its checks share one time budget (GST-3201). (#4527)
- /ship's review collector no longer reads a reviewer's own scope list or list of passed checks as findings (GST-3201). (#4527)
- Merge-queue stack tests fail when git cannot list a PR's added test files (GST-3201). (#4527)
- /ship can now review results PRs that publish raw session data. (#4511)
- /ship's review pass works when the checkout you ship isn't the project's default checkout. (#4507)
- A second review round no longer reports done when nothing was reviewed. (#4520)
- The prompt-routing hook now suggests skills you install yourself, and no longer sends ordinary bug reports to /qa (GST-3123). (#4506)
- gstack's script runner and merge-queue checks now also refuse cut-off output. (#4502)
- A failed second-model review reports the cause that actually stopped it, and the runner never deletes its own prompt. (#4519)
- `gstack-ship-queue ready` now fills in the trial merges and the second-model trail read instead of printing lines to fill. (#4529)
- Five small ship-tool fixes. (#4529)
- A ruling the owner gives while a `/ship` run is open can be relayed. (#4529)
- /ship no longer kills a working test run as stalled when the test command starts its workers in their own session. (#4533)
- /ship no longer stops a review because the state-machine reader started its reply the way its instructions say. (#4504)
- When Claude Code runs /verify right before a commit, it now gets a quick check instead of the full fix loop (GST-3150). (#4508)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant