Skip to content

fix(eval-plan): pilots with a seat-metered arm can be approved, and each arm is projected at its own price (#3653) - #3751

Closed
garrytan wants to merge 10 commits into
mainfrom
gst-566-eval-plan-seat-metered
Closed

garrytan wants to merge 10 commits into
mainfrom
gst-566-eval-plan-seat-metered

Conversation

@garrytan

@garrytan garrytan commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner
{
  "schema_version": 1,
  "pr": 3751,
  "head": "15142402632cf2f4bd6291c9aadff25d43469a6e",
  "version": "1.92.2.0",
  "base": "main",
  "gate": {
    "evalplan": {
      "pass": 640,
      "fail": 0,
      "skip": 0
    },
    "guards": {
      "pass": 13,
      "fail": 0,
      "skip": 0
    }
  },
  "session_kind": "unattended",
  "head_tree": "1037735809b0e95dda5a59b138f4e80f63143704",
  "product_patch_id": "fa06cfd6628fbece4695e949aab4ea3540220fc1",
  "gstack_version": "1.91.379.0",
  "review_depth": {
    "depth": "standard",
    "class": "code",
    "runtime_lines": 97,
    "rules": "150435b0a543019a",
    "base": "4eff47c5cf8464c1537194817ffa8984139d3ca0",
    "registry": "4eff47c5cf8464c1537194817ffa8984139d3ca0",
    "gate_scan": {
      "tagged": 0,
      "head": "15142402632cf2f4bd6291c9aadff25d43469a6e"
    },
    "reasons": [
      "runtime=97>50"
    ]
  },
  "pregate": "requires-remote cleared @1037735809b0 (1 checks, current for 15142402632c)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/41250874342",
    "windows-free-tests.yml/windows-free-shard: https://github.com/garrytan/gstack/actions/runs/41250874501",
    "pregate.json/remote_gate: remote-gate tree=1037735809b0 tests=18185"
  ],
  "issues": {
    "fixes": [
      3653
    ],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "updated",
    "scope": "docs/, README.md, AGENTS.md, CONTRIBUTING.md"
  },
  "review_loop": {
    "passes": 2,
    "blocking_found": 0,
    "deferred": 0,
    "noted": 2,
    "fixed": 1,
    "scope": "frozen",
    "fix_checks": 4,
    "dispatches": {
      "specialist": 2
    }
  }
}

What changed

pilot report with one priced arm and one arm on a seat (no per-call charge): main refuses approval and doubles the projection; this branch projects the priced arm alone and is ready to approve

The same pilot (2 attempts each on gpt-5.4-mini at $0.02 and on gpt-6-astra settled unknown, 100 items, $5 cap) reported on main and on this branch. Main can never approve it and projects $4.00; this branch projects the priced arm at $2.00, lists the seat arm on its own line and says it fits.

Before/after as text
--- Before · main 3bfe566
+++ After · this branch 2321dc5
-PILOT: n=2 arms=gpt-5.4-mini,gpt-6-astra settled=2 unknown=2 status=reserved
+PILOT: n=2 arms=gpt-5.4-mini,gpt-6-astra settled=2 unknown=0 status=reserved
-| $0.0400 | $0.0000 (0 open) | 2 ($0.8193 at reservation) | $1.2689 (1 worker × $0.4096) | $5.00 | $0.0000 |
+| $0.0400 | $0.0000 (0 open) | 2 ($0.0000 at reservation) | $0.0597 (1 worker × $0.0197) | $5.00 | $0.0000 |
-PILOT_PROJECTION: items=100 per_item=$0.0200 sd=$0.0000 projected=$4.0000 band=[$4.0000, $4.0000] cap=$5.00 verdict=unknown
-APPROVAL: not ready — unknown charges make the band meaningless; settle, then rerun report
+PILOT_ARM: arm=gpt-5.4-mini n=2 per_item=$0.0200 sd=$0.0000 projected=$2.0000 band=[$2.0000, $2.0000]
+PILOT_SEAT_METERED: arm=gpt-6-astra calls=2 settled=2 (declared seat-metered: no incremental charge, outside the cap and the projection; usage-limited)
+PILOT_PROJECTION: items=100 known_arms=1/1 per_item=$0.0200 sd=$0.0000 projected=$2.0000 band=[$2.0000, $2.0000] cap=$5.00 verdict=within
+APPROVAL: run 100 items at projected $2.0000 (95% band $2.0000–$2.0000) under cap $5.00: fits; ask the owner before any further priced call

Why

#3653: a paid eval that compares a priced API model with claude -p on a subscription could never be approved, because the subscription arm settles unknown and pilot report refused any pilot with unknown charges. The same report also pooled every arm into one mean, so a priced arm's cost was multiplied across arms that cost nothing, and two arms at different prices got a band wide enough to fail a pilot that fit the cap. Now a subscription arm is declared up front with pilot start --seat-metered <arm>, and each priced arm is projected from its own attempts.

Blast radius

Only gstack-eval-plan pilot start and pilot report and the /eval-plan skill prose change; the spend ledger (lib/spend-ledger.ts) and the dry run are untouched, and a pilot started without --seat-metered keeps every arm priced, so existing pilots only see the per-arm projection (script ran: the 640-test eval-plan lane, plus the base/head reports above). A seat arm still needs a lib/pricing.ts row, and an attempt on it that settles a real charge counts as spent and puts the arm back in the projection, so the cap can't be dodged by declaring an arm seat-metered (walked failure path and pinned in test).

Approvals

No approvals recorded for branch gst-566-eval-plan-seat-metered.

Summary

"A seat-metered arm makes every pilot unapprovable" was the whole of #3653's first half; its second half was the pooled mean. The fix projects each arm on its own and sums them, adds --seat-metered so an arm with no per-call charge holds none of the cap and gets its own PILOT_SEAT_METERED: line, and fixes the declaration per --out so a restart can't quietly change it. The skill also tells the operator to stop a seat-metered run at its first quota or rate-limit error, with no retry. That stop rule is the idea from #3273 by @merit-blake, rewritten here as skill prose; none of #3273's code was carried over (its batch runner, environment key handling and workflow edit were declined).

Security read: nothing here makes network calls, reads environment variables or secrets, widens permissions, edits workflows or install scripts, or adds a dependency.

Test Coverage

7 new tests in test/pricing-eval-plan.test.ts, all in the existing priced-pilot suite:

  • each arm projected on its own, so arms at different prices that fit the cap are not told they exceed it
  • a priced arm's mean is never multiplied across an arm settled unknown
  • a declared seat-metered arm holds none of the cap, stays out of the verdict and gets its own line
  • a charge on a seat-metered attempt still counts as spent; the declaration must name listed arms and is fixed for its --out
  • an all-seat pilot spends nothing and fits; one priced attempt is not enough for a verdict
  • a seat-metered arm still needs a price row, as the dry run does (guard, passes at main by design)
  • pilot start --seat-metered reaches the ledger through the CLI

Tests: 1571 test files in the repo, 0 new files; 7 tests added to an existing file.
Regression proof — fails at HEAD's base code: 6 of 7 fail against main 3bfe566's lib/eval-plan.ts (26 pass, 6 fail with this branch's test file copied onto main); the price-row guard passes at main by design · passes after fix: yes (32 pass, 0 fail).
Tests ran: evalplan 640
Tests ran: guards 13

Pre-Landing Review

Review found and fixed two cap bypasses before this PR opened; nothing blocking is left.

  • Pass 1 (specialists and the cross-family outside voice): an informational test gap (nothing pinned that an unpriced seat arm still needs a price row), fixed in b6d4744; a Low on the summed deviation, fixed in 71f116c.
  • Pass 2: the outside voice found a High. Re-running pilot start in the same --out without --seat-metered reused the $0 reservations, so priced calls got past the cap. Fixed in 8c9b61c: a pilot's seat set is fixed when it starts, and a change needs a new --out. The same commit fixes its Medium (a seat arm still needs a price row, because the dry run requires one) and the adversarial Medium (a real charge on a seat attempt could not be recorded; it now counts as spent).
  • Adversarial pass 3: a High. A seat arm that settled a real charge was counted as spend but left out of the projection, so the verdict could approve a run its own data said would exceed the cap. Fixed in 3c9eeca: that arm is projected as priced.
  • A fix-check and an adversarial pass at 3c9eeca (granted by GStack Minder under crew rule 23), and again at tree e4d7923 after the prose trim in 0dce80e: no blockers.
  • Base merge 1514240 (GStack Minder's rule-19 grant): v1.92.2.0 evals: no-gstack arms for the model panel, arm-benchmark, safety and routing evals #3708 (no-gstack arms) changed the same bin/gstack-eval-plan help line and docs/skills.md paragraph; both sides are kept. Merged, eval-plan/SKILL.md measures 5021 tokens against its 4887 eager ceiling: each PR fit alone, and v1.92.2.0 evals: no-gstack arms for the model panel, arm-benchmark, safety and routing evals #3708 used the headroom. The ceiling goes to 5021 (+134) in test/fixtures/context-budget.json, and only that one entry changes. A fix-check on the resolution hunks found nothing (tree 1037735).
  • Skipped on purpose: projected_usd in the JSON report is a subtotal when an arm has no numeric settles. eval-plan: seat-metered pilot arms remain unapprovable; the mixed-arm projection extrapolates known costs across unknown arms #3653 asks for that subtotal, the text labels it known_arms=k/m, and the verdict stays unknown.

REVIEW_LOOP: passes=2 blocking_found=0 deferred=0 noted=2 fixed=1 scope=frozen fix_checks=4 dispatches=2
Gate edits: one test file extended (test/pricing-eval-plan.test.ts, additions only); no test, threshold, budget or CI gate relaxed. The first remote pregate caught a context-budget overrun, fixed in 0dce80e by trimming the skill prose, not by raising the ceiling.
Outside review: the cross-family outside voice ran in passes 1 and 2; its High and Medium are fixed above.

Review notes

Non-blocking findings, listed and not looped (crew rule 23 floor):

  • The "not ready" hint can tell the operator to rerun with --seat-metered <arm> for an arm that is already declared, when that seat arm also settled a real charge. Display only.
  • Open attempts on a seat arm that has settled a real charge still hold $0 reservations, so the worst-case column under-counts that arm until they settle. Spent money is still counted, and an overrun still exits 1.
  • An all-seat pilot reports "fits" at a $0 projection before any attempt settles. Deliberate and pinned in tests; a wrong seat declaration shows up once a real charge settles.
  • The PILOT: line's unknown= excludes seat arms, while the spend table's "unknown charges" includes them (visible above: unknown=0 beside 2). Display only.
  • Nothing checks that a declared seat arm really runs on a subscription. A wrong declaration shows up when its first real charge settles and the arm moves into the projection.
  • The fixed-declaration check reads pilot.json outside the ledger lock, so two concurrent starts in one --out, or a start that failed partway through reserving, could mix seat sets. Both need an unusual earlier event.
  • The summed band and deviation assume arms move together, so they lean high (could say exceeds for a run that fits). That is the safe direction for a cap.

Exploratory QA

Field Value
Date / branch / revision 2026-10-31 / gst-566-eval-plan-seat-metered / 2321dc5 (clean tree); original probes ran before fixes 8c9b61c and 3c9eeca and were re-run at 2321dc5
Caller / authority / depth ship; report-only probes in a private fixture root; smoke
Surfaces / scope CLI: gstack-eval-plan pilot start / settle / report
Runtime / native tools bun 1.4.4; bun --no-env-file --config=/dev/null bin/gstack-eval-plan
Fixture ownership / destinations ~/.capy/work/shipA/qa-probe/fx* (owned, synthetic dry run; no provider calls)
Probe budget / stop reason 12 smoke probes, 4 used in the original run; complete

Contract outcomes (re-run at 2321dc5)

Contract and source Probe Expected → observed Outcome
seat arm reserves none of the cap (#3653) pilot start --arms gpt-5.4-mini,gpt-6-astra --n 2 --seat-metered gpt-6-astra seat attempts at $0.0000 → estimate=$0.0000 ×2, status=reserved pass
a real charge on a seat attempt counts as spent (8c9b61c) pilot settle --attempt pilot-gpt-6-astra-1 --usd 0.5 settles and counts → SETTLED … spent=$0.5000; the report then projects gpt-6-astra as priced pass
--seat-metered must name a listed arm pilot start --arms gpt-5.4-mini --seat-metered gpt-6-astra refusal, nothing written → exit 2, no --out dir pass
seat set fixed per --out second pilot start on the same --out without --seat-metered refusal → exit 2, "needs a new --out" pass
per-arm projection and approval 2 priced + 2 seat settles, pilot report --total 100 one PILOT_ARM, one PILOT_SEAT_METERED, verdict within → as shown above pass

Discoveries

  • checkpoint 002 — the seat arm reserved $0, so the next probe tried to settle it with dollars.
  • checkpoint 003 — at that tree the dollar settle was refused; review later decided a real charge must count as spent (8c9b61c), so this result is superseded history. The re-run above shows the current behavior.
  • checkpoint 004 — an unlisted seat arm is refused with nothing reserved; next probe read the report.

Coverage limits

No live provider or claude -p call ran; costs are synthetic settles. No new tests came from QA; the behaviors above are each pinned by a unit test listed under Test Coverage.

Design Review

No frontend files changed — design review skipped.

Eval Results

No paid eval ran in this ship. The skill prose change was checked by the free gates: gen-skill-docs freshness, context-budget ratchet, parity suite and the skill size guard. The paid screen runs at gstack-ship-queue ready --screen.

Plan Completion

Plan: GSTA-218 (approved). #3653's per-arm projection and --seat-metered (D1) and #3273's stop rule as prose only (D2) are done here; #3651 (D3) ships as its own PR.

Issues

Fixes #3653

Verification Results

The plan's checks: failing tests first (6 fail at main, pass here), the #3653 repro (~/.capy/work/repro3653, now approvable), and the QA probes above: all PASS. Remote pregate on Ubicloud: 18185 tests, 0 failed, at tree 1037735 (merge commit 1514240). Two remote-only lanes (macos-named-regressions, windows-free-shard) clear from this PR's Actions runs.

TODOS

No TODO items completed in this PR.

Documentation

Status: updated. One factual edit to docs/skills.md: a seat-metered arm stays out of the projection only until an attempt settles a real charge, matching lib/eval-plan.ts.

Scope: base 3bfe566, head 0dce80e, no staged/unstaged/untracked changes; committed changes in bin/gstack-eval-plan, lib/eval-plan.ts, eval-plan/SKILL.md(.tmpl), docs/skills.md, changelog.d entry and the test file. Docs touching eval-plan or the pilot subcommands were checked.

Documentation health:

  • docs/skills.md: updated
  • eval-plan/SKILL.md.tmpl: current
  • eval-plan/SKILL.md: current (generated; regeneration not checked)
  • changelog.d/gst-566-eval-plan-seat-metered.md: current (not edited)
  • docs/eval-models.md: current
  • docs/troubleshooting.md: current

Coverage debt: the EVAL_BUDGET_EXHAUSTED entry in docs/troubleshooting.md omits --seat-metered. Diagram drift: none found; Step 1.5 diagram check not run.

Test plan

  • bun test test/pricing-eval-plan.test.ts test/eval-plan-no-gstack-arm.test.ts test/eval-plan-prereg-diagnostics.test.ts test/eval-models.test.ts test/catalog-budget.test.ts test/gstack-skill-start.test.ts test/auq-error-fallback-hook.test.ts test/context-budget-ratchet.test.ts test/gen-skill-docs.test.ts: 640 pass, 0 fail at 1514240
  • bun test test/parity-suite.test.ts test/module-size-ratchet.test.ts test/skill-md-size-guard.test.ts test/spawnsync-timeout-tripwire.test.ts: 13 pass, 0 fail at 1514240
  • UBI_OWNER=gsta218 gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands --base main: 18185 tests, 0 failed (tree 1037735)

🤖 Generated with Claude Code

garrytan and others added 9 commits October 31, 2026 18:49
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…d CLI wiring

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…mmed band (arms share rows)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…ut, price row still required, a real charge settles as spent)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…d arm

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…s no $N token

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…a charge

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Oct 31, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

E2E Evals: ✅ PASS

[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 0/0 · behavior 0/0 · judge 0/0 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 0m00s · cost $0.00

0 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $0 rule/judge cost | reconcile exit: 0

Profile: pr / mode pr; selected behaviors: 0, judges: 0. Deferred to scheduled/release coverage: 11 behaviors and 2 changed prompt files. Deferred checks did not run and receive no PR-pass credit.

Rule and judge shards
Shard Automated result Manual/unscored Executed Reused Status Cost
Fail-closed reconciliation
[test:paid] report: 1/1 slices, 0 planned shards, tier=gate, attempt 1
[test:paid] coverage: profile=pr mode=pr; selected E2E=0, judges=0
[test:paid] deferred: 11 broad behaviors, 2 changed prompts without quick live coverage; these are not PR passes
[test:paid] collector final outcomes: 0 executed, 0 reused; 0 passed, 0 failed, 0 manual accepted (unscored; no score-cache credit) (0 attempt records from 0 collectors; every record counts)
[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 0/0 · behavior 0/0 · judge 0/0 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 0m00s · cost $0.00
[test:paid] exercised-path map (shadow, non-blocking): 0 MAP_GAP line(s) over 0 executed case(s) with recorded commands
[test:paid] report: every planned shard accounted and passed

Sliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry.

Dispatched narrow screen (gstack-ship-queue ready --screen) of merge commit 4e4621d90f327a333b366518f84de038346b5a99.

Resolves bin/gstack-eval-plan help and docs/skills.md by keeping both sides (#3708's results --check and this PR's per-arm report and --seat-metered text). Raises the eval-plan eager context ceiling 4887 -> 5021 (+134 tok): #3708 and this PR each fit alone; together eval-plan/SKILL.md measures 5021.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai

capy-ai Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Frozen in wave autoplan-e97beb7bf577 (branch wave/autoplan-e97beb7bf577) at 1514240. Push nothing here: a moved head drops this PR from the wave. Fixes go on the wave branch through its integrator.

capy-ai Bot added a commit that referenced this pull request Nov 2, 2026
* fix(eval-plan): per-arm pilot projection and --seat-metered arms (#3653)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): guard tolerates Claude Code's cwd-reset footer and recovers after an unreadable init (#3651)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (journal reader strips the cwd-reset footer; an init with no text block no longer locks the session)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: journal reader skips a failed or unreadable init, as invocation() does

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(autoplan): --ceo-only runs the CEO review alone with zero questions (GST-537)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(eval-plan): all-seat pilot, single-attempt arm and --seat-metered CLI wiring

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(eval-plan): summed deviation adds arm deviations, matching the summed band (arms share rows)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (test pins an unpriced seat-metered arm)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (seat-metered declaration fixed per --out, price row still required, a real charge settles as spent)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: classify autoplan-ceo-only in the Codex host metadata

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: a seat-metered arm that settles a charge is projected as a priced arm

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: select autoplan-ceo-only on the skills it runs; say it needs Claude Code

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(skills): /eval-plan --seat-metered and per-arm projection

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(eval-plan): skill prose stays under its context budget and carries no $N token

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: run autoplan-ceo-only in the marathon lane

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(skills): a seat-metered arm leaves the seat set once it settles a charge

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: changelog entry for /autoplan --ceo-only

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): init_failed covers unreadable init output and the cwd-reset line (#3651)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: count the autoplan-ceo-only paid file in the shard inventory

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(autoplan): #3651 tests pass the test-code type ratchet

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(dx): CAP finding titles carry a one-line description (GBRA-223 claim 2)

The DX error contract told both autoplan DX voices to title capped findings
exactly CAP4:<path>:<item>, so gates, tasks and reconcile saw a bare code.
The title is now CAP4:<path>:<item> <what fails>; the grader already parses
that shape.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): reconcile pairs same-file findings that carry no line (GBRA-223 claim 1)

Plan reviews name files, rarely lines, and reconcile associated by location
only when both findings had a line, so findings both voices raised on one
file were filed new and native-only. A same-file pair with a missing line is
now associated as unverified (basis file), never confirmed; a CAP code no
longer counts in title comparison. DRIVER.md names submit --reconcile.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): open Critical and High findings reach the gate as apply items (GBRA-223 claim 3)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap binds on every unattended host; --estimate-usd persists; DRIVER names --usage-usd (GBRA-223 claim 4)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): run binds complete api results, names a truncated response, --max-tokens; DRIVER documents run (GBRA-223 claim 6)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(autoplan): carry an item-by-item answer when the later gate asks nothing new (GBRA-223 claim 7)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(dx): trim the compact error contract so plan-devex-review stays under its size pin

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (p1b applies finding fixes, api attempts reserve no host default, cross-file title pairing, paired severity counted)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: review fixes (priced api reservations, no file-only twins, two-pass pairing, partial replies keep p1-dependent defaults open, outside refusal degrades)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* P7: record whose words answered a gate (--words-by agent:<id>), never carry an agent's own words, import each row under its own answerer (GEVA-2)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: review pass 2 (p1b needs --rec-hash after recommend; api calls send the output limit they reserve for)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: fragment for the /autoplan field-report wave (GST-672)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: captain record notes --words-by agent attribution

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: follow finding apply items in approval-class and Captain flow tests; bind run-api limit to DEFAULT_MAX_TOKENS

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* wave autoplan: lower the gemini plan-devex-review pin to its exact size after #3781's DX trim

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): CAP titles on different paths never match by title in reconcile or confirm rounds (wave review)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a truncated api call settles its cost and a retry is admitted afresh against the cap

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): an applied finding stays open until the revised plan is reviewed; a shared location alone never twins two findings

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(approval): a plan gate answered in an agent's own words is not owner approval

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate): a new answer replaces the earlier answer's authorship and words

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a failed api call with no usage keeps its attempt open; reservations price the bound prompt (wave fix-check)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap fails closed: only a call refused before sending keeps its attempt; a sent call with unknown usage settles at its reservation

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): the 16384 output default applies to both providers; changelog for the wave's integrator fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): how run settles a failed api call; changelog scopes it to the native voice

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap fails closed on every exit of run; an unpriced model reserves the default once a cap is set

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): every sent call settles; an unpriced model under a cap reserves the default

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): DRIVER and help say a cap reserves in any session kind and how run settles a sent call

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai

capy-ai Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Landed in wave #3788 as v1.92.50.0 (90a170c), merged from this PR's pin 1514240. Integrator changes touching this PR's files: none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

eval-plan: seat-metered pilot arms remain unapprovable; the mixed-arm projection extrapolates known costs across unknown arms

1 participant