Skip to content

Keep a numbered review finding's closing paragraph on that finding after a nested list (GST-2174) - #4237

Merged
capy-ai[bot] merged 3 commits into
mainfrom
capy/keep-a-numbered-review
Nov 9, 2026
Merged

capy-ai[bot] merged 3 commits into
mainfrom
capy/keep-a-numbered-review

Conversation

@garrytan

@garrytan garrytan commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

/ship's review gate no longer blocks a merge on a Low finding just because the native reviewer wrote it as a numbered item with a nested list and a closing paragraph. This serves gstack's goal of a review loop that stops only on real blockers.

What changed

Before: the closing paragraph split off and the Low finding blocked /ship as CRITICAL. After: one Low record, no blocker.

Before: the closing paragraph split off and the Low finding blocked /ship as CRITICAL. After: one Low record, no blocker.

Before/after as text: Before: the closing paragraph split off and the Low finding blocked /ship as CRITICAL. After: one Low record, no blocker.
--- Before · before.txt · captured output
+++ After · after.txt · captured output
 $ collect adversarial-native reply (numbered FIXABLE item, nested list, closing "Severity: Low. Blocks: no.")
+records: 1
+  Low          1. **FIXABLE. The retry queue drops a job on shutdown** (`queue.ts`). 
-records: 2
-  CRITICAL  <- blocks /ship  1. **FIXABLE. The retry queue drops a job on shutdown** (`queue.ts`). 
-  Low          So a job queued in the last second is lost, which is rare. Severity: L
 

Why

An alphaclaw /ship run stopped on a finding its own reviewer rated "Severity: Low. Blocks: no." The reviewer wrote six findings and the gate collected nine, because the closing paragraph of each numbered item split off into its own record and left the item's FIXABLE label to count as CRITICAL. Now that paragraph stays on its finding, so a Low FIXABLE collects at Low and /ship keeps going.

Blast radius

Only parseLabelled in lib/review-ledger.ts changed, and every reviewer's output goes through it. The new join runs only for the native reviewer's numbered items with no fields of their own and stops after the first closing paragraph, so other readers' output parses exactly as on main. The 22 test files that import the ledger pass (216 tests in the 14-file collector lane, run at 3159fbe and again at the final tree), and the new test fails on the previous head.

Summary

The native reviewer often writes a finding as 1. **FIXABLE. …**, a short sub-list, then a closing paragraph that holds Severity and Blocks. The collector treated that paragraph as a new finding, so the item kept no fields and its FIXABLE label made it a blocker. The collector now keeps the indented lines and the first closing paragraph on the numbered item, so the paragraph's fields bind to it.

The first review round found three ways the first version of that join could hide a real blocker. It took in a separate bold-led finding, it kept absorbing later paragraphs, and it applied to every reader and could lower an item that names CRITICAL. The join is now narrow. It applies only to the native reviewer, ends after the first closing paragraph, never takes in a bold-led paragraph or a verdict line, and never lowers an item that names CRITICAL or P0. A new test covers each case.

Test Coverage

All new code paths have test coverage (deterministic coverage audit: the one changed runtime file has a changed test naming it).
Regression proof: test/review-native-nested-list.test.ts fails at main f75ce6f (2 of 2), and its third test fails at 1109bf5, the head before the review fixes. All three pass now.
Tests ran: collector 216

Pre-Landing Review

Pre-Landing Review: 9 issues — 0 auto-fixed, 3 asked (3 fixed, 0 skipped), 0 deferred, 6 noted

Round 1 (full panel: batched testing/security/state-machines specialist and Red Team) found three blockers in the join, all fixed in 3159fbe:

  • a bold-led margin paragraph that starts a separate finding was absorbed, so its Severity: Low and Blocks: no lowered the FIXABLE blocker;
  • the join applied to every reader and could lower an item that names CRITICAL to an absorbed paragraph's Severity;
  • the join kept absorbing later paragraphs until a field came, so a separate finding's fields bound to the item.

Round 2 (the same panel on the fix range) found no blocker. The Step 11 voices (native adversarial reviewer, outside voice on openai/gpt-6.1-sol and the targeted money-path reader) ran on the final code tree.

REVIEW_LOOP: passes=2 blocking_found=8 deferred=0 noted=9 fixed=3 scope=frozen ledgers_read=4 dispatches=8

Gate edits: none detected (1 listed, 0 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)

Outside review: completed (host subagent, openai/gpt-6.1-sol), 1 finding, listed under Review notes. Targeted reader: NO FINDINGS.

Review notes

These are open notes, shipped after the two-round cap. None would fail a test, break production or lose data; the first two can make the gate block on a finding that is not real (it fails closed), and none can hide a real blocker.

  • lib/review-ledger.ts:281 (outside voice, native reviewer): an indented paragraph after the nested list uses up the one allowed join, so fields in a later margin paragraph split off and the item collects a false CRITICAL. The suggested fix (afterGap && !indented) reopens round 1's fail-open case and fails test/review-native-nested-list.test.ts, so it was not taken.
  • lib/review-ledger.ts:278 (native reviewer): a closing section of two margin paragraphs before own-line fields splits again, as it did before 1109bf5. Picking the better default needs a replay of real native outputs.
  • lib/review-ledger.ts:183 (native reviewer, round 2 Red Team): the comment says an item naming CRITICAL is never lowered, but Severity and Blocks on their own lines still lower it through the existing field-line path and ruling A's relaxed pass. That behaviour is unchanged since main; only inline fields in the joined paragraph are pinned.
  • lib/review-ledger.ts:279 (round 2 Red Team): the verdict-line exclusion also applies to indented lines, so an indented verdict inside an item splits off and takes the item's later field lines (a false CRITICAL, fail-closed).
  • lib/review-ledger.ts:301 (native reviewer): non-native readers no longer get the join, by design after round 1; their output parses as on main.
  • Collector artifacts: three reviewer prose paragraphs (round 2 Red Team scope note, native scope and closing notes) were collected as CRITICAL records; they state no defect.
  • Round 1 Red Team limit: one plain paragraph right after the nested list that carries its own Severity and Blocks: no still binds to the item; no parser rule can tell it from the intended shape.

Exploratory QA

Functional smoke, 3 probes on parseFindings against synthetic reviewer replies (verdict: pass):

  • the intended case (numbered FIXABLE, nested list, closing Severity: Low. Blocks: no.) collects one Low record: pass;
  • a second paragraph after the closing one stays its own finding and the FIXABLE stays CRITICAL: pass;
  • a specialist's numbered item naming CRITICAL is never joined to a later Low paragraph: pass.
    Limit: collect and ledgerBlockers are covered by the unit test, not by a QA capture.

Design Review

No frontend files changed — design review skipped.

Eval Results

No prompt-related files changed — evals skipped.

Review bot comments

No review bot comments.

Scope Drift

Scope Check: CLEAN

Plan Completion

Plan completion audit: not run (no plan is bound to this branch and no docs/designs/ file matches). Fix: add "Plan: " to the PR body, or run /autoplan.

Verification Results

No plan obligations: 0 PASS, 0 FAIL, 0 BLOCKED, 0 NOT RUN.

TODOS

No TODO items completed in this PR.

Documentation

Status: updated. One factual edit to docs/review-policy.md, no blockers.

Scope: base f75ce6f to head 3159fbe on capy/keep-a-numbered-review. Audited lib/review-ledger.ts, test/review-native-nested-list.test.ts and the new changelog.d/capy-keep-a-numbered-review.md fragment. Nothing was staged or unstaged.

Documentation health:

  • docs/review-policy.md: Updated. It now says the native reviewer's numbered item with a nested list keeps its first closing paragraph, so that paragraph's Severity and Blocks bind to the item (ALPH-93).
  • docs/troubleshooting.md: Current. Its 'Numbered findings' section is about verdict counting, not the collector.
  • changelog.d/capy-keep-a-numbered-review.md: Current (reviewed only).

Coverage debt: none. The collector rule is internal and has reference coverage in docs/review-policy.md. Diagram drift: none.

Test plan

  • bun test --timeout=60000 over the 14 collector files (review-native-nested-list, review-native-finding-heading, review-gst1031, review-ledger-invariant, review-closing-summary, review-ledger-verdict-line, review-fixable-class, review-negated-severity, review-prose-severity, review-scope-ledger-count, review-loop-binding, review-pass, review-dispatch, module-size-ratchet): 216 pass, 0 fail
  • Pre-push guards: typecheck, module-size and conflict-markers pass
  • CI is the full free-suite run at the final head

🤖 Generated with Claude Code

Agent notes

Refs GST-2174

Approvals

No approvals recorded for branch capy/keep-a-numbered-review.
Plan: exempt: bug fix
Plan check: exempt bug fix

REVIEW_LOOP: passes=2 blocking_found=8 deferred=0 noted=9 fixed=3 scope=frozen ledgers_read=4 dispatches=8
Gate edits: none detected (1 listed, 0 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)

  • Review plan: REVIEW_DEPTH full (risk:money path lib/review-ledger.ts), panel=all, pass 1 plus one full pass on the fix range, then Step 11 voices and the targeted reader on tree c09db559; the targeted reader read the final tree ba43705d again after the docs commit and returned NO FINDINGS. The round-2 plan rerun with the voices' findings open printed NEXT: stop at the round cap; the brief rules ship with open notes after round 2.
  • Docs: the Step 14.5 audit added one factual paragraph to docs/review-policy.md; one scoped reader read it clean.
  • The two review-dispatch.test.ts subprocess tests exceed Bun's 5 s default on a 4-core Capy machine, on main too; the lane runs them with --timeout 60000.
  • Thread GSTA-460.
{
  "schema_version": 1,
  "pr": 4237,
  "head": "340142fe78c500907770ccfe12662399ea9d2330",
  "version": "1.98.0.0",
  "base": "main",
  "gate": {
    "collector": {
      "pass": 216,
      "fail": 0,
      "skip": 0
    }
  },
  "session_kind": "unattended",
  "head_tree": "ba43705d0ea3c602255adb28b76fee0f643b5667",
  "product_patch_id": "3b41a32bc143f94b2479609ebf431f9edf9aca7e",
  "gstack_version": "1.98.0.0",
  "review_depth": {
    "depth": "full",
    "class": "code",
    "runtime_lines": 30,
    "rules": "66e5174161eecc2c",
    "base": "f75ce6fdcc14e6398b00645c1608e834d766a922",
    "registry": "f75ce6fdcc14e6398b00645c1608e834d766a922",
    "gate_scan": {
      "tagged": 0,
      "head": "340142fe78c500907770ccfe12662399ea9d2330"
    },
    "reasons": [
      "risk:money",
      "runtime=30"
    ],
    "risk": [
      "money"
    ],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @ba43705d0ea3 (8 checks, current for 340142fe78c5)",
  "requires_remote": [
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/42699924822/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=ba43705d0ea3 tests=36881 skipped=89 runs=42699924822/1 profiles=ci-reuse/free-tests heads=340142fe78c5 verified=2026-11-09T21:33:29.852Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "updated",
    "scope": "docs/, README.md, CLAUDE.md, AGENTS.md (docs/review-policy.md updated)"
  },
  "review_loop": {
    "passes": 2,
    "blocking_found": 8,
    "deferred": 0,
    "noted": 9,
    "fixed": 3,
    "scope": "frozen",
    "rounds": 2,
    "ledgers_read": 4,
    "dispatches": {
      "specialist": 6,
      "red-team": 2
    }
  },
  "review_targeted": {
    "classes": [
      "money"
    ],
    "rules": "8811fd300bf305f0",
    "brief_sha": "7f92a0d0a8319c3c0e076fb3b1a4689c45120b2be2c14dfc7d4cb03aafa9d6a5",
    "product_patch_id": "3b41a32bc143f94b2479609ebf431f9edf9aca7e",
    "status": "done",
    "findings": 0,
    "raised_by": "targeted"
  }
}

…ter a nested list (GST-2174)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 2 commits November 9, 2026 19:49
…CAL item

The join that keeps a numbered native finding's closing paragraph now ends after that first paragraph, skips bold-led margin paragraphs and verdict lines, applies only to the native voice's items, and never lowers an item that names CRITICAL or P0. Each of these let a separate finding's Severity and Blocks: no lower a FIXABLE blocker to Low (GST-2174).

Review-Finding: review/specialist-batched#1 reader=testing
Review-Finding: review/red-team#2 reader=red-team
Review-Finding: review/red-team#3 reader=red-team
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…entry

docs/review-policy.md now says how the collector keeps a native numbered finding's first closing paragraph after a nested list (GST-2174).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant