Skip to content

Review collector reads JSON readers as JSON only and re-asks prose it can't read (GST-2237) - #4269

Closed
garrytan wants to merge 2 commits into
mainfrom
capy/review-collector-reads-json
Closed

garrytan wants to merge 2 commits into
mainfrom
capy/review-collector-reads-json

Conversation

@garrytan

Copy link
Copy Markdown
Owner

The review collector now reads the five JSON readers' replies as JSON only, re-asks a prose reply it could not read instead of calling it clean, and counts an unknown severity as a blocker. This keeps /ship's review gate from inventing blockers that cost owner rulings (#4208, ALPH-93, ALPH-11) and from passing a real finding it dropped.

JSON readers collect only JSON. The specialists, the batched specialist, the Red Team, the targeted reader and fix-check already must answer with one JSON object per finding or NO FINDINGS. Their prose (coverage notes, a summary of their own JSON) is no longer read for severity words, so "Critical paths (checkout, refund) are covered" or "I found nothing at the blocking floor" can't become a CRITICAL. Their JSON now collects in any layout: pretty-printed (a lowercase "critical" used to vanish), nested, in an array or code fence, or cut off mid-line.

Unread prose fails closed. A prose voice's reply whose only clean marker is the Recommendation: line, that collected no finding but names a severity word, FIXABLE, a file:line or VERDICT: FINDINGS, is now COLLECT_NOT_REVIEWED ... reason=prose_not_collected and verify reports NOT_REVIEWED. The audit's 1. [Fixable] lib/pay.ts:40 charges the card twice… reply used to count as a clean review. Unstructured prose in such a reply collects only next to a structured finding, so the outside voice's "Critical paths … are covered" reply is re-asked instead of becoming a blocker that judgment can't lower. An explicit NO FINDINGS stays clean, and in that reply the native voice's FIXABLE counts only as a tag, so "the two FIXABLE items from pass 1; both are fixed" no longer collects a CRITICAL.

What blocks. A label no vocabulary ranks (CRITICAL (blocks shipping), the brief's own wording, or BLOCKER) counts as CRITICAL in the ledger count and in judge, replacing rank(label) ?? 0. The outside voice's High and P1 block, as its brief says. Blocks: no keeps its GST-1031 meaning.

No reviewer prompt or brief changed. docs/review-policy.md's collector paragraph describes the new contract.

Replay of the saved ledgers. 92 of the 95 drive ledger directories have a ledger, holding 219 reader outputs. 178 of those have their raw reply saved and were collected with main (dcebc44) and with this branch. 3 verdicts change: GSTA-189 red-team (a prose summary row dropped, still a review, 0 blockers), GST-986 r2 fix-check (phantom CRITICAL "I found nothing at the blocking floor" dropped, blockers 1 → 0), and #4212 p1 adversarial-native ("NO FINDINGS. Range…" with a "(not a finding, Low, Blocks: no)" note is now re-asked instead of collecting a Low). The 41 outputs with no saved raw reply can't be replayed. Their merged rows hold 3 more JSON-reader prose rows (GSTA-197 run2), one of them the second phantom CRITICAL. No saved row carries an unranked label or an outside-voice High or P1.

Agent notes

Open in Capy

… can't read (GST-2237)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 10, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

A JSON reader's finding whose object does not parse (a trailing comma, or the severity key
below the opening brace) now collects at the severity it names instead of being dropped.

Review-Finding: p1/specialist-batched#1 reader=testing
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai

capy-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Closing without merging. The JSON-only collection drops CRITICAL findings that main's prose fallback still catches, in four reproduced shapes: a trailing comma in a pretty-printed object, a severity key below the opening brace, a trailing comma in a findings wrapper, and a stray brace before a one-line array. A retry after 2026-11-21 will keep the prose fallback whenever an object fails to parse but names a severity.

@capy-ai capy-ai Bot closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant