Repository navigation
v1.94.22.0 feat(skills): when-to-use descriptions, wider triggers and contract blocks for /investigate, /review, /qa, /verify (GST-1088 lane B) - #3909
Conversation
…locks for /investigate, /review, /qa, /verify (GST-1088 lane B) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
…s; bare gstack-skill-contract name until lane A ships it Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…llows the small-diff skip (review fix round) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… stop rules (Step 11 voices) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…CKER_SHIPPED flag Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This reverts commit cc1539c.
… qa parity cap Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
E2E Evals: ✅ PASS125 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $33.52 rule/judge cost | reconcile exit: 0 Profile: pr / mode Rule and judge shards
Fail-closed reconciliationSliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry. Dispatched narrow screen (gstack-ship-queue ready --screen) of merge commit |
… ships, and re-pin /review by the measured 240 bytes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…check have landed Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…al agent_dispatch marker in the contract resolver), cap comments cite gate rev 2, pins at the combined measurement Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…r (match alone gates it), shared-preamble cut check, changelog at post-#3909 sizes, pins at measured (review 56,716, ratio 1.2979) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…-resolved bin path (GST-1088 lane B follow-up) (#3957) * feat(skills): when-to-use descriptions, wider triggers and contract blocks for /investigate, /review, /qa, /verify (GST-1088 lane B) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(skills): keep core trigger phrases in /review and /qa descriptions; bare gstack-skill-contract name until lane A ships it Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(skills): Done-means list leaves out steps a host suppresses and allows the small-diff skip (review fix round) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: Done-means honors loadout section exclusions and the skill's own stop rules (Step 11 voices) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: the Done-means checker sentence follows an explicit CONTRACT_CHECKER_SHIPPED flag Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: cite GST-1088 gate rev 2 for the /review cap raises Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Revert "test: cite GST-1088 gate rev 2 for the /review cap raises" This reverts commit cc1539c. * fix: trim /qa's opening sentence by 4 bytes so the branch fits main's qa parity cap Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: flip CONTRACT_CHECKER_SHIPPED now that bin/gstack-skill-contract ships, and re-pin /review by the measured 240 bytes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: the changelog fragment says lane A's routing hook and contract check have landed Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(contracts): run gstack-skill-contract through the host-resolved bin path The Done-means check/waive sentence named a bare gstack-skill-contract, which a standard install does not put on PATH (fix-check finding on #3909 at 58aeee7). Render it through ctx.paths.binDir like every other gstack bin, with a failing-first test across every host. Re-pin /review by the measured +56 skeleton bytes and +24 chars per non-Claude host render. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: changelog fragment wording for the bin-path fix Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
What changed
/review now opens with a "Done means" list rendered from its
contract:block, so the agent knows what must exist before it stops.Before/after as text: /review now opens with a Done means list rendered from its contract: block, so the agent knows what must exist before it stops
Why
Untold, Opus 5.5 never opened /investigate in 252 trials (Run D, gstack-evals#140), and a when-to-use description raised GPT-6.1 Sol's open rate from 9.5% to 27.0% in the same run. Claude Code's catalog shows only a description's first sentence, so /investigate, /review, /qa and /verify now open with a "Use when ..." sentence. /investigate and /review also say up front what "done" means, so a run that skips the specialists, the adversarial pass or the report is visibly incomplete.
Blast radius
Every host render of these four skills changes, plus
gstack/llms.txt, the Hermes trigger lead and router lines inlib/harness-register.ts. That is safe because the generator owns all of it:gen:skill-docs --dry-runreports 0 stale files, and the catalog, context, host-config, size-guard and parity tests pass at 981a3ce (script ran). The Done-means list drops any item whose step the host or the loadout removes, so no render asks for a step it doesn't contain (red-team read all 13 renders, file read).Approvals
Approved: yes (record r-74843981c3d6). The full approval record stays private: this repository is public or its visibility is unknown.
APPROVALS_RENDER: redacted (repository visibility public); gstack-approvals list shows the full record
Summary
GST-1088 lane B (plan: GStack Minder drive
renders/GSTA-256/plan-v2.md, approved by Garry with "do it" on gate rev 1). Agents weren't opening the right skill because the catalog line said what each skill is, not when to use it. This PR makes the first sentence a when-clause, widens the trigger lists people actually type, addscontract:blocks to /investigate and /review, and renders each block as a "Done means" list at the top of the skill.Descriptions
Claude Code's catalog keeps only the first sentence (
applyCatalogTrim), so the when-clause comes first. For the core skills /review and /qa,test/proactive-core-set.test.tsalso requires that first sentence to carry one of the skill's own trigger phrases and the full description to carry every one. /review's "Proactively suggest before a merge." sits on its own line becauseskill-validationrequires it.Full descriptions (Codex, Gemini, Copilot, Hermes, OpenClaw and other full-description hosts)
after: Use when a test fails, code errors or output is wrong: find the root cause, then fix it. Systematic debugging; Iron Law: no fixes without root cause. Fits a failing build, test, script, service or config, an error message or stack trace, wrong or missing output, or broken data, before any edit that fixes it. (gstack)
after: Use when you review this PR's diff. Checks it for SQL safety, LLM trust boundaries, conditional side effects and bugs that pass CI and break production. Fits "look over my diff", "review my changes", "code review", "check my diff" or "pre-landing review". Proactively suggest before a merge. (gstack)
after: Use when coding: run the checks and tests /ship picks until green. Build loop: run the repo's checks and the tests /ship would pick, fix, repeat. Never green when no test covers a change. Fits "verify", "check my work", "run the checks", "make it green", or after each change before /ship. (gstack)
Triggers
Where these reach (
lib/skill-triggers.ts,scripts/gen-skill-docs.tswithTriggerLead,lib/harness-register.tsskillDirs):triggers:norcontract:. It sees only the new first sentence and the Done-means list.descriptionLead) gets a 57-characterUse for: <triggers>lead built from the trigger list. For example, /investigate's lead becomes "Use for: debug this, fix this bug, test fails." and /review's becomes "Use for: review this pr, look over my diff, code review.".triggers:in its frontmatter.lib/harness-register.ts(dots, generic and OpenClaw router mode) now take the new "Use when ..." sentence. The Claude-trimmed description used to have no trigger sentence, so those lines used to fall back to the trigger list. The widened triggers therefore no longer reach router lines for these four skills.triggers:.Contract blocks and "Done means"
The
contract:blocks follow the frozen v1 schema ({ version: 1, items: [{ id, kind, ... }] }):report_labels(DEBUG REPORTwith Symptom, Root cause, Fix, Evidence, Regression test and Status) andcheck_after_last_edit.selected_specialists_dispatched(markerDispatching N specialists:from Step 4.5),agent_dispatch(Think like an attacker and a chaos engineer, from Step 4.8's adversarial prompt) andreport_labels(Pre-Landing ReviewwithGate edits).{{CONTRACT_DONE_MEANS}}(scripts/resolvers/contract-done-means.ts, registered inscripts/resolvers/index.ts) renders each block at the top of the body:REVIEW_ARMY,ADVERSARIAL_STEP) or the loadout excludes its section (section:review/review-army,section:review/adversarial).report_labelsitems allownone: <why>where the skill's own stop rule leaves a label empty.gstack-skill-contract checkandwaivefollows an explicitCONTRACT_CHECKER_SHIPPEDflag, andtest/skill-descriptions.test.tsfails while the flag and the bin disagree. Main has shippedbin/gstack-skill-contractsince v1.94.6.0 feat(hooks): opt-in skill routing and contract Stop hooks with gstack-skill-contract (GST-1088 lane A) #3912 (v1.94.6.0), so this PR merges main at bf2bcf1 (GStack Minder's rule 19 grant), sets the flag to true and renders the 240-character sentence under the list in /investigate and /review. The flag left at false is what turned train 102 red (runs 41625996288 and 41627137651).skill_contractsoff switch (gate rev 2 d5a). A placeholder without a block, invalid YAML or a schema break stops the render withCONTRACT_INVALID <file>: <problem>. Fix: edit <skill>/SKILL.md.tmpl, then bun run gen:skill-docs.Raised caps
The growth comes from F1's "Done means" list and the
contract:block, both from GST-1088's plan (renders/GSTA-256/plan-v3.md line 74, r2-d6 "Done-means list"). Garry said "do it" on gate rev 1 and on gate rev 2 (2026-11-03), which made F1 and D1 final. GStack Minder ruled that the PR making the checker flip re-pins /review by the measured amount with no slack. Lane C's diet is expected to bring /review back under the old caps. Every value below is measured on the merged tree at 981a3ce.maxSkeletonBytesmaxSizeRatioagy/gstack-reviewcodex/gstack-reviewcopilot/gstack-reviewcursor/gstack-reviewfactory/gstack-reviewgemini/gstack-reviewkiro/gstack-reviewopencode/gstack-reviewslate/gstack-reviewNo fixed cap moved: the 5,315-token ratchet, DRIVER.md's 12,288 bytes, the 61,420-byte global skeleton cap, the 800-line module size and PART_MAX 24,576 all pass (
context-budget-ratchet,module-size-ratchet,ship-parts). The carve-guards comments now cite the flip and plan-v3 line 74.Sibling PR #3837 also edits the allowlist (12
gstack-autoplanpins down by 17). A trial merge of this head with #3837's head 93bba94 conflicts only because the edited lines sit next to each other: the two PRs change disjoint keys (12 autoplan keys there, 9 review keys here, no overlap). Resolution: keep both sets. On the resolved tree,skill-md-size-guard,parity-suiteandskill-descriptionspass (115 pass, 0 fail)./qa gets no cap raise. Its trigger widening was dropped, and its opening sentence reads "... and fix bugs." rather than "... and fix its bugs.": main at 106fd39 already sits exactly at qa's cap (95,983 union bytes), so this PR adds no bytes to /qa. Merged with main, /qa measures 95,983, a ratio of 1.141989 against the 1.1420 cap.
Test change
test/harness-register.test.ts:251changes fromnot.toMatch(/^Use /)tonot.toMatch(/^Use for: /). The old check stood in for "OpenClaw doesn't get Hermes's trigger lead", and it relied on review's own description not starting with "Use". That description now starts with "Use when", so the check now tests for Hermes'sUse for:lead directly.Test Coverage
test/skill-descriptions.test.ts(new) was written first and failed 14 times before the frontmatter edits. It covers the when-clause openings within the catalog, lead and host caps, trigger length and overlap, ten sample prompts that each route to exactly one skill, the v1 schema and printed markers for every contract block, each renderer item kind, host and loadout filtering, the checker flag, and malformed blocks failing with the file and the fix.Tests ran: targeted 1526 pass, 0 fail, 21 skip at 981a3ce (the 23 files below plus skill-contract, skill-contract-stop-hook, skill-route and the QA-only cross-host test); the preflight lane ran 1327 tests, exit 0. CI's Free Tests run 41628918123 is green at tree 07edccb5c6c2.
Pre-Landing Review
Pass 1 (full: batched testing, security and state-machines specialist, Red Team, native adversarial, outside voice openai/gpt-6.1-sol): 3 informational findings fixed (5255dd9) and 3 findings at the blocking floor fixed (93d0a20, 3d16d82): excluded loadout sections stayed required in the list, the list left no honest stop for an empty report label, and the rendered text depended on whether
bin/gstack-skill-contractexists. Pass 2 (full on the fix range, asgstack-review-planprinted): clean, one informational note. After main moved, CI at d2f1e29 failed /qa's parity cap on the merge ref; GStack Minder granted one scoped fix-check ("Lane B #3909: grant fix-check qa trim.") for the 4-byte /qa trim in cc9b9b9, and that fix-check was clean (one informational note about this draft, fixed). After the rule 19 merge of main and the checker flip, GStack Minder granted one scoped fix-check on the flip hunk; it was clean with one informational note (below).REVIEW_LOOP: passes=2 blocking_found=1 deferred=0 noted=3 fixed=4 scope=frozen unrecorded=1 ledgers_read=5 fix_checks=3 dispatches=8
Gate edits: 4 listed, 3 read, 0 findings — 0 kept, 0 restored, 0 open. The listed edits are the owner-approved re-pins above (
test/helpers/carve-guards.tsreview caps,test/fixtures/skill-md-size-allowlist.jsongstack-review rows) and thetest/harness-register.test.ts:251assertion.Outside review: completed (openai/gpt-6.1-sol), one finding, fixed.
Review notes
gstack-skill-contractbare, and a standard install doesn't put that bin on PATH (other bins in these skills go through~/.claude/skills/gstack/bin/). A waive run as written exits 127 and its waiver is dropped, so the Stop hook blocks once more. It fails no test and loses no data; rendering the host-resolved path changes the shared sentence and the pins, so it is left to lane A (flip fix-check, informational).docs/skill-routing-and-contracts.mddoesn't mention the Done-means list, and its /investigate example uses idrepro-after-fixwhere the template usescheck-after-last-edit(docs audit, optional).ADVERSARIAL_STEPis outside this lane (adversarial, low).Exploratory QA
Not run: no runnable surface changed; the change is skill text and its generator.
Design Review
No frontend files changed — design review skipped.
Eval Results
Not run in this lane.
bun run eval:selectselects 113 paid behavior probes plus 24 judges becausescripts/resolvers/index.tsrestores every gate case, which is past the $10 lane cap. The paid screen runs throughgstack-ship-queue ready --screen(pre-approved), and the routing and contract evals belong to lane A's PR.Review bot comments
No review bot comments.
TODOS
No TODO items completed in this PR.
Documentation
Scope. Read-only audit at head 58aeee7 on capy/gst-1088-lane-b against base bf2bcf1. The 16 committed changed files were checked: four skill descriptions and triggers, the
contract:frontmatter on /investigate and /review, the newCONTRACT_DONE_MEANSresolver,gstack/llms.txt, the changelog fragment, and tests. Nothing is staged, unstaged or untracked. Three documents were read in full (changelog fragment,gstack/llms.txt,docs/skill-routing-and-contracts.md), and two more were checked by grep (docs/PROJECT_STRUCTURE.md,CLAUDE.md). The other candidate docs were grepped for the old and new description strings and for the contract and resolver names.Result. The docs are current.
gstack/llms.txtalready carries the new when-clause descriptions for /investigate, /qa, /review and /verify. The changelog fragment accurately describes the change, including that the routing hook and contract check landed in lane A, and it points todocs/skill-routing-and-contracts.md. That doc's schema, item kinds and exit codes match what the templates declare. No other tracked doc quotes the old descriptions.Debt.
docs/skill-routing-and-contracts.mddoes not mention the{{CONTRACT_DONE_MEANS}}resolver or the "Done means" list rendered at the top of the skill. Its /investigate example uses the idrepro-after-fix, while the template usescheck-after-last-edit; the doc presents it as an example. No edits were made, because the mode is read-only.Test plan
bun run typecheck: exit 0;bun run gen:skill-docs --dry-run: 0 staleAlso changed:
gstack/llms.txt, regenerated bygen:skill-docs.