Repository navigation
v1.94.6.0 feat(hooks): opt-in skill routing and contract Stop hooks with gstack-skill-contract (GST-1088 lane A) - #3912
Merged
Merged
Conversation
…ll-contract and gstack-skill-route (GST-1088 lane A) Both switches default off. UserPromptSubmit routing hook scores prompts against installed triggers; Stop hook blocks once when a contract skill ends without its outputs, failing open on every state error. setup gains --[no-]skill-routing and --[no-]skill-contracts mirroring the timeline gate. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
…t-of-test ratchet) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…-contract fixture as free (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…op_hook_active, read last_assistant_message, per-agent completion, met runs stay met, installed-only SKILL.md loads, program-based check commands, safe specialist names, report rechecks the saved run (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…t, the shell test builtin does not; met_after_block is final (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…0 selects no specialists (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…line, parenthesized notes are not specialist names (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
… the contract (GST-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…T-1088 lane A) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Two opt-in Claude Code hooks so a model actually runs gstack's skills (GST-1088, lane A of plan-v2). Both switches default off, so this PR changes no behavior when it merges.
hosts/claude/hooks/skill-route-hook, UserPromptSubmit,skill_routing). Scores the prompt against the installed skills'triggers:frontmatter (no model call, no network) and, at or above the 0.66 threshold, adds one additionalContext line that names its source and its off switch and asks the model to say in one sentence that gstack routed it. Slash-command prompts, prompts over 8,000 characters and every error produce no output; it always exits 0 (never 2) under a 1.5 s internal deadline. Trace and per-session failure counter in$GSTACK_STATE_ROOT/hooks/skill-route.log, never the prompt text, never stderr.hosts/claude/hooks/skill-contract-stop-hook, Stop,skill_contracts). Checks the most recently loaded skill that declares acontract:block (schema v1 frozen in plan-v2:report_labels,check_after_last_edit,selected_specialists_dispatched,agent_dispatch) and blocks once per skill run with the gate rev 2 E1 text, which never names the off switch. It checks only a final turn (a STATUS line, a headless or unattended session, or a turn that does not end on a question), allows at once onstop_hook_active, and allows while Agent dispatches are still running. Every read, write, parse or timeout error fails open with one stderr line ending(see docs/skill-routing-and-contracts.md#state-error); an unknown transcript shape also bumps a canary counter. Records are written atomically per Claude Code session id before any block.gstack-skill-contract check|report|waive:checkis a pure function of a transcript JSONL (evals can reuse it);waiveis stateless (G1) andcheckreads waivers from the waive Bash calls in the transcript, counting them as misses for follow;reportlists misses and re-checks a blocked run against its transcript. Stable exit codes 0–6, each with a docs anchor.gstack-skill-route --explain "<prompt>": candidates, scores, threshold, route and the exact hint (a dry run, works with the switch off).--[no-]skill-routingand--[no-]skill-contracts, mirroring the timeline_stop_hook gate: flag > env (GSTACK_SKILL_ROUTING/GSTACK_SKILL_CONTRACTS) > config (skill_routing/skill_contracts, on|off) > default off; a flag persists, a typo warns and is not persisted, off and--no-teamremove the registration, and the first registration prints one line. TwoKNOWN_HOOKSrows inbin/gstack-settings-hook.docs/skill-routing-and-contracts.md(precedence, dry run, schema with a worked example, host support table, check/report/waive, an anchor per error code).Field names match lane B (#3909):
markeris literal text where a whole-word N stands for a number (Dispatching N specialists:), and a "Small diff ... specialists skipped." line counts as zero specialists selected, a pass.Security note (install surface)
setupgains two flags and a gate function, andbin/gstack-settings-hookgains two identity rows. With a switch on, setup writes a hook entry into~/.claude/settings.jsonthrough the existing hardened settings-hook (lock, backup-on-change, atomic write, fail-closed on a corrupt file). With both off (the default) setup only callsremove-source, which writes nothing when no entry exists.bin/gstack-session-kind(1 s cap) only when a turn ends on a question. Both read the transcript and SKILL.md files the host already has; logs and records are mode 0600 under the state root and never hold prompt text.Scorer precision/recall on Run D (d7) and hint pilot (d6)
d7, deterministic scorer, no model. Inputs: the 37 Run D tasks (gstack-evals 8cd4973
quick/laneb/tasks/final.json, less crack-7z-hash, adaptive-rejection-sampler and largest-eigenval), instructions from terminal-bench-2 69671fb read as text. Fit stratum (should route to /investigate): custom-memory-heap-crash, overfull-hbox, fix-code-vulnerability, fix-git, db-wal-recovery. The other 32 are decoys. Triggers: lane B #3909 at b772dc6 and again at its current head cc1539c (identical rows: cc1539c drops two /review triggers that never fired on these tasks), and main's (54606c5) as the before.The last row is why 93b17ed limits routing to the four discovery targets in plan decision 4 (/investigate, /review, /qa, /verify): with all 63 skills routable, 17 of 32 decoys got a hint for some unrelated skill. At 0.66 the four decoy routes are extract-elf and constraints-scheduling to /investigate, and torch-pipeline-parallelism and tune-mjcf to /verify. The three fit misses (overfull-hbox, fix-git, db-wal-recovery) are short task specs that share no /investigate trigger. Recall is lane B's trigger list, not the scorer, and 0.66 stays provisional until GEVA-214 freezes it.
d6, hint pilot. 36
claude -ptrials, Opus 5.5 (claude-opus-5-5, Claude Code 2.1.317, its own claude_code system prompt), lane B's skills installed, a two-file Node repo with one failing test, three bug-fix prompts × 3 reps per arm,--max-turns 2. Opened = the first tool call isSkill: investigate. Run twice: on lane B b772dc6 ($3.03) and on its head cc1539c ($3.23), with the same result both times; lane spend $6.41 with the smoke trial.The hint moves Opus from 0/9 to 9/9, and it said "gstack routed this to /investigate" as asked. The three wordings tie at the ceiling, so
name, the shortest, stays the default. Why null for the wording comparison: exposure was confirmed (the hook logged a route in every hinted trial and /investigate was in every session's skill list), but these prompts are easy and saturated all three arms, so the pilot can't rank the wordings. Verdict: KEEP the hint; FIX-METHOD for the wording comparison (harder, ambiguous prompts in GEVA-214's run). The CLAUDE.md-block fallback isn't triggered, because the hint did move Opus.Review (gstack-review-plan, full depth, class gate)
reportlost a fixed run once the skill was loaded again. Both are fixed in 798045b, along with six informational fixes:stop_hook_activeStops now recordmet_after_block/miss(never block), the payload'slast_assistant_messagecovers a transcript that lags one message, a met run stays met, only aSKILL.mdread under askills/directory counts as a load, specialist selection lines come only from assistant text or Bash results with plain names (closes a prompt-injection path into the block text), and check commands match by the program they run.bundle exec rspec,uv run pytest,docker compose run app pytest) count again, andmet_after_blockis final too. The review plan stops at the round-2 cap.remove-sourcerewrites a settings.json that holds empty hook arrays on every./setup(corebin/gstack-settings-hookbehavior, out of this lane's scope). (4)npm install,node --versionandmake cleanstill count as check commands, and one Agent prompt naming two specialists satisfies both; both err toward allowing the stop. (5) When another Stop hook continued the turn first and our contract is unmet, nothing is recorded for that stop. (6) Edits made through Bash (sed -i, heredocs) don't count as edits forcheck_after_last_edit.e318060, c19a685, e49c9be, 5c7d678 (lane B and lane C requests):
Dispatching N specialists:reads N as a number, and N = 0 is met with no dispatches; parenthesized notes are not names (Dispatching 1 specialists: State Machines (small diff)).agent_dispatchtakes an optionalmarkerand prefers it (Dispatching adversarial pass: nativefollowed by an Agent call), keepingmatchas the fallback. An early exit meets the contract: a finalSTATUS: BLOCKED(for /investigate,Status: BLOCKEDin the DEBUG REPORT) or a final message line starting "Nothing to review" (/review's early exit). Also from e318060: the checker's default marker strings now live in one table,DEFAULT_MATCHERSinlib/skill-contract.ts, so lane C's fixed markers are a one-line edit;Dispatching 0 specialists: (small diff)counts as zero selected. The ledger-backed targeted reader ran after each of these heads; its informational notes are folded into the list above, and on the final head 5c7d678 it reported no findings.Tests
test/skill-route.test.ts,test/skill-contract.test.ts,test/skill-contract-stop-hook.test.ts,test/setup-skill-hooks-gate.test.ts(fixture skills carry the contract blocks; no skill template is touched): unwritable state root, corrupt record, stop_hook_active, a multi-turn interactive fixture that is never blocked, the J1 /investigate fixture that is blocked, Agent readers still running, the pinned block message (test/fixtures/skill-contract/block-message.txt), routing error cases (malformed triggers, unreadable config, bad JSON, over-long prompt, stdin over cap), hook output schema and settings.json entry schema, and both off switches.