Skip to content

fix(seats): Hermes refuses in plain words (F116) - #4125

Open
lilyshen0722 wants to merge 2 commits into
mainfrom
stack/f116-hermes-plain-refusal
Open

lilyshen0722 wants to merge 2 commits into
mainfrom
stack/f116-hermes-plain-refusal

Conversation

@lilyshen0722

@lilyshen0722 lilyshen0722 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

A person who attached Hermes as their own agent got refusals worded as operator notes. Growth's 11-04 harness walk found Hermes 0.22.5 refused at attach. F116 keeps that refusal and makes it plain.

Before and after

Case Before After
Public seat "Hermes public or sandbox-declared seats are unsupported: no enforced confinement adapter exists" "Commonly has no sandbox for Hermes yet, so Hermes can't run as a public seat. Pick another harness for a public seat."
A sandbox mode (same as above) "This environment file asks for sandbox.mode=…, and Commonly has no sandbox for Hermes yet, so Hermes can't run under it."
A provider block "Hermes adapter does not consume environment.provider; supply OPENROUTER_API_KEY in the host environment" "Hermes connects through OpenRouter only, so it can't use the provider block in this environment file. Remove that block and set OPENROUTER_API_KEY where this agent runs."
No key "Supply the operator OPENROUTER_API_KEY through Secret Manager before running Hermes" "Hermes connects through OpenRouter only. Set OPENROUTER_API_KEY where this agent runs, then start it again."
An mcp block "Hermes adapter does not yet consume environment.mcp declarations" "Commonly can't give Hermes extra MCP servers yet, so this environment file can't list any. Remove the mcp block to run Hermes."

The checks themselves are unchanged. The single public-or-sandbox check is split in two so each can name its own reason.

The CLI's shared provider check (providerRefusal) runs before the adapter's own check at attach, at run and in the daemon's adapter swap, so it used to answer a Hermes provider block with "requires a provider-capable adapter". An adapter that takes no provider block can now name its own refusal (noProviderRefusal). Hermes does, and every other adapter keeps the shared message.

Tests

  • cli/__tests__/adapters.hermes.test.mjs adds a test that every refusal is plain, with no internal names, and one for the missing-key message.
  • A test runs the shared provider check against the real Hermes adapter.
  • The two existing tests now pin the new wording, as does seats-package.test.mjs.
  • 5 of these tests fail on main.
  • The Hermes, seats-package, attach and run-loop tests pass (462), as do the other CLI files that mention Hermes (412). Lint is clean.

Docs

The Hermes row in docs/agents/byo-support-matrix.md now says what is refused at attach, and why.

packages/seats is bundled into the CLI tarball, so a merge publishes the CLI to @latest. Press it in the post-PH cli window. Tier 1. Sonnet review: HOLD at 960431a (the shared provider check answered first, so the new provider text never reached attach), fixed at d136629.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N4VWLDwn18YFTTgyP91mNW

Hermes's refusals reach the person attaching their own agent, and they read
as operator notes: 'no enforced confinement adapter exists', 'through Secret
Manager', 'environment.provider'. Each now says what Hermes needs and what to
do: Commonly has no sandbox for Hermes yet (public seat, or a sandbox mode,
named), and Hermes connects through OpenRouter only (provider block, or a
missing OPENROUTER_API_KEY). The checks themselves are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4VWLDwn18YFTTgyP91mNW
@coderabbitai

coderabbitai Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between ac58a16 and d136629.

Files selected for processing: 5

  • cli/__tests__/adapters.hermes.test.mjs
  • cli/__tests__/seats-package.test.mjs
  • cli/src/lib/environment.js
  • docs/agents/byo-support-matrix.md
  • packages/seats/src/adapters/hermes.js
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

… (F116 review)

providerRefusal runs before validateEnvironment at attach, run and the
daemon's swap, so a Hermes provider block still got 'requires a
provider-capable adapter'. An adapter that takes no provider block may now
name its own refusal (noProviderRefusal); Hermes uses its plain text and every
other adapter keeps the shared message. The environment.mcp refusal is plain
too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4VWLDwn18YFTTgyP91mNW

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One review round (Tier 1: Sonnet, effort medium). The reviewer read REVIEW.md and checked logic, truth of the wording, where each message is shown, and the tests.

Round 1 at 960431a: HOLD.

  • The CLI's shared providerRefusal runs before validateEnvironment at attach, so a Hermes provider block still got "requires a provider-capable adapter". The new provider text never reached the person, and the matrix line overclaimed.
  • Low: the environment.mcp refusal was still jargon.
  • Checked clean:
    • the split public/sandbox check is exactly equivalent, so nothing new passes;
    • sandbox.mode interpolation is enum-bounded and local;
    • pod turn-failure notices use fixed strings, so nothing leaks into a pod and classification is unchanged;
    • no other test or doc quotes the old strings.

Round 2 at d136629: PASS.

  • providerRefusal returns adapter.noProviderRefusal only for an adapter without provider support that sets it. Every other adapter keeps the shared message.
  • The check runs at attach, run and the daemon swap, so all three paths now use the plain wording.
  • HERMES_NO_PROVIDER is one const shared with validateEnvironment, so the two can't drift.
  • The mcp refusal is plain and true. Hermes gets no default MCP, so only a person's own block hits it.
  • The matrix line is accurate.
  • 34/34 in the Hermes and seats-package files.

Nit, open: noProviderRefusal isn't listed in the adapter contract doc.

GATE: PASS

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant