Skip to content

feat(hermes): let the integration name a retain strategy - #4570

Merged
nicoloboschi merged 2 commits into
vectorize-io:mainfrom
0xble:feat/hermes-retain-strategy
Oct 6, 2026
Merged

nicoloboschi merged 2 commits into
vectorize-io:mainfrom
0xble:feat/hermes-retain-strategy

Conversation

@0xble

@0xble 0xble commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

The gap

Named retain strategies steer extraction per content type. The capability exists end to end except in this integration:

  • the API exposes it — MemoryItem.strategy: "Named retain strategy for this item. Overrides the bank's default strategy for this item only."
  • the generated client carries it
  • hindsight-integrations/hermes/_build_retain_kwargs builds the item and never sets it

So every transcript the Hermes integration stores falls through to unmissioned concise extraction, and an operator has no way to change that short of editing the plugin.

It shows up in this repo's own catalog too: neither hermes-orchestrator nor hermes-gateway-bot can express a strategy, only a bank-wide retain_mission, because the integration cannot send one.

Why it matters

Measured on a live bank running this integration, comparing documents retained under a strategy matched to their content type against the unmissioned path on the same bank and model:

path docs facts facts/doc
missioned 2,814 4,053 1.6
no strategy 10,285 249,069 90.2

Agent session transcripts run to several MB and are largely raw tool output. Without a mission the extractor faithfully turns diffs and test assertions into durable memory.

There's a second effect that a bank-level retain_mission cannot reach: a strategy also carries retain_chunk_size. Without one, a long session is chunked at the 3000 default into fragments, so the extractor never sees a full proposal-to-revision arc and the same decision gets re-extracted per fragment.

The change

Adds a retain_strategy setting, read through _scoped_setting like the other retain-shaping values (retain_source, retain_user_prefix, retain_assistant_prefix) so a multiplexed secondary profile cannot inherit the default profile's value.

Empty by default; behavior is unchanged. When unset the item omits the key entirely and the bank keeps deciding through its own retain_default_strategy. Existing deployments are unaffected.

The bank must already define the name under retain_strategies. An unknown name is ignored server-side and the item falls back to unmissioned extraction — called out in the setting description so a typo surfaces as a documented behavior rather than a silent no-op.

Tests

Three added to hindsight-integrations/hermes/tests/test_provider.py, driven through the real retain path rather than calling the builder directly:

  • test_retain_omits_strategy_by_default — no key on any stored item
  • test_retain_sends_the_configured_strategy — the value rides on every item
  • test_retain_strategy_is_exposed_as_a_setting — operators can set it without editing code
uv run --frozen pytest tests/ -q
18 passed

Baseline was 11 test functions in that file; now 14. No existing test changed.

Possible follow-up

If this lands, the hermes-orchestrator template could gain a retain_strategies entry so the recommended setup ships a session strategy rather than only a bank-wide mission. Kept out of this PR to keep it to the capability.

0xble and others added 2 commits October 6, 2026 11:08
Named retain strategies steer extraction per content type, and the
capability exists everywhere except here. `MemoryItem.strategy` is in the
API ("Named retain strategy for this item. Overrides the bank's default
strategy for this item only") and in the generated client, but this
integration's `_build_retain_kwargs` never set it, so every transcript it
stores falls through to unmissioned `concise` extraction.

The gap is visible in this repo too: neither `hermes-orchestrator` nor
`hermes-gateway-bot` in the Bank Templates catalog can express a
strategy, because the integration has no way to send one. They set only a
bank-wide `retain_mission`.

Measured on a live bank, documents retained under a strategy matched to
their content type produced 1.6 facts each; the same bank's unmissioned
path produced 90.2. Agent session transcripts are largely raw tool
output, so without a mission the extractor emits facts about diffs and
test assertions as durable memory. A strategy also carries
`retain_chunk_size`, so without one a long session splits at the 3000
default and the extractor never sees a full proposal-to-revision arc.

Adds `retain_strategy`, read through `_scoped_setting` like the other
retain-shaping values so a multiplexed secondary profile cannot inherit
the default profile's value.

Empty by default and behavior is unchanged: the item omits the key and
the bank keeps deciding through its own `retain_default_strategy`. An
unknown name is ignored server-side and falls back to unmissioned
extraction, which the setting description states so a typo is not a
silent no-op.
- README table row + regenerated docs page and skills mirror
- module docstring lists HINDSIGHT_RETAIN_STRATEGY
- rebased onto main (test-file conflict: kept both sides)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants