[allyblockcast]

Unpins scripts/test-hermes-compat.sh from Hermes v2026.9.21 (pinned in #32 / BLO-47312) by porting the gate that upstream vectorize-io/hindsight already redesigned for Hermes main.

Why a port, not a new design: upstream's gate passes against Hermes main today — job 117087267621 (2026-10-16, hermes-agent @ 848fc0ad): Hermes no longer bundles a hindsight provider → provider resolves to the external plugin → Status: available ✓ → PASS. The fork had zero local changes in hindsight-integrations/hermes, hindsight-embed, or the clients, so the port is conflict-free.

What it brings (upstream vectorize-io#4575, vectorize-io#4769, vectorize-io#4838):

• 3.14 venv, HERMES_REF defaults to main.
• Installs this checkout's hindsight-integrations/hermes as an external plugin, runs hermes plugins validate, asserts the loaded provider is ours.
• Asserts the four Hindsight distributions in the venv are the locally built wheels (not PyPI).
• Step 7 drives the MemoryProvider contract; skipped with an explicit message when no LLM key.
• hindsight-embed _http_probe fix: 0.10.0's probe cleared the caller's event loop and broke local_embedded.

PM runtime: not needed. The plugin is loaded from $HERMES_HOME/plugins inside a venv where Hermes and this checkout's wheels are co-resolved — the compatibility property the gate exists for.

Verifying signal

test-hermes-compat green on this PR with Hermes main, log showing local wheel versions.

🤖 Generated with Claude Code

[allyblockcast]

🔗 Paperclip issue: BLO-47316
🔗 Paperclip issue: BLO-47312

[allyblockcast]

Ally — Consolidated PR Review

Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex (applied directly; no nested CLI in the k8s runtime).
Reviewed head: 0695977

Looks good. scripts/test-hermes-compat.sh at this head is byte-identical to upstream vectorize-io/hindsight main (checked by diff), so the port is faithful. The provider surface step 7 calls matches what's actually there: sync_turn/prefetch/recall_status/handle_tool_call/shutdown in hindsight-integrations/hermes/__init__.py (L1357, L1224, L1237, L1460, L1554), and on_session_end comes from Hermes' MemoryProvider base (agent/memory_provider.py L183). The integrations-hermes filter output it now triggers on exists (test.yml L56/L229).

Critical Issues (0)

Important Issues (0)

Suggestions (2)

• [pr-review-toolkit/tests] hindsight-embed/tests/test_http_probe.py:70 — the new regression test for the _http_probe loop-state fix doesn't run on this PR. test-embed is gated on has_secrets == 'true' (test.yml L4651) and came back skipped at this head. The only CI coverage of the fix is indirect, through step 7 of test-hermes-compat, and only when an LLM key is present. Worth running pytest hindsight-embed/tests/test_http_probe.py locally or on a secrets-bearing run before relying on it.
• [gstack/review] .github/workflows/test.yml:4963 — HERMES_COMPAT_LLM_API_KEY: ${{ secrets.OPENAI_API_KEY }} now gives steps 6–7 a real key on same-repo PRs, so this job makes paid OpenAI calls (gpt-4o-mini, one short extraction, polled for up to 180s) whenever ci/embed/core/integrations-hermes changes. That's small and intended. Just noting that the job is no longer zero-cost on same-repo PRs.

Strengths

• _http_probe.probe_get now always runs the probe on its own loop in a helper thread. That's the right fix: asyncio.run on the caller's thread leaves it with no current event loop, which broke hindsight-client's sync _run_async (the session stayed bound to the old loop). The test pins exactly that invariant.
• Step 3 removes Hermes' bundled copy and asserts find_provider_dir resolves to the external plugin. Without that, every later step would silently test Hermes' code instead of this branch.
• The direct_url.json file:// assertion makes sure all four distributions are the locally built wheels, not PyPI. It guards the gate's own premise against the plugin's self-install path.
• Exit code 3 for a key-less partial run, with its own closing line, so a fork PR never claims a round-trip that didn't happen.

Recommended Action

1. No blocking changes requested.
2. Merge once the remaining required CI checks finish green.