Repository navigation
fix(summarize): token-budget chunking, Session Summary reuse, noop predicate, per-operation breaker - #45
Merged
Conversation
added 10 commits
September 21, 2026 15:54
fetchWithTimeout clamped every budget to a 170s HARD_BUDGET_CAP_MS while openai.ts still reported the unclamped value in its timeout error, so an AGENTMEMORY_LLM_TIMEOUT_MS of 480000 aborted at 170s and claimed 480000ms. Broker-side records confirm it: 86 client-disconnect 499s on the general route, p50 duration 169,996ms, p90 170,004ms, none past 400s. The cap existed to stay under iii's 180s invocation timeout. ADR 0001 deleted that runtime — the in-process Engine has no invocation timeout, so the ceiling bounded against a limit that no longer exists and turned slow local models into provider failures, which then tripped the circuit breaker and failed unrelated calls. The caller's timeout is now the total budget for all attempts plus sleeps, honored exactly. test/fetch-timeout.test.ts asserted the clamp; it now asserts the opposite. Generated-By: Claude Opus 5 (1M context)
groupKey joined its two parts with a raw NUL byte. One NUL is enough for git and grep to classify the whole file as binary, so every search over src/engine/inproc/sdk.ts silently returned nothing and `file` reported it as data rather than source. U+001F separates just as well. The key is only ever a streamItems Map key and is never persisted, so nothing migrates. Generated-By: Claude Opus 5 (1M context)
CLAUDE.md opened straight into architecture, so every feature argument started from mechanism. Lead with the goal instead: right Memory, right moment, right scope, least record that restates it. Generated-By: Claude Opus 5 (1M context)
Score every open upstream issue on severity x difficulty, cross-link the PRs that claim to fix each one, and score the 168 PRs that reference no open issue by how much of their diff lands in files this fork tracks. Every row carries a disposition: wont-fix for what is out of scope here (no iii engine, no npm publish, no Windows CI, no deploy tree, no adapter for the host), already-fixed for what this tree already carries, candidate for the working set. The 17 high-severity/low-difficulty issues and the 35 high-relevance candidate PRs were then opened against this tree and read. Rows checked that way carry a status whose leading token says what the read found; 15 claims turned out to be already fixed here and moved out of the candidate set. A row with no status is still upstream's claim, not a verified defect. Generated-By: Claude Opus 5 (1M context)
Generated-By: Claude Fable 5.1
Every provider is wrapped as resilient(<inner>), so the === "noop" check never matched and a keyless install ran the whole summarize pipeline. One predicate looks through the wrapper; summarize and graph Extraction both use it. Closes #32. Refs #31, rohitg00#1367. Generated-By: Claude Fable 5.1
Chunks were cut at a fixed 400 Observations, but per-Observation size varies 9x across the store, so a capped Session could exceed the broker's practical latency and time out. Observations are now measured with the provider's tokenize route (llama.cpp-style POST /tokenize, forwarded outside the circuit breaker) and estimated at chars/3 when there is none, then packed in order to SUMMARIZE_CHUNK_TOKENS (default 50,000) with the system prompt's overhead reserved. SUMMARIZE_CHUNK_SIZE is gone. The chunk concurrency default drops from 6 to 2: it was tuned for iii's 180s cap, and the timeout basis is measured at 2 on one GPU. Closes #33. Refs #31. Generated-By: Claude Fable 5.1
…g it A Session Summary is current until an Observation arrives after it was written; Observations are append-only within a Session, so a stored summary whose Observation count matches the Session's is returned without an LLM call. force: true on mem::summarize and POST /agentmemory/summarize bypasses the reuse. This closes the retry loop on Sessions that summarize and are stopped again with nothing new. Closes #34. Refs #31, rohitg00#572. Generated-By: Claude Fable 5.1
A Session whose summarize calls fail three times used to open the single breaker and stop compress and extraction for the recovery window, so one oversized Session stalled every Observation behind it. The resilient wrapper now keeps a breaker per operation (compress, summarize, describeImage). circuitState reports the most severe operation so /agentmemory/health and the CLI keep their shape; circuitStates exposes the per-operation map and health returns it as circuitBreakers. describeImage is now forwarded through the wrapper, which it never was. Closes #35. Refs #31, rohitg00#1350. Generated-By: Claude Fable 5.1
- Restore CRLF line endings in src/triggers/api.ts; the earlier edit had rewritten the whole file to LF. - Session Summary reuse now keys on Observation count AND the last Observation id (new optional SessionSummary.lastObservationId); a delete followed by a new Observation no longer passes as current, and a summary stored before the field existed is regenerated. - Always measure tokens when the provider can count; the estimate is only the fallback. OpenAIProvider remembers a 404 from /tokenize and stops asking for the life of the instance. - Measure once per summarize, outside the 2-attempt retry, via planChunks. - Charge each Observation its prompt separator (4 tokens) and floor SUMMARIZE_CHUNK_TOKENS at the prompt overhead plus one. - Chunk fan-out goes through mapWithConcurrency; chunk offsets use a for loop. - isNoopProvider unwraps only balanced resilient(...) layers. Refs #31 Generated-By: Claude Fable 5.1
Owner
Author
Code review:
|
This was referenced Sep 22, 2026
possiblyneal
pushed a commit
that referenced
this pull request
Sep 22, 2026
Generated-By: Claude Fable 5.1
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Summarize cluster of #31. Four tracer bullets on top of
fix/llm-timeout-hard-cap:isNoopProvider(), which unwraps balancedresilient(...)layers, so a noop deployment no longer sends prompts to nothing./tokenizeviacountTokens, 3-chars/token estimate only when the count fails; a 404 is remembered per provider instance) against a 50k budget with 400 tokens reserved per chunk for the prompt and 4 per Observation for its separator. Tokens are measured once per summarize, outside the retry.SUMMARIZE_CHUNK_TOKENS(floored at the prompt overhead + 1) andSUMMARIZE_CHUNK_CONCURRENCY(default 2) replace the fixed-count chunker.SessionSummary.lastObservationId) both match the Session is returned without an LLM call; a summary stored before the id was recorded is regenerated.force: trueonmem::summarize/POST /agentmemory/summarizebypasses reuse.circuitStatereports the most severe so health output keeps its shape,circuitStatesexposes the map (health returns it ascircuitBreakers).describeImageis now forwarded through the wrapper.Base commit
01bd9e3carries the timeout hard-cap removal; those commits are shared with that branch.Test plan
npm test— 1913 passed, 1 skippednpm run build/tokenizeprobe against the broker (2250 chars → 501 tokens in 38 ms)AGENTMEMORY_LLM_TIMEOUT_MS=300000, rebuild, restart, re-summarize a 500-Observation Session (Roll out: merge the four cluster PRs, set the 300s timeout, rebuild, restart #43)Closes #31 partially. Refs rohitg00#1367, rohitg00#572, rohitg00#1350.
🤖 Generated with Claude Code