You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: allow a cheaper model for compression than for summarization #899
All four LLM-backed pipelines — per-observation compression, session summarization, consolidation synthesis, and reflection — share a single model setting (OPENAI_MODEL / ANTHROPIC_MODEL / etc.). But their cost profiles and quality bars are very different.
Measured on one active day of real usage (several parallel Claude Code sessions, AGENTMEMORY_AUTO_COMPRESS=true, v0.9.27):
Pipeline
LLM calls
Share
Observation compression
1 908
~87%
Graph extraction
147
~7%
Session summarization
123
~6%
Consolidation + reflect
37
~2%
Compression is the bulk of spend, and — as the README's own "Cost-aware model selection" section puts it — it is "a summarization task with relatively loose quality bars". Summaries, consolidation output, and lessons, on the other hand, are exactly what gets injected into future sessions, so quality there matters most.
Today the only lever is downgrading the single shared model, which trades away quality on the 2-6% of calls where it matters to save money on the 87% where it doesn't. I couldn't split the workload because there is no per-task model setting (closest prior art: #613 documented cost tiers, #767 proposes a token budget — neither separates models per task).
The codebase is actually already shaped for this: MemoryProvider has distinct compress() and summarize() methods, and AgentMemoryConfig.compressionModel exists but always mirrors provider.model.
Proposed solution
A single provider-agnostic env var:
OPENAI_MODEL=your-main-model# summaries, consolidation, lessonsAGENTMEMORY_COMPRESS_MODEL=your-cheap-model# per-observation compression etc.
compress() callers (observation compression, graph extraction, query expansion, sliding window) use the override when set.
summarize() callers (session summaries, consolidation pipeline, reflect, crystallize, compress-file, skill-extract) stay on the main model.
Activity
feat(providers): per-lane LLM model via AGENTMEMORY_SUMMARIZE_PROVIDER (
refactor(providers): simplify summarize-lane wiring (rohitg00#899)
fix(providers): isolate compress and summarize lanes (rohitg00#899)