Repository navigation
fix(core): bound the recent part a compaction keeps - #54649
superbiche wants to merge 1 commit into
Conversation
The kept part started at the newest user message before the cut, however far back that message was. After one message followed by a long run of tool calls, the whole run was kept verbatim: observed at 434k characters against a 15k-token allowance, with the next compaction re-reading that text as its input. With no user message since the previous checkpoint, everything was kept. Start at the user message only when it is within another allowance of the cut; otherwise, and when nothing was asked since the previous checkpoint, start at the cut. Measure the kept part in characters, separators included, so the stored text fits the allowance as the token estimate counts it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
The following comment was made by an LLM, it may be inaccurate: |
|
Thanks for the careful write-up and the measurements in #54648! I think this duplicates #53233, which is still open. It fixes the same unlimited walk-back in If you think starting at the cut is the better trade-off (for example, it keeps some verbatim context in long single-prompt runs), please make that case on #53233 or #43250. Your measurements from multi-compaction sessions, and a test of #53233 against them, would also help there. |
Issue for this PR
Closes #54648
Type of change
What does this PR do?
recentStartfinds the cut that fitskeeptokens, then moves the start back to the newest user message before it so the kept part opens with what was asked. That walk-back has no bound, so one message followed by a long tool run is kept whole (434k characters against a 15k-token allowance in the linked issue), and with no user message since the previous checkpoint everything is kept.Now the user message is used as the start only when it is within another
keepof the cut; otherwise, and in the no-user-message case, the kept part starts at the cut. Tool calls stay with their results because both live in one assistant message. The kept part is measured in characters including separators (Token.chars, the inverse ofToken.estimate's ratio) so the stored text itself fits the allowance. The newest entry is still always kept, as before.How did you verify your code works?
New test in
packages/core/test/session-compaction.test.ts: user message within reach of the cut (still opens the kept part), 120 tool steps after it (kept part starts at the cut and stays withinkeep; the message goes to the summarizer), a checkpoint followed only by tool steps (stays withinkeep), and an exact-fit pair. It fails on the unpatched code.bun run testandbun typecheckinpackages/core,bun run checkat the root.Screenshots / recordings
Not a UI change.
Checklist
@superbiche · user · drafted with Claude Fable 5.1, reviewed before posting; the voice and decisions are mine.