core: transcript shortening clips every tool output in an exchange at once, leaving most of the window unused #61371

Activity

1. pjiang000-eng commented on Nov 3, 2026

   

   pjiang000-eng

   on Nov 3, 2026

   Author

   More actions

   The test referenced above, for anyone wanting to run it directly. It goes in packages/core/test/session-compaction.test.ts alongside the other summary tests:

      }
    })
    
   const toolAssistant = (outputs: string[]) =>
     Schema.decodeUnknownSync(SessionMessage.Assistant)({
       id: SessionMessage.ID.create(),
       type: "assistant",
       agent: Agent.defaultID,
       model: { id: "summary-model", providerID: "test" },
       content: outputs.map((text, index) => ({
         type: "tool",
         id: `call_${index}`,
         name: "read",
         state: { status: "completed", input: {}, content: [{ type: "text", text }] },
         time: { created: 0 },
       })),
       time: { created: 0, completed: 1 },
     })
   
   it.effect("transcript shortening clips the largest tool output first, not every one at once", () =>
     Effect.gen(function* () {
       // One exchange: no user message after the last compaction, so the level loop has no older exchange to cut first.
       const fixture = yield* summaryFixture([
         textUser("Read the three files"),
         toolAssistant([tokenText(150_000), tokenText(15_000), tokenText(15_000)]),
         measured("Latest answer", summaryWindow + summaryBuffer),
       ])
       expect(yield* fixture.run()).toEqual({ status: "completed" })
       const transcript = JSON.stringify(requests.at(-1)?.messages)
       // Only the excess needs cutting, so the two small results should stay verbatim.
       expect(transcript.match(/characters omitted/g)?.length).toBe(1)
     }),
   )
   
    for (const background of [false, true]) {
      it.effect(`summary cache prefix uses measured input with old reasoning (background=${background})`, () =>
        Effect.gen(function* () {

   Observed here: Expected: 1, Received: 3 — all three tool outputs clipped in one pass.
2. opencodejr commented on Nov 3, 2026

   

   opencodejrBot

   on Nov 3, 2026

   More actions

   Thanks for tracing this through the code and including a failing test.

   You're asking for the summary transcript to be shortened in smaller steps inside one exchange, for example by clipping the largest tool output first, so that when a request is only slightly over its target the smaller tool outputs stay verbatim and the summary can use the rest of the window.

   As written, flattenAndDropOldest deliberately moves a whole exchange from one shortening level to the next. So what you describe is how it was built, not a malfunction, and I've marked this as an improvement request for the maintainers to consider. The test you included will help if they pick it up.

   Related:

   • [BUG] V2: Auto-compaction accepts task-empty summary after shortened transcript, silently losing active objective (Ollama/Qwen) #60663: an automatic compaction after a shortened transcript (flattenAndDropOldest, no new user message since the previous compaction) produced a summary that lost the active task. Coarse clipping like this may have played a part.
   • fix(core): honor compaction retained-text budget #53233: a PR about honouring the compaction retained-text budget.
3. IgorGanapolsky commented on Nov 3, 2026

   

   IgorGanapolsky

   on Nov 3, 2026

   More actions

   Nice trace, and the failing test is exactly what makes this actionable.

   While the maintainers weigh finer-grained shortening inside an exchange, there's a stopgap that fits this failure mode: instead of clipping tool outputs down to fit the transcript, shunt oversized tool outputs out of the session and hand the agent a pointer it can reopen selectively. The clip decision then happens per tool output rather than per exchange, so the 15k outputs stop costing window without being destroyed, and the summary keeps the room it actually needs.

   Disclosure: I'm the founder of ThumbGate, a guardrails layer for agent sessions that does selective tool-output shunting. You can wire it onto a session with npx thumbgate init; for teams running this in production, the direct route is https://thumbgate.ai/go/pro-direct.