[BUG] V2: Auto-compaction accepts task-empty summary after shortened transcript, silently losing active objective (Ollama/Qwen) #60663

Activity

1. opencodejr commented on Oct 30, 2026

   

   opencodejrBot

   on Oct 30, 2026

   More actions

   Thanks for the detailed write-up and the token and Ollama data, they made this much easier to trace.

   I rebuilt your scenario on the latest dev (cb1e58f) with a simulated model at your limits (65,536 context, 16,384 output):

   • A user message sets an objective, and compaction feat: compact and other improvements #1 writes a summary with a full ## Objective.
   • With no new user message, a large tool step triggers automatic compaction Roadmap & Existing Issues #2. It goes through the shortened text path (flattenAndDropOldest).
   • The scripted model replies with a task-empty summary.

   What I found:

   • The previous summary is sent. The compaction Roadmap & Existing Issues #2 request starts with the previous summary, Objective included, inside <conversation-checkpoint>. Then come the shortened transcript and the "Update the existing checkpoint…" prompt. So on our side the objective isn't dropped. In your session, the model received it and ignored it (your mechanism b).
   • This likely explains the small 8,397-token request. With no user message since the last compaction, the whole history counts as one exchange. Shortening works one exchange at a time, so every tool output in it is cut to 2,000 characters at each end in one go, even though much more room was left. That is coarse, but it is what the code was written to do. It is not lost content.
   • The empty summary is accepted. It is stored as completed because OpenCode only checks that the reply follows the template (any required heading). It doesn't check that the content is meaningful.

   The last point is how it was built rather than a malfunction, so it may be better filed as a feature request. For example: reject or retry a summary that drops a previous non-empty Objective, or carry the Objective forward. If you can share the first lines of the actual compaction #2 request, or show that its previous summary was missing, please reply and we'll look again.

   Related: #57240 (empty checkpoint replies marked succeeded), #50816 (stale summary accepted), #44080 (empty summary accepted on v1).
2. ivanfc commented on Oct 30, 2026

   

   ivanfc

   on Oct 30, 2026

   Author

   More actions

   I disagree that this is a feature request. From the user's side the result is silent data loss: the active objective disappears, the compaction is stored as "completed", and the agent restarts as if the session were new, with no warning and no way to notice except that it suddenly greets you. "Working as built" doesn't make that acceptable behavior; a summary that throws away a previously recorded objective should never silently replace a valid context.

   For the record: on this machine all 17 automatic compactions so far are from local Qwen at a 64K window, and 2 of them came out task-empty. That's not a rare edge case for local models, which are exactly the ones most likely to ignore a checkpoint inside a long "update" prompt.

   The minimum fix I'd expect as a bug fix, not an enhancement: if the previous summary has a non-empty ## Objective and the new one doesn't, don't mark it completed: retry once, and if it still fails, keep the previous compaction and surface an error. Please keep this open as a bug.
3. John-Rood commented on Oct 31, 2026

   

   John-Rood

   on Oct 31, 2026

   More actions

   Three things in this thread are still open at the code level, and all three resolve in the v2.2.3 tree (082ac62, unchanged on current dev). Two of them come out your way.

   Where the 8,397 tokens went

   The target was not small. For an automatic compaction the budget is calculateCeiling (compaction.ts:1134). With your limits, context 65,536 and output 16,384, that is 65,536 - 16,000 = 49,536, and summaryRoom (:1148) returns the same number because 15% of the window is 9,830, under RESERVE_MIN. The summary request aimed at about 48.5k once prompt overhead is subtracted.

   What collapsed is the transcript, and the exact line is the level loop.

   TRANSCRIPT_CLIP = 2_000 (:150) with clip() (:1074) keeps 1,000 characters from the start and 1,000 from the end of a cut value, so it is 2,000 total, not 2,000 at each end.

   Shortening walks whole levels over each exchange (:742-751): whole, tool outputs clipped, tool outputs removed, user text only, omitted. There is no state between those levels.

   With no user message after compaction #1, recentStart (:1005) keeps nothing, splitConversation (:985) returns keepID: undefined, and flattenAndDropOldest sees one group, because grouping starts a new group when the list is empty or the message is a user message (:715). Your entire post-checkpoint history is that one group.

   Level 0 is the full 55k, over room. The next reachable state is OUTPUTS_CLIPPED, which cuts every tool result to 2,000 characters in a single pass. Eight tool results across the two steps is 16,000 characters, about 4,000 tokens, sitting in front of the roughly 9 KB checkpoint that lead rebuilds (:702-704) at about 2,250 tokens, plus the summary prompt and the system message. 8,397 is what that adds up to. Nothing is removed elsewhere.

   That also answers your point 5. The count behind compaction #2 was 38,963 input + 16,598 cache read + 3,240 output = 58,801, against Ollama's n_tokens = 58800. estimatePrompt (:1103) takes the last measured assistant row and adds an estimate of what came after it, so the figure was current to within a token, and due (:422) fires at >= ceiling. There is no stale trigger to fix.

   The checkpoint was in the request

   Not only in your rebuild. lead is built from previousCompaction (:967) and rendered by to-llm-message.ts:293 as <conversation-checkpoint><summary>. No branch on the flattened path drops it. So your mechanism (b) is the one that happened: the model was given the objective and did not repeat it, and the code then accepted the result.

   The acceptance check is the defect

   filled (:469) is text.split("\n").some((line) => headings.includes(line.trim())). One line that equals a template heading is enough, so a summary whose Objective reads "(none, no task has been established yet)" passes on the first attempt and publish stores it as completed (:921).

   Keeping the previous context on failure is already the behaviour, one line further down. A compaction that is not completed renders as no messages at all (to-llm-message.ts:294), and history loads from the latest completed one, so a rejected summary leaves the old checkpoint as the lead. The auto path then fails the step (runner/llm.ts:243), which surfaces as a compaction error instead of a greeting.

   Your point 1 and point 3 are one change, and it is a predicate.

   The predicate

   One condition keeps it from failing legitimate compactions: only enforce it when no user turn has arrived since the checkpoint. After a user turn the update prompt allows the objective to change ("Newer history always takes precedence over the existing checkpoint", buildPrompt :233).

   const objectiveOf = (text: string) => {
     const lines = text.split("\n")
     const start = lines.findIndex((line) => line.trim() === "## Objective")
     if (start < 0) return ""
     const body: string[] = []
     for (const line of lines.slice(start + 1)) {
       if (line.trim().startsWith("## ")) break
       body.push(line)
     }
     return body.join("\n").replace(/^\s*[-*]\s*/gm, "").trim()
   }
   const emptyObjective = (value: string) => value === "" || /^\(?\s*none\b/i.test(value)
   
   // inside summarize(), beside `previous`
   const userSince = messages.slice(previous === undefined ? 0 : 1).some((message) => message.type === "user")
   const keepsObjective = (text: string) =>
     previous === undefined ||
     userSince ||
     emptyObjective(objectiveOf(previous.summary)) ||
     !emptyObjective(objectiveOf(text))
   
   const filled = (text: string) =>
     keepsObjective(text) && text.split("\n").some((line) => headings.includes(line.trim()))

   The existing send already provides the retry you asked for. A reply that fails filled gets NUDGE once, and a second failure becomes compaction.failed. With the predicate in place your task-empty summary gets one more attempt, and if it comes back empty again the turn fails visibly. Worth giving that case its own failure string so "did not match the required template" does not describe it.

   What I ran

   I pulled the real SUMMARY_TEMPLATE and the real filled out of the tag instead of retyping them, then ran six cases: your summary passes today and is rejected with the predicate; a healthy summary still passes; a first compaction with no previous checkpoint is unaffected; a checkpoint whose objective is already "(none)" is unaffected; a compaction after a user turn is unaffected; a reply carrying only a legacy heading is still rejected. Six of six.

   A second bug worth its own issue

   The coarse step. When the newest exchange is the only exchange, the loop cannot shorten anything older first, so it wipes detail inside the exchange the summary most needs while about 40k tokens of room go unused. Your incident has exactly that shape, which is why the compaction request looks nothing like the conversation it summarized. Shortening within an exchange, clipping the largest tool results before the rest, would recover the room. It is not required to stop the silent loss, but it is a real second defect and it is the one that made the request numbers look impossible.

   Keep this open as a bug. The predicate cannot tell a usable summary from one that says there is no task, and the failure path you asked for already exists, so changing one boolean is not a design decision.

   The cross-session version of the same loss is the problem I built MemoryRouter for: the conversation kept outside the context window and across compaction, so a checkpoint that comes back thin is not the only copy of the work. https://docs.memoryrouter.ai/coding-tools?utm_source=github&utm_medium=issue&utm_campaign=anomalyco-opencode
4. opencodejr

   closed this as not plannedon Nov 2, 2026
5. opencodejr commented on Nov 2, 2026

   

   opencodejrBot

   on Nov 2, 2026

   More actions

   Closing this as not planned, since we haven't heard back. On the latest dev (cb1e58f), the shortened compaction request still put the previous summary's ## Objective in front of the model. The task-empty summary was the model ignoring it. OpenCode stored it as completed because it only checks the template structure, which is how it was built.

   If you'd like OpenCode to reject or carry forward a summary that drops a previous Objective, please open a feature request. If you can show the compaction #2 request from your session was missing the previous summary, comment here or open a new issue with that and we'll look again. Related: #57240, #50816.
6. opencodejr

   reopened this on Nov 3, 2026