Repository navigation
fix(operations): let provider switches survive pasted images in both directions - #31
Merged
Merged
Conversation
…ction before refusing a shrink A recorded Claude transcript (1,100 entries, 2,458,176 estimated characters) could not be switched to a 288,000-character OpenCode target: after clearing 256 tool outputs, trimming 3 inputs and dropping 15 turns the ladder still held a 549,620-character final turn — a 15-character prompt plus one base64 screenshot — and threw ConversationUnfittableError. No rung read message content, and the drop rung's fallback kept the protected newest turn whole. Rung 3 now replaces image, document and opaque content items inside messages with a text placeholder, oldest first, under the same protection and net-savings guard as the tool-output rung. The ladder runs its clearing rungs a second time with the protection lifted when the drop rung cannot fit any complete turn, on the pre-drop conversation so older turns that now fit are kept, and throws only if the newest turn's own text is still too large. Both are reported (clearedAttachments, clearedAttachmentChars, liftedRecentTurnProtection) so no lossy step is silent. Clearing a tool output is now idempotent, which the second pass requires for the report to describe what the user lost rather than the rung's own arithmetic. The character estimate is unchanged; #30 records why it overstates images. Fixes #28 Refs Juliusolsson05/agent-code#998 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…f copying them verbatim
The Claude native-resume projector pushed every neutral image/document value
into the transcript unchanged. The OpenCode decoder wraps its native
{type:'file', url:'data:…'} part and the Codex decoder its
{type:'input_image', image_url} item, so an OpenCode → Claude switch wrote a
'file' block that the Claude API rejected on the very next prompt
("400 … Input tag 'file' found using 'type' does not match any of the
expected tags"), leaving the session unusable; before that the model could
not see the image at all.
Base64 data URLs in `url` or `image_url` become the observed Claude
{type:'image'|'document', source:{type:'base64', media_type, data}} block,
recorded as a repaired change. Claude-shaped values pass through as before.
Remote URLs and non-PDF documents are dropped with the existing change record
rather than guessed into an unobserved source shape, and a message left empty
by that is dropped as it always was.
Fixes #29
Refs Juliusolsson05/agent-code#999
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 18, 2026
fix(provider-switch): survive pasted images in both switch directions
Juliusolsson05/agent-code#1000
Merged
…mage re-encoding Four read-only reviewers found no blocker and several real defects. The second pass counted one loss twice. A tool input whose bulk is nested can never get under the cap, so its string member was already the bare marker when the lifted pass visited it again; re-truncating the marker saved one character, rewrote "103 characters omitted" as the length of the previous marker, reported three trims for two tool calls and set liftedRecentTurnProtection on a switch where nothing real was removed. The trim marker is now terminal, as the output placeholder already was, and that placeholder is matched whole rather than by prefix so an output that merely begins with those words can still be cleared. "The lift removed something" is read from the counts instead of from object identity, which three separate return statements had been keeping true by accident. The lift also took more than it needed. Each rung stops only when the whole conversation fits, which inside the lifted range it rarely can, so rung 2 cleared every recent tool output on its way to the pasted image that was the cause: 256 to 262 outputs on the recorded transcript for no extra retained history. liftProtection now measures the full lift, then tries the cheaper rung subsets in ladder-cost order and keeps the first that fits and drops no more entries. The recorded transcript keeps its six newest outputs and 32,605 characters instead of 6,975. The Claude projector accepted any image/* and any payload. Images are now the API's closed set (png, jpeg, gif, webp), the payload must be plain base64 and within the 5 MB base64 limit, and everything else drops with a change record: an HEIC or SVG part would otherwise have produced the same permanent 400 this work exists to end. An opaque item carrying such a data URL is re-encoded too, which lets a Claude transcript the old projector already poisoned be healed by a Claude to Claude duplicate or rewind. Also: two rung references the renumbering missed, the header's stale design pointer, a census sentence credited with something it did not measure, and tests for the pre-drop rerun, the PDF gate, the mime fallback and rung idempotency. Every new regression test was mutation-checked. Plan marked complete with the as-built notes and declined findings. Refs #28, #29, #30, #32 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
One user episode on 2026-09-18 hit two defects in this package in sequence, and the debug bundle records both:
{type:'file', mime, url:'data:…', id:'prt_…', …}part landed in the Claude transcript unchanged. Claude answered "I can't view images in this session" twice, and on the next prompt the API rejected the whole conversation:400 messages.985.content.1: Input tag 'file' found using 'type' does not match any of the expected tags: …, 'document', 'image', …. The block stays in history, so the session was unusable from then on. Codexinput_imageitems take the same path.ConversationUnfittableError: … the smallest complete suffix the ladder can produce is 549785 charactersagainst the 288,000-character budget. The transcript (1,100 entries, 2,458,176 estimated characters) ended with a 15-character prompt carrying a 549,526-character base64 screenshot, two turns after a prompt carrying the 713,997-character opaquefilepart from defect 1. The ladder cleared 256 outputs, trimmed 3 inputs, dropped 15 turns — and had no rung that reads message content, while the drop rung's fallback kept the protected newest turn whole.Implemented behaviour
operations/shrink.ts(commit 1, fixes #28)clearAttachments: replacesimage,documentandopaquecontent items inside messages with[<image|document|attachment> omitted during provider switch], oldest first, under the same recent-turn protection and net-savings guard as rung 2. Text items are never touched. The rung sits between clearing outputs and trimming inputs because a stale screenshot is consumed input like a stale tool output, while tool-call inputs hold the session's edits. Report:clearedAttachments(per item),clearedAttachmentChars(net).keepRecentTurns: 0) on the pre-drop conversation and the drop rung runs again; the ladder throws only if the newest turn's own text still exceeds the budget. Re-running on the pre-drop conversation matters: once recent payload is on the table, old turns the first attempt discarded may fit. The lift does not simply run all three clearing rungs: each stops only when the whole conversation fits, which inside the lifted range it rarely can, so that cleared every recent tool output on its way to the image that was the cause.liftProtectionmeasures the full lift, then tries the cheaper rung subsets in ladder-cost order and keeps the first that fits and drops no more entries. Report:liftedRecentTurnProtection, true only when the second pass removed something, read from the counts rather than object identity.clearToolResultsandtrimToolInputsare now idempotent: the output placeholder (matched whole, not by prefix) and the trim marker are terminal. The second pass exposed that a second visit "saved" a character or two, counted one loss twice, rewrote the trim marker to quote the previous marker's length, and flipped the lift flag on a conversation where nothing real was removed.claude/project/nativeResume.ts(commit 2, fixes #29)claudeMessageContentconverts foreign image/document values: a Claude-shaped value passes through; a base64 data URL inurl(OpenCode) orimage_url(Codex) becomes the observed{type:'image'|'document', source:{type:'base64', media_type, data}}block for the API's closed image set (png, jpeg, gif, webp) andapplication/pdf, when the payload is plain base64 and an image is within the API's 5 MB base64 limit, recorded asnative-resume.content.<kind>.repaired; anything else is dropped with the existingdroppedchange, and a message left empty is dropped as before. Remote URLs are not guessed intosource.type: 'url'because no observed Claude fixture has that shape. Anopaqueitem carrying such a data URL is re-encoded too, so a Claude transcript the old projector already poisoned is healed by a Claude → Claude duplicate or rewind.Design decisions
value. Anopaqueitem counts because every cross-provider projector drops it on arrival, so leaving it charged the budget for bytes the target never sees.claude-message-block-image/claude-message-block-documentare emitted.Tests
Written first and failing first (14 red, then green). Census caveat 1 (redacted fixtures prove shape, not size) means every size-driven case is built in the test from the recorded shape;
pastedImageTailConversation()intesting/engine/fixtureConversations.tsdocuments the real transcript it scales down, and the shared entry builders replaced a private copy incontextBudget.test.ts.shrink.test.ts:clearAttachments(oldest first, protection honoured, all three content kinds, text untouched, net-savings guard, no-op when fitting); recorded-shape regression for bug(operations): shrink ladder cannot remove message attachments and never revisits protected turns, so one pasted image makes a conversation unfittable #28 (fits, both attachments cleared, lift reported, drop rung still fires, newest prompt kept with placeholder, never a fragment); lift for a final turn dominated by tool output; no lift when dropping suffices (protected entries identical); still throws when the newest turn's own text exceeds the budget.contextBudget.test.ts:planConversationContext(…, { allowSourceTurns: false })returnsshrunkwith the extended report for the recorded shape.nativeResumeProjection.test.ts: OpenCodefilepart → Claude image block (no"type":"file", noprt_id on the wire); Codex data URL → image block, remote URL dropped with a change; Claude-shaped passthrough + PDF → document block; empty message dropped.Verification
npm run checkon Node 24: contract satisfied, typecheck clean, 26 files / 152 tests passed, packed surface verified. CI green on the final head.planConversationContext(…, 'opencode', 288_000, { allowSourceTurns: false }):shrunkin 70 ms,estimatedCharactersAfter: 32605, report{ clearedResults: 256, clearedAttachments: 2, clearedAttachmentChars: 1263463, trimmedInputs: 3, droppedTurns: 13, droppedEntries: 1055, liftedRecentTurnProtection: true }(the six newest tool outputs survive; the first version of the lift cleared them and kept 6,975 characters); retained history opens on the marker then a complete user turn, and the newest user message isThis [Image #1]+[image omitted during provider switch]. (Scratch script against a personal transcript, not committed.)describeShrink, design doc) runs the app's provider-switch suite against this branch.Review round
Four read-only reviewers across this PR and the host PR, no blocking findings. Fixed in
ec514bc: the non-idempotent input trim and its false lift flag; the wasteful full lift (subset search); over-wide image media types and unvalidated payloads; healing of already-poisoned Claude transcripts; prefix-only placeholder detection; two rung references the renumbering missed; the header's stale design pointer; a census sentence credited with something it did not measure; missing tests (older turns survive the lift, PDF gate,mimefallback, direct rung idempotency). Every new regression test was mutation-checked: reverting its fix fails exactly that test.Declined, with reasons: RFC 2397 parameters and upper-case
;BASE64,are dropped rather than parsed (safe direction, no recorded producer emits them). "Protection outranks the drop rung when the first pass fits" is recorded on #30, where the image overcharge that makes it common is addressed. The PR title keeps theoperationsscope although the second fix isclaude; the repo merges with merge commits, so the title never becomes a commit subject.Known limitations / follow-ups
file_pathwith the marker and still leaves the input over the cap.maxInputCharswould keep more and is a tuning question for the live probe.codexImagedoes not read OpenCodeurl,filePartdoes not read Codeximage_url). They drop with a change record rather than emitting invalid blocks, so they are not in this PR's blast radius; the neutral-hub work (Restructure pairwise translation (A↔B) into neutral-hub codecs (A↔C↔B) with lossless intermediate transcript #5) is the natural home for one shared attachment normaliser.Refs Juliusolsson05/agent-code#998, Juliusolsson05/agent-code#999
🤖 Generated with Claude Code