Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/plans/2026-09-27-codex-exec-wrapped-output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Codex exec_command results survive in resumed history (#1321)

## Evidence
- **Census of 2,541 local rollouts.** It found 85,355 `function_call_output` lines whose output is wrapped: `Chunk ID:`, `Wall time:`, `Process exited with code N`, `Original token count:`, then `Output:`. It found **0** `exec_command_end` events in any file.
- **Current Codex never writes that event.** codex-rs `rollout/src/policy.rs` puts `EventMsg::ExecCommandEnd` under "Transient, non-durable events". rust-v0.107.0 through v0.136.0 did persist it in extended-history mode (review a), and none of the local rollouts use that mode.
- **The drop rule.** `mapCodexRolloutToFeedEntries` drops every wrapped output that has an exit line (`isCodexExecWrapperOutput`), on the stated belief that "the correlated `exec_command_end` event carries the same result". That belief is false, so the dropped line was the only copy. Every `exec_command` card in resumed history showed no output and no exit status. That covers Codex through 0.144: `exec_command` wrapped outputs occur in 0.1xx through 0.144, and 0.15x uses the `exec` tool.
- **A second bug.** The predicate searched the WHOLE string for "Process exited with code". A still-running chunk whose command output contains those words was dropped too.

## Change
- **Header-only parsing.** `codexExecWrapperExitCode(output)` reads the exit code from the header only, the part before `\nOutput:\n`. It returns null for unwrapped output and for a still-running chunk.
- **Keep the result.** A wrapped, finished output now maps to a tool result with:
- the stripped output (possibly empty);
- `is_error = exitCode !== 0`;
- the same `codex` metadata the event branch produces (`kind: 'exec_command_end'`, `exitCode`, empty `parsedCmd` and `command`, null `cwd`).

So the command adapter reads it as the native transport it is: exit-proven, the output owned, and an empty success absorbed by the row dispatcher as before.
- **Comments corrected.** The stale comments that say the wrapper comes from, or is duplicated by, `exec_command_end` are fixed.

## Tests
`execWrappedOutput.test.ts` runs three real 0.132.0 pairs (exit 0 with output, exit 1, exit 0 with no output) through the real mapper and the real `fromCodexCommandOperation`, plus a still-running chunk whose body contains the exit words. All four are red on main.

## Not in scope
The 0.15x `exec` tool's `custom_tool_call_output` results are a different carrier, already handled by the code-mode envelope path.

## Review a (round 1)
- **P1: a running chunk read as a success.** A wrapper whose header says "Process running with session ID N" is a partial chunk. The command's exit arrives on a later `write_stdin` result with another call_id. The adapter's native branch claimed `exitProven: true` for every result, so the card said success. The mapper now marks such a chunk `codex.kind: 'exec_command_running'`, and the adapter does not claim a proven exit for it, so it shows `unknown`. A first attempt, "no exit code means unproven", also flipped synthetic Git-formatter test results that carry no metadata. The explicit mark keeps the change to the one shape the review found. Correlating the later `write_stdin` exit back to the card by session id is a separate feature, not done here.
- **P2: two carriers.** In extended-history rollouts, one call can have both the event and the wrapper. `createCodexTranscriptEntryMapper` keeps the first exec terminal result per call_id, remembering the last 512 per stream. A pair split across a history page boundary is not caught.
- **P3: no `Output:` marker.** Without the LF `\nOutput:\n` marker the wrapper is not parsed, so no exit claim and no stripping. None of the 85,355 local wrappers lacks it.
- **Unpinned metadata.** The exact `codex` metadata is now asserted, which kills the kind/parsedCmd/command/cwd mutations.

## Review b (round 2)
- **P2: a wrapper with no LF `Output:` marker still painted success.** The mapper returned a plain, metadata-less result, and the native adapter proves every unmarked result. Any `Chunk ID:` wrapper whose header cannot be parsed is now marked `exec_command_unparsed` and kept whole; the adapter shows `unknown` for it, as for `exec_command_running`. Test: that wrapper, through the adapter, shows `unknown` with a null exit.
- **P2: first-carrier dedupe kept the truncated event.** The persisted event's `aggregated_output` is sanitized to 10,000 bytes, and extended mode persisted it through v0.136.0, not v0.131.0. The wrapper now wins. An event that arrives after its wrapper is dropped. A wrapper that follows its event is kept, and later-wins `buildToolResultIndex` hands the card the wrapper. Codex emits the event before the function result (`ToolEventEmitter::finish`).
- **P2: page and burst boundaries bypass the per-mapper memory.** Accepted and documented. Across a boundary both results are kept and later-wins decides, which again favours the wrapper in Codex's emit order. No local rollout has both carriers.
25 changes: 18 additions & 7 deletions src/providers/codex/renderer/adapters/command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -708,13 +708,24 @@ function commandResultEvidence(
output: materialized,
exitCode: nativeExit,
failed,
// Native exec_command_end results always carry exit-derived evidence:
// rollout.ts computes is_error from `exit_code !== 0 || status ===
// 'failed'` and stamps codex.exitCode. is_error === false is therefore a
// proven success on THIS transport — unlike code-mode
// custom_tool_call_output, whose is_error never reflects the inner
// command.
exitProven: true,
// Native exec_command_end results carry exit-derived evidence:
// rollout.ts computes is_error from the exit code on both terminal
// carriers (the exec_command_end event, and the wrapped
// function_call_output with a "Process exited with code N" header) and
// stamps codex.exitCode. is_error === false is therefore a proven
// success on THIS transport — unlike code-mode custom_tool_call_output,
// whose is_error never reflects the inner command.
//
// EXCEPT a still-running chunk (#1395 review a, P1). A wrapper whose
// header says "Process running with session ID N" is partial: the
// command goes on in that session, and its exit arrives on a later
// write_stdin result with a different call_id, which this card cannot
// see. rollout.ts marks it `exec_command_running`. Claiming success for
// it painted long-running and later-failing commands green (9,896 such
// chunks in the local corpus); its honest state is "unknown". The same
// holds for a wrapper whose header could not be parsed
// (`exec_command_unparsed`, #1395 review b).
exitProven: codex?.kind === 'exec_command_running' || codex?.kind === 'exec_command_unparsed' ? failed : true,
running: false,
owned: true,
}
Expand Down
59 changes: 48 additions & 11 deletions src/providers/codex/renderer/transcript/entries.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,24 +133,61 @@ function partToBlock(part: ResultPart): { type: string; text?: string; [key: str
}
}

/** Codex's `exec_command_end` wraps its output in a "Chunk ID: …\nOutput:\n<real output>"
* envelope. The user never wants to see that wrapper — strip it. */
/** Codex's `exec_command` result (the `function_call_output` rollout line,
* through 0.144) wraps its output in a "Chunk ID: …\nOutput:\n<real output>"
* envelope. The user never wants to see that wrapper — strip it. (This used
* to say the envelope came from `exec_command_end`; that event is never
* persisted, #1321.) */
export function stripCodexExecWrapper(output: string): string {
const marker = '\nOutput:\n'
const idx = output.indexOf(marker)
if (!output.startsWith('Chunk ID:') || idx === -1) return output
return output.slice(idx + marker.length)
}

/** True for ANY exec-wrapped output ("Chunk ID: …" with a "Process exited
* with code …" line), stdout or not. The rollout mapper drops these
* `function_call_output` lines because the correlated `exec_command_end`
* event carries the same result, with exit code and command, and renders
* the card; keeping both would duplicate it. (This comment used to say
* "only the wrapper and nothing else", which the code never did; #1298
* review B.) */
export function isCodexExecWrapperOutput(output: string): boolean {
return output.startsWith('Chunk ID:') && output.includes('\nProcess exited with code ')
/** The exit code in an exec-wrapped output's header ("Chunk ID: …" …
* "Process exited with code N" … "Output:"), or null when the output is not
* wrapped or the process was still running ("Process running with session
* ID …", a partial chunk a later write_stdin/poll continues).
*
* WHY only the header is read: the body is the command's own bytes, and a
* command can print "Process exited with code 0" itself (a cat of a captured
* transcript). The old `includes` test scanned the whole string.
*
* WHY this replaced `isCodexExecWrapperOutput` (#1321): that predicate made the
* rollout mapper DROP every wrapped result, on the belief that a correlated
* `exec_command_end` event carried the same result. Current Codex never
* persists that event (codex-rs `rollout/src/policy.rs` lists
* `EventMsg::ExecCommandEnd` as transient), and a census of 2,541 local
* rollouts found 0 of them against 85,355 wrapped outputs with an exit line.
* (rust-v0.107.0 through v0.136.0 did persist it in extended-history mode;
* the transcript mapper then prefers this wrapper, the fuller carrier.) The drop therefore
* removed the ONLY copy of every `exec_command` result (Codex through 0.144)
* from resumed history, and the card showed no output or exit status. */
export function codexExecWrapperExitCode(output: string): number | null {
if (!output.startsWith('Chunk ID:')) return null
// WHY the marker is required (#1395 review a, P3): without it there is no
// header/body boundary, so the "header" would be the whole string and the
// unstripped wrapper would render as a finished result's output. Every one
// of 85,355 finished local wrappers has the LF marker; anything else is not
// a shape we have seen, and it falls back to a plain result with no exit
// claim (the adapter then shows an unproven outcome, not a success).
const outputMarker = output.indexOf('\nOutput:\n')
if (outputMarker === -1) return null
const header = output.slice(0, outputMarker)
const match = /\nProcess exited with code (-?\d+)(?:\n|$)/.exec(header)
return match ? Number(match[1]) : null
}

/** True for a wrapped exec output whose header says the process is still
* running ("Process running with session ID N"): a partial chunk whose exit
* arrives later, on a write_stdin result with another call_id. Header only,
* for the same reason as codexExecWrapperExitCode. */
export function isCodexExecWrapperRunning(output: string): boolean {
if (!output.startsWith('Chunk ID:')) return false
const outputMarker = output.indexOf('\nOutput:\n')
if (outputMarker === -1) return false
return /\nProcess running with session ID \S+(?:\n|$)/.test(output.slice(0, outputMarker))
}

/** Build a Claude-shaped assistant Entry containing a single
Expand Down
Loading
Loading