Skip to content

fix(server): Codex turns that fail with an unlisted error code no longer stay running - #28040

Open
juliusmarminge wants to merge 4 commits into
mainfrom
fix/issue-28032
Open

juliusmarminge wants to merge 4 commits into
mainfrom
fix/issue-28032

Conversation

@juliusmarminge

Copy link
Copy Markdown
Member

Problem

When Codex fails a turn with a codexErrorInfo code our generated schema doesn't list, the T3 run stays running forever with no error item. A ChatGPT-account Codex rejecting a model sends modelNotFound (reported case: gpt-5-codex). Both the error and the failed turn/completed notifications fail schema decode, and dispatchNotification drops them with "Dropped a Codex App Server notification that failed to decode". The adapter never sees the terminal event. This is not a registration race.

Reproduced on main (e5909e5) with the real dev server and the real codex-cli 0.163.0 app-server. Codex's stdout went through a filter that rewrites "codexErrorInfo":"other" to "modelNotFound", which matches the reporter's wire shape. Then t3 thread start <project> "say hi" --provider <that instance> --model gpt-5-codex. The thread was still running 50 s after the native turn/completed {status: "failed"}. Control: the same flow without the filter (code "other") ends as failed.

Change

CodexErrorInfo is a closed list of string codes plus a few structured variants. The generator now widens the string-code branch of the fetched upstream definition to any string (OpenStringEnumDefinitions in packages/effect-codex-app-server/scripts/generate.ts), and schema.gen.ts is regenerated from the same pinned ref. The structured variants stay typed. The only generated change is Schema.Literals([...]) → Schema.String in the 16 copies of CodexErrorInfo. Adapter consumers already compare codes as strings, so an unknown code is classed as a plain provider error and the run fails with Codex's message.

Scope and approval

Fixes #28032

Verification

  • New test fails the turn when Codex reports an error info the schema does not list (CodexAdapterV2.test.ts). It replays error (willRetry: false) then a failed turn/completed, both with codexErrorInfo: "modelNotFound". On main it fails with expected undefined to equal 'failed'; it passes on this branch. With "other" it passes on main too, so it isolates the unlisted code.
  • CodexAdapterV2.test.ts 123/123, packages/effect-codex-app-server tests pass. vp run check:changed passes: format, lint, knip, typecheck of 4 packages, tests.
  • node scripts/generate.ts reproduces the committed generated files (apart from the generator's existing 2-line header).
  • Server-only fix with no visible surface, so there's no capture matrix. Runtime transcript from the real dev server and real Codex app-server, using the filter above:
Before (main e5909e5) After (65c7c6a)
native error / turn/completed modelNotFound / failed modelNotFound / failed
"Dropped a Codex App Server notification that failed to decode" 2 (one per notification) 0
T3 thread after native completion status: running, activeRunId set, 50 s later status: failed, activeRunId: null; thread start --wait exits 1 with Codex's message
# before
2026-10-31T07:49:25.297Z incoming error          codexErrorInfo="modelNotFound"
2026-10-31T07:49:25.310Z incoming turn/completed codexErrorInfo="modelNotFound" status=failed
[07:49:25.299] WARN Dropped a Codex App Server notification that failed to decode
[07:49:25.310] WARN Dropped a Codex App Server notification that failed to decode
# thread show at 07:50:15: {'status': 'running', 'activeRunId': 'run:thread:mcp%3Aaf9fcc0a-…:ordinal:1', 'itemCount': 3}
# after
{"status":"failed","reply":null,"error":"{\"error\":{\"code\":\"model_not_found\",\"message\":\"unknown provider for model gpt-5-codex\",…}}"}
# thread show: {'status': 'failed', 'activeRunId': None, 'itemCount': 3}

Filter used as the Codex instance's binary path (scratch, not committed):

#!/usr/bin/env bash
exec codex "$@" | sed -u 's/"codexErrorInfo":"other"/"codexErrorInfo":"modelNotFound"/g'

Not checked: a live ChatGPT-account Codex. The VM routes Codex through an API proxy, so the failure text above is the proxy's instead of "The 'gpt-5-codex' model is not supported when using Codex with a ChatGPT account." The decode path is the same.

Claude Opus 5.5 (issue lead, red test, implementation) and GPT-6.1 Sol (review) in Claude Code / Codex, through T3 Code.

🤖 Generated with Claude Code

juliusmarminge and others added 4 commits October 31, 2026 07:52
…rror info

A Codex turn that completes failed with codexErrorInfo "modelNotFound"
(sent by ChatGPT-account Codex when it rejects a model) never reaches
turn.terminal: the generated schema's closed CodexErrorInfo union rejects
the value, so the error and turn/completed notifications are dropped at
decode. This test reproduces #28032 and fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex sends codexErrorInfo codes the pinned protocol schema does not list,
such as "modelNotFound" when a ChatGPT account rejects a model. The closed
literal union failed decode, so the client dropped both the error and the
failed turn/completed notifications and the turn stayed running forever.

Override CodexErrorInfo in the generator so its string branch accepts any
code, keeping the typed struct variants, and regenerate the schema.

Fixes #28032

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ing a copy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Oct 31, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 65c7c6a

Macroscope's review found this PR approvable — Unknown Codex error codes can reach existing failure handling instead of leaving failed turns running. The change is limited to error-code decoding and preserves existing turn-status and retry rules.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB −6 B (−0.1%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.3 KiB 1.2 KiB −6 B (−0.5%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.6 KiB 21.6 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 2 2 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB +29 B (+0.6%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.3 KiB +29 B (+2.3%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB +41 B (+0.2%) 24.4 KiB ✅
Claude Live turn messages 1 2 +1 (+100.0%) 8 ✅

Baseline: 1aa46d0 · PR result: 65c7c6a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@coderabbitai

coderabbitai Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 1aa46d0 and 65c7c6a.

Files ignored due to path filters: 1
  • packages/effect-codex-app-server/src/_generated/schema.gen.ts is excluded by !**/_generated/**
Files selected for processing: 2
  • apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.test.ts
  • packages/effect-codex-app-server/scripts/generate.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

Walkthrough

The generated CodexErrorInfo schema now accepts unlisted string values unless an override applies. A Codex adapter replay test covers a failed turn with modelNotFound and checks its terminal status and error message.

Changes: Codex error decoding

Layer / File(s) Summary
Open the error schema and test failed-turn replay
packages/effect-codex-app-server/scripts/generate.ts, apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.test.ts
The generator replaces constrained string variants in CodexErrorInfo with an unconstrained string schema when no definition override applies. The replay test checks that a turn with modelNotFound ends as failed and retains the provider error message.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: maria-rcks

Merge Risk: Minimal · up to 65c7c

Unknown Codex error codes can now produce failed turns with the provider message. No merge-blocking issue remains identified.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary fix: Codex turns with unlisted error codes now reach a failed state instead of remaining running.
Description check Passed The description includes the required Problem, Change, Scope and approval, and Verification sections. It provides detailed reproduction steps, implementation details, issue scope, test results, runtim…
Linked Issues check Passed Issue #28032 requires unsupported Codex model turns to reach a failed terminal state and retain Codex's error message. The PR widens the generated CodexErrorInfo string branch to accept unlisted cod…
Out of Scope Changes check Passed The changed generator logic, generated schema, and regression test directly support issue #28032. The changes preserve typed structured variants and do not introduce unrelated behavior or surface chan…
Full details: Description check

Explanation

The description includes the required Problem, Change, Scope and approval, and Verification sections. It provides detailed reproduction steps, implementation details, issue scope, test results, runtime evidence, and known verification limits.

Full details: Linked Issues check

Explanation

Issue #28032 requires unsupported Codex model turns to reach a failed terminal state and retain Codex's error message. The PR widens the generated CodexErrorInfo string branch to accept unlisted codes, updates the generated schema, and adds a regression test for modelNotFound on both error and failed turn/completed notifications. The reported test and package checks pass. The excluded generated path prevents direct inspection of that file, but the change summary and generator update establish the intended generated change.

Full details: Out of Scope Changes check

Explanation

The changed generator logic, generated schema, and regression test directly support issue #28032. The changes preserve typed structured variants and do not introduce unrelated behavior or surface changes.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@juliusmarminge juliusmarminge added bugfix Fix PR opened by an issue lead; enters the PR loop pr:ready Every gate passed; waits for Julius's land label labels Oct 31, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fix PR opened by an issue lead; enters the PR loop pr:ready Every gate passed; waits for Julius's land label size:XL 500-999 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Codex V2 run stays "running" after the native turn fails immediately (modelNotFound)

1 participant