Skip to content

[codex] Harden overflow auto-compaction recovery #22141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aibrahim-oai merged 13 commits into from
May 11, 2026
Merged

[codex] Harden overflow auto-compaction recovery #22141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
a175208
core: auto-compact on sampling context overflow
aibrahim-oai May 11, 2026
1482455
core: harden overflow compaction retries
aibrahim-oai May 11, 2026
c568bf1
Revert "core: harden overflow compaction retries"
aibrahim-oai May 11, 2026
a568d99
core: bound overflow compaction retries in sampling
aibrahim-oai May 11, 2026
b20b795
core: keep sampling recovery self contained
aibrahim-oai May 11, 2026
6c445f6
core: restore overflow input after compact retry
aibrahim-oai May 11, 2026
c3a51fb
core: restore retry input in sampling
aibrahim-oai May 11, 2026
fb557f6
core: preserve images in compacted user history
aibrahim-oai May 11, 2026
4d17876
core: reuse compacted user text helper
aibrahim-oai May 11, 2026
4b691eb
core: keep empty compacted user text
aibrahim-oai May 11, 2026
2aa791e
Merge branch 'main' into dev/auto-compact-context-overflow
aibrahim-oai May 11, 2026
c2bfd5e
core: preserve compacted text segment order
aibrahim-oai May 11, 2026
77769ef
core: fix compact truncation test clone
aibrahim-oai May 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 46 additions & 16 deletions codex-rs/core/src/compact.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,15 +386,15 @@ pub fn content_items_to_text(content: &[ContentItem]) -> Option<String> {
}
}

pub(crate) fn collect_user_messages(items: &[ResponseItem]) -> Vec<String> {
pub(crate) fn collect_user_messages(items: &[ResponseItem]) -> Vec<Vec<UserInput>> {
items
.iter()
.filter_map(|item| match crate::event_mapping::parse_turn_item(item) {
Some(TurnItem::UserMessage(user)) => {
if is_summary_message(&user.message()) {
None
} else {
Some(user.message())
Some(user.content)
}
}
_ => None,
Expand Down Expand Up @@ -465,7 +465,7 @@ pub(crate) fn insert_initial_context_before_last_real_user_or_summary(

pub(crate) fn build_compacted_history(
initial_context: Vec<ResponseItem>,
user_messages: &[String],
user_messages: &[Vec<UserInput>],
summary_text: &str,
) -> Vec<ResponseItem> {
build_compacted_history_with_limit(
Expand All @@ -478,39 +478,33 @@ pub(crate) fn build_compacted_history(

fn build_compacted_history_with_limit(
mut history: Vec<ResponseItem>,
user_messages: &[String],
user_messages: &[Vec<UserInput>],
summary_text: &str,
max_tokens: usize,
) -> Vec<ResponseItem> {
let mut selected_messages: Vec<String> = Vec::new();
let mut selected_messages: Vec<Vec<UserInput>> = Vec::new();
if max_tokens > 0 {
let mut remaining = max_tokens;
for message in user_messages.iter().rev() {
if remaining == 0 {
break;
}
let tokens = approx_token_count(message);
let message_text = user_message_text(message);
let tokens = approx_token_count(&message_text);
if tokens <= remaining {
selected_messages.push(message.clone());
Comment on lines +492 to 495

@chatgpt-codex-connector chatgpt-codex-connector Bot May 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Count preserved images against the compaction budget

The selection budget only counts text tokens, but the new structured path keeps every non-text item. Image-only messages have tokens == 0, so the reverse scan can preserve an unbounded number of historical images without reducing remaining, and mixed messages keep all images even when text is truncated. Image-heavy threads can still overflow after “successful” compaction.

Useful? React with 👍 / 👎.

remaining = remaining.saturating_sub(tokens);
} else {
let truncated = truncate_text(message, TruncationPolicy::Tokens(remaining));
let truncated = truncate_user_message(message, remaining);
selected_messages.push(truncated);
break;
}
}
selected_messages.reverse();
}

for message in &selected_messages {
history.push(ResponseItem::Message {
id: None,
role: "user".to_string(),
content: vec![ContentItem::InputText {
text: message.clone(),
}],
phase: None,
});
for message in selected_messages {
history.push(ResponseItem::from(ResponseInputItem::from(message)));
}

let summary_text = if summary_text.is_empty() {
Expand All @@ -529,6 +523,42 @@ fn build_compacted_history_with_limit(
history
}

pub(crate) fn user_message_text(message: &[UserInput]) -> String {
message
.iter()
.filter_map(|item| match item {
UserInput::Text { text, .. } => Some(text.as_str()),
_ => None,
})
.collect::<Vec<_>>()
.join("")
}

fn truncate_user_message(message: &[UserInput], remaining_tokens: usize) -> Vec<UserInput> {
let mut remaining_tokens = remaining_tokens;
let mut truncated = Vec::with_capacity(message.len());
for item in message {
match item {
UserInput::Text { text, .. } if remaining_tokens > 0 => {
let token_count = approx_token_count(text);
if token_count <= remaining_tokens {
truncated.push(item.clone());
remaining_tokens = remaining_tokens.saturating_sub(token_count);
} else {
truncated.push(UserInput::Text {
text: truncate_text(text, TruncationPolicy::Tokens(remaining_tokens)),
text_elements: Vec::new(),
});
remaining_tokens = 0;
}
}
UserInput::Text { .. } => {}
_ => truncated.push(item.clone()),
}
}
truncated
}

async fn drain_to_completed(
sess: &Session,
turn_context: &TurnContext,
Expand Down
90 changes: 82 additions & 8 deletions codex-rs/core/src/compact_tests.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ fn content_items_to_text_ignores_image_only_content() {
}

#[test]
fn collect_user_messages_extracts_user_text_only() {
fn collect_user_messages_preserves_user_content() {
let items = vec![
ResponseItem::Message {
id: Some("assistant".to_string()),
Expand All @@ -68,17 +68,34 @@ fn collect_user_messages_extracts_user_text_only() {
ResponseItem::Message {
id: Some("user".to_string()),
role: "user".to_string(),
content: vec![ContentItem::InputText {
text: "first".to_string(),
}],
content: vec![
ContentItem::InputImage {
image_url: "file://image.png".to_string(),
detail: Some(DEFAULT_IMAGE_DETAIL),
},
ContentItem::InputText {
text: "first".to_string(),
},
],
phase: None,
},
ResponseItem::Other,
];

let collected = collect_user_messages(&items);

assert_eq!(vec!["first".to_string()], collected);
assert_eq!(
vec![vec![
UserInput::Image {
image_url: "file://image.png".to_string(),
},
UserInput::Text {
text: "first".to_string(),
text_elements: Vec::new(),
},
]],
collected
);
}

#[test]
Expand Down Expand Up @@ -117,7 +134,13 @@ do things

let collected = collect_user_messages(&items);

assert_eq!(vec!["real user message".to_string()], collected);
assert_eq!(
vec![vec![UserInput::Text {
text: "real user message".to_string(),
text_elements: Vec::new(),
}]],
collected
);
}

#[test]
Expand All @@ -126,9 +149,13 @@ fn build_token_limited_compacted_history_truncates_overlong_user_messages() {
// that oversized user content is truncated.
let max_tokens = 16;
let big = "word ".repeat(200);
let user_message = vec![UserInput::Text {
text: big.clone(),
text_elements: Vec::new(),
}];
let history = super::build_compacted_history_with_limit(
Vec::new(),
std::slice::from_ref(&big),
std::slice::from_ref(&user_message),
"SUMMARY",
max_tokens,
);
Expand Down Expand Up @@ -162,10 +189,57 @@ fn build_token_limited_compacted_history_truncates_overlong_user_messages() {
assert_eq!(summary_text, "SUMMARY");
}

#[test]
fn truncate_user_message_preserves_text_segment_order_around_images() {
let before_image = "before ".repeat(8);
let after_image = "after ".repeat(200);
let before_image_tokens = approx_token_count(&before_image);
let after_image_token_budget = 16;
let remaining_tokens = before_image_tokens + after_image_token_budget;
let user_message = vec![
UserInput::Text {
text: before_image.clone(),
text_elements: Vec::new(),
},
UserInput::Image {
image_url: "file://image.png".to_string(),
},
UserInput::Text {
text: after_image.clone(),
text_elements: Vec::new(),
},
];

let truncated = super::truncate_user_message(&user_message, remaining_tokens);

assert_eq!(
vec![
UserInput::Text {
text: before_image,
text_elements: Vec::new(),
},
UserInput::Image {
image_url: "file://image.png".to_string(),
},
UserInput::Text {
text: truncate_text(
&after_image,
TruncationPolicy::Tokens(after_image_token_budget)
),
text_elements: Vec::new(),
},
],
truncated
);
}

#[test]
fn build_token_limited_compacted_history_appends_summary_message() {
let initial_context: Vec<ResponseItem> = Vec::new();
let user_messages = vec!["first user message".to_string()];
let user_messages = vec![vec![UserInput::Text {
text: "first user message".to_string(),
text_elements: Vec::new(),
}]];
let summary_text = "summary text";

let history = build_compacted_history(initial_context, &user_messages, summary_text);
Expand Down
33 changes: 29 additions & 4 deletions codex-rs/core/src/session/turn.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ use crate::compact::InitialContextInjection;
use crate::compact::collect_user_messages;
use crate::compact::run_inline_auto_compact_task;
use crate::compact::should_use_remote_compact_task;
use crate::compact::user_message_text;
use crate::compact_remote::run_inline_remote_auto_compact_task;
use crate::compact_remote_v2::run_inline_remote_auto_compact_task as run_inline_remote_auto_compact_task_v2;
use crate::connectors;
Expand Down Expand Up @@ -310,7 +311,7 @@ pub(crate) async fn run_turn(
Vec::new()
} else {
let initial_input_for_turn: ResponseInputItem = ResponseInputItem::from(input.clone());
let response_item: ResponseItem = initial_input_for_turn.clone().into();
let response_item: ResponseItem = initial_input_for_turn.into();
let user_prompt_submit_outcome = run_user_prompt_submit_hooks(
&sess,
&turn_context,
Expand Down Expand Up @@ -918,7 +919,11 @@ pub(super) fn filter_connectors_for_input(
return Vec::new();
}

let mentions = collect_tool_mentions_from_messages(&user_messages);
let user_message_texts = user_messages
.iter()
.map(|message| user_message_text(message))
.collect::<Vec<_>>();
let mentions = collect_tool_mentions_from_messages(&user_message_texts);
let mention_names_lower = mentions
.plain_names
.iter()
Expand Down Expand Up @@ -1041,6 +1046,7 @@ async fn run_sampling_request(
Arc::clone(&turn_diff_tracker),
)
.await;
let max_retries = turn_context.provider.info().stream_max_retries();
let mut retries = 0;
let mut initial_input = Some(input);
loop {
Expand Down Expand Up @@ -1074,7 +1080,27 @@ async fn run_sampling_request(
}
Err(CodexErr::ContextWindowExceeded) => {
sess.set_total_tokens_full(&turn_context).await;
return Err(CodexErr::ContextWindowExceeded);
if retries >= max_retries {
return Err(CodexErr::ContextWindowExceeded);
}
retries += 1;
let reset_client_session = match run_auto_compact(
&sess,
&turn_context,
client_session,
InitialContextInjection::BeforeLastUserMessage,
CompactionReason::ContextLimit,
CompactionPhase::MidTurn,
)
.await
{
Ok(reset_client_session) => reset_client_session,
Err(_) => return Err(CodexErr::TurnAborted),
};
if reset_client_session {
client_session.reset_websocket_session();
Comment on lines 19 to +1101

@pakrym-oai pakrym-oai May 11, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need to do this.

}
continue;

@chatgpt-codex-connector chatgpt-codex-connector Bot May 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore the overflowing user turn before retrying

After this continue, the retry is built from compacted history, but the original input was already consumed and is never reinstalled. Local/remote compaction rebuilds history from text summaries, so an overflowing turn with images or text beyond the compaction limit can be retried with content dropped or truncated before the model ever sees it.

Useful? React with 👍 / 👎.

}
Err(CodexErr::UsageLimitReached(e)) => {
let rate_limits = e.rate_limits.clone();
Expand All @@ -1091,7 +1117,6 @@ async fn run_sampling_request(
}

// Use the configured provider-specific stream retry budget.
let max_retries = turn_context.provider.info().stream_max_retries();
if retries >= max_retries
&& client_session.try_switch_fallback_transport(
&turn_context.session_telemetry,
Expand Down
Loading
Loading