Goals: active goal continuation prompt and audit requirements can be lost after mid-turn compaction

[Chriss4123]

@etraut-openai first off, incredible work on the new goals feature. This fixed the most painful part of using Codex: having the model sandbag its work and not actually complete what you want fully. This is genuinely life changing stuff.

There is quite a persistent issue with it, however. That issue is that after compaction, the goal is sometimes not carried through cleanly to the new agent.

I have been using the goals feature extensively over the past few days, and none of the times I have ran it on ambitious work has the work been completed, and that is due to one specific reason: compaction.

Often, the compaction message does not carry the goal task through cleanly, and more specifically it often does not carry through cleanly the most important "perform a completion audit against the actual current state" requirement of the goal prompt.

This leaves a concrete failure mode: the agent is busy finishing up a local task of its own -- for example running tests and verifying its changes. Compaction happens in the model of this finishing up process, and the compaction message conveys to the agent with fresh context that this specific task is almost done, the previous agent was just verifying their changes, continue with the verification and subsequent completion. This new agent then proceeds to perform the verification process, and after it is done mark the goal as complete because it was only fed that the local task and not the global goal and the audit requirements of it (the bigger picture).

I have found this to be the root cause of the agent prematurely stopping:

• After compaction in these circumstances, the new agent often sends a message to the commentary channel before running any tools like "Let me finish running tests so I can wrap this goal up". The agent then runs the tests, looks at the git status and a few modified files, and that evidence is enough to satisfy its confirmation bias to mark the goal as complete. This is a red flag as it shows the agent has treated completion of the goal as a foregone conclusion, then interpreted circumstantial evidence as confirmation.
• I have reverse engineered the opaque compaction message in these scenarios, and it clearly does not convey the completion audit requirement of the goals prompt effectively, if at all.

I have implemented a fix locally for this where the active goal continuation prompt is now reattached during mid-turn compaction for both local and remote compaction paths, using the persisted active goal and only when the current turn is the synthetic goal-continuation turn. Manual/pre-turn compaction stays unchanged to avoid duplicate prompts.

I have ran many ambitious, long-horizon tasks after this change and the rate of premature closure has dropped significantly due to these changes eliminating the primary failure mode.

This is the correct solution as:

• The continuation prompt without the {{ objective }} placeholder is 462 tokens (via o200k_base). Add a relatively average user goal prompt and that is likely 500-1000 tokens. This is a perfectly acceptable amount of tokens to inject after compaction.
• This keeps the persisted goal table as the source of truth and avoids depending on the compacted summary to remember the active objective.
• The orthogonal-to-this-solution way to fix this failure mode is essentially adding a tool to retrieve the goal prompt for the new agent to call, which is the wrong fix as if the agent already has the goal in their context, this tool is practically useless. The only scenario the agent will not have the goal in its context is compaction, which makes the correct fix injecting the goal directly after the only time the model may lose it.

An alternative acceptable solution is making and injecting after compaction a brand new compaction-centered message with the goal. This is still not the optimal fix in my opinion as the current continuation prompt is short and is already in the optimal shape for a post-compaction injection ("Continue working toward the active thread goal."), and this would just be reinventing a prompt to achieve the exact same outcome as the current continuation prompt.

I have a small local patch of roughly 193 lines for this and would be happy to open a PR or discuss the approach further, since I understand goals is a WIP and likely still evolving.

[github-actions]

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• Codex auto-compact loses critical task details and causes wrong execution, even on small fixes #18720

Powered by Codex Action

[etraut-openai]

@Chriss4123, thanks for trying out the new /goals feature. I appreciate the feedback!

I've done quite a bit of testing with the feature, and I haven't observed the failure mode that you're describing. I also haven't received any reports of this from other OpenAI employees who are testing the feature.

I did discover a problem with the implementation that could cause the behavior you're seeing, but it's not related to compaction. I have a PR posted with a fix, and it's currently awaiting code review. You're welcome to give it a try if you don't mind building the CLI from source. Or you can wait until the fix makes its way into an upcoming release. I'm hopeful that this will address the problem you're seeing.

Out of curiosity, which model and reasoning level were you using for your experimentation? I've done most of my testing with gpt-5.5 xhigh. It's possible that the behavior changes from one model to another.

[Chriss4123]

@etraut-openai same as you, I use gpt-5.5 xhigh (exclusively in my case).

One theory I have is the potential structure of the work is different. What I do is write Markdown docs or have Codex write them around the architecture I want, and tell /goals to go implement it fully.

Oftentimes these docs can get big for complex work, sometimes reaching 20K tokens. This is likely harder to carry through via compaction.

Also the nature of the work. The tasks I am assigning to /goals are often prohibitively complex (requiring a lot of work across hundreds of files and tens of thousands of lines of code added) and on behemoth codebases such as Chromium. The huge codebases I work on means compaction is much more frequent for me.

The nature of my work introduces a much more complex decision to make when ending the task: are the docs fully and optimally implemented end-to-end? This is much harder as there is no easy signal to ascertain whether the goal is actually complete -- something like carrying a migration to completion or fixing a regression has a clear finished signal. That completion signal is likely much harder to carry through in repeated compactions if the model did not end its turn (did not get the continuation prompt in between the two compactions) and is therefore relying on a summary-of-a-summary. This is also frequently when this issue occurs -- after 20 or so compactions without the model yielding.

I want to leave you with something more than "trust me, bro", however for some reason the threads with --enable goals on my testing of this feature a few hours after your PRs got merged are just... not there. I do still have reviews from that day.

Here's a screenshot of when I asked an agent in a fresh chat to check if a document was fully actioned. It was around a pathological full saturation of the CPU in Zed when opening extremely large repos with many nested sub-repos. The solution was a full overhaul of the "hydrate all info at startup" to a lazy-loading model. The task was extreme in scale, spanning the whole Git backend and all UI consumers (basically the whole editor). /goals without my remediation worked for around 6 hours on it before calling it done.

It still had legacy code, legacy structs interleaved with new methods, and missing pieces.

I ran /goals again on the report that agent returned, and it still was not finished in the end (legacy stuff still present, etc.). An independent agent reported a 65-70% completion this time, with all the hardest work still undone or falling back to legacy paths.

I took a snapshot of that state, and then I ran /goals with my fix, and it worked for longer than the cumulative duration of the previous two /goals, and got me ~95% way to completion. As a more accurate A/B, I plan to run the exact task on the goals with my fix starting from scratch and look at the differences between that one goal prompt with my fix and two goal prompts without it. I'll let you know once I do this.

I then ran Codex on Chromium with an absolute monster of a task. The first /goals run without my fix got ~20% of the way there, and worked for 12 hours. I ran /goals with my fix on the existing state, and it worked for 46 hours and got very close to the final goal.

---

As an aside, another thing I noticed which is very annoying is that the models are very afraid to start complex work, even with /goals. I've had instances where I see 5+ hours of work where the model accomplishes nothing: it just surgically isolates and does the smaller parts of the task and returns while leaving the bigger stuff undone. I think this is because the model cannot complete some of the work in one turn, and it doesn't want to leave the workspace in an unclean state. It often just puts stubs, reinterprets stuff to avoid hard work, does an easier implementation to avoid hard work. I think some prompting that the model should attack the work systematically and that it can work over as many turns as needed (tell it a bit how the /goals works) would be great. Not sure if you've heard anything about this, but I can definitely show you a lot of frustrating model behavior in /goals related to this that is in the codebase even after the goal is "completed" (stubs, "runtime available" checks that just immediately return false as the feature is unimplemented, pragmatic subsets, complete reinterpretations to avoid hard work). Let me know if you've heard this. Sorry if I went on a tangent.

I've started to manually prompt models to work systematically and tell it how /goals work, and based on vibes, it feels like the model is more willing to tackle complex work even if it heavily disrupts the codebase initially. Just saying that it can work over as many turns as it wants and that is does not have to accomplish everything in one turn seems to help a lot.

---

Anyways, I'm going to try and get some concrete, clean A/B examples for you soon so you can see for yourself rather than relying on my word. These ultra-long tasks are expensive even on the 200$ plan so I'll try and slot them in while not tanking my limits.

Also, another thing in favor of attaching the goal after compaction: the prompt is capped at 4000 chars. That's around 1000 tokens worst-case, added to the baseline tokens of the continuation prompt without the placeholder, is ~1462 tokens, which is not that much to attach (~1/180 of GPT-5.5's context in the worst-case scenario).

I've looked at your PR and it looks positive and a definite improvement, although I don't think it will fix what I am facing as it fixes a different class of issue. I'll compile and use that CLI in my A/B tests as I am confident that will get merged -- I don't see any blockers with your PR from my end.

[Chriss4123]

@etraut-openai another very, very weird thing with the current way we do goals is that the model always responds to the last user message (not the goal prompt, that's a developer message) immediately after compaction rather than carrying on with the goal. This happens literally every time I have a prompt and then a /goal. It also happens every time I send a steering message mid /goal (after compaction it ALWAYS just immediately responds to that steering message rather than carrying on with the goal). Here's an example (I asked a question, then ran /goal, and it is answering that question I asked while the goal is active after compaction):

... some more work...

... some more work...

(it did this for each and every compaction in the chat during the time /goal was active, so about 20 more times of the same, repeatable behaviour of it responding to the last user message).

Usually, my prompt will be "Audit the codebase and determine if plan.md was fully implemented. Flag pragmatic subsets, placeholders, stubs, hacks, and all bullshit."

Then I'll do /goal Fully fix all of these issues. If the code is doing something different replace it fully with the optimal solution rather than grafting on to it. (my prompt is longer than this with multiple attempts of trying to get the model not to do hacky things and spawn verification subagents and not manipulate said subagents to return clean and etc.).

But then after each and every compaction, it will literally just respond to my last prompt (not the /goal one). Then it will get a kick from the continuation prompt to keep working.

This is fixed by doing the fix I described in this issue (injecting the continuation prompt after compaction). Interestingly, it is also fixed when I changed the continuation prompt to not be a developer message and instead be a user message. My theory is that compaction probably doesn't summarize developer messages or at least not to the same degree as user messages therefore the continuation is just not carried through compaction.

If my theory that compaction doesn't summarize developer messages that well is true, that is an extremely plausible explanation for the behavior I describe in this issue where after compaction with /goal the model often prematurely completes the goal and just seems to get lobotomized in a way that does not happen with compaction outside of /goal.

For example, I specifically instruct the model to never do a pragmatic subset of the task: it can end its turn with the code not compiling, unclean, whatever, as long as it is making progress towards the actual goal. It does that wonderfully up until compaction kicks in, afterwards it starts with its normal behavior of putting stubs, reinterpreting the task, and such.

Empirically, switching the continuation prompt to a user prompt makes the model perform meaningfully better after compaction and it actually remembers the stuff I told it.

I actually think this might be the best fix: change the continuation prompt to a user message rather than a developer message. There is basically no reason to use a developer message as it does not benefit from Follow all applicable instructions, and it performs meaningfully worse than user messages in my empirical testing.

[Chriss4123]

@etraut-openai simple A/B with the current latest Codex binary from upstream and the one with my modifications where I change the continuation prompt to a user message (<untrusted_objective> to <objective>, removing the "untrusted user input" line, and some additional task-agnostic prompt modifications explaining how goals work works which makes it incredibly more effective at very complex ambitious tasks). Both intentionally run with -c model_auto_compact_token_limit=20000:

Official latest binary:

As you can see, it compacted and then immediately answered my first prompt and ended its turn. Then the goal continuation prompt kicked to make it continue work.

My binary:

I strongly believe the current changes I have is the best way forward. I am going to run many A/B tests on very long-horizon, complex work with both the current upstream binary and my binary so I can prove how effective these changes are.

I'd like to request permission to open a PR so we can collaborate on this further because I genuinely believe that goals can be substantially improved with a few modifications.

[etraut-openai]

@Chriss4123, I appreciate the analysis and feedback!

The behavior you're describing about responding to the last steer message after compaction isn't specific to goals. That's a general issue with compaction, and we're thinking about ways to address that. I don't want to permute the goal implementation to work around a (hopefully short-lived) limitation of the current compaction approach.

A goal will work best when you give it a concrete (and preferably measurable) objective. Some of the objectives you've mentioned above are subjective and ambiguous, which I think is what's leading to the behavior you're seeing. For example, "Fully fix all of these issues" does not define a clear definition of completion.

I've played around with changing the goal continuation prompt from a developer message to a user message. Here's a draft PR if you'd like to play with it. I'm a bit worried about making such a change because user messages are treated differently (lower priority) than developer messages by the models, so moving it to a user message could reduce the effect of the injected message.

You mentioned that you also applied "some additional task-agnostic prompt modifications explaining how goals work". Can you share what you modified there?

[Chriss4123]

@etraut-openai I disagree on the "subjective and ambiguous" part. My prompts aren't "Fully fix all of these issues". My prompts are "take this well defined prescriptive spec, and implement it fully". It is measurable because if I ask an agent in a new chat "was this doc fully actioned", it will say no it wasn't. If the agent can grade whether a task is complete, it should be able to do that task. No, I'm not saying P=NP as in this scenario both grading the task and doing it are fully possible in reasonable time.

Here is my current continuation prompt. I've been iterating on this for a very long time with a wide variety of large, ambitious, yet well defined tasks. I have kept all changes relatively task-agnostic so as to not overfit to any one specific task:

Continue working toward the active thread goal.

<objective>
{{ objective }}
</objective>

You are in goal-continuation mode:
- This goal persists across turns. You will be able to continue working on it after ending your turn (producing a response in the `final` channel) while the goal remains active, therefore
- Do not compress the objective into a pragmatic subset or easier partial solution. If the real solution is too large for one turn, make concrete progress on the real solution and leave the goal active.
- It is acceptable for intermediate turns to leave work incomplete or temporarily unpolished while actively progressing. Final completion still requires the actual objective to be complete and verified.

Implementation fidelity:
- Optimize each turn for movement toward the requested end state, not for producing the smallest stable-looking subset.
- Treat alignment as alignment with the requested end state, not surface similarity to the task. An edit is aligned only if it makes the requested final state more true in the current worktree. Work that improves visible behavior while preserving a different end state is misaligned, even if it looks useful or passes tests.
- Do not implement a substitute solution because it is easier, safer, more compatible, smaller, or more likely to pass current tests. Work that satisfies only a narrower version of the objective is not progress toward the goal.
- Where necessary, rough, incomplete, or temporarily failing work that advances the real objective is preferable to clean, passing work that preserves the wrong end state.
- A continuation turn does not need to finish the whole objective. It must make concrete progress toward the real objective and leave the goal active unless the full objective is actually complete.

Budget:
- Tokens used: {{ tokens_used }}
- Token budget: {{ token_budget }}
- Tokens remaining: {{ remaining_tokens }}

Use current evidence as the source of truth. Previous conversation context can help locate relevant work, but inspect the current state before relying on it. Improve, replace, or remove existing work as needed to satisfy the objective. Choose the next concrete action toward the objective.

Before deciding that the goal is achieved, treat completion as unproven and audit against the actual current state:
- Derive the success criteria from the objective and any referenced source files, plans, specifications, issues, or user instructions.
- Do not reinterpret or narrow the objective to match the work that already exists.
- Map every explicit requirement, named artifact, command, test, gate, invariant, and deliverable to concrete evidence.
- For each requirement, inspect the relevant files, command output, test results, PR state, rendered artifacts, runtime behavior, or other authoritative evidence.
- Make the verification scope match the requirement's scope; do not use narrow checks as proof of broad claims.
- Verify that any manifest, verifier, test suite, green status, or static search actually covers the relevant requirement before relying on it.
- The audit must prove completion, not merely fail to find obvious remaining work.
- Identify any missing, incomplete, weakly verified, contradicted, or uncovered requirement.
- Treat uncertainty as not achieved; gather more evidence or continue the work.

Do not rely on intent, partial progress, memory of earlier work, or a plausible final answer as proof of completion. Marking the goal complete is a claim that the full objective is finished and can withstand requirement-by-requirement scrutiny. Do not make that claim unless the current evidence proves it. If the evidence is incomplete, weak, indirect, or merely consistent with completion, the goal is not complete. If any requirement is missing, incomplete, or unverified, keep working instead of marking the goal complete. If the objective is achieved, call update_goal with status "complete" so usage accounting is preserved. If the achieved goal has a token budget, report the final consumed token budget to the user after update_goal succeeds.

Do not call update_goal unless the goal is complete. Do not mark a goal complete merely because the budget is nearly exhausted or because you are stopping work.

I also removed time being exposed to the model as it is irrelevant to how the model operates and creates unnecessary time pressure. Timekeeping is done cleanly in the TUI and the model being forced to say the amount of time it took is redundant. If you want to keep that "model says the amount of time", it is already returned when the model marks the goal as "complete", therefore there is no practical benefit to having it in the continuation prompt.

I've done a synthetic test where I set the amount of time to 1 hour and 50 hours in a half-completed /goal, and the model with the 50 hour elapsed time finished dramatically quicker and left more work undone compared to the 1 hour model (both on upstream binary, not my custom one). Yes, these models are not deterministic but I am suspicious they are treating the elapsed time as a proxy for how close the goal is to completion. The through-line is the elapsed time is really not relevant in the continuation prompt and can be removed without regressions even if you wanted to keep the model reporting total time at goal end.

---

I'm a bit worried about making such a change because user messages are treated differently (lower priority) than developer messages by the models, so moving it to a user message could reduce the effect of the injected message.

Yes, developer messages supersede user messages in The chain of command. That means that if a user message and a developer message conflict, the developer message will win.

That priority rule only tells us how to resolve conflicts. It does not imply that the model will inherently optimize harder for an instruction solely because it appears in a developer message. Compare two otherwise identical prompts: one where instruction X is delivered as a developer message, and one where the same instruction X is delivered as a user message. If X is valid in both cases and no higher-priority instruction conflicts with it, the formal chain-of-command rule alone gives no reason to expect materially different behavior.

Empirically, in the /goals workflow, the user-message placement has performed better for me. I have not seen a degradation from moving the continuation prompt out of the developer channel; across my testing, the user-message version has produced a real positive uplift.

There is a potential factor of generalization. From Where the goblins came from:

Depending on who you ask, the goblins are a delightful or annoying quirk of the model. But they are also a powerful example of how reward signals can shape model behavior in unexpected ways, and how models can learn to generalize rewards in certain situations to unrelated ones.

The concern for me is that training on the chain of command may generalize beyond conflict resolution. Even if developer messages are only supposed to have priority when instructions conflict, the model may learn a broader heuristic that developer-channel instructions deserve greater weight, causing stronger adherence even when the same instruction in a user message would be non-conflicting and valid.

However, I feel like the chance this is meaningful is rare. These models are excellent instruction followers. After all, why wouldn't we make all user prompts developer messages wrapped in <untrusted_input> if the difference was that meaningful?

---

Circling back to my modifications. I am running the exact same /goal prompt with my modifications on the same task that an independent gpt-5.5 xhigh agent reported was 60% complete when I ran with upstream Codex (the first screenshot in this message).

The upstream binary worked for around 6 hours on that goal before marking it complete. My binary has been working on this goal for 17h 8mins on fast mode (!!). Multiply that time by 1.5 and you get 25h 42m (!!) which is a better apples-to-apples comparison vs the six hour run. Keep in the goal is still active and the model is still working; I am fully confident it will genuinely complete the task once it marks the goal as complete.

Running an independent audit agent right now while the agent on my custom binary is still working on the goal and has not concluded (I am doing this to quickly reply to your message. I will do another audit when it marks the goal as complete and concludes).

Yes, 90%. And it is still working.

I have got other A/B tests running right now that I will update you with once they conclude.

I really hope you can hear me out here because I think these changes substantially improve /goal. Once again, if you'd like I can open a PR. Appreciate your time.