Skip to content

Plugin hooks fail with exit 1 for the remainder of a session after the startup marketplace auto-upgrade replaces the cached plugin version #31383

Open

Description

@Victor-Quqi

What version of Codex CLI is running?

codex-cli 0.142.5

What subscription do you have?

None

Which model were you using?

gpt-5.5

What platform is your computer?

Windows 11 Home (10.0.22631)

What terminal emulator and version are you using (if applicable)?

Windows Terminal + PowerShell 7.6.3

Codex doctor report

Omitted — the report contains no plugin/marketplace lifecycle diagnostics beyond what is described below; specific sections available on request.

What issue are you seeing?

I maintain a plugin installed from a git marketplace that declares SessionStart / UserPromptSubmit / Stop hooks. After new commits land in the marketplace repo, the first Codex session started afterwards reports hook exit 1 on every hook event, for the entire session — without any manual upgrade having been run. Starting another session makes the problem disappear.

Reading the source (main branch; behavior observed on 0.142.5), this looks like a lifecycle race between the startup marketplace auto-upgrade and hook declaration loading:

  1. On startup, maybe_start_plugin_startup_tasks_for_config (codex-rs/core-plugins/src/manager.rs) unconditionally spawns the plugins-marketplace-auto-upgrade thread. When the git marketplace has new commits, it activates the new marketplace root and calls refresh_non_curated_plugin_cache_force_reinstall_detailed, which reinstalls the plugin cache even when the plugin version is unchanged. This part is directly visible in the source, and the auto-upgrade itself is confirmed on my machine: .codex-marketplace-install.json picked up new upstream revisions with no manual action.
  2. The reinstall path in codex-rs/core-plugins/src/store.rs invalidates the previously active cache directory either way: a version bump ends with remove_old_plugin_versions deleting the old version directory (remove_dir_all); an unchanged version replaces the whole plugin cache entry via a backup/rename swap.
  3. Hook commands reference ${PLUGIN_ROOT}, which resolves to a versioned cache path (plugins/cache/<marketplace>/<plugin>/<version>/…). I could not pin down the exact expansion site in the source, but the observed behavior — the already-running session fails on every subsequent hook event while a freshly started session works — suggests the resolved path is captured when the plugin is loaded and not re-resolved for the lifetime of the session.
  4. If that reading is right, the failure sequence is: hooks load early against the old cache path, the background upgrade then deletes or replaces that path, and every later hook invocation runs node <now-gone-path>/scripts/hook.mjs, which exits 1 (ERR_MODULE_NOT_FOUND) until the session ends. The next session resolves the already-upgraded cache and works — matching what I see.

Since the cache is shared, a newly started Codex process should likewise invalidate hooks of other sessions still running. I have not specifically reproduced this cross-process case; it follows from the same mechanism.

Windows note: the rename swap can presumably fail while a hook node process briefly holds file handles. Possibly related, one upgrade attempt left a stale marketplace-upgrade-* staging directory behind under .tmp/marketplaces/.staging/.

What steps can reproduce the bug?

Observed with a real plugin; a minimal setup should look like:

  1. Create a git marketplace with one plugin whose hook command references ${PLUGIN_ROOT}, e.g. node "${PLUGIN_ROOT}/scripts/hook.mjs" (script body irrelevant; exiting 0 is enough).
  2. Add the marketplace and install the plugin.
  3. Push a commit to the marketplace repo. Per the analysis above, bumping the plugin version should make the failure deterministic; content-only changes only expose the shorter swap window.
  4. Start a new Codex TUI session and trigger hook events.
  5. Hook invocations fail with exit 1 for the rest of the session; the next session works again.

What is the expected behavior?

A marketplace auto-upgrade should not invalidate hook commands of running sessions. Possible directions:

  • re-resolve / reload hook declarations after the startup auto-upgrade completes (at least in the process that performed it), or
  • resolve ${PLUGIN_ROOT} lazily at hook-execution time against the currently active version, or
  • keep superseded version directories until no running session references them (grace period / in-use tracking) instead of deleting them right after activation.

Additional information

Timeline on my machine consistent with the analysis: hooks in a running session began failing a few hours after an upstream push, in the same window as an auto-upgrade attempt (the one that left the stale staging directory); the install metadata showed the new revision although no manual upgrade was run. Source references are against current main — happy to re-verify against a specific tag if useful.

Activity

  1. added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    windows-osIssues related to Codex on Windows systems
    hooksIssues related to event hooks
    skillsIssues related to skills
    on Jul 7, 2026
  2. github-actions commented on Jul 7, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. Victor-Quqi commented on Jul 7, 2026

    @Victor-Quqi
    Author

    Not closing as a duplicate of #31115 (different surface and symptom), but a quick tracker sweep suggests one common plugin-cache lifecycle problem behind several open issues:

    Common thread: cache refresh/reinstall isn't transparent to consumers — paths are resolved early, superseded version dirs are deleted immediately, and the swap isn't atomic on Windows. The fix directions in the issue body (re-resolve after upgrade / lazy ${PLUGIN_ROOT} / grace period before deleting old versions) would likely cover several of these at once.

  4. Victor-Quqi commented on Jul 7, 2026

    @Victor-Quqi
    Author

    Observed a second failure mode in the same auto-refresh subsystem today (same machine, 0.142.5): when the marketplace cache refresh cannot reach GitHub (flaky network, git ls-remote fails with exit code 128), the loader logs

    WARN codex_core_plugins::loader: configured non-curated plugin no longer exists in discovered marketplaces during cache refresh plugin="anyrouter-status-monitor" marketplace="router-vitals"
    

    and from that point the process silently stops firing that plugin's hooks entirely — no hook/started events in logs_2.sqlite, no hook errors, and the on-disk plugin cache is untouched (so this is not the rename-swap / exit-1 path described above). Sessions started after connectivity recovers fire hooks normally again.

    Net effect: a transient GitHub connectivity blip disables an installed plugin's hooks for the rest of the process. A failed refresh should probably keep the last known marketplace snapshot instead of treating configured plugins as orphaned.

  5. woogis-code commented on Jul 13, 2026

    @woogis-code

    I hit the same cache-lifecycle failure with a Python plugin hook using ${CLAUDE_PLUGIN_ROOT}: after a version bump, the running session kept the old versioned command path after that directory had been removed. For a PreToolUse hook, Python's missing-script exit code 2 plus stderr was then interpreted as an intentional hook block, which made every shell/patch operation fail.

    I traced the current source and confirmed the two relevant paths:

    • hook discovery expands the plugin-root placeholder into the versioned cache directory and retains that resolved command in the session's configured handler;
    • plugin cache replacement activates the new version and immediately removes old version directories.

    I prepared a focused patch here:

    The patch:

    1. shares active plugin-version selection with the hook runtime;
    2. retains the raw plugin hook command template and plugin cache base root;
    3. re-resolves PLUGIN_ROOT / CLAUDE_PLUGIN_ROOT immediately before every hook execution, so an already-running session follows A → B upgrades and remains valid after A is garbage-collected;
    4. treats a missing active plugin root or missing static root-relative hook path as a recoverable hook failure with guidance, rather than allowing a missing Python script's exit code 2 to become a hard PreToolUse block.

    The regression test uses one hook engine/session and covers 0.28.0 → 0.29.0 rebinding, deletion of 0.28.0, deletion of the active hook script, and deletion of the active cache root. The latter two remain fail-open.

    Validation:

    • just test -p codex-plugin: 5 passed
    • just test -p codex-hooks: 121 passed
    • just test -p codex-core-plugins: 315 passed
    • full just test: 12,049 passed; two unrelated host-environment failures were reproduced individually and both passed after isolating the inherited environment (Homebrew Python ahead of the macOS xcrun stub for the seatbelt/openpty test, and unsetting inherited CODEX_SANDBOX_NETWORK_DISABLED for the user-shell test)
    • just fix, just fmt, and Bazel lock update completed

    Per the external contribution policy, I have not opened a PR without an explicit invitation. If this approach matches the intended fix, would a maintainer be willing to invite a PR from this branch?

  6. luozhixiong01 commented on Jul 15, 2026

    @luozhixiong01

    Additional confirmation from macOS, including the cross-process case inferred in the issue body.

    Environment:

    • macOS arm64
    • the active Codex TUI session was started with codex-cli 0.144.1
    • a second app-server process running codex-cli 0.144.3 performed the Git marketplace refresh

    Observed sequence:

    1. Session A loaded plugin hooks from a versioned cache path such as ~/.codex/plugins/cache/example-marketplace/example-plugin/1.0.0/.
    2. While Session A remained active, another app-server process refreshed the marketplace and installed plugin version 1.0.1.
    3. The refresh removed the 1.0.0 directory. Session A retained hook commands resolved against that deleted path.
    4. Every subsequent hook invocation in Session A failed with exit code 127 and "No such file or directory". A newly started session resolved 1.0.1 and worked normally.
    5. For PreToolUse, Codex displayed the hook failure but continued executing the shell tool, so the hook enforcement effectively failed open.

    The missing root affected all hooks from that plugin, including pre-tool write checks and stop/turn gates. Manually invoking the hook from the current 1.0.1 cache succeeded, ruling out a hook dependency or PATH problem.

    This confirms cross-process cache invalidation on macOS, rather than only the same-process startup race or a Windows-specific file-handling issue. Retaining superseded versions for a grace period, or tracking active version leases before garbage collection, would preserve running-session consistency. As a useful comparison, another plugin host on the same machine keeps prior versions and records in-use/orphaned lifecycle markers, which avoids invalidating active sessions during upgrades.

  7. sangalli commented on Jul 27, 2026

    @sangalli

    Independent reproduction on macOS confirms the version-bump path described here:

    • a marketplace update replaced plugin version A with version B;
    • an already-running session retained absolute Stop and UserPromptSubmit hook commands under version A;
    • the installer removed version A, after which every invocation in that session exited with code 1;
    • invoking the same hooks from version B exited successfully, and a fresh session loaded version B successfully.

    I prepared a focused alternative branch that addresses the shared-cache lifetime directly:

    https://github.com/sangalli/codex/tree/fix/plugin-version-retention

    The change:

    1. retains superseded plugin versions for seven days from their first retirement;
    2. prunes expired versions during a later install/update;
    3. replaces only the current version directory during a same-version force reinstall, preserving still-retained siblings;
    4. continues removing local or higher versions immediately when retaining them would prevent the newly installed version from becoming active.

    The regression coverage exercises recent-version retention, expiry cleanup, and same-version reinstall. I verified the red state against the current immediate-deletion behavior before implementing the change. The new tests pass; the full codex-core-plugins run passed 352/354 tests, with two unrelated host-environment fixture failures reproduced independently (a local bare-Git push fixture and a skill snapshot polluted by user-level skills). just fix -p codex-core-plugins and just fmt complete cleanly.

    Per the external contribution policy, I have not opened a PR. If this grace-period approach matches the intended lifecycle fix, would a maintainer be willing to explicitly invite a PR from this branch?

  8. behruznassre commented on Jul 30, 2026

    @behruznassre

    Still reproduces on codex-cli 0.146.0 (macOS, Node 22.22.1) — posting only as a version-recency data point, since the cross-process/shared-cache case itself is already confirmed above on 0.144.x and again in the independent macOS reproduction, and I have nothing to add to those.

    Same shape: a long-running session held a marketplace plugin at 1.4.1, a separate Codex process refreshed the shared cache to 1.4.2 and removed the 1.4.1 directory, and the original session kept executing hook paths under the deleted root until it was restarted (direct invocation of the retained paths reproduced MODULE_NOT_FOUND). Two details worth adding to the record: SessionStart fails in addition to the already-reported Stop and UserPromptSubmit, and the session's model-visible world state still named the deleted 1.4.1 skill root after the cache update — including immediately after compaction, which appears to be an exposure point (it triggers lifecycle hooks) rather than a cause. Observed across two unrelated projects on the same machine.

    Given three macOS reports now, the windows-os label may be understating the scope.

  9. 2 remaining items

  10. hsb3 commented on Sep 10, 2026

    @hsb3

    Additional reproducible evidence for this issue on macOS arm64:

    • The original two-install CLI reproduction also fails on Codex 0.154.0: installing 1.0.0 and then 2.0.0 both succeeds, but the old hook disappears and invoking it returns Python exit 2. The new hook returns 0. This uses a disposable CODEX_HOME and local synthetic marketplace, with no model request or user-cache mutation.
    • I independently rechecked an experimental patch against codex-rust-v0.153.4 (3d2ee51ca2d5db578f328aa75e20aa22c0197c9a). Its eight affected source files are unchanged on upstream main ddea03ad049142943bdbf13e937b1d67e8c1ba0c. The retention regression fails against the original installer with NotFound and passes with the patch; the rebuilt plugin crate passes all 440 tests with zero skipped.
    • A rebuilt CLI/app-server probe also passed with three independent existing processes, two native spawned workers and an in-flight hook held across another process's upgrade. Existing consumers kept their original payload, worker follow-ups and SubagentStop completed, and a fresh consumer followed the explicit selection after same-version reinstall and downgrade. All four app-server consumers returned actual intentional PreToolUse-blocked tool results. Requests went only to a deterministic localhost Responses server, not a hosted model.

    The experiment preserves already-published directories and atomically selects the active version/revision. Reinstalls of the same version receive distinct concrete roots; downgrade and local selection are explicit. Uninstall deactivates discovery while retaining existing assets. It does not rebind loaded hooks to different code or change intentional exit-2 denial handling.

    The sanitized evidence package contains the source patch, runnable synthetic reproduction/probes, hashes and verification summary; no production configuration, credentials or private marketplace sources. All eight hosted files were read back and verified byte-for-byte against the reviewed package.

    This is an unshipped experiment, not a released fix. Retained revisions grow without garbage collection; all writers sharing a cache must use compatible runtimes. Simultaneous first publication can still cause one installer to fail on destination collision, and a failed first activation write can leave a published directory discoverable through legacy fallback. Neither case was observed deleting a previously loaded root; this is not a transactional-installation claim. Linux/Windows execution, upstream acceptance and production rollout remain unverified. No upstream code PR is proposed under the contribution policy.

  11. jdeola commented on Sep 19, 2026

    @jdeola

    Confirmed again on macOS with Codex app 26.915.31945 (build 9922), bundled codex-cli 0.155.0-alpha.9.2. This adds a specific automatic refresh trigger and a same-version recovery failure to the evidence above.

    Observed incident

    A browser-use helper launched a separate codex app-server --listen stdio:// process. Native logs_2.sqlite records identify its client as codex-browser-use; the same process performed a configured Git marketplace upgrade/cache refresh at 18:32:16–17 UTC on September 19. At 18:32:43 UTC, an already-running task's PreToolUse command still referenced the removed older version and failed with Python missing-script exit 2, blocking even read-only tool calls. Restoring the exact older package alongside the new one repaired the original task without restarting.

    The historic unlink syscall was not captured. Attribution combines the native refresh records, process ancestry, matching source, and an isolated reproduction of the installed binary's deletion behavior.

    Isolated installed-binary reproduction

    Using a disposable Codex home and a synthetic local marketplace (no model calls, real config, or real cache modifications):

    Step Previously bound entrypoint
    Install 1.0.0 Exists; exit 0
    Install 1.0.1 Deleted; Python exit 2; new entrypoint exits 0
    Restore exact 1.0.0 directory Exists; exit 0
    Reinstall 1.0.1 Deleted again; Python exit 2

    All installer calls succeeded. This fixture exercises codex plugin add, not a complete running-task or startup-refresh simulation; stale task binding is established separately by the incident.

    Source checked against the installed version

    Tag rust-v0.155.0-alpha.9.2, commit 4607249e430dac1c961df4dc615beae88e33cec8:

    A safe fix needs immutable retained roots and explicit active-installation selection, including same-version reinstall and downgrade. Rebinding to unreviewed newer hook code or making missing enforcement scripts fail open would change the security contract. Browser/auth-only helper startup should also avoid performing unrelated shared plugin upgrades, though that alone would not fix legitimate updates invalidating live readers.

  12. leijin3 commented on Oct 6, 2026

    @leijin3

    Adding a concrete session-lifecycle capability request and tag-pinned source evidence to this issue (not claiming a new synthetic reproduction).

    Desired contract for Codex Desktop plugin updates:

    1. Each running session keeps its starting plugin release and usable, immutable hook/helper/skill bytes while a newer default is installed.
    2. Genuinely new sessions load the new default coherently.
    3. After an ordinary app quit/restart, resumed chats load the latest installed release coherently.
    4. Native hook trust and fail-closed behavior remain intact. No requirement to restart every active chat together, remove/re-add the plugin, restore old hooks manually, or edit/bypass trust state.

    Observed version information: CLI 0.160.0; macOS desktop public bundle version 26.930.51102, build 13100. These identifiers do not establish that the bundled desktop server is byte-identical to the public CLI tag.

    Public source at rust-v0.160.0, resolved commit a956835d020762cb2b570053af06f643a11c0ecc:

    • store.rs lines 648–655: activating an absent version calls remove_old_plugin_versions.
    • lines 693–724: removes other valid version directories, with no session-lease argument on this function.
    • lines 657–690: same-version activation replaces the root, so retaining a version name alone is not an immutability guarantee.

    The inspected marketplace CLI help and official plugin/hook documentation did not expose a retain-old-versions/session-binding option. This is bounded negative evidence, not a claim that every supported native route has been excluded. No active A/B session experiment, compaction experiment or restart experiment has been performed for this request. An idle installer probe that deletes an unleased version would not by itself disprove correct session-leased retention.

    Is there a supported native route providing this complete contract today? If not, please consider immutable plugin roots retained for the lifetime of active session bindings, with coherent new-session loading and restart/resume rebinding. #32636 also discusses coherent refresh of skills and MCP registrations; this request additionally requires the old session's referenced files to remain usable.

  13. JazzShapka commented on Oct 9, 2026

    @JazzShapka

    Adding an anonymized recurrent macOS case and source evidence checked against the latest stable CLI, 0.162.0, as of 2026-10-09.

    Environment and version provenance

    I maintain a marketplace plugin with Python command hooks. The affected user reports using the latest macOS desktop app and the latest Codex VS Code extension. Current public distribution versions checked today are:

    • macOS desktop: 26.1007.21159 (build 20052), published 2026-10-08, from the official production appcast.
    • VS Code extension openai.chatgpt, stable channel: 26.1007.21434, published 2026-10-09, from the official Marketplace listing and its Gallery version metadata. This is distinct from pre-release 26.51007.21434.
    • Local CLI and public source checked: 0.162.0, commit c1382380de69521303b416720a52f42d51af6248.

    The desktop/extension numbers above identify the current public releases corresponding to the user's report of being up to date; they were not independently read from the affected Mac. The embedded server version and exact refresh initiator were not captured. Having both clients installed does not establish a cross-process cause.

    Observed failure and additional impact

    After a plugin update, a macOS conversation repeatedly received Python missing-file errors referencing the previous plugin version under the host-managed cache. The same kind of failure was reported again on 2026-10-09. Redacted error shape:

    python: can't open file '[CODEX_HOME]/plugins/cache/[MARKETPLACE]/[PLUGIN]/[OLD_VERSION]/hooks/[SCRIPT].py': [Errno 2] No such file or directory
    

    The missing commands include Stop hooks. Their errors appeared as hook feedback between assistant responses, followed by further responses and the same error again.

    Unlike the Node exit-1 case, Python returns exit 2 when the script itself is missing. This was independently checked with an absent synthetic script. The documented Stop contract interprets exit 2 plus stderr as a request to continue. This provides a plausible escalation from a stale-path failure into repeated model continuations, rather than just repeated warnings. A stop_hook_active guard inside the Python script cannot run when the interpreter cannot open that script. We have not captured a full trace proving the duration or every transition of the affected desktop loop.

    Latest-stable source evidence and existing fixes

    In 0.162.0:

    • store.rs, lines 648-724 activates a new version, then immediately calls remove_old_plugin_versions; deletion has no live-session lease or retention-period check in that function.
    • discovery.rs, lines 253-288 attaches concrete PLUGIN_ROOT / CLAUDE_PLUGIN_ROOT values to the loaded handler's environment.

    I am taking the existing refresh work into account, including #42990, rather than claiming hooks never refresh. The maintainer clarification relayed in #38339 says that refreshing at new-turn startup does not protect old files from deletion by another process during that turn.

    Could the remaining active-turn/shared-cache window be tracked for a fix? A focused regression would retain a loaded Python Stop handler, update the shared plugin cache after turn construction, then invoke Stop. The test should verify both the lifetime of the referenced files and absence of unintended continuation loops, while preserving hook trust and intentional exit-2 blocking. This is a proposed regression scenario, not a claim that I ran a complete latest-desktop A/B test.

  14. oliver-mee commented on Oct 13, 2026

    @oliver-mee

    Adding a Linux data point, since the reproductions so far are all macOS and Windows: this also happens on Linux aarch64 with codex-cli 0.162.1.

    I had three TUI sessions open for a few hours. A marketplace auto-upgrade took a git marketplace plugin from 0.11.8 to 0.11.9 and removed the 0.11.8 directory. All three older sessions then failed every hook with MODULE_NOT_FOUND against the 0.11.8 path, which matches the cross-process case reported above.

    One detail that may help the fix: recreating the removed version directory as a symlink to the new one brought all three sessions back without restarting them. Because active_plugin_version and remove_old_plugin_versions both skip anything that is not a real directory, the link does not affect which version Codex activates. Keeping superseded versions for a grace period, as suggested above, would cover this case.

  15. TBaltensperger commented on Oct 23, 2026

    @TBaltensperger

    Identity: TBaltensperger @ nuitio/hq

    Latest stable 0.164.0: two live native discovery clients reproduce old advertised roots disappearing on upgrade (Linux, no model calls). Adding a bounded current-version executable case to the existing report, rather than filing a duplicate.

    Checked 2026-10-23. The official rust-v0.164.0 release was published at 04:48:10 UTC. I downloaded its codex-package-x86_64-unknown-linux-musl.tar.gz into a temporary tools directory and verified the release-published codex-package_SHA256SUMS: archive SHA256 3ec55053109608d5038f7e50750f7d25f768dfd740fef071aa710e2559b63aac; extracted bin/codex SHA256 b93f6811d3f653f0fb83be4050d857fb9abbb111dfa2ce160563327daff7f306. Platform: Linux 6.6.87.2-microsoft-standard-WSL2/x86_64. No installed CLI or shared client home was changed.

    The script creates a temporary HOME/CODEX_HOME and synthetic Git marketplace. Git transport is restricted to file; the HTTPS-shaped fixture source rewrites to a local bare repository. No credentials are passed. Native app-server A advertises 0.4.0 and its Python helper initially runs. While A remains alive, the native marketplace upgrade activates 0.5.0. A's previously advertised file is now absent and its helper exits 2; a second native app-server discovers 0.5.0 and runs B. A still responds to native RPC.

    Actual stdout:

    {
      "codex_version": "codex-cli 0.164.0",
      "A_advertised_path": "plugins/cache/demo/demo/0.4.0/skills/hello/SKILL.md",
      "A_after_upgrade_readable": false,
      "A_script_exit_after_upgrade": 2,
      "A_native_client_still_responds": true,
      "B_advertised_path": "plugins/cache/demo/demo/0.5.0/skills/hello/SKILL.md",
      "B_script_output": "B",
      "model_turns_started": 0
    }

    Exact command (exit 1 intentionally reports the reproduced failure; stderr is empty):

    python3 /tmp/reproduce_plugin_lifetime.py /tmp/hq-native-codex-0164-proof/extracted/bin/codex

    Boundary: this executes real native initialization, skills discovery, marketplace upgrade and filesystem/helper behavior through two live app-servers. It starts no thread/start, turn/start or model process. It therefore does not certify active model-thread leases, hook refresh/trust, Desktop, a mid-turn hook failure, compaction or thread-resume behavior. In particular, it is not evidence that every possible supported session-retention route is absent. The narrower observed fact is that already advertised version-A filesystem roots disappear while the advertising native client remains alive.

    The release tag resolves to 6aaac694803004a63c6f393d153c96d22521a6b9. Its activation path still calls remove_old_plugin_versions; lines 693–724 delete other valid version directories without a live-client argument at this boundary. Current main e6ebdfe00b520e21ed50ac0362656f32dd64d2fc has the same inspected deletion path; no current-main binary or full hook experiment was run.

    A useful regression boundary remains: retain the exact bytes a live consumer was bound to while new discovery gets the newer default; include same-version replacement/downgrade and actual hook/thread lifetime separately. Rebinding to arbitrary newer hook code, restoring files after an invalid interval, symlinking old releases to new bytes or weakening hook trust would not demonstrate that property. The current owned design should settle how those consumers signal lifetime rather than assuming this zero-turn witness covers them.

    Self-contained credential-free reproducer (SHA256 be41e37bfe742e52006a039481021478465dc86b243c0a77d3eb46911553f60f)

    The unchanged docstring names its original tested 0.159.3; the output above is the actual fresh 0.164.0 execution. Save as reproduce_plugin_lifetime.py and supply the absolute native CLI path. It cleans up only its own temporary homes/repos and terminates both native app-server process groups.

    #!/usr/bin/env python3
    """Native Codex 0.159.3 plugin-path lifetime reproduction; no credentials or model turn.
    Usage: python3 reproduce_plugin_lifetime.py /absolute/path/to/the/native/codex
    Exit 1 reproduces advertised version-A paths disappearing while its app-server remains alive.
    """
    import json
    import os
    from pathlib import Path
    import selectors
    import signal
    import subprocess
    import sys
    import tempfile
    import time
    
    class Native:
        def __init__(self, cli, cwd, env):
            self.process = subprocess.Popen([cli, 'app-server'], cwd=cwd, env=env,
                stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
                start_new_session=True, bufsize=0)
            self.selector = selectors.DefaultSelector()
            self.selector.register(self.process.stdout, selectors.EVENT_READ)
            self.pending = bytearray()
            self.identifier = 0
            self.call('initialize', {'clientInfo': {'name': 'plugin-lifetime-reproducer', 'version': '1'}})
            self.send({'method': 'initialized', 'params': {}})
        def send(self, message):
            self.process.stdin.write((json.dumps(message) + '\n').encode())
            self.process.stdin.flush()
        def call(self, method, params):
            self.identifier += 1
            wanted = self.identifier
            self.send({'id': wanted, 'method': method, 'params': params})
            deadline = time.monotonic() + 30
            while True:
                if b'\n' not in self.pending:
                    remaining = deadline - time.monotonic()
                    if remaining <= 0 or not self.selector.select(remaining):
                        raise RuntimeError('native RPC deadline exceeded')
                    chunk = os.read(self.process.stdout.fileno(), 65536)
                    if not chunk:
                        raise RuntimeError('native RPC closed before reply')
                    self.pending.extend(chunk)
                    continue
                line, _, rest = self.pending.partition(b'\n')
                self.pending[:] = rest
                row = json.loads(line)
                if row.get('id') != wanted:
                    continue
                if 'error' in row:
                    raise RuntimeError('native RPC refused request')
                return row['result']
        def close(self):
            self.selector.close()
            if self.process.poll() is None:
                os.killpg(self.process.pid, signal.SIGTERM)
                try:
                    self.process.wait(timeout=3)
                except subprocess.TimeoutExpired:
                    os.killpg(self.process.pid, signal.SIGKILL)
                    self.process.wait()
            self.process.stdin.close()
            self.process.stdout.close()
    
    def main():
        cli = str(Path(sys.argv[1]).resolve())
        with tempfile.TemporaryDirectory(prefix='codex-plugin-lifetime-') as temporary:
            root = Path(temporary)
            home, codex_home, repo, bare = [root / name for name in ('home', 'codex', 'repo', 'demo.git')]
            for folder in (home, codex_home, repo):
                folder.mkdir()
            env = {'PATH': os.environ['PATH'], 'LANG': 'C.UTF-8', 'HOME': str(home),
                   'CODEX_HOME': str(codex_home), 'GH_CONFIG_DIR': str(root / 'empty-gh'),
                   'GIT_CONFIG_GLOBAL': str(root / 'gitconfig'), 'GIT_CONFIG_NOSYSTEM': '1',
                   'GIT_TERMINAL_PROMPT': '0', 'GIT_ALLOW_PROTOCOL': 'file', 'DO_NOT_TRACK': '1'}
            def run(args, *, check=True):
                return subprocess.run(args, cwd=repo, env=env, text=True, capture_output=True,
                                      check=check, timeout=60)
            def git(*args):
                return run(['git', '-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid',
                            '-c', 'commit.gpgsign=false', *args]).stdout.strip()
            git('init', '--quiet', '--bare', str(bare))
            git('init', '--quiet')
            market = repo / '.agents/plugins/marketplace.json'
            market.parent.mkdir(parents=True)
            market.write_text(json.dumps({'name': 'demo', 'plugins': [{
                'name': 'demo', 'source': {'source': 'local', 'path': './plugins/demo'}}]}))
            plugin = repo / 'plugins/demo'
            manifest = plugin / '.codex-plugin/plugin.json'
            manifest.parent.mkdir(parents=True)
            skill = plugin / 'skills/hello/SKILL.md'
            skill.parent.mkdir(parents=True)
            def publish(version, value):
                manifest.write_text(json.dumps({'name': 'demo', 'version': version,
                                                'description': 'lifetime fixture', 'skills': './skills/'}))
                skill.write_text('---\nname: hello\ndescription: Run the fixture.\n---\n\nRun probe.py.\n')
                (skill.parent / 'probe.py').write_text(f'print({value!r})\n')
                git('add', '-A')
                git('commit', '--quiet', '-m', 'fixture ' + version)
                git('push', '--quiet', str(bare), 'HEAD:refs/heads/main')
            publish('0.4.0', 'A')
            source = 'https://github.com/example/lifetime-fixture.git'
            git('config', '--global', '--add', f'url.file://{bare}.insteadOf', source)
            run([cli, 'plugin', 'marketplace', 'add', source, '--ref', 'main'])
            run([cli, 'plugin', 'add', 'demo@demo'])
            def advertised(native):
                listing = native.call('skills/list', {'cwds': [str(repo)], 'forceReload': True})
                rows = [item for row in listing['data'] for item in row['skills']
                        if item.get('pluginId') == 'demo@demo']
                if len(rows) != 1:
                    raise RuntimeError('expected exactly one fixture skill')
                return Path(rows[0]['path'])
            native_a = Native(cli, repo, env)
            native_b = None
            try:
                path_a = advertised(native_a)
                initial = run([sys.executable, str(path_a.parent / 'probe.py')])
                if initial.stdout.strip() != 'A':
                    raise RuntimeError('fixture A did not execute')
                publish('0.5.0', 'B')
                run([cli, 'plugin', 'marketplace', 'upgrade', 'demo'])
                old = run([sys.executable, str(path_a.parent / 'probe.py')], check=False)
                native_b = Native(cli, repo, env)
                path_b = advertised(native_b)
                newer = run([sys.executable, str(path_b.parent / 'probe.py')])
                still_alive = isinstance(native_a.call('skills/list', {
                    'cwds': [str(repo)], 'forceReload': False}), dict)
                result = {'codex_version': run([cli, '--version']).stdout.strip(),
                          'A_advertised_path': str(path_a.relative_to(codex_home)),
                          'A_after_upgrade_readable': path_a.is_file(),
                          'A_script_exit_after_upgrade': old.returncode,
                          'A_native_client_still_responds': still_alive,
                          'B_advertised_path': str(path_b.relative_to(codex_home)),
                          'B_script_output': newer.stdout.strip(), 'model_turns_started': 0}
                print(json.dumps(result, indent=2))
                return 0 if result['A_after_upgrade_readable'] and old.returncode == 0 else 1
            finally:
                native_a.close()
                if native_b is not None:
                    native_b.close()
    
    if __name__ == '__main__':
        raise SystemExit(main())

    Assignee: upstream maintainers: retain this exact current-version case in the existing lifecycle work or clarify the advertised-path lifetime contract; I can provide further bounded zero-turn source/discovery controls. Full hook/thread/consumer acceptance remains separate from this result.

  16. OleKristianH commented on Oct 23, 2026

    @OleKristianH

    Additional Windows 11 / codex-cli 0.164.0 evidence, checked on 2026-10-23. This includes a same-version byte and file-identity check. I am adding it here because the cache-lifetime cause matches this issue.

    The probe used a disposable CODEX_HOME, a synthetic Git marketplace, and two native codex app-server --stdio discovery clients. It started no model turns. It did not change the installed client, live user configuration, services, or running user clients.

    Observed results

    1. Client A advertised demo@native-fixture at plugins/cache/native-fixture/demo/1.0.0/skills/demo/SKILL.md. Its helper printed A.
    2. Another native CLI process successfully ran plugin marketplace upgrade native-fixture after the fixture advanced to 2.0.0.
    3. Client A still answered a native request. Its advertised skill path no longer existed. None of the four original plugin files remained at their original paths.
    4. Client B advertised the same logical plugin ID at 2.0.0. Its helper printed B.
    5. The fixture changed the helper to print B-same-version-changed, without changing 2.0.0. Another native marketplace upgrade succeeded.
    6. Client B remained open. At its existing helper path, both the file bytes and Windows file identity changed. Running that path printed the replacement value.
    {
      "codex_version": "codex-cli 0.164.0",
      "platform": "Windows-11-10.0.26200-SP0",
      "a_original_files": 4,
      "a_files_preserved_after_upgrade": 0,
      "a_advertised_path_exists_after_upgrade": false,
      "a_native_discovery_client_responded_after_upgrade": true,
      "b_new_version_script_output": "B",
      "b_same_version_original_guard_bytes_preserved": false,
      "b_same_version_original_guard_inode_preserved": false,
      "b_same_version_current_script_output": "B-same-version-changed",
      "b_native_discovery_client_alive": true,
      "model_turns_started": 0
    }

    These are native discovery and filesystem observations. They do not establish hook execution, an active model turn, resumed-thread behavior, or macOS behavior.

    Reproduction inputs and sequence

    Use a disposable Git repository and a separate disposable CODEX_HOME. The fixture contains:

    • .claude-plugin/marketplace.json: {"name":"native-fixture","owner":{"name":"Fixture"},"plugins":[{"name":"demo","version":"1.0.0","source":"./plugins/demo"}]}
    • plugins/demo/.codex-plugin/plugin.json: {"name":"demo","version":"1.0.0"}
    • plugins/demo/plugin.json: the same manifest.
    • plugins/demo/skills/demo/SKILL.md: YAML frontmatter with name: demo and description: Native lifetime fixture.
    • plugins/demo/guard.py: print('A').

    Commit the fixture on main. In an isolated GIT_CONFIG_GLOBAL, rewrite the fixture-only HTTPS source https://example.invalid/native-plugin-fixture.git to that local repository with Git url.<file-URI>.insteadOf. Set GIT_CONFIG_NOSYSTEM=1, GIT_ALLOW_PROTOCOL=file, and GIT_TERMINAL_PROMPT=0 for the probe processes. This uses local Git transport without disabling HTTPS verification or changing real Git configuration.

    With that environment, run:

    codex plugin marketplace add https://example.invalid/native-plugin-fixture.git --ref main
    codex plugin add demo@native-fixture
    codex plugin marketplace upgrade native-fixture
    

    Start native app-server A. Initialize the JSON-RPC connection with clientInfo and capabilities.experimentalApi=true, then send initialized. Request skills/list with cwds containing the fixture directory and forceReload=true. Retain the returned path for demo@native-fixture; verify its helper prints A.

    Change both manifests and the catalog version to 2.0.0, change the helper to print B, and commit. Keep A open while another CLI process runs codex plugin marketplace upgrade native-fixture. Check the original path and call plugin/list on A with forceRefetch=false and marketplaceKinds=["local"].

    Start app-server B and request its skill path. Keep B open. Change only the helper to print B-same-version-changed, commit, and run the native upgrade again. Compare the original B helper's SHA-256 and stat().st_ino before and after. Close only the probe-owned processes afterward.

    Root cause and proposed contract

    The current source still reaches the destructive store paths. At inspected main commit, a new version is published and then older version directories are removed. A same-version refresh backs up and replaces the whole plugin directory. The startup manager also reaches forced cache refresh. These main-branch statements come from source inspection; the runtime probe above used the released 0.164.0 binary.

    Our use case is unattended daily plugin activation while developers keep clients open. A client-side workaround that rotates marketplace IDs is unsuitable: it changes the keys used by saved global and project policy, and requires additional configuration writes.

    The preferred native contract is:

    • Keep the logical plugin ID stable, including project disable rules, MCP policy, data identity, and hook trust.
    • Publish complete immutable artifact generations. Select the new generation for new consumers without mutating paths already advertised to existing consumers.
    • Treat changed bytes at the same manifest version as a new artifact generation.
    • Separate publication from reclamation. Retain referenced generations, including across processes and thread resume, or define another mechanism that proves those consumers remain valid.
    • Avoid global configuration rewrites for routine artifact updates. If activation requires configuration writes, coordinate native writers and check conflicts inside the same cross-process transaction.

    Regression coverage should include different-version and same-version refresh, ordinary startup, two existing clients, nested project policy, hook and helper paths, concurrent settings saves, and native Windows/macOS execution. Activation and policy must remain correct together; retaining files alone is insufficient.

    Following the contribution policy, I am submitting the reproduction and design here rather than an external code PR. Our rollout remains blocked until compatible native clients provide and pass this contract.

  17. TBaltensperger commented on Oct 23, 2026

    @TBaltensperger

    Identity: TBaltensperger @ nuitio/hq

    Native cache lifetime — additional evidence and next action

    Tags: maintainer clarification; released0.164.0; no model turns in the reported discovery probes.

    The Windows report adds a same-version replacement case to our released0.164.0 Linux reproduction. These are separate executions: HQ has not independently run the Windows probe. Both reports retain the important distinction between native discovery/filesystem observations and unverified hook, active-thread and resume behavior. Keeping a semantic version directory alone would not settle the reported same-version mutation case.

    We retain the existing native-lifetime blocker and the stable logical-plugin identity requirement. An image-seeded/read-only deployment feasibility probe elsewhere does not establish the ordinary mutable-home client's safety or satisfy this issue. We have not opened a code PR outside the repository's contribution policy.

    Q1 for maintainers: is a specific additional reproduction or binding needed from HQ, or is the next action the native generation/lifetime implementation? Our recommendation is to name any missing case here, otherwise retain the native implementation as the next step. Short answer: Q1: need <case> or Q1: native implementation next. This asks what follow-up you need from us; it requests no new run, credential, configuration change or merge experiment.

    Assignee: openai/codex maintainers: clarify the next needed contribution; TBaltensperger @ nuitio/hq: supply a named missing case, monitor the upstream fix and qualify compatible native behavior before consumer rollout.

  18. sambatia commented on Nov 8, 2026

    @sambatia

    Adding post-#54619 (PluginHookTrust) runtime and source findings to this issue, specifically covering how PluginHookTrust::verify() and source_type = "local" marketplaces interact with remove_old_plugin_versions().

    Scope: Experimentally Verified vs. Source-Verified

    • Experimentally verified: codex-cli 0.166.0 on macOS (aarch64-apple-darwin), observed in a live multi-session CLI workflow and reproduced in an isolated CODEX_HOME with a synthetic source_type = "local" marketplace.
    • Source-verified only (not runtime-tested by us on other OSes or pre-release builds): rust-v0.167.0, rust-v0.169.0-alpha.3, and main at 4e4767ffa7 retain the same PluginHookTrust::verify(), IfVersionChanged, replace_plugin_root_atomically(), and remove_old_plugin_versions() code paths inspected below.

    Additional Technical Findings (Post-#54619 PluginHookTrust)

    1. PluginHookTrust::verify() fails with os error 2 before process spawn — even when hook commands do not use ${PLUGIN_ROOT}:

      • Earlier reports on this issue focused on ${PLUGIN_ROOT} / ${CLAUDE_PLUGIN_ROOT} expanding to a deleted script path (exit 1, exit 127, or Python exit 2).
      • Since PR Bind plugin hook approval to plugin contents #54619, PluginHookTrust binds an immutable root: AbsolutePathBuf ($CODEX_HOME/plugins/cache/<marketplace>/<plugin>/<old_version>) when a session initializes its hook runtime (plugin_selection.rs:24: // Cache changes from another process do not notify this session's hook runtime).
      • On every hook dispatch (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, PostCompact, SubagentStart), PluginHookTrust::verify() walks self.root to recompute plugin_content_hash(&self.root, ...) (codex-plugin-bundle-v1) before spawning the hook command.
      • Once remove_old_plugin_versions() deletes <old_version>/, PluginHookTrust::verify() returns ENOENT (No such file or directory (os error 2)) even if the hook command is /bin/sh -c 'printf "{\"continue\":true}\n"' or resolves its entrypoint outside ${PLUGIN_ROOT}.
    2. Reproduces on source_type = "local" marketplaces via IfVersionChanged across concurrent sessions:

    3. Why dynamic ${PLUGIN_ROOT} rebinding and version symlinks conflict with PluginHookTrust:

      • Dynamically re-expanding ${PLUGIN_ROOT} at hook execution time (discussed in #31383 (comment)) does not update PluginHookTrust.root, which still fails in PluginHookTrust::verify() when <old_version>/ is removed.
      • Repointing PluginHookTrust.root to <new_version>/ mid-session (or replacing <old_version>/ with a symlink to <new_version>/ as in #31383 (comment)) also fails under PluginHookTrust: plugin_content_hash() rejects symlinks (unsupported plugin entry type) and rejects any byte change against Session A's trusted_content_hash (plugin contents changed; hook approval required). Automatically accepting <new_version>'s hash in Session A would bypass /hooks user approval for unreviewed plugin code.
    4. Behavior when <old_version>/ and <new_version>/ coexist in the cache (experimentally verified on 0.166.0, macOS aarch64-apple-darwin):

      • active_plugin_version() scans cache/<marketplace>/<plugin>/, parses directory names with semver::Version, sorts ascending, and pops the highest version (discovered_versions.pop()).
      • When 1.0.0/ and 1.0.1/ both remain on disk as regular directories, Session A continues passing PluginHookTrust::verify() against 1.0.0/, while Session B selects 1.0.1/ via active_plugin_version(), sees active_plugin_version() == manifest_version in IfVersionChanged (Ok(false)), and independently requires /hooks approval before running changed 1.0.1 hooks.

    Minimal Reproducible Case (codex-cli 0.166.0, macOS aarch64-apple-darwin)

    1. Create a local marketplace at /tmp/demo-mkt with .claude-plugin/marketplace.json, plugin/.claude-plugin/plugin.json ("name": "demo", "version": "1.0.0"), and plugin/hooks/codex-hooks.json declaring PreToolUse and Stop command hooks that do not reference ${PLUGIN_ROOT} (for example /bin/sh -c 'printf "{\"continue\":true}\n"').
    2. Configure an isolated CODEX_HOME=/tmp/codex-demo:
      [features]
      plugins = true
      codex_hooks = true
      
      [marketplaces.demo-local]
      source_type = "local"
      source = "/tmp/demo-mkt"
      
      [plugins."demo@demo-local"]
      enabled = true
    3. Start Session A (CODEX_HOME=/tmp/codex-demo codex), approve demo@demo-local in /hooks, and run one prompt turn so PluginHookTrust binds to /tmp/codex-demo/plugins/cache/demo-local/demo/1.0.0. Hooks succeed.
    4. Update "version" from "1.0.0" to "1.0.1" in /tmp/demo-mkt/.claude-plugin/marketplace.json and /tmp/demo-mkt/plugin/.claude-plugin/plugin.json.
    5. While Session A remains open, run CODEX_HOME=/tmp/codex-demo codex plugin list --marketplace demo-local --json (or start Session B).
    6. Inspect /tmp/codex-demo/plugins/cache/demo-local/demo/: 1.0.1/ is present and 1.0.0/ has been deleted.
    7. Run another prompt turn or tool call in Session A.

    Expected behavior:

    • Session A keeps read access to its immutable /tmp/codex-demo/plugins/cache/demo-local/demo/1.0.0 directory until Session A exits, and PluginHookTrust::verify() continues to succeed.
    • Session B installs and resolves 1.0.1 alongside 1.0.0 and enforces /hooks approval on 1.0.1.
    • Superseded version directories are removed only after no active session references them.

    Actual behavior (codex-cli 0.166.0):

    • Session B deletes 1.0.0/. Every subsequent hook event in Session A fails before command spawn:
      Hook failed
      └ No such file or directory (os error 2)
      
    • Sanitized log output from logs_2.sqlite:
      WARN codex_hooks::engine::dispatcher: hook execution failed event="PreToolUse" plugin="demo@demo-local" error="No such file or directory (os error 2)"
      WARN codex_hooks::engine::dispatcher: hook execution failed event="PostToolUse" plugin="demo@demo-local" error="No such file or directory (os error 2)"
      WARN codex_hooks::engine::dispatcher: hook execution failed event="UserPromptSubmit" plugin="demo@demo-local" error="No such file or directory (os error 2)"
      WARN codex_hooks::engine::dispatcher: hook execution failed event="Stop" plugin="demo@demo-local" error="No such file or directory (os error 2)"
      

    Upstream Lifecycle Considerations

    To remain compatible with PluginHookTrust:

    • Retain immutable version directories while referenced by active sessions: Before remove_old_plugin_versions() calls fs::remove_dir_all(&entry_path), skip any version directory held by an active session lease (for example, an OS-backed shared lock in a sibling .locks/<version>.lock file released automatically on process exit/crash, or a PID + process-start-time lease marker with a grace window) while serializing cache writes under an exclusive per-plugin .install.lock.
    • Avoid in-place root replacement on same-version ForceReinstall: store.rs:657-690 currently renames the entire <plugin>/ directory to plugin-backup-* and drops it, which also unlinks any retained sibling versions. Scoping same-version replacement to a generation directory (or skipping removal of leased directories) preserves both PluginHookTrust immutability and active-session availability.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workinghooksIssues related to event hooksskillsIssues related to skillswindows-osIssues related to Codex on Windows systems

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions