Skip to content

cli: service skill/compatibility watcher self-loops on .agents/skills, ~37% idle CPU #54114

Open

Description

@s524797336

Description

Summary

The background service (opencode serve --service) continuously tears down and rebuilds the inotify watch for a compatibility skills directory (<project>/.agents/skills), about 1–2 times per second. It never settles while otherwise idle, pinning the service at ~35–40% CPU and flooding the log.

Environment

  • opencode version: v2.0.26 (channel latest)
  • OS: Linux 6.18.40.1-microsoft-standard-WSL2 (x86_64) — Docker Desktop
  • Terminal: TERM_PROGRAM=vscode, TERM=xterm-256color, COLORTERM=truecolor
  • Shell: fish
  • Install/channel: latest
  • Active plugins: one local plugin (worktree-plugin.js under ~/.config/opencode/plugins/). The looping code is in the built-in opencode.config.* plugins, not this plugin.

Reproduction

  1. Use a project that exposes compatibility skills via ./.agents/skills/<name>/SKILL.md (the project also has a ./.opencode/ directory).
  2. Start the background service: opencode service start (or launch the app / TUI).
  3. Watch the service log: tail -f ~/.config/opencode/log/opencode.log.
  4. Observe watcher stopped → watcher subscribe → watcher started repeating for path=<project>/.agents/skills, ~1–2×/s, indefinitely — even though no file in that directory is changing.
  5. Count watcher subscribe per path in the log: .agents/skills accumulates thousands of occurrences; every other path subscribes once or twice and stays stable.

Expected Behavior

The skills/compatibility watcher subscribes to the directory once and stays idle when nothing changes — the same way ~/.config/opencode/skills and <project>/.opencode/skills behave.

Actual Behavior

The watch is cleared and rebuilt continuously. In opencode.config.compatibility, refresh() calls iQ(l) (closes all watch resources) and then re-subscribes join(dir, "skills") for each compatibility directory. The watch rebuild appears to emit events that are fed back through the queue → 100 ms debounce → refresh() again, forming a self-sustaining loop. opencode.config.skill and Config.reconcileWatches (which publishes config.updated) participate in the same feedback.

Evidence — watcher subscribe occurrences per path over one service run:

<N>   path=<project>/.agents/skills      <- loops, grows without bound
  2   path=~/.config/opencode/skills
  2   path=<project>/.opencode/skills
  2   path=<project>/.opencode
  2   path=<project>/AGENTS.md
  ...  (every other path: 1–2 occurrences)
  • No file under .agents/ was modified during the observation window (find . -newermt '-2 minutes' returned nothing); no external process held the directory open. So real filesystem changes are not the trigger.
  • The only log lines produced in the window were watcher subscribe / watcher started / watcher stopped (no skills rescanned or config.updated), i.e. the loop is at the low-level watcher layer.
  • Measured steady state while otherwise idle: ~37 watcher events per 20 s, service CPU ~37%.

Additional Context

  • Impact: opencode serve --service sits at ~35–40% CPU while otherwise idle, and opencode.log grows to tens of MB/day from watcher spam alone.
  • Workaround that works — disable the file watcher via service env:
    opencode service set env OPENCODE_FILEWATCHER_DISABLE 1
    opencode service start
    After restart: .agents/skills watcher events drop to 0 per 20 s and service CPU falls to ~14%. Downside: config/skills/AGENTS.md hot-reload is disabled.
  • opencode debug config does not list .agents as a config directory; the subscription comes from the compatibility plugin ([...claude, ...agents].map(z => join(z, "skills"))).
  • Minor related issue: opencode service set env <KEY> <VALUE> stops the service before writing config and does not restart it, so it must be run outside the agent/session.

Activity

  1. opencode-agent commented on Oct 9, 2026

    @opencode-agent
    Contributor

    Thanks for the detailed report. I tried to reproduce this on 2.0.26 on native Linux and couldn't trigger the loop.

    What I tried:

    • Setup: a project with .agents/skills/<name>/SKILL.md, a .opencode/ directory and AGENTS.md, plus a local plugin in ~/.config/opencode/plugins/.
    • Run: opencode serve --service, with the project loaded through the API, then left idle for a few minutes.
    • Result: <project>/.agents/skills subscribed once and stayed idle, and the service sat at about 3–4% CPU. Writing to the skill file, .opencode/ or ~/.config/opencode/ caused a few re-subscribes, which then settled.

    Your environment (Docker Desktop on WSL2) is the main thing I couldn't match. These details would help narrow it down:

    1. Location: where does the project directory live? Is it inside the Linux filesystem, or a Windows path (or a bind mount of one, e.g. /mnt/c/..., or a Docker volume from the Windows side)? Does the loop also happen if you copy the project to a native Linux path such as ~/proj?
    2. Plugin: does worktree-plugin.js write any files, such as state, logs or worktree metadata? If so, where? Writes anywhere under ~/.config/opencode/ trigger a compatibility reload (see core: any file write inside the config directory triggers an un-debounced compatibility reload — ~/.claude/skills watcher resubscribe loop (v2.0.23) #53393). Does the loop stop if you temporarily move the plugin out of plugins/?
    3. Logs: a short excerpt of opencode.log covering a few loop cycles, from a run started with --log-level debug if you can. Please include the lines just before each watcher stopped.

    Related reports, most likely first:

  2. s524797336 commented on Oct 9, 2026

    @s524797336
    Author

    Thanks for the detailed report. I tried to reproduce this on 2.0.26 on native Linux and couldn't trigger the loop.

    What I tried:

    • Setup: a project with .agents/skills/<name>/SKILL.md, a .opencode/ directory and AGENTS.md, plus a local plugin in ~/.config/opencode/plugins/.
    • Run: opencode serve --service, with the project loaded through the API, then left idle for a few minutes.
    • Result: <project>/.agents/skills subscribed once and stayed idle, and the service sat at about 3–4% CPU. Writing to the skill file, .opencode/ or ~/.config/opencode/ caused a few re-subscribes, which then settled.

    Your environment (Docker Desktop on WSL2) is the main thing I couldn't match. These details would help narrow it down:

    1. Location: where does the project directory live? Is it inside the Linux filesystem, or a Windows path (or a bind mount of one, e.g. /mnt/c/..., or a Docker volume from the Windows side)? Does the loop also happen if you copy the project to a native Linux path such as ~/proj?
    2. Plugin: does worktree-plugin.js write any files, such as state, logs or worktree metadata? If so, where? Writes anywhere under ~/.config/opencode/ trigger a compatibility reload (see core: any file write inside the config directory triggers an un-debounced compatibility reload — ~/.claude/skills watcher resubscribe loop (v2.0.23) #53393). Does the loop stop if you temporarily move the plugin out of plugins/?
    3. Logs: a short excerpt of opencode.log covering a few loop cycles, from a run started with --log-level debug if you can. Please include the lines just before each watcher stopped.

    Related reports, most likely first:

    Thanks — I think this lines up with #53393, and I found the differentiator.

    Root cause in my environment: the config directory and the data directory are the same directory.

    opencode debug paths reports:
    config ~/.config/opencode
    data ~/.local/share/opencode
    db ~/.local/share/opencode/opencode.db
    log ~/.local/share/opencode/log

    But here both paths resolve to the same directory (identical inodes) — this
    devcontainer bind-mounts the same host folder at both locations. As a result,
    opencode's own data files — opencode.db, opencode.db-wal, opencode.db-shm and
    log/opencode.log — all live inside the config directory. So every DB/WAL/log
    write is a write under the config dir, which per #53393 triggers the
    un-debounced compatibility reload; that calls iQ() to clear and re-subscribe the
    skill watchers (opencode.config.compatibility.refresh()), and the subscribe/log
    activity produces more writes → self-sustaining loop. On a normal install these
    two dirs are separate, which likely explains why you can't reproduce on native
    Linux.

    While idle, the only files touched under the config dir are opencode.db,
    opencode.db-wal, opencode.db-shm and log/opencode.log; /.agents/skills
    itself is never modified.

    Suggested repro on native Linux: force the data dir to equal the config dir —
    e.g. run XDG_DATA_HOME=$HOME/.config opencode serve --service (or symlink
    ~/.local/share/opencode → ~/.config/opencode) — with a project that has
    .agents/skills//SKILL.md. I'd expect the loop to reproduce with no bind
    mount involved.

    Answers to your questions:

    1. Location: project files are on the container's overlay fs (not a Windows
      /mnt/c path). The difference vs your setup is the merged config/data dir
      above, not the project mount.
    2. Plugin: worktree-plugin.js does write under the config dir
      (~/.config/opencode/worktrees/repos//{worktree-sessions.json,.repo-id}
      and worktrees/.migrated-v4), but only during setup and worktree
      create/delete — not while idle. The idle writers are the DB WAL and the log,
      so the plugin is probably not the trigger. Can confirm by moving it out.
  3. opencode-agent commented on Oct 9, 2026

    @opencode-agent
    Contributor

    Thanks for finding the trigger. Making the data directory the same as the config directory was enough to reproduce this on native Linux, no bind mount needed.

    Steps:

    • 2.0.26, a project with .agents/skills/demo/SKILL.md and a .opencode/ directory.
    • XDG_DATA_HOME set to the same directory as XDG_CONFIG_HOME, so opencode.db and the log end up in ~/.config/opencode.
    • Started opencode serve, loaded the project through the API, then left it idle.

    Result: <project>/.agents/skills was torn down and re-subscribed about once per second (about 30 times in 30 s) with no file changes. Every other path subscribed once. With separate data and config directories it subscribes once and stays put. It still happens on the latest v2 (2f3ba04, run from source). The script below reproduces it.

    This looks like the same un-debounced compatibility reload as #53393, with OpenCode's own database and log writes in the config directory as the trigger. Open PR #53764 for that issue may also fix this case. Until there is a fix, the workarounds are to keep the data directory separate from the config directory (e.g. point XDG_DATA_HOME somewhere outside ~/.config), or the OPENCODE_FILEWATCHER_DISABLE=1 setting you already use.

    Reproduction script (repro.sh)
    #!/usr/bin/env bash
    # Repro for #54114: when the OpenCode data dir is the same directory as the config dir,
    # the idle service keeps tearing down / re-subscribing the compatibility skills watcher.
    # Usage: bash repro.sh            (uses `opencode` from PATH)
    #        OC_DEV=/path/to/checkout bash repro.sh   (runs the CLI from source)
    set -u
    ROOT=${ROOT:-/tmp/opencode/repro54114}
    pkill -f "serve --port 4777" 2>/dev/null; sleep 1
    rm -rf "$ROOT"; mkdir -p "$ROOT/home/.config/opencode" "$ROOT/proj/.agents/skills/demo" "$ROOT/proj/.opencode"
    printf -- '---\nname: demo\ndescription: demo skill\n---\nhello\n' > "$ROOT/proj/.agents/skills/demo/SKILL.md"
    (cd "$ROOT/proj" && git init -q)
    
    export HOME="$ROOT/home"
    export XDG_CONFIG_HOME="$HOME/.config"
    export XDG_DATA_HOME="$HOME/.config"   # data dir == config dir (~/.config/opencode), like the author's bind mount
    export XDG_STATE_HOME="$HOME/.local/state" XDG_CACHE_HOME="$HOME/.cache"
    export OPENCODE_PASSWORD=pw
    
    if [ -n "${OC_DEV:-}" ]; then
      oc() { bun --conditions=browser --preload="$OC_DEV/packages/tui/node_modules/@opentui/solid/scripts/preload.js" "$OC_DEV/packages/cli/src/index.ts" "$@"; }
    else
      oc() { opencode "$@"; }
    fi
    
    cd "$ROOT/proj"
    oc --version
    oc serve --port 4777 > "$ROOT/serve.out" 2>&1 &
    SERVER=$!
    sleep 10
    oc api --server http://localhost:4777 GET /api/skill -H "x-opencode-directory:$ROOT/proj" | head -c 200; echo
    LOG=$(ls "$XDG_DATA_HOME"/opencode/log/opencode*.log | head -1)  # opencode.log (release) or opencode-local.log (source)
    count() { grep 'watcher subscribe' "$LOG" | grep -c "$ROOT/proj/.agents/skills"; }
    A=$(count); sleep 30; B=$(count)
    echo "subscribes for .agents/skills: t=0 $A, t=30s $B (idle, no file changes)"
    ps -eo pid,pcpu,cmd | grep "serve --port 4777" | grep -v grep
    grep 'watcher subscribe' "$LOG" | grep -o 'path=[^ ]*' | sort | uniq -c | sort -rn | head -5
    pkill -f "serve --port 4777" 2>/dev/null

    Run it: bash repro.sh

  4. argszero commented on Oct 11, 2026

    @argszero
    Contributor

    I'd like to work on this.

    The cause is visible in packages/core/src/config/plugin/compatibility.ts: refresh() starts with FiberMap.clear(watches) and then re-subscribes every directory. That blanket clear is what defeats the onlyIfMissing: true on the FiberMap.run inside watch() — so every event that reaches the 100 ms debounce tears down and rebuilds all compatibility watches, and each rebuild publishes again. Nothing has to change on disk for that to keep going.

    The sibling paths in this codebase already reconcile instead of clearing — Config.reconcileWatches in packages/core/src/config.ts and ConfigSkillPlugin.watchDirectory both compute the desired key set, FiberMap.remove the keys that are gone, and let onlyIfMissing keep the ones that remain. I'll bring the compatibility watcher in line with that, so an unchanged directory is never re-subscribed and the loop has no event to feed on.

    That also makes the failure testable with the existing Watcher.Test layer, which records every subscribe call: emit one update, let the debounce settle, and assert the compatibility directory was not subscribed a second time.

    Opening a PR against dev.

  5. argszero commented on Oct 11, 2026

    @argszero
    Contributor

    Opened a PR implementing this: #54450 — reconciles the compatibility watches instead of clearing and rebuilding them on every event.

  6. larspage commented on Oct 26, 2026

    @larspage

    Reproducing on Linux (Fedora), opencode v2.1.3 and v2.1.4, with the same symptom and a measurable per-path leak.

    Setup. Two skill roots, both real and non-empty, different inodes:

    drwxr-xr-x  634  ~/.claude/skills   (176 files)
    drwxr-xr-x   22  ~/.agents/skills   (26 files, includes use-railway)
    

    OpenCode watches both conventions by design, so each agent session subscribing to both is expected. The problem is that the subscriptions are never released.

    What I measure. The log records two relevant messages, and only two:

    message="watcher subscribe"   registry grows
    message="watcher stopped"     registry shrinks
    

    There is no watcher unsubscribe message in this build. If you grep for one you get 0 and conclude nothing is ever released. The release event is watcher stopped.

    Net registry per server instance, over a single ~4-day server lifetime:

    subscribes = 98,153
    stops      =  8,397
    net        = 89,756   <- still held, still growing
    

    Releases do happen, roughly 1 per 11 subscribes. The net is what grows without bound.

    Per path, same instance:

    path subscribes stops net
    ~/.claude/skills 45,394 2,366 43,028
    ~/.agents/skills 45,389 5,010 40,379
    ~/.claude/skills/unslop/SKILL.md 273 1 272
    ~/.config/opencode/skill 233 2 231

    The two skill roots account for 83,407 of the 89,756 held subscriptions.

    Rate. A fresh server accumulates roughly 1,240 net subscriptions in its first 5 minutes, about 13,600/hour, so the registry reaches the 4-day level again in under 7 hours.

    The registry is per-process. Restarting the opencode serve --service process resets the count to zero and it immediately starts climbing again. Restarting the TUI client does nothing, which is what made this look like it was not release-related.

    Two details that may help:

    • The ~/.claude/skills and ~/.agents/skills counts are near-identical (45,394 vs 45,389), which looks like each session subscribes to both roots.
    • Individual SKILL.md files also accumulate, at 273 subscribes against 1 stop each.

    Also note the version situation: v2.1.4 does not fix this. I upgraded to it specifically to test that, and the 2.1.3 to 2.1.4 diff (72 commits, 300 files) touches no watcher or file-watching source. The post-upgrade rate is the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions