Skip to content

fix(cli): wait for background jobs in opencode run - #60699

Merged
rekram1-node merged 1 commit into
devfrom
run-background-jobs
Nov 4, 2026
Merged

rekram1-node merged 1 commit into
devfrom
run-background-jobs

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

opencode run exits as soon as the model's first reply ends. The shell tool tells the model to start long work with background: true and end its reply because it will be woken when the command finishes. In run there is nobody left to wake: training runs, test suites and evals get cut off before the model sees their results.

Change

After each reply, run checks the whole session tree with GET /api/experimental/job?sessionID=...&descendants=true: the main session, its subagents, their subagents, and every background shell or image any of them started.

a reply finishes anywhere in the tree
  ├─ nothing running, no wake queued → exit
  └─ any job in the tree still running → keep waiting
       └─ it finishes → its session is woken, replies → check again
  • A job that finished but hasn't woken its session yet still holds the run. Its row is removed only after the wake is queued, so the check also looks at queued inbox items and active sessions in the tree.
  • Ctrl-C while waiting now exits with code 130. Before, it hung when the session was already idle.

Visual

<svg xmlns="http://www.w3.org/2000/svg" width="240" height="72" viewBox="0 0 240 72" fill="none" stroke="currentColor" font-family="ui-monospace,monospace" font-size="9">
  <style>
    .job{animation:job 4s linear infinite}
    .wake{opacity:.25;animation:wake 4s linear infinite}
    @keyframes job{0%{width:0}60%,100%{width:120px}}
    @keyframes wake{0%,60%{opacity:.25}70%,100%{opacity:1}}
    @media (prefers-reduced-motion:reduce){.job,.wake{animation:none}.job{width:120px}.wake{opacity:1}}
  </style>
  <rect x="8" y="14" width="44" height="14" rx="3" opacity=".6"/>
  <text x="14" y="24" fill="currentColor" stroke="none">reply</text>
  <rect class="job" x="60" y="16" width="120" height="10" rx="3" fill="currentColor" fill-opacity=".25" stroke="none"/>
  <text x="60" y="42" fill="currentColor" stroke="none" opacity=".7">background job running</text>
  <rect class="wake" x="188" y="14" width="44" height="14" rx="3"/>
  <text class="wake" x="194" y="24" fill="currentColor" stroke="none">reply</text>
  <text x="60" y="60" fill="currentColor" stroke="none" opacity=".5">run stays open until the wake</text>
</svg>

Out of scope

Verification

  • bun test test/run/noninteractive.test.ts in packages/cli: 15 pass, and 150 pass with --rerun-each 10
  • bun run typecheck:tsc in packages/cli
  • oxlint and prettier --check on the changed files

Added by the opencode bot (AI-generated).

Fixes #48361
Fixes #59223

Related: #60286

Public API: opencode run now waits for background jobs anywhere in its session tree before exiting; no new flags or endpoints

@rekram1-node
rekram1-node force-pushed the run-background-jobs branch 2 times, most recently from b3b5fd2 to 1010b17 Compare October 30, 2026 21:54
@opencode-agent
opencode-agent Bot force-pushed the background-job-idle branch from f2da732 to e1f45df Compare November 3, 2026 03:28
Base automatically changed from background-job-idle to dev November 3, 2026 16:32

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach looks sound: asking the server for background jobs before treating session.execution.succeeded as the end of the run, then checking active sessions and the inbox. The ordering that makes this race-free checks out in the code: the shell notifier calls sessions.synthetic before completeBackground deletes the job row. I couldn't reproduce the problem end to end. My drive script never got the simulated background shell call to reach the CLI, so this review rests on reading the code plus the tests (noninteractive.test.ts 15/15, server job.test.ts 1/1) and typecheck:tsc in packages/cli (no errors).

Public API (all additions, backwards compatible):

  • New GET /api/experimental/job (experimental.job.list) with optional query params sessionID, kind (shell|subagent|image), status (running|completed|error|cancelled) and descendants ("true"|"false"). It returns { data: Job.Info[] } and gives 404 for an unknown session.
  • New client method client.experimental.job.list (promise and effect clients), plus the ExperimentalJobList* types.
  • New @opencode/schema/job module (Job.Kind, Job.Status, Job.Time, Job.Shell, Job.Subagent, Job.Image, Job.Info).
  • Behaviour change in opencode run: it now stays running until background jobs from the run finish, and Ctrl-C while waiting exits with 130.

Main concern: opencode run can now hang forever when a prompt starts a background job that never finishes (see the inline comment).

Fixes #48361
Fixes #59223
Related: #60286

Comment thread packages/cli/src/run/noninteractive.ts Resolved

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit fixes the earlier concern: background shells now hold opencode run open for at most 10 minutes after the last reply. After that the run stops them, prints a warning and exits, and subagents and images still wait until they finish. I checked by reading the code, plus noninteractive.test.ts (16/16) and typecheck:tsc in packages/cli (no errors). I didn't run it end to end.

Public API: unchanged by this pull request now that the job-list endpoint and client are in the base. The only change is that opencode run waits for its session's background jobs (shells for at most 10 minutes).

Concerns: the 10-minute timer can run out while a woken reply is still streaming and cut it off (inline). The first new test is large and depends on how many ticks Bun.sleep(0) takes (inline).

  • packages/cli/src/run/noninteractive.ts:176: The 10-minute timer can run out while the model is still writing a reply. shellWait is only reset on session.execution.succeeded. Say one shell finishes near the 10-minute mark and wakes the model, and another shell is still running. If the timer runs out during that reply, this check sees only running shells and returns "shells" before it looks at session.active. The expired branch then stops the other shell and returns from consume, so the rest of the reply is never printed. Could you check active/inbox before treating the session as idle with only shells left, or clear the timer when a new step starts (for example on session.step.started for this session)?
  • packages/cli/test/run/noninteractive.test.ts:695: This test is about 100 lines and builds its own event stream and mocks, which the backgroundRun helper used by the other new tests already covers. It also relies on two await Bun.sleep(0) calls being exactly enough ticks before it asserts exited === false. That timing is fragile, and the test can pass trivially if the run hasn't reached the job check yet. Could it use backgroundRun and wait for job.list to be called (or for another explicit signal) instead of counting ticks?

@rekram1-node
rekram1-node force-pushed the run-background-jobs branch 2 times, most recently from 1feba95 to b706e45 Compare November 3, 2026 21:22

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix works: I ran the #48361 reproduction (two background subagents, 15s each) from source. On the base, run exited after 3s without the results. With this pull request it waited 18s, printed the parent's resumed FINAL: got TOKEN_A and TOKEN_B and exited 0. noninteractive.test.ts (15/15) and typecheck:tsc in packages/cli both pass.

The earlier point about run hanging forever on a long-lived shell stays addressed: shells are capped at 10 minutes, and run now just exits without them.

Public API: unchanged. The only change is that opencode run waits for its session's own background jobs (shells for at most 10 minutes).

Remaining concern: the shell timer can still run out during a woken reply, and it now aborts with no check (inline).

  • packages/cli/src/run/noninteractive.ts:586: This timer is still only cleared on session.execution.succeeded, and it now aborts with no check at all. Say one shell finishes near the 10-minute mark and wakes the model while another shell is still running. The timer then fires in the middle of the woken reply and consume stops. reconcile backfills that reply's text, but nothing handles what the turn does after that: permission or question requests go unanswered, so session.wait could hang, and background jobs it starts are ignored. Clearing the timer when this session starts a new step (or re-checking session.active before aborting) would avoid this.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix works: in the #48361 reproduction (two background subagents, 15s each, run from source), the base exited after 2s without the results. With this pull request, run waited 18s, printed the parent's resumed FINAL: got TOKEN_A and TOKEN_B and exited 0. noninteractive.test.ts (14/14) and typecheck:tsc in packages/cli both pass.

The 10-minute shell timer is gone, so the earlier problem of it running out during a woken reply can no longer happen. Without that cap, the open thread about a background shell that never exits keeping run open stays open; the description now lists it as out of scope.

Public API: unchanged. The only behaviour change is that opencode run waits for its session's own background jobs.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix still works: in the #48361 reproduction (two background subagents, 15s each, run from source), the base exited after 5s without the results. With this pull request, run waited 20s, printed the parent's resumed FINAL: got TOKEN_A and TOKEN_B and exited 0. noninteractive.test.ts (15/15) and typecheck:tsc in packages/cli both pass.

The new tree-wide check found no new problems. A finished job is only removed after its session's wake-up is queued, so checking the queued wake-ups of every session that owned a job and the active sessions under the main one covers the handoff without a gap. The new subagent-shell test is small and doesn't depend on timing.

The open thread on unbounded waits now also covers servers started by subagents, since their jobs count too.

Public API: unchanged. The only behaviour change is that opencode run waits for background jobs anywhere in its session tree.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This push is a rebase onto newer dev, and the pull request's own commit is unchanged since my last review. On the rebased head, the #48361 reproduction still works: run waits for the background subagents, prints the resumed FINAL: got TOKEN_A and TOKEN_B and exits 0. noninteractive.test.ts (15/15) and typecheck:tsc in packages/cli both pass. No new concerns.

Public API: unchanged. The only behaviour change is that opencode run waits for background jobs anywhere in its session tree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Background Subagents in Standalone headless mode opencode2 run and background: true tasks may cause premature exit

1 participant