Skip to content

fix(opencode): give opencode serve a startup budget that survives a loaded machine - #1367

Merged
Juliusolsson05 merged 10 commits into
mainfrom
fix/opencode-serve-startup-under-load
Sep 27, 2026
Merged

Juliusolsson05 merged 10 commits into
mainfrom
fix/opencode-serve-startup-under-load

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Refs #1355. This fixes the opencode serve startup timeout. Orchestration children whose start takes over 30 s still hit the orchestration bridge's own create-agent deadline, so #1355 stays open until #1370 lands.

Plan: docs/plans/2026-09-27-opencode-serve-startup-under-load.md (first commit).

Problem

orchestration_create_agent with kind opencode failed repeatedly under load: 3 times from 3 workers in about an hour, and twice more for my own reviewers. Each failure read "Timed out waiting for …/opencode serve to report its URL". The only deadline on this path is SpawnedServer's package default of 10 s. The app never passed its own, and nothing in the app wraps headless.start() in another deadline.

Evidence

Reproduced on the real bundled binary (OpenCode 1.18.31), timing opencode serve --port 0 until its listen line:

  • Idle: 0.57 s, 0.73 s, 0.86 s.
  • CPU contention (16 burners at nice 10, on a machine other workers were already loading): 43.1 s, 16.7 s and 18.3 s, and every start became ready.

The server was healthy but starved; the fixed wall-clock deadline was what killed it.

What merges

  • OpencodeSession passes startupTimeoutMs: OPENCODE_SERVE_STARTUP_TIMEOUT_MS (120 s) to OpencodeHeadless. The app owns the process and the package already takes the option, so this needs no package change.
  • Why 120 s: it is about 3× the worst healthy start measured, and it still bounds a server that stays alive but never listens.
  • A dying server still fails at once through the package's exit path.
  • Not built: a CPU-progress watchdog. It needs a per-platform child CPU probe and is only worth building if a real hang is recorded; the WHY comment says so.

Not in this PR

  • The orchestration bridge's 30 s create-agent deadline (bug(orchestration): a child whose provider starts in >30 s is adopted without its bootstrap prompt #1370). It is shared by every provider. A start slower than 30 s makes the create outcome unknown, and the late-adopted child gets no bootstrap prompt. Fixing it means choosing between late bootstrap delivery and MCP clients' own tool-call timeouts. With this PR, a start of 10–30 s now succeeds through orchestration too; two of the three recorded contended starts took 16.7 and 18.3 s.

Tests

  • opencodeSession.readiness.test.ts: the spawned headless gets the startup wait, and the wait is between two and four times the worst measured healthy start, and a start that rejects stops the server it spawned.
  • opencodeServeStartup.system.test.ts: the real SpawnedServer and a real child process, where a stub opencode prints OpenCode's listen line 11 s late. It becomes ready under the app's wait, and it is rejected under the package default: the exact failure users hit.
  • opencodeServeStartup.system.test.ts, through the package: OpencodeHeadless forwards the app's wait to the spawned serve, and dropping that forwarding fails the test (review a).
  • Red check: both new cases were red before the fix.
  • Gates: on 8f7a5ec6 (main merged), npx tsc -b is clean and both suites pass. The slow-serve system tests wait as long as the app does.

🤖 Generated with Claude Code

Juliusolsson05 and others added 3 commits September 26, 2026 21:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ling it at 10 s

The package's serve readiness default (10 s) was sized for an idle machine.
Measured on the bundled binary, a healthy server reported its URL in
0.6-0.9 s idle but 16.7-43.1 s under load, so orchestration's OpenCode agents
failed to spawn. The app owns the process and now passes a 120 s wait; a
server that dies still fails at once.

Refs #1355 (the orchestration bridge deadline is #1370)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05 Juliusolsson05 added provider:opencode Affects OpenCode integration type:bug Something works wrong class:C3-silent-failure The app knows it failed and does not say sev:P2 Real bug with a workaround labels Sep 27, 2026
…stant's margin

#1367 review a: dropping OpencodeHeadless's forwarding of startupTimeoutMs,
or a 44 s constant, left every test green. The orchestration bridge's own
30 s create-agent deadline is filed as #1370; this PR becomes Refs #1355.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
…stant's margin

#1367 review a: dropping OpencodeHeadless's forwarding of startupTimeoutMs,
or a 44 s constant, left every test green. The orchestration bridge's own
30 s create-agent deadline is filed as #1370; this PR becomes Refs #1355.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05
Juliusolsson05 force-pushed the fix/opencode-serve-startup-under-load branch from 85abb7e to bdfa24b Compare September 27, 2026 06:50
Juliusolsson05 and others added 2 commits September 26, 2026 23:51
#1367 review b: a fixed 30 s test budget could fail on a loaded runner
while the app (120 s) would still be waiting. The positive cases now
follow OPENCODE_SERVE_STARTUP_TIMEOUT_MS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… rollback

#1367 review c: a 1_200_000 constant and removing the rollback's
headless.stop() both left every test green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05

Copy link
Copy Markdown
Owner Author

Review disposition. Round 1: a FIX-BEFORE-MERGE, b MERGE-READY, c (pi) MERGE-READY. Verification of a: FIX-BEFORE-MERGE in round 2 (one new blocker), then MERGE-READY in round 3.

Scoped out and filed

Fixed

  • a, round 2 (blocker): the fix commit's message still said Fixes #1355, so a merge commit would have closed bug(opencode): orchestration-created OpenCode agents fail to start (opencode serve spawn timeout) under load #1355. I rewrote the messages of this PR's own commits (git filter-branch --msg-filter, origin/main..HEAD only) and force-pushed with a lease. The tree is byte-identical, and GitHub now lists no closing issue. a confirmed this in round 3.
  • a (minor): the package's forwarding of startupTimeoutMs was untested. A system test now drives OpencodeHeadless.resolveServerUrl with the app's option, and it is red when the forwarding is dropped.
  • a (surviving mutant): a 44 s constant passed. The constant is now at least 2× the worst healthy start (43.1 s).
  • c (minor): a 20-minute constant passed. It is now also at most 4× the worst healthy start.
  • c (minor): the rejected-start rollback was untested. A start that rejects must stop the server it spawned, and the test is red without headless.stop().
  • b (minor): a fixed 30 s test budget could fail on a loaded runner while the app would still be waiting. The positive slow-serve tests now wait OPENCODE_SERVE_STARTUP_TIMEOUT_MS + 10 s.

🤖 Generated with Claude Code

@Juliusolsson05
Juliusolsson05 merged commit 0c696af into main Sep 27, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the fix/opencode-serve-startup-under-load branch September 27, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

class:C3-silent-failure The app knows it failed and does not say provider:opencode Affects OpenCode integration sev:P2 Real bug with a workaround type:bug Something works wrong

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant