Repository navigation
Description
What happened
Two OpenCode threads on one T3 server failed in the same second (2026-11-15 21:39:34 UTC) with The OpenCode event stream was lost and could not reconnect. The T3-owned opencode serve process stayed alive and answered requests again about 90 s later. This is the same class as #23962 and recurs on a build that contains #25596.
Diagnosis
T3 drops /api/event after 45 s of silence, then reconnects 5 times, 2 s apart. Each attempt borrows the connection and subscribes before reconciling, and the subscribe has a 10 s timeout. All five attempts died in the subscribe step, so reconcile() never ran and failAll failed every OpenCode thread on the shared session. #25596's unbounded retry for a live owned process only covers reconcile(), so it never applied.
Steps to reproduce
Deterministic, no memory pressure needed: freeze the owned opencode serve with SIGSTOP for longer than about 100 s while a turn is active.
- Start an isolated T3 server so your normal one is untouched:
t3 serve --mode web --port <port> --host 127.0.0.1 --no-browser --base-dir <base-dir> - A fresh base dir has the OpenCode provider disabled. Enable it in
<base-dir>/userdata/settings.json:{"providerInstances":{"opencode":{"driver":"opencode","enabled":true,"config":{"binaryPath":"opencode"}}}} - Add a project and start a thread whose turn stays active:
t3 project add <workspace> --base-dir <base-dir> t3 thread start <workspace> "Run this exact command and wait for it to finish: sleep 600" --base-dir <base-dir> --provider opencode --model opencode-go/deepseek-v4.1-flash - Once the
sleep 600tool call is running, stop theopencode servethat this T3 server started (<opencode-port>is the port T3 picked for it, visible inps), wait 120 s, then resume it:pid=$(pgrep -f 'opencode serve --hostname=127.0.0.1 --port=<opencode-port>') kill -STOP "$pid" for i in $(seq 12); do sleep 10; awk '{print $3}' "/proc/$pid/stat"; done # T every time kill -CONT "$pid"
Expected: T3 waits for the live owned server and the turn continues after SIGCONT.
Actual: after 45 s of silence and five 10 s subscribe timeouts, the run fails with transport_error / The OpenCode event stream was lost and could not reconnect. while the process is still in state T. The full timeline from this repro is in the comment below.
How it happened in the real incident
The agents cgroup was at its limit. T3 caps app-t3code-agents.slice at 3.35 GiB soft / 5.03 GiB hard on this 6.7 GiB WSL2 VM, and that covers both provider servers, their tool children and the preview browser. A build with a 3 GB Node heap ran in one OpenCode thread. Slice counters: memory.events high 103941, memory.peak 3.68 GB, swap.peak 2.1 GB, swap.events fail 28018492, oom_kill 0. Nothing was killed; the server stayed throttled longer than T3's window of up to 58 s of reconnects. The T3 server outside the slice stayed responsive.
Version
0.1.10-nightly.20261115.4749 (commit 295a0a7, includes #25596). OpenCode 2.3.0.
Environment
WSL2, 6.7 GiB RAM; T3 desktop app on the Windows host, server inside WSL.
Evidence
21:36:38.253 last OpenCode reply before the stall
21:37:56 last /api/event frame (inferred from the 45 s watchdog)
21:38:41.413 reconnect 1 7.2 s no response headers
21:38:50.641 reconnect 2 10.0 s
21:39:02.646 reconnect 3 10.0 s
21:39:14.652 reconnect 4 7.2 s
21:39:23.869 reconnect 5 10.0 s
21:39:33.88 failAll; provider session status=error
21:41:02.278 next successful OpenCode reply (GET /api/model, 61.8 s)
OpenCode logged eight /api/event responses closing at 21:41:02 (the five reconnects plus three probes), minutes after T3 gave up.
Related issues
#23962 was closed by #25596; its 2026-11-14 comment reports this remaining subscribe gap. #30201 added the 10 s subscribe timeout that now bounds each reconnect. #25751 is the defect variant that leaves the session ready, not this case. #26171: the shared Codex app-server sits in the same agents slice.
Fix applied or workaround
Server restart cleared it. Local: raise the WSL memory limit, run large builds outside the agents slice, lower the Node heap.
Filed by
DeepSeek V4.1 Flash (OpenCode) via t3 triage, reviewed by Claude Opus 5.5 (Kiro) in T3 Code.
Activity
juliusmarminge commentedon Nov 15, 2026
I tried to reproduce this on the current build and could not.
Commit 69f8f59 (origin/main). I did not run the app against a frozen OpenCode server. Instead I wrote an adapter test in apps/server/src/orchestration-v2/Adapters/OpenCode2AdapterV2.test.ts, 'recovers an owned server after more than five event subscriptions time out'. It runs the real OpenCode2 V2 adapter against the replay server with an owned (live) process. The first /api/event subscription opens the session, the stream then ends, and the next six subscriptions never return headers. The test advances the TestClock 12 s per attempt (the 10 s SUBSCRIPTION_TIMEOUT plus the 2 s RECONNECT_DELAY), then lets the following subscription and reconcile succeed. On main the turn ends 'failed': after 5 reconnect attempts, failAll ran with 'The OpenCode event stream was lost and could not reconnect.', and reconcile never ran. With a temporary unbounded reconnect retry for owned servers the same test passed ('completed'); I reverted that and pushed no fix. The rest of that file and the OpenCode integration tests (190) still pass with the testkit change.
Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.
satyalyadav commentedon Nov 16, 2026
Reproduced on 0.1.10-nightly.20261115.4749 (commit 295a0a7) with a plain SIGSTOP and no memory pressure.
Setup: a second T3 server on its own port and base dir, so the normal server was untouched. It ran one OpenCode thread whose first turn runs
sleep 600, which keeps the turn active while the server is otherwise idle.Steps:
<base-dir>/userdata/settings.jsonbefore starting the thread:{"providerInstances":{"opencode":{"driver":"opencode","enabled":true,"config":{"binaryPath":"opencode"}}}}sleep 600tool call is running, freeze theopencode servethat the isolated server started for 120 s, then resume it.<opencode-port>is the port T3 picked for it (visible inps).Result: the run failed at 02:23:31.761Z with
transport_error,The OpenCode event stream was lost and could not reconnect., the same failure as the report. Times are UTC; reconnect start times and durations are from the server trace spans.The process was alive the whole time (state T, never exited), and it answered every reconnect within a second of SIGCONT.
reconcilenever ran, so the #25596 recovery never applied.The memory-throttled cgroup from the original report is only how the server got frozen in the real incident. Any freeze of the owned process longer than about 100 s reproduces this: at most 45 s of silence, then five 10 s subscribes with 2 s gaps (45 + 5×10 + 4×2 = 103 s).
Why I think this is a real bug and not a no-repro:
failed, with failAll and no reconcile. It passed only with the unbounded owned retry, which was reverted. The run above is the same failure against a real ownedopencode serve, frozen rather than mocked.reconcilegot the unbounded owned retry.Could this be reopened?