thread payload hangs every request ~325s -> [504] Provider timed out, retry loop wedges the session, no circuit breaker #54538

what happens

one long-lived thread (v2.0.23, windows, model opencode/mimo-v2.6-flash-free) started failing every single turn: each request hangs in prefill, never streams a byte, and surfaces after ~5.5 min as:

⚠ Retry due · attempt 2 · Streaming response failed: [504] Provider timed out after 356295ms (numbers vary: 325351ms, 362397ms seen too)

the retry banner then re-sends the identical payload, which hangs again. meanwhile the same model in any fresh session answers in 2-15s (tested repeatedly while the thread was failing).

what we ruled out (measured, same day, same box)

• provider/backend down: fresh session + same model + text ping = 3.1s
• the pasted image in that thread: fresh session + same model + the exact same png attached = 14.1s, fine
• aborted assistant turns: fresh session, streamed a bit, interrupted mid-flight (aborted assistant turn now in history), pinged again = 2.4s, fine
• quota: reproduced inside a fresh free window; quota exhaustion surfaces a different error string anyway
• thread size: the thread was compacted - its last healthy call had 177 input tokens

what fixed it

the trigger is something in that thread's specific payload - context-dependent, not reproducible with the individual pieces in isolation. the fix that worked: POST /session/{id}/interrupt (the retry loop holds the session busy - every revert/stage/commit returns SessionBusyError 409 until an explicit interrupt), then revert to a point before the corruption. after the revert: same ping, 26s, works.

parts that feel like app bugs regardless of the trigger

1. no circuit breaker: identical-payload timeouts retry forever. after ~3 identical timeouts the run should stop and say so (ideally with a payload hash so people can correlate attempts).
2. retry attempts are invisible in logs - only final drain failures land in opencode.log, so a hanging-then-recovering loop leaves zero trace. one line per retry attempt would have made this debuggable in minutes instead of a day.
3. a pre-first-token hang leaves the session wedged for everything (busy 409s across the api) until an explicit interrupt.

related

#53747 #45514 #49030 #50675

environment

v2.0.23, windows, opencode/mimo-v2.6-flash-free (zen free lane). happy to run instrumentation builds or share the pre-fix session export - it is exported and intact.