Repository navigation
fix(core): clean up orphaned MCP processes after a crash - #59398
rekram1-node wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
The fix works. On the base, kill -9 of serve left a stdio MCP server that ignores stdin EOF, and its child, running with PPID 1. With this PR they're all gone within 3s. Disconnect, reconnect and SIGTERM shutdown still leave nothing behind. The core typecheck passes, and the MCP tests match the base: the one failure there also fails on the base.
Public API: unchanged.
Worth knowing: a nonexistent command now fails as MCP server process exited with code 127: …: not found instead of the spawner's NotFound error. It's still readable, since stderr is included.
The SIGKILL test in process-guard.test.ts fails here on every run. The guard does its job, but the test counts zombies as alive (see inline).
Related: #50363
|
|
||
| // SIGTERM ends the descendant at once; the leader ignores it until the SIGKILL two seconds later. | ||
| const deadline = Date.now() + 10_000 | ||
| while (pids.some(alive) && Date.now() < deadline) await Bun.sleep(50) |
There was a problem hiding this comment.
This fails every run in our sandbox, even though the guard works. The killed leader ends up a zombie (STAT Z, PPID 1) because PID 1 there doesn't reap orphaned processes, and process.kill(pid, 0) succeeds for zombies. Containers without an init process behave the same way. Could alive treat zombies as dead (e.g. check the state in /proc/<pid>/stat, or ps -o stat=)? Then the test checks that the processes stopped, not whether PID 1 reaped them. The real 2s/10s waits also make it slow; a test with only the descendant (no TERM-ignoring leader) would avoid the 2s escalation.
22c9e79 to
6bcb5fe
Compare
There was a problem hiding this comment.
Still fixes the crash case. I reran the serve + kill -9 repro with a stdio MCP server that ignores stdin EOF. On the base it stays alive with PPID 1; with this PR it and its child are gone right away. Disconnect, reconnect and SIGTERM shutdown still leave nothing behind. The core typecheck passes, and the MCP tests match the base.
The earlier point about the flaky SIGKILL test is resolved: that test and process-guard.ts were removed, and the guard is now inline in stdio.ts.
Public API: unchanged.
No new concerns.
|
Closing this in favor of a smaller approach: track the process groups the spawner starts and kill them in an exit hook, instead of wrapping every spawn in sh. I'll open that as a separate PR. |
Problem
If the opencode server crashes or gets SIGKILLed, local MCP servers that don't exit on stdin EOF (like
npxwrappers) keep running forever, reparented to PID 1. They pile up across restarts. Part of #50363.Normal shutdown already kills the MCP process group, but nothing does that when the server dies without running its cleanup.
Change
I start each local MCP server through a small
shguard inpackages/core/src/mcp/stdio.ts. The server gives the guard an extra pipe (fd 3) and never writes to it. The kernel closes that pipe however the server exits, so a watcher in the guard sees EOF and SIGKILLs its own process group.kill 0hits the watcher's own group, which exists as long as the watcher does, so a reused PID can't be hit.Out of scope
Verification
servewith a stdio MCP server that ignores stdin EOF, thenkill -9the server. Before: the MCP server stays alive with PPID 1. After: it's gone immediately.bun run script/test.ts test/mcp.test.tsandbun typecheckinpackages/core,bun run checkfrom the root.Visual
Added by the opencode bot (AI-generated).
Related: #50363