Skip to content

Stop a language SDK task's children when the task is stopped - #75545

Closed
namanjain24-sudo wants to merge 1 commit into
apache:mainfrom
namanjain24-sudo:fix-language-sdk-child-process-group
Closed

namanjain24-sudo wants to merge 1 commit into
apache:mainfrom
namanjain24-sudo:fix-language-sdk-child-process-group

Conversation

@namanjain24-sudo

Copy link
Copy Markdown
Contributor

The coordinators start a language runtime with subprocess.Popen and leave it in the supervisor's own
process group, so stopping the task signals the runtime alone. Anything the task started — a CLI spawned
from a TypeScript handler, for instance — is re-parented to init and keeps running alongside the next try,
and holds the runtime's event loop open until the SIGKILL escalation fires after killed_task_cleanup_time.

Python tasks stopped leaking this way in #65505: the task process runs in its own process group so the
supervisor can signal the whole group. WatchedSubprocess._signal_subprocess already does that with
os.killpg whenever new_process_group was recorded at start time, so the runtime only needs the same
treatment — process_group=0 on the Popen, and the flag set on the subprocess object.

This is direction 1 of the two the issue lists. It fixes every path that stops the task, since they all go
through _signal_subprocess: clearing a running task, execution_timeout, repeated heartbeat failures and
task-success overtime.

process_group is POSIX-only, which matches the rest of this machinery — the Python path already calls
os.posix_spawn(..., setpgroup=0) and os.fork() + os.setpgid unconditionally.

closes: #75533

Testing

Checked the real behaviour rather than only the call, with a runtime that spawns a child the way the
issue's handler does:

WITHOUT process_group (old): own_pgid=False  grandchild_alive_after_stop=True
WITH    process_group=0 (fix): own_pgid=True  grandchild_alive_after_stop=False
  • New test asserts the runtime is started with its own process group and that the subprocess records it, so
    _signal_subprocess takes the killpg path. It fails without the change (KeyError: 'process_group').
  • task-sdk/tests/task_sdk/coordinators/ — 475 passed.
  • task-sdk/tests/task_sdk/execution_time/test_supervisor.py — 374 passed, 1 skipped.
  • breeze run mypy task-sdk/src/airflow/sdk/coordinators/_subprocess.py — clean.
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Opus 5) following the guidelines

The coordinators start a language runtime with subprocess.Popen and leave it in
the supervisor's own process group, so stopping the task signals the runtime
alone. Anything the task started -- a CLI spawned from a TypeScript handler, say
-- is re-parented to init and keeps running alongside the next try, and holds
the runtime's event loop open until the SIGKILL escalation fires.

Python tasks stopped leaking this way in apache#65505 by running in their own process
group so the supervisor can signal the whole group. Giving the runtime the same
treatment is all that _signal_subprocess needs to reach it with os.killpg.
@boring-cyborg boring-cyborg Bot added area:coordinator Coordinator: The interface to spawn Lang-SDK subprocesses area:task-sdk labels Oct 24, 2026
@uranusjr

Copy link
Copy Markdown
Member

#75707 is more progressed.

@uranusjr uranusjr closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:coordinator Coordinator: The interface to spawn Lang-SDK subprocesses area:task-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Child processes of language SDK tasks survive when the task is stopped

2 participants