You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Make Task SDK supervisor subprocess a session leaderFix supervisor session leader - #70950
In task-sdk/src/airflow/sdk/execution_time/supervisor.py, the _subprocess_main function is the entry point for the child process that runs the user's task code. An existing TODO comment indicated that this child process should be made a session leader.
If the process is not a session leader, it remains in the same process group as the parent supervisor. This means that if a signal (like SIGINT or SIGTERM) is sent to the parent process group, both the parent and the child would receive it independently, potentially leading to double-firing of signal handlers and race conditions during shutdown or interruption.
The Solution
This PR resolves the TODO by making the child process a session leader immediately upon startup.
We introduced a call to os.setsid() inside a try...except OSError block, guarded by hasattr(os, "setsid") to ensure cross-platform compatibility (especially for Windows environments where setsid might not be available). This isolates the child process into its own session and process group, ensuring that signals are handled explicitly through the supervisor's lifecycle management rather than broadcasted by the OS terminal.
The changes gracefully fall back if os.setsid() is unavailable or fails.
CI/CD pipelines (via breeze and pre-commit) will verify that syntax is correct and that process isolation continues to function properly across supported operating systems.
Thanks for the PR. A bare setsid() here would regress graceful shutdown: the official image runs dumb-init with DUMB_INIT_SETSID=1, which broadcasts SIGTERM to the whole process group on container stop, and that group co-membership is how the task subprocess receives the signal (see "Signal propagation" in the docker-stack entrypoint docs). With the child in its own session, the task never gets SIGTERM, so its on_kill() handler never runs before the grace period expires and everything is SIGKILLed. #69034 recently made "signal only the supervisor" an explicit opt-in via DUMB_INIT_SETSID=0; this change would force it everywhere.
On the double-firing concern from #70949: _reset_signals() a few lines down already resets the inherited handlers to SIG_DFL in the child, so handlers can't fire twice.
There's also an open PR covering this ground: #65738 moved from os.setsid() to os.setpgid(0, 0) during review, scoped it to task execution only, paired it with os.killpg in kill(), and removes this TODO. Closing in favor of that one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background & Problem
In
task-sdk/src/airflow/sdk/execution_time/supervisor.py, the_subprocess_mainfunction is the entry point for the child process that runs the user's task code. An existingTODOcomment indicated that this child process should be made a session leader.If the process is not a session leader, it remains in the same process group as the parent supervisor. This means that if a signal (like
SIGINTorSIGTERM) is sent to the parent process group, both the parent and the child would receive it independently, potentially leading to double-firing of signal handlers and race conditions during shutdown or interruption.The Solution
This PR resolves the
TODOby making the child process a session leader immediately upon startup.We introduced a call to
os.setsid()inside atry...except OSErrorblock, guarded byhasattr(os, "setsid")to ensure cross-platform compatibility (especially for Windows environments wheresetsidmight not be available). This isolates the child process into its own session and process group, ensuring that signals are handled explicitly through the supervisor's lifecycle management rather than broadcasted by the OS terminal.Impacted Files
[MODIFY] task-sdk/src/airflow/sdk/execution_time/supervisor.py
Testing & Verification
os.setsid()is unavailable or fails.closes: #70949
Was generative AI tooling used to co-author this PR?
Generated-by: Antigravity IDE following the guidelines