fix: forward task logs to stdout in LocalExecutor#63960
fix: forward task logs to stdout in LocalExecutor#63960taranlu-houzz wants to merge 1 commit intoapache:mainfrom
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
|
@taranlu-houzz This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack. |
1af4a04 to
1a722cb
Compare
|
The CI failures are all from Pushed a fix that updates the test assertion to expect the new parameter. Edit: Claude sent this message. Still trying to figure out the best way to be transparent about working via an agent. |
LocalExecutor's `_execute_work()` calls `supervise()` without passing `subprocess_logs_to_stdout=True`, so task logs are only written to log files and never reach the container's stdout. This breaks log collection in Kubernetes deployments that rely on container stdout for aggregation (e.g., Fluentd, Coralogix, Datadog). The containerized executor (`execute_workload.py`) already passes this flag (added in apache#47731), but LocalExecutor was not updated. Closes apache#54501
7590962 to
142ad7c
Compare
|
Closing in favor of #64067 |
…tor log forwarding in Airflow 3 This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. closes: apache#63960 closes: apache#49863 closes: apache#54501
…rwarding (#64067) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. (cherry picked from commit f1495c2) Co-authored-by: Subham <subhamsangwan26@gmail.com> closes: #63960 closes: #49863 closes: #54501
…rwarding (apache#64067) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. (cherry picked from commit f1495c2) Co-authored-by: Subham <subhamsangwan26@gmail.com> closes: apache#63960 closes: apache#49863 closes: apache#54501
…rwarding (#64067) (#64592) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. (cherry picked from commit f1495c2) closes: #63960 closes: #49863 closes: #54501 Co-authored-by: Subham <subhamsangwan26@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…ache#64067) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. closes: apache#63960 closes: apache#49863 closes: apache#54501
…ache#64067) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. closes: apache#63960 closes: apache#49863 closes: apache#54501
…rwarding (#64067) (#64592) This restores the ElasticsearchTaskHandler in airflow_local_settings.py and ensures LocalExecutor forwards task logs to stdout by passing subprocess_logs_to_stdout=True to the supervisor. (cherry picked from commit f1495c2) closes: #63960 closes: #49863 closes: #54501 Co-authored-by: Subham <subhamsangwan26@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
LocalExecutor's
_execute_work()callssupervise()without passingsubprocess_logs_to_stdout=True, so task logs are only written to logfiles and never reach the container's stdout. This breaks log collection
in deployments that rely on container stdout for log aggregation.
The containerized executor (
execute_workload.py) already passes thisflag (added in #47731), but LocalExecutor was not updated.
closes: #54501
related: #49863
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code following the guidelines