Skip to content

Downgrade "XCom not found" log to warning in Task SDK client - #73201

Closed
robertorobles wants to merge 1 commit into
apache:mainfrom
robertorobles:downgrade-xcom-not-found-log-level
Closed

robertorobles wants to merge 1 commit into
apache:mainfrom
robertorobles:downgrade-xcom-not-found-log-level

Conversation

@robertorobles

Copy link
Copy Markdown

closes: #73175

Description

TaskInstance.xcom_pull() from within a running task goes through airflow.sdk.api.client.py::XComOperations.get() (and get_sequence_item()). When the requested key/task_id has no matching XCom, the API server returns a 404, which is caught and handled gracefully: the call returns None, matching Airflow 2.x behavior, and no exception propagates to the task.

Before returning, though, this was logged at ERROR level:

log.error(
    "XCom not found",
    dag_id=dag_id, run_id=run_id, task_id=task_id, key=key,
    map_index=map_index, detail=e.detail, status_code=e.response.status_code,
)

This means any task using the common, valid pattern of pulling an optional upstream XCom (e.g. with a default, or just checking whether something was pushed) emits a task-log ERROR even though nothing failed. This is confusing for monitoring/alerting setups that key off ERROR-level task logs, and misrepresents an expected, already-handled condition as an error.

This behavior was introduced in #45344 (closing #45341), which fixed a real bug — the task crashing entirely on a missing XCom — by catching the 404 and returning None. That PR's discussion focused on not crashing; the log level itself wasn't discussed.

Change

Downgrade the two "XCom not found" log.error(...) calls in XComOperations.get() and XComOperations.get_sequence_item() to log.warning(...). No behavior change — the call still returns None/an ErrorResponse as before, only the log level changes.

Scope is intentionally limited to these two XCom call sites (as opposed to also touching the unrelated "Asset not found" log in the same file), to keep this focused and easy to review.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

TaskInstance.xcom_pull() gracefully returns None when the requested
XCom key doesn't exist (matching Airflow 2.x behavior), but the Task
SDK's HTTP client logged this expected, handled condition at ERROR
level. This produces noisy, misleading ERROR lines in task logs for a
common and valid pattern (checking for an optional upstream XCom),
which is confusing for monitoring/alerting setups that key off
ERROR-level task logs.

Related: apache#45341, apache#45344 (which introduced this behavior
while fixing a task crash on missing XCom).

Closes: apache#73175

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@boring-cyborg

boring-cyborg Bot commented Sep 15, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@robertorobles

robertorobles commented Sep 15, 2026 •

Copy link
Copy Markdown
Author

Closing this as a duplicate — #73198 was opened shortly before this one for the same issue (#73175) and additionally includes test coverage for the changed log level, which this PR was missing. Let's consolidate review effort there.

@robertorobles

Copy link
Copy Markdown
Author

Superseded by #73198.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xcom_pull logs at ERROR level for an expected "XCom not found" case in Airflow 3.x Task SDK

2 participants