[darshil929]

When a custom XCom backend stores values externally (e.g. in S3), a branch operator stops skipping some of its downstream tasks on Airflow 3 — specifically dynamically-mapped tasks and tasks that are cleared and re-run.

Why: the branch records its skip decision as an XCom, and that XCom is pushed through your custom backend, which only persists a pointer to the real value. The scheduler reads that XCom as plain JSON to decide what to skip, can't interpret the pointer, and lets the task run instead of skipping it. It's the same root cause as the original 2.10.5 NoCredentialsError — on Airflow 3 the scheduler no longer calls the backend, so the crash turned into a silent skip failure.

Fix: keep the skip decision out of the custom backend. It's now stored in the same plain form Airflow uses by default, so the scheduler can always read it. Nothing changes when no custom backend is configured, and there's no Execution API change.

Added a regression test that runs a real branch operator with an externalizing backend configured — it fails before this change and passes after.

Separate from #52219, which was a different skip bug already fixed in #53455.

closes: #50491

---

Was generative AI tooling used to co-author this PR?

• Yes — claude sonnet 4.6

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[eladkal]

Please provide evidance of before and after images of a running dag with the setup of custom backend.

[darshil929]

Before/after on a running dag with a value-externalizing custom XCom backend.

Setup (Breeze, SQLite). The built-in object-storage backend with the threshold set to 0, so every value, including the branch decision is externalized to a reference instead of stored inline:

AIRFLOW__CORE__XCOM_BACKEND=airflow.providers.common.io.xcom.backend.XComObjectStorageBackend
AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH=file:///files/xcoms
AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_THRESHOLD=0

The branch skips should_be_skipped directly on the worker on the first run, so the regression only surfaces once the scheduler re-evaluates the skip via NotPreviouslySkippedDep i.e. after the task is cleared. So each run is triggered, then should_be_skipped is cleared.

Cleared task — before vs after

Without the fix the cleared task runs (Try 1); with the fix it stays skipped (Try 0).

Before — no fix:

After — with fix:

Task log — before vs after

Before — no fix (body executed):

After — with fix (task was skipped):

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

[darshil929]

Hey @potiuk @eladkal @amoghrajesh,

This PR got closed by the stale bot, no review was ever left on it. It's been marked ready for maintainer review since Jul 8. Would you be able to reopen and review the solution whenever you get a chance?

[kaxil]

Verified locally: the new test fails on main (the pointer string ends up in the DB and the mapped downstream runs) and passes with the fix. Thanks for the before/after evidence.

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.