Skip to content

Fix task retry logic to respect retries for all exit codes#58384

Merged
potiuk merged 1 commit intoapache:mainfrom
astronomer:correct-upforretry-conditional
Nov 17, 2025
Merged

Fix task retry logic to respect retries for all exit codes#58384
potiuk merged 1 commit intoapache:mainfrom
astronomer:correct-upforretry-conditional

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

Why

Tasks that fail with non zero exit codes (like exit code 1) were not retrying even when retries were configured. This occurred when processes exited abnormally in a task, such as:

  • External command failures (e.g., git fetch failing with exit code 1)
  • Network timeouts in subprocess operations
  • Any process that exits with a positive exit code before Python exception handling

Previously, only tasks terminated by signals (negative exit codes) would be retried. Normal failures with positive exit codes would immediately fail without checking if retries were available.

Testing

DAG used:

from airflow.sdk import DAG
from datetime import datetime
from airflow.providers.standard.operators.bash import BashOperator

dag = DAG(
    "test_retry_exit_code",
    start_date=datetime(2024, 1, 1),
    schedule=None,
    catchup=False,
)

task = BashOperator(
    task_id="git_fetch_fail",
    bash_command="git fetch invalid-remote || exit 1",
    retries=3,
    dag=dag,
)

Works as expected
image


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@potiuk potiuk merged commit 93c7348 into apache:main Nov 17, 2025
84 checks passed
@amoghrajesh
Copy link
Copy Markdown
Contributor Author

Forgot to cherry pick

amoghrajesh added a commit that referenced this pull request Nov 19, 2025
#58384)

(cherry picked from commit 93c7348)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@amoghrajesh amoghrajesh self-assigned this Nov 19, 2025
@amoghrajesh amoghrajesh added this to the Airflow 3.1.4 milestone Nov 19, 2025
@amoghrajesh
Copy link
Copy Markdown
Contributor Author

Manual cherry pick: #58478

amoghrajesh added a commit that referenced this pull request Nov 19, 2025
#58384)

(cherry picked from commit 93c7348)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
amoghrajesh added a commit that referenced this pull request Nov 19, 2025
ephraimbuddy pushed a commit that referenced this pull request Nov 20, 2025
aaron-wolmutt pushed a commit to aaron-wolmutt/airflow that referenced this pull request Nov 20, 2025
ephraimbuddy pushed a commit that referenced this pull request Dec 3, 2025
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
itayweb pushed a commit to itayweb/airflow that referenced this pull request Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants