Skip to content

Narrow exception handling in Glue job run fallback scan - #74082

Open
AshleyImmanuel wants to merge 2 commits into
apache:mainfrom
AshleyImmanuel:fix-glue-fallback-71489
Open

Narrow exception handling in Glue job run fallback scan#74082
AshleyImmanuel wants to merge 2 commits into
apache:mainfrom
AshleyImmanuel:fix-glue-fallback-71489

Conversation

@AshleyImmanuel

@AshleyImmanuel AshleyImmanuel commented Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Fix infinite loop in GlueJobOperator on job run failure

This caps the _find_job_run_id_by_task_uuid retry loop at 100 iterations to prevent infinite hanging when Glue fails. It also narrows exception handling to ClientError to prevent suppressing non-AWS exceptions.

closes: #71489

Was generative AI tooling used to co-author this PR?
  • Yes — Antigravity

Generated-by: Antigravity following the guidelines

@AshleyImmanuel
Fix AWS Glue job retry fallback logic and exceptions (apache#71489)
de839fe 
- Cap pagination loop in _find_job_run_id_by_task_uuid at 100 pages to prevent infinite loops.
- Narrow exception catching from broad Exception to botocore.exceptions.ClientError in _find_previous_job_run.
- Change log levels from warning to exception.
- Add IAM permission documentation to glue.rst.
@boring-cyborg

boring-cyborg Bot commented Oct 2, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

SameerMesiah97
SameerMesiah97 reviewed Oct 2, 2026
View reviewed changes

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have left some comments.

Comment thread providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py Resolved
@AshleyImmanuel
AshleyImmanuel force-pushed the fix-glue-fallback-71489 branch from 9252fa7 to 5f71b8c Compare October 4, 2026 08:33
@AshleyImmanuel
Narrow exception handling in Glue job run fallback scan
6c0652f 
Keep while True for the task-UUID pagination scan since the natural NextToken-based termination already handles exit. Narrowing except Exception to except ClientError prevents non-AWS errors from being silently swallowed.

closes: apache#71489
@AshleyImmanuel
AshleyImmanuel force-pushed the fix-glue-fallback-71489 branch from 5f71b8c to 6c0652f Compare October 4, 2026 09:00
@AshleyImmanuel

AshleyImmanuel commented Oct 4, 2026

Copy link
Copy Markdown
Author

Thanks for the review! I've updated the PR based on your feedback:

  1. Reverted to while True -- the natural NextToken-based pagination termination already handles exit, so the arbitrary 100-page cap was unnecessary and could have prevented finding a legitimate match.
  2. Kept the except Exception -> except ClientError narrowing, so non-AWS errors (e.g., TypeError, KeyError) won't be silently swallowed.
  3. Updated the test to verify that pagination exhausts all pages and returns None when no match is found.

Drafted-by: Antigravity; reviewed by @AshleyImmanuel before posting

@AshleyImmanuel AshleyImmanuel changed the title Fix infinite loop in GlueJobOperator on job run failure Narrow exception handling in Glue job run fallback scan Oct 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@o-nikolas o-nikolas Awaiting requested review from o-nikolas o-nikolas is a code owner
1 more reviewer
@SameerMesiah97 SameerMesiah97 SameerMesiah97 left review comments
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

area:providers kind:documentation provider:amazon AWS/Amazon - related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bound and harden the task-UUID scan fallback

2 participants

@AshleyImmanuel @SameerMesiah97