Skip to content

DAG Processor: bundle.refresh() network and auth failures are swallowed with 0 import errors recorded #76552

Open

Description

@meel-ayush

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.4.0rc3 (and main)

What happened and how to reproduce it?

Description

When bundle.refresh() fails in the DAG processor manager (such as a git network timeout, GitHub/GitLab rate limiting, or expired credentials), DagFileProcessorManager._refresh_dag_bundles() catches the exception and logs it to daemon logs, but records nothing in the metadata database.

Exactly 0 rows are written to ParseImportError or DagWarning. Because of this, the Airflow Web UI displays 0 import errors and reports the environment as healthy, even though DAG synchronization from the bundle has stopped completely.

Affected Code

In airflow-core/src/airflow/dag_processing/manager.py:934-939:

try:
    bundle.refresh()
    any_refreshed = True
except Exception:
    self.log.exception("Error refreshing bundle %s", bundle.name)
    continue

Steps to Reproduce

  1. Configure a GitDagBundle pointing to a remote repository.
  2. Trigger a connection failure (ConnectionResetError) or authentication failure during bundle.refresh().
  3. The DAG processor logs Error refreshing bundle ... to stderr and continues its loop.
  4. Check the metadata DB: 0 rows in import_error (ParseImportError) and 0 rows in dag_warning (DagWarning).
  5. The UI shows 0 import errors while the bundle remains out of date.

Expected Behavior

When a bundle fails to refresh, a ParseImportError or DagWarning (e.g. DagWarningType.BUNDLE_REFRESH_FAILED) should be recorded in the database for that bundle so operators can see the failure directly on the UI dashboard.

Proposed Fix

In airflow-core/src/airflow/dag_processing/manager.py, catch the exception in _refresh_dag_bundles and persist a warning or import error row to the database.

I have verified this behavior with a local test and am working on a PR with regression tests to address this.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. added
    kind:bugThis is a clearly a bug
    needs-triagelabel for new issues that we didn't triage yet
    on Nov 5, 2026
  2. amoghrajesh commented on Nov 6, 2026

    @amoghrajesh
    Contributor

    Can this happen in practice? It's possible if there is rate limiting from GitHub or like, an expired credential, and if thats real, this is a good improvement to have.

  3. meel-ayush commented on Nov 6, 2026

    @meel-ayush
    Author

    Yeah, in environments using Git bundles with deploy tokens or GitHub sync, rate limits or expired tokens cause bundle.refresh() to fail. Right now it logs to daemon logs but leaves the UI showing 0 import errors, so users are left wondering why new DAGs aren't syncing.

    Opened #76559 to record an import error on the bundle when refresh or discovery fails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions