Skip to content

Warn in UI when the same dag_id is registered from multiple files - #68791

Merged
bbovenzi merged 11 commits into
apache:mainfrom
takayoshi-makabe:warn-on-duplicate-dag-id
Jul 9, 2026
Merged

bbovenzi merged 11 commits into
apache:mainfrom
takayoshi-makabe:warn-on-duplicate-dag-id

Conversation

@takayoshi-makabe

@takayoshi-makabe takayoshi-makabe commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

This is a follow-up to #68678, which proposed raising an ImportError when the same dag_id is found in two different files.

Currently, when two files define the same dag_id, Airflow silently overwrites the existing Dag entry with whichever file is parsed last — no error, no warning, no trace in the UI. This means a mis-deployed file can invisibly destroy a production Dag, leaving operators with no indication of what happened or where the conflict originated.

@uranusjr pointed out that an ImportError would fire transiently on every legitimate file rename, since the old DB entry persists until stale_dag_threshold passes. This PR avoids that entirely by recording a DagWarning instead of blocking the parse.

The warning shows both file names and the dag_id so users can self-diagnose:

dag_id 'hoge' is now served from 'file_b.py' (bundle: 'default'), previously registered from 'file_a.py'
(bundle: 'default'). If 'file_a.py' was renamed or moved, this notice will clear on the next parse
cycle once the old file is no longer observed. If both files coexist with the same dag_id, rename one of
them to avoid non-deterministic behavior.

For file moves the warning is transient and self-clearing; for real duplicates it persists until resolved.


Was generative AI tooling used to co-author this PR?

  • Yes — Claude Code (Sonnet 4.6)

@takayoshi-makabe

takayoshi-makabe commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor Author

The CI failure was in CI image checks / Static checks — the check-metrics-synced-with-registry hook requires every metric used in code to be registered in the metrics registry YAML, but dag_processing.duplicate_dag_id was missing.

Fixed by adding the metric to
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml.

fed0d5e

@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

Hi @potiuk,
I have fixed the issue that caused the previous CI failure. Could you please approve and trigger the tests again when you have a moment? Thank you!

@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@uranusjr
Thank you for the review. I've made some revisions based on your comments.

@takayoshi-makabe
takayoshi-makabe requested a review from uranusjr July 2, 2026 03:23
@takayoshi-makabe

Copy link
Copy Markdown
Contributor Author

@uranusjr
Thank you for the approval. Could you please trigger the CI and merge the changes?

@potiuk potiuk added the ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. label Jul 8, 2026
@bbovenzi
bbovenzi merged commit 48c6bb4 into apache:main Jul 9, 2026
146 checks passed
@bbovenzi bbovenzi added this to the Airflow 3.4.0 milestone Jul 9, 2026
@takayoshi-makabe
takayoshi-makabe deleted the warn-on-duplicate-dag-id branch July 9, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dag-processor ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants