Skip to content

Keep .airflowignore rules when the Dags folder symlink is switched - #74853

Merged
potiuk merged 1 commit into
apache:mainfrom
rammmaa:airflowignore-symlink-swap
Oct 18, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
rammmaa:airflowignore-symlink-swap

Conversation

@rammmaa

@rammmaa rammmaa commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

git-sync publishes each revision by repointing the Dags folder symlink to a new checkout. When that happens during file discovery, child directories resolve into the new checkout and no longer find the ignore rules inherited from their parent, so files under ignored directories are picked up for that scan.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

When git-sync switches the Dag folder symlink during file discovery, subdirectories can lose the rules inherited from .airflowignore. The rules are stored under resolved paths, so switching to a new checkout causes subsequent lookups to miss. Files that should be ignored can then be discovered and parsed.

Key inherited rules by the walked path so they remain available across a symlink switch. Keep resolved paths in a separate set to preserve the existing recursive-link detection.

Add a deterministic regression test that atomically switches the symlink after discovery has stored the parent’s rules. The test fails without the fix and passes with it. All 35 tests in shared/module_loading/tests/module_loading pass.

Closes #74851

git-sync publishes each revision by repointing the Dags folder symlink
to a new checkout. When that happens during file discovery, child
directories resolve into the new checkout and no longer find the ignore
rules inherited from their parent, so files under ignored directories
are picked up for that scan.
@potiuk
potiuk merged commit 04e68bf into apache:main Oct 18, 2026
79 checks passed
@boring-cyborg

boring-cyborg Bot commented Oct 18, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-4-test PR Link

vatsrahul1001 pushed a commit that referenced this pull request Oct 24, 2026
… switched (#74853)

git-sync publishes each revision by repointing the Dags folder symlink
to a new checkout. When that happens during file discovery, child
directories resolve into the new checkout and no longer find the ignore
rules inherited from their parent, so files under ignored directories
are picked up for that scan.
(cherry picked from commit 04e68bf)

Co-authored-by: Haram Lee <yoonsoi041909@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request Oct 24, 2026
… switched (#74853) (#75466)

git-sync publishes each revision by repointing the Dags folder symlink
to a new checkout. When that happens during file discovery, child
directories resolve into the new checkout and no longer find the ignore
rules inherited from their parent, so files under ignored directories
are picked up for that scan.
(cherry picked from commit 04e68bf)

Co-authored-by: Haram Lee <yoonsoi041909@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.airflowignore rules are lost when a Dag folder symlink switches during a scan

2 participants