Repository navigation
Avoid a full asset table scan per request in simple auth manager - #72934
Merged
Merged
Conversation
Lee-W
force-pushed
the
asset-auth-filter-perf
branch
from
October 5, 2026 12:00
0fc3dce to
4fa1cf2
Compare
Scoping asset API responses to what a user may read (apache#72682) gave BaseAuthManager a default that loads the id, name and uri of every asset and then asks is_authorized_asset about each row. Simple auth manager grants asset access by role alone and never inspects the asset it is asked about, so that loop re-derives a single constant answer once per row, along with the name and uri it only loads to build the details it then ignores. The auth managers whose is_authorized_asset makes a remote call were given batched overrides at the time; simple auth manager, which is the default, was left on the generic path.
Simple auth manager now overrides get_authorized_assets, and it is the auth manager the API tests run against. The asset route tests that scope responses to readable assets patched the method on BaseAuthManager, so the override bypassed the patch and the tests stopped exercising the filtering they assert on.
Lee-W
force-pushed
the
asset-auth-filter-perf
branch
from
October 6, 2026 16:13
4fa1cf2 to
7b1a2c2
Compare
Lee-W
marked this pull request as ready for review
October 6, 2026 20:59
vincbeck
approved these changes
Oct 7, 2026
1 task done
1 task done
Merged
1 task done
potiuk
added a commit
that referenced
this pull request
Oct 19, 2026
…e base-class patch (#75138) * Mock the simple auth manager in the dependencies graph permission tests SimpleAuthManager overrides get_authorized_assets since #72934, so patching the BaseAuthManager method left the real one in place, every asset stayed readable, and the tests added in #73008 failed on main. Generated-by: Claude Opus 5 * Reject patching BaseAuthManager methods that SimpleAuthManager overrides Twice in one day a permission test patched BaseAuthManager.get_authorized_assets, which SimpleAuthManager overrides, so the mock never applied and main broke after merge. The hook reads which methods SimpleAuthManager overrides, so it keeps up when more are added. Generated-by: Claude Opus 5
potiuk
added a commit
that referenced
this pull request
Oct 19, 2026
…gs the caller may read (#72924) (#75108) * [v3-4-test] Scope upstream asset events of a Dag run to assets and Dags the caller may read (#72924) GET /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamAssetEvents returned every asset event a run consumed as long as the caller could read the consuming Dag. Events of assets, or produced by source Dags, the caller is not allowed to read were exposed, including their name, uri, extra and source Dag/task ids, even though GET /assets/events hides the same events. Query the consumed events through the dagrun_asset_event table and apply the same per-asset and per-source-Dag readability filters as GET /assets/events, so hidden events are excluded from both the response and total_entries. (cherry picked from commit 6f408e1) Co-authored-by: Henry Chen <henrychen@apache.org> * Mock the simple auth manager in the upstream asset events permission test (#75126) SimpleAuthManager overrides get_authorized_assets since #72934, so patching the BaseAuthManager method left the real one in place, every asset stayed readable, and the test added in #72924 failed on main. Generated-by: Claude Opus 5 (cherry picked from commit 33e6b5f) --------- Co-authored-by: Henry Chen <henrychen@apache.org> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001
pushed a commit
that referenced
this pull request
Oct 20, 2026
…gs the caller may read (#72924) (#75108) * [v3-4-test] Scope upstream asset events of a Dag run to assets and Dags the caller may read (#72924) GET /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamAssetEvents returned every asset event a run consumed as long as the caller could read the consuming Dag. Events of assets, or produced by source Dags, the caller is not allowed to read were exposed, including their name, uri, extra and source Dag/task ids, even though GET /assets/events hides the same events. Query the consumed events through the dagrun_asset_event table and apply the same per-asset and per-source-Dag readability filters as GET /assets/events, so hidden events are excluded from both the response and total_entries. (cherry picked from commit 6f408e1) Co-authored-by: Henry Chen <henrychen@apache.org> * Mock the simple auth manager in the upstream asset events permission test (#75126) SimpleAuthManager overrides get_authorized_assets since #72934, so patching the BaseAuthManager method left the real one in place, every asset stayed readable, and the test added in #72924 failed on main. Generated-by: Claude Opus 5 (cherry picked from commit 33e6b5f) --------- Co-authored-by: Henry Chen <henrychen@apache.org> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scoping asset API responses to what a user may read (#72682) gave
BaseAuthManagera default that loads the id, name and uri of every asset and then asksis_authorized_assetabout each row. Simple auth manager grants asset access by role alone and never inspects the asset it is asked about, so that loop re-derives a single constant answer once per row, and listing one page of assets costs time proportional to the size of the asset table rather than to the page.The auth managers whose
is_authorized_assetmakes a remote call were given batched overrides at the time; simple auth manager, which is the default, was left on the generic path.Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.