Skip to content

Add GET /triggers to list every trigger of every kind and its status - #76260

Merged
bbovenzi merged 6 commits into
apache:mainfrom
aws-mwaa:ghaeli/to7-triggers-api
Nov 4, 2026
Merged

bbovenzi merged 6 commits into
apache:mainfrom
aws-mwaa:ghaeli/to7-triggers-api

Conversation

@seanghaeli

@seanghaeli seanghaeli commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

There's no way to see what runs on the triggerers: GET /assets shows only a watcher's name and trigger id, so a failing watcher's status has no visibility. #76101 records each trigger's failures and owner on its row; this reads them.

  • GET /api/v2/triggers and GET /api/v2/triggers/{trigger_id} list every trigger the caller can see, with its kind, status, triggerer, failures, last error, last event and owner. Kwargs, which can hold credentials, are never loaded or returned.
  • status is worked out when read: pending if no triggerer has it, orphaned if its triggerer stopped heartbeating, failing while it has failures since it last recovered, otherwise running.
  • Filters: kind, status, triggerer_hostname, dag_id, asset_id and teams.
  • No new permission: each kind is gated like the routes that show its owner. A task's trigger needs Task Instance read on its Dag, a callback's trigger Dag Run read on its Dag, and a watcher's trigger read on one of its assets. A shared trigger only lists the watchers on assets you can read. If you can read none of those you get a 403.
  • Watchers in GET /assets gain status.

How to review: get_triggers and _build_response in routes/public/triggers.py, then the access rule PermittedTriggerFilter in core_api/security.py and the status in Trigger.status / status_expression in models/trigger.py.

There is ~600 lines of source, the rest is generated code, docs and tests.

Breeze on Postgres, every kind of trigger running with one watcher crash-looping, then FAB users with one role each:

--- GET /api/v2/triggers -> HTTP 200
  total_entries=5
  id=1   kind=asset_watcher status=running  host=trig-a dag_id=None fails=0 last_event=None | watchers healthy_asset/healthy_watcher
  id=2   kind=asset_watcher status=running  host=trig-a dag_id=None fails=0 last_event=None | watchers shared_trigger_a_asset/shared_trigger_b_watcher,shared_trigger_b_asset/shared_trigger_b_watcher
  id=3   kind=asset_watcher status=failing  host=trig-a dag_id=None fails=38 last_event=None | watchers expired_credentials_asset/expired_credentials_watcher
         last_error='ConnectionError: the security token included in the request is expired'
  id=5   kind=callback      status=running  host=trig-a dag_id=example_deadline_callback fails=0 last_event=2026-10-29T12:11:17.236983Z | callback 01a18792 dag=example_deadline_callback run=manual__2026-10-29T12:11:03.871280+00:00 cb_state=running
  id=4   kind=task          status=running  host=trig-a dag_id=example_deferred_task fails=0 last_event=None | task example_deferred_task.wait_for_trigger try=1 ti_state=deferred
  ?kind=asset_watcher -> HTTP 200 ids=[1 2 3]
  ?status=failing -> HTTP 200 ids=[3]
  ?dag_id=example_deadline_callback -> HTTP 200 ids=[5]
  ?asset_id=3 -> HTTP 200 ids=[2]
  ?kind=sensor -> HTTP 422 ids=[]

# FAB auth manager, users named for what they can read
--- reads_deferred_task_dag: list HTTP 200
    total_entries=1 visible=4:task(wait_for_trigger)
--- reads_deadline_dag: list HTTP 200
    total_entries=1 visible=5:callback
--- reads_assets_only: list HTTP 200
    total_entries=3 visible=1:asset_watcher(kwargs_with_secret_watcher), 2:asset_watcher(healthy_watcher), 3:asset_watcher(expired_credentials_watcher)
--- no_access: list HTTP 403
    detail='Forbidden'

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Opus 5.5) following the guidelines

@seanghaeli

Copy link
Copy Markdown
Contributor Author

Tagging @fat-catTW and @potiuk since this follows #76101 which you reviewed

There is no way to see what runs on the triggerers. Airflow 3 dropped the
Browse > Triggers page, GET /assets shows a watcher's name and trigger id and
nothing else, so a watcher failing for days looks healthy, and a failing
deferred task's or callback's trigger only shows in that one owner's view.

One collection, filtered by kind, status, triggerer, Dag, asset or team,
answers what is failing, since when and why, the same way for every kind. Kind
and status are worked out from the trigger's row, its owner and its triggerer's
heartbeat, not stored: a trigger is failing from its first failure until it
recovers. Sorting by failures or times puts a trigger that never failed or
fired in the same place on every database. A trigger is shown to whoever can
see its owner, with no new permission; team access comes with that check, so
the team filter only narrows the list. Kwargs are never returned, and a
trigger shared by several assets lists only the watchers on assets the caller
can read. An asset's watchers carry their status in GET /assets.
…ge was read

GET /triggers read a trigger's task instance and its public map index in two
statements; a Dag or run deleted between them made the map index lookup raise
KeyError, a 500. Such a trigger is now left out, like one whose owner let go.
… run read

A trigger was shown to whoever could read its Dag, but a task trigger shows its
task instance's state and try and the trigger's last error, which the task
instance routes need Task Instance read for, and a callback trigger shows what
the deadlines of a Dag run need Dag Run read for. Each kind is now gated like
the routes that show its owner; an asset watcher's trigger still needs asset
read. A caller who can read none of them gets a 403, and the permission tables
list that instead of no permission.
Comparing Trigger.kind_expression() in each branch of the permission filter
made Postgres evaluate its CASE and watcher subquery once per branch for every
row. Testing the owner keys directly applies the same rule without it, so
unfiltered pages are back to their previous speed with many triggers.
@seanghaeli
seanghaeli force-pushed the ghaeli/to7-triggers-api branch from 17cf2a1 to 38bfc3c Compare November 3, 2026 21:20
@bbovenzi
bbovenzi merged commit 7b00cf0 into apache:main Nov 4, 2026
152 checks passed
@o-nikolas o-nikolas mentioned this pull request Nov 5, 2026
1 task done
hussein-awala added a commit to hussein-awala/airflow that referenced this pull request Nov 7, 2026
apache#76260 turned the sqlalchemy import in trigger.py into a list that already
has and_, so take its version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants