Skip to content

Add support for filtering Dags by any DagRun state - #68657

Merged
guan404ming merged 3 commits into
apache:mainfrom
guan404ming:fix-64827-dag-stats-running-count
Jul 9, 2026
Merged

Add support for filtering Dags by any DagRun state#68657
guan404ming merged 3 commits into
apache:mainfrom
guan404ming:fix-64827-dag-stats-running-count

Conversation

@guan404ming

@guan404ming guan404ming commented Jun 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Related Issue

Why

During a backwards backfill the home dashboard reported no running Dags because the running/queued counts only looked at each Dag's latest run, missing an older run still active.

How

  • Add support for filtering Dags by any DagRun state
  • Make the filter to be dropdown select to prevent the ui being too compact
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Claude Code with Opus4.8
  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Jun 17, 2026
@guan404ming
guan404ming marked this pull request as ready for review June 17, 2026 09:31
@eladkal eladkal added this to the Airflow 3.3.0 milestone Jun 17, 2026
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Jun 17, 2026
pierrejeambrun
pierrejeambrun reviewed Jun 17, 2026
View reviewed changes

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just one question

Comment thread airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dashboard.py Resolved
@Vamsi-klu

This comment was marked as spam.

Sign in to view
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch 2 times, most recently from fd62e72 to 3622d5e Compare June 22, 2026 10:02
@guan404ming

guan404ming commented Jun 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Makes sense. thanks for catching this, you're right that the count and the card link diverged.

I went with the first option: I added a dedicated dag_run_state filter on the /ui/dags endpoint with "any active DagRun" semantics, separate from last_dag_run_state (which still matches the latest run only). The Running/Queued dashboard cards now link to dags?dag_run_state=running / queued, so the count and the click-through stay aligned in the backfill case. The Dags-list state toggle also routes Running/Queued through this any-run filter (Failed/Success keep latest-run semantics) and reflects whichever param is active, so landing from a card highlights the right button.

Just pushed the change and feel free to provide any suggestion about the changes.

@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch from 413d307 to 65fabdf Compare June 23, 2026 08:18
pierrejeambrun
pierrejeambrun reviewed Jun 24, 2026
View reviewed changes

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks a few suggestions

Comment thread airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx Resolved
Comment thread airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx Resolved
@guan404ming

guan404ming commented Jun 27, 2026

Copy link
Copy Markdown
Member Author

Thanks for the review. Here are some update:

  • Split the single run-state control into two distinct dropdowns: Last run (latest run, all states) and
    Any run (any run). They're independent now, so combinations like latest = success + has a running run
    work, and you can still filter latest = running/queued.
  • Limited the Any run filter to running/queued on both ends — the backend _AnyDagRunStateFilter now rejects other states with a 400 (only these have the partial index on dag_run; others would full-scan), and the UI only offers those two.

Screenshot:

Screen.Recording.2026-06-27.at.2.37.02.PM.mov

@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch 5 times, most recently from adc4548 to 11d57f1 Compare June 29, 2026 08:23
@guan404ming
guan404ming requested a review from potiuk as a code owner June 29, 2026 08:23
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch from 11d57f1 to 935bb34 Compare June 29, 2026 08:25
pierrejeambrun
pierrejeambrun approved these changes Jun 30, 2026
View reviewed changes

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, a couple of small suggestion. Then we can merge.

Comment thread airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx Resolved
Comment thread airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/RunStateSelect.tsx Resolved
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch from 67ecba1 to aede937 Compare June 30, 2026 16:07
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch from aede937 to 20f3f91 Compare July 2, 2026 23:55
@guan404ming
guan404ming requested a review from henry3260 as a code owner July 2, 2026 23:55
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch 3 times, most recently from 4178a6b to 9e2a3c8 Compare July 8, 2026 05:57
@guan404ming
guan404ming force-pushed the fix-64827-dag-stats-running-count branch from 9e2a3c8 to 9feb968 Compare July 9, 2026 04:39
@guan404ming

guan404ming commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

I think the comments have been resolved and it looks good to merge. Please feel free to let me know or open pr/issue to refine this one, thanks!

@guan404ming guan404ming changed the title Fix dashboard running/queued counts ignoring non-latest runs Add support for filtering Dags by any DagRun state Jul 9, 2026
@guan404ming
guan404ming merged commit e05d6cd into apache:main Jul 9, 2026
167 of 168 checks passed
@guan404ming
guan404ming deleted the fix-64827-dag-stats-running-count branch July 9, 2026 08:01
This was referenced Jul 23, 2026
Merged
Merged
Merged
pierrejeambrun added a commit that referenced this pull request Jul 23, 2026
@pierrejeambrun @guan404ming
Add support for filtering Dags by any DagRun state (#68657) (#70292)
239a639 
* Add support for filtering Dags by any DagRun state

* Update airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx



* Switched the fixed width to fit-content

---------


(cherry picked from commit e05d6cd)

Co-authored-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
vatsrahul1001 pushed a commit that referenced this pull request Aug 5, 2026
@pierrejeambrun @guan404ming @vatsrahul1001
Add support for filtering Dags by any DagRun state (#68657) (#70292)
8e6e666 
* Add support for filtering Dags by any DagRun state

* Update airflow-core/src/airflow/ui/src/pages/DagsList/DagsFilters/DagsFilters.tsx



* Switched the fixed width to fit-content

---------


(cherry picked from commit e05d6cd)

Co-authored-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
@vatsrahul1001 vatsrahul1001 mentioned this pull request Aug 7, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner
@rawwar rawwar Awaiting requested review from rawwar rawwar is a code owner
@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner
@bugraoz93 bugraoz93 Awaiting requested review from bugraoz93 bugraoz93 is a code owner
@shubhamraj-git shubhamraj-git Awaiting requested review from shubhamraj-git shubhamraj-git is a code owner
@choo121600 choo121600 Awaiting requested review from choo121600 choo121600 is a code owner
@bbovenzi bbovenzi Awaiting requested review from bbovenzi bbovenzi is a code owner
@ryanahamilton ryanahamilton Awaiting requested review from ryanahamilton ryanahamilton is a code owner
@vatsrahul1001 vatsrahul1001 Awaiting requested review from vatsrahul1001 vatsrahul1001 is a code owner
@pierrejeambrun pierrejeambrun Awaiting requested review from pierrejeambrun pierrejeambrun is a code owner
@henry3260 henry3260 Awaiting requested review from henry3260 henry3260 is a code owner

Assignees

No one assigned

Labels

area:API Airflow's REST/HTTP API type:bug-fix Changelog: Bug Fixes

Projects

None yet

Milestone

Airflow 3.3.1

Development

Successfully merging this pull request may close these issues.

Backwards backfill UI incorrectly shows “no running tasks” while older backfill task instances are still running

5 participants

@guan404ming @Vamsi-klu @pierrejeambrun @vatsrahul1001 @eladkal