UI - Improve refresh rate strategy of the Grid view

[pierrejeambrun]

The grid view is sending a lot of requests to the API. (1 for each dag runs displayed).

With the current refresh rate, it's kind of spamming the API server, for instance if all 10 runs displayed are 'running' 10 requests are send every 3 seconds.

We should probably adjust the refresh rate of the grid view. The grid really is an overview and we probably don't need it's entire information refreshed at the same refresh rate as the rest of the App. This could remove some stress from the API.

For instance we could have the 'selected' dag run refreshing at the default refresh interval (if 'pending' status of course) and have the other dag runs refreshed at a much lower frequency.

Note: Also with all these request happening it actually delays other requests. (http1 -> 6 connections limit per origin)

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[vishakha1411]

Hi, may I work on this? please assign this to me.

[pierrejeambrun]

@vishakha1411 feel free to work on this, you don't need to be assigned to start working on an issue or open a PR.

[antonio-mello-ai]

Hi @pierrejeambrun, I'd like to work on this.

After reading the code, the main bottleneck is useGridTISummaries being instantiated per-run in TaskInstancesColumn, generating N requests per refresh cycle. The 5x slowRefreshMultiplier for non-selected runs helps but doesn't address the core issue.

Proposed approach (incremental, frontend-only):

1. Skip refetch for runs in final states (success, failed, upstream_failed) — their TI summaries won't change, so refetchInterval: false. This alone should eliminate most requests when the majority of displayed runs are completed.

2. Reduce grid/structure refresh frequency — DAG structure rarely changes mid-session. Could increase its interval or make it conditional on DAG version changes.

3. Simple backoff for long-running runs — if a TI summary response hasn't changed across consecutive fetches, progressively slow down the polling for that run.

I intentionally left out batching (new backend endpoint) and WebSocket/SSE as they'd be larger changes. Happy to tackle those in a follow-up if you think the scope is right.

Does this approach make sense, or would you prefer a different direction? Please let me know before I start coding.

[pierrejeambrun]

This PR #62369 will convert this endpoint (summaries) to a streaming request. Removing the N+1 query problem