Skip to content

Feature expand health endpoint reporting - #70416

Merged
vincbeck merged 21 commits into
apache:mainfrom
JH-A-Kim:feature-expand-health-endpoint-reporting
Aug 28, 2026
Merged

vincbeck merged 21 commits into
apache:mainfrom
JH-A-Kim:feature-expand-health-endpoint-reporting

Conversation

@JH-A-Kim

@JH-A-Kim JH-A-Kim commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Description

These changes act as an implementation for the suggested changes in the design proposal:

Design propsoal

and from the Discussion from the following thread and lazy consensus on the dev list:

DISCUSS
LAZY CONSENSUS

These changes were co authored by @JH-A-Kim and @Tegh25

The changes are the following:

  • extended monitor.py to support instances and detailed status reporting in the JSON output for the health endpoint
  • added helper functions to get information from the metadata database for running components, aggregate statuses from a list of component instances, and a legacy health reporter.
  • extended get_airlfow_health to include lists of components and report their aggregated status.
  • added aggregated status reporting in a three tier model for detailed_status, healthy > degraded > down. These are all instances are healthy, at least one instance is healthy and at least one is unhealthy, and then down is where all instances are down.

Side Note: There was a slight change from using unhealthy to down in detailed_statusfor clarity.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [Sonnet 5] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:CLI area:db-migrations PRs with DB migration area:Triggerer labels Jul 24, 2026
@JH-A-Kim
JH-A-Kim force-pushed the feature-expand-health-endpoint-reporting branch 10 times, most recently from 1399c6d to a49a683 Compare July 29, 2026 22:39
@JH-A-Kim
JH-A-Kim marked this pull request as ready for review July 30, 2026 02:46
JH-A-Kim and others added 18 commits August 27, 2026 16:32
When a triggerer or Dag processor job is created, the team/bundle name field is added into its
Job instance. These fields have been added to the Job class as strings and list of strings.
Without these assertions, regressions in how triggerer team_name and
Dag processor bundle_names flow into Job rows and the monitor health
response could slip through. including degraded multi-instance status
and get_jobs_health filtering/ordering.
Operators using HA components need the documented response shape so they
can rely on detailed_status and instances instead of assuming a single
scheduler/triggerer/Dag processor heartbeat.
Removed state=RUNNING and added end_date=None to query for jobs. This matches the
pattern in trigger.py.
Removed the logic which returns status=None if the retrieved job list is empty.
Updated tests to check for this new logic, and added DB tests that do not mock
the job retrieval.
team_name as an arg to the class is redundant since it has been added to Job.
Callers for this class will be required to set team_name on Job.
@Tegh25
Tegh25 force-pushed the feature-expand-health-endpoint-reporting branch from ee97a18 to 1bc2326 Compare August 27, 2026 21:38
@Tegh25

Tegh25 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Could you work on the conflicts please?

Resolved 🙂

@ferruzzi ferruzzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thanks to both of you for seeing this through to the end, it's been a long ride.

@JH-A-Kim

JH-A-Kim commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Thank you for being an incredible mentor! I'll see you both around the repo and slack (and eventually when I come back from my internship lol) (and also on my other pr that I haven't had time to get back to)

@vincbeck
vincbeck merged commit 9da89e7 into apache:main Aug 28, 2026
156 checks passed
@Tegh25
Tegh25 deleted the feature-expand-health-endpoint-reporting branch August 28, 2026 19:30
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Sep 11, 2026
@pierrejeambrun pierrejeambrun added this to the Airflow 3.4.0 milestone Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:CLI area:db-migrations PRs with DB migration area:Triggerer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants