Repository navigation
Look up task logs by task try UUID - #74204
Merged
Merged
Conversation
These handlers previously identified a task's logs by coordinates (dag, task, run, map index, try number). Once a task can run in several loop passes that share those coordinates, the identity has to come from something that is unique per attempt, and with the ownership change every try already has its own UUID. Elasticsearch and OpenSearch entries now carry the task instance's id as ``ti_id`` and Cloud Logging entries carry it as a label of the same name, next to the existing log_id or labels, so the human-readable identity and any Kibana or Logs Explorer, etc. queries built on them continue to work. Readers match on that id alone, or fall back to the log_id or labels for entries that have no ``ti_id``, so logs written before this change stay readable without reindexing. ``ti_id`` is where the task process has bound the UUID in to its own log lines since 3.3, so entries from 3.3 onward already match. Nothing earlier wrote it, and most lines the supervisor logs itself do not carry it either, so the Elasticsearch and OpenSearch uploaders add it to every line they ship. Readers only match on the id from Airflow 3.4. Before then clearing a task instance gives the live row a new id, so it may no longer be the id its logs were written under, and earlier versions keep matching on log_id or labels alone. Downloading every try on Google Cloud Logging drops the id condition just as it already drops the try number. Co-Authored-By: Claude <noreply@anthropic.com>
boring-cyborg
Bot
added
area:logging
area:providers
provider:elasticsearch
provider:google
eladkal
approved these changes
Oct 4, 2026
olegkachur-e
approved these changes
Oct 5, 2026
kaxil
approved these changes
Oct 5, 2026
kaxil
left a comment
kaxil
left a comment
Member
There was a problem hiding this comment.
LGTM, a few non-blocking comments inline.
ashb
requested review from
hussein-awala,
jedcunningham and
jscheffl
as code owners
October 5, 2026 13:56
potiuk
approved these changes
Oct 5, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Oct 8, 2026
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These handlers previously identified a task's logs by coordinates (dag,
task, run, map index, try number). Once a task can run in several loop
passes that share those coordinates, the identity has to come from
something that is unique per attempt, and with the ownership change
every try already has its own UUID.
Elasticsearch and OpenSearch entries now carry the task instance's id as
ti_idand Cloud Logging entries carry it as a label of the samename, next to the existing log_id or labels, so the human-readable
identity and any Kibana or Logs Explorer, etc. queries built on them
continue to work. Readers match on that id alone, or fall back to the
log_id or labels for entries that have no
ti_id, so logs writtenbefore this change stay readable without reindexing.
ti_idis where the task process has bound the UUID in to its own loglines since 3.3, so entries from 3.3 onward already match. Nothing
earlier wrote it, and most lines the supervisor logs itself do not
carry it either, so the Elasticsearch and OpenSearch uploaders add it to
every line they ship.
Readers only match on the id from Airflow 3.4+. Prior to that, in 99.99% of cases the UUID was unique, (possibly 100% of cases that could write task logs) but it felt safer to make this conditional based on the release containing #73554 to be safe.
I used Claude Sonnet 5.5 on this one for help with writing the query syntax and tests of the ES/OS branches.