Repository navigation
Propagate OpenLineage parent and root job types to triggered Dags - #75303
Merged
mobuchowski merged 6 commits intoOct 26, 2026
Merged
mobuchowski merged 6 commits into
mobuchowski merged 6 commits into
Conversation
kacpermuda
marked this pull request as ready for review
October 21, 2026 11:34
kacpermuda
force-pushed
the
fix-trigger-dagrun-openlineage-root-job-type
branch
from
October 21, 2026 11:42
6ed9e41 to
7684b09
Compare
ieCecchetti
reviewed
Oct 21, 2026
kacpermuda
force-pushed
the
fix-trigger-dagrun-openlineage-root-job-type
branch
from
October 21, 2026 15:40
f362269 to
0ab02e1
Compare
mobuchowski
reviewed
Oct 23, 2026
When a Dag is triggered by TriggerDagRunOperator, the OpenLineage parent info injected into the triggered Dag run conf dropped rootParentJobType. The root job type set by an upstream caller was lost, so downstream OpenLineage events could not report the correct root job type. Co-Authored-By: Claude <noreply@anthropic.com>
…Dags Reading rootParentJobType straight from the Dag run conf ignored the validation the OpenLineage macros apply to root identifiers, so a job type belonging to a rejected root could be paired with a different job. Resolving the types in the OpenLineage provider keeps them consistent with the root identifiers they describe. Co-Authored-By: Claude <noreply@anthropic.com>
Job types forwarded through the Dag run conf were only exposed by the macros, so events of a triggered Dag did not describe the type of their parent and root jobs. Only complete job type information is accepted from the conf, as a partial JobTypeJobFacet would not be a reliable description of the job. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Task event parent facets and SQL child events did not describe the type of their parent and root jobs even when it was known, and the review of the job type handling found a few places where a type could be paired with a different job or dropped. Co-Authored-By: Claude <noreply@anthropic.com>
kacpermuda
force-pushed
the
fix-trigger-dagrun-openlineage-root-job-type
branch
from
October 26, 2026 13:23
32787d1 to
d258e85
Compare
mobuchowski
approved these changes
Oct 26, 2026
7 of 36 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
lineage_job_typeandlineage_root_job_typemacros to the OpenLineage provider, and haveTriggerDagRunOperatorinjectparentJobTypeandrootParentJobTypeinto the triggered Dag run conf. When present and complete in the conf, the job types are added as thejobTypefacet of the parent and root jobs in theParentRunFacet. Root job type is only inherited when the conf root is the one actually used, so a type is never paired with a different job.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Sonnet 5.5) following the guidelines