Repository navigation
Add a TypeScript SDK ADR for TaskFlow and native Dag authoring - #72047
Merged
jason810496 merged 6 commits intoSep 10, 2026
Merged
Conversation
…ge and native Dags The cross-SDK decisions on argument binding and the TaskFlow Dag DSL were settled against Go and Java, both of which can recover a task function's parameter list, Go with reflect at run time, Java with an annotation processor at compile time. TypeScript can do neither: its idiomatic handler takes a single destructured object whose parameter positions do not survive to run time. That constraint changes the answer to several questions the earlier records treat as settled, so the reasoning needs to live somewhere reviewers and future SDK authors can find it. The mixed-language argument-binding surface and native-Dag authoring are different enough problems to warrant separate ADRs. ADR-0001 covers wiring a TypeScript task body under a Python-declared Dag (MixedLangDag); ADR-0002 covers a Dag whose graph and task bodies are both TypeScript (Dag). Each leads with the call-site code a user actually writes, and each records the ctx/client-as-getters question as open rather than settled, since resolving it would also reshape the binding mechanism and open the door to positional wiring.
pierrejeambrun
left a comment
pierrejeambrun
left a comment
Member
There was a problem hiding this comment.
Thanks for the well though documents.
Just a few questions / suggestions.
jason810496
marked this pull request as draft
September 7, 2026 02:14
jason810496
added a commit
to jason810496/airflow
that referenced
this pull request
Sep 8, 2026
The review on apache#72047 found three things these ADRs described but should not have. The SDK's context and client shared a handler's single argument object, so every typed handler had to intersect its own data type with the SDK's. A mixed-language Dag was authored through a second Dag class, when TypeScript declares no Dag at all in that mode. And a graph whose every edge carried data had no way to express an edge that carries none, which left a fan-in of order-only dependencies unanswerable and pushed the examples into nesting that stops reading well before a Dag grows large. Recording what the review arrived at keeps these documents a reference for the interface the SDK will ship, before either interface reaches users.
The review on apache#72047 found three things these ADRs described but should not have. The SDK's context and client shared a handler's single argument object, so every typed handler had to intersect its own data type with the SDK's. A mixed-language Dag was authored through a second Dag class, when TypeScript declares no Dag at all in that mode. And a graph whose every edge carried data had no way to express an edge that carries none, which left a fan-in of order-only dependencies unanswerable and pushed the examples into nesting that stops reading well before a Dag grows large. Recording what the review arrived at keeps these documents a reference for the interface the SDK will ship, before either interface reaches users.
jason810496
force-pushed
the
docs/ts-sdk/adr-taskflow-native-dag
branch
from
September 8, 2026 07:02
ffb564e to
de939e2
Compare
jason810496
commented
Sep 8, 2026
jason810496
left a comment
jason810496
left a comment
Member
Author
There was a problem hiding this comment.
Thanks @pierrejeambrun for the review.
jason810496
marked this pull request as ready for review
September 8, 2026 07:25
1 task done
jason810496
marked this pull request as draft
September 9, 2026 05:27
A bundle usually provides both native Dags and bodies for tasks Python declares, and the earlier draft made it describe itself twice: one verb per kind, with a collector named after only one of them. Naming that collector Bundle, letting a single register call take either kind, and hanging serve off the same object leaves an author with one thing to hold and one place to look. StubHandler says what a mixed-language registration is - a body for a Python @task.stub task - instead of borrowing the word Dag for something TypeScript does not own. Both records also open with their decision now, with the AsyncLocalStorage dispatch scope, the name-folding proxy, and the rest of the mechanics moved to an appendix, so a reviewer can see what was decided without reading how it works first.
The example declared a bare Dag and bare tasks, which reads as a more verbose way to write the mixed-language case. Carrying a schedule on the Dag and retries on a task shows the reason the native mode exists: the Dag file owns the configuration a Python stub Dag would otherwise own, and a handler for a Python-declared task can carry neither. The fields are recorded as forward-looking. DagSpec and TaskSpec accept only the empty object today, so the record says where an author writes this configuration without claiming to choose the fields themselves.
jason810496
marked this pull request as ready for review
September 9, 2026 08:24
The name should say what the SDK registers, and what it registers is a task: Python declares it with @task.stub, gives it a task_id, and the SDK supplies only its body. "Handler" described the callable rather than the thing it stands for, leaving these docs using a word no other part of Airflow uses for a task.
Python declares the task; an SDK supplies only its body. Naming the SDK object after the task overloads a word the native surface already uses for something the user creates with dag.Task, so it names the role the SDK actually plays instead.
imrichardwu
pushed a commit
to imrichardwu/airflow
that referenced
this pull request
Sep 11, 2026
…e#72047) * Add TypeScript SDK ADRs for TaskFlow argument binding on mixed-language and native Dags The cross-SDK decisions on argument binding and the TaskFlow Dag DSL were settled against Go and Java, both of which can recover a task function's parameter list, Go with reflect at run time, Java with an annotation processor at compile time. TypeScript can do neither: its idiomatic handler takes a single destructured object whose parameter positions do not survive to run time. That constraint changes the answer to several questions the earlier records treat as settled, so the reasoning needs to live somewhere reviewers and future SDK authors can find it. The mixed-language argument-binding surface and native-Dag authoring are different enough problems to warrant separate ADRs. ADR-0001 covers wiring a TypeScript task body under a Python-declared Dag (MixedLangDag); ADR-0002 covers a Dag whose graph and task bodies are both TypeScript (Dag). Each leads with the call-site code a user actually writes, and each records the ctx/client-as-getters question as open rather than settled, since resolving it would also reshape the binding mechanism and open the door to positional wiring. * Record the reviewed TypeScript SDK task and Dag authoring interfaces The review on apache#72047 found three things these ADRs described but should not have. The SDK's context and client shared a handler's single argument object, so every typed handler had to intersect its own data type with the SDK's. A mixed-language Dag was authored through a second Dag class, when TypeScript declares no Dag at all in that mode. And a graph whose every edge carried data had no way to express an edge that carries none, which left a fan-in of order-only dependencies unanswerable and pushed the examples into nesting that stops reading well before a Dag grows large. Recording what the review arrived at keeps these documents a reference for the interface the SDK will ship, before either interface reaches users. * Collapse the TypeScript SDK's registration surface onto the bundle A bundle usually provides both native Dags and bodies for tasks Python declares, and the earlier draft made it describe itself twice: one verb per kind, with a collector named after only one of them. Naming that collector Bundle, letting a single register call take either kind, and hanging serve off the same object leaves an author with one thing to hold and one place to look. StubHandler says what a mixed-language registration is - a body for a Python @task.stub task - instead of borrowing the word Dag for something TypeScript does not own. Both records also open with their decision now, with the AsyncLocalStorage dispatch scope, the name-folding proxy, and the rest of the mechanics moved to an appendix, so a reviewer can see what was decided without reading how it works first. * Show what a native TypeScript Dag owns that a stub handler cannot The example declared a bare Dag and bare tasks, which reads as a more verbose way to write the mixed-language case. Carrying a schedule on the Dag and retries on a task shows the reason the native mode exists: the Dag file owns the configuration a Python stub Dag would otherwise own, and a handler for a Python-declared task can carry neither. The fields are recorded as forward-looking. DagSpec and TaskSpec accept only the empty object today, so the record says where an author writes this configuration without claiming to choose the fields themselves. * Rename StubHandler to StubTask in the TypeScript SDK ADRs The name should say what the SDK registers, and what it registers is a task: Python declares it with @task.stub, gives it a task_id, and the SDK supplies only its body. "Handler" described the callable rather than the thing it stands for, leaving these docs using a word no other part of Airflow uses for a task. * Rename StubTask to TaskHandler in the TypeScript SDK ADRs Python declares the task; an SDK supplies only its body. Naming the SDK object after the task overloads a word the native surface already uses for something the user creates with dag.Task, so it names the role the SDK actually plays instead.
xvega
pushed a commit
to xvega/airflow
that referenced
this pull request
Sep 13, 2026
…e#72047) * Add TypeScript SDK ADRs for TaskFlow argument binding on mixed-language and native Dags The cross-SDK decisions on argument binding and the TaskFlow Dag DSL were settled against Go and Java, both of which can recover a task function's parameter list, Go with reflect at run time, Java with an annotation processor at compile time. TypeScript can do neither: its idiomatic handler takes a single destructured object whose parameter positions do not survive to run time. That constraint changes the answer to several questions the earlier records treat as settled, so the reasoning needs to live somewhere reviewers and future SDK authors can find it. The mixed-language argument-binding surface and native-Dag authoring are different enough problems to warrant separate ADRs. ADR-0001 covers wiring a TypeScript task body under a Python-declared Dag (MixedLangDag); ADR-0002 covers a Dag whose graph and task bodies are both TypeScript (Dag). Each leads with the call-site code a user actually writes, and each records the ctx/client-as-getters question as open rather than settled, since resolving it would also reshape the binding mechanism and open the door to positional wiring. * Record the reviewed TypeScript SDK task and Dag authoring interfaces The review on apache#72047 found three things these ADRs described but should not have. The SDK's context and client shared a handler's single argument object, so every typed handler had to intersect its own data type with the SDK's. A mixed-language Dag was authored through a second Dag class, when TypeScript declares no Dag at all in that mode. And a graph whose every edge carried data had no way to express an edge that carries none, which left a fan-in of order-only dependencies unanswerable and pushed the examples into nesting that stops reading well before a Dag grows large. Recording what the review arrived at keeps these documents a reference for the interface the SDK will ship, before either interface reaches users. * Collapse the TypeScript SDK's registration surface onto the bundle A bundle usually provides both native Dags and bodies for tasks Python declares, and the earlier draft made it describe itself twice: one verb per kind, with a collector named after only one of them. Naming that collector Bundle, letting a single register call take either kind, and hanging serve off the same object leaves an author with one thing to hold and one place to look. StubHandler says what a mixed-language registration is - a body for a Python @task.stub task - instead of borrowing the word Dag for something TypeScript does not own. Both records also open with their decision now, with the AsyncLocalStorage dispatch scope, the name-folding proxy, and the rest of the mechanics moved to an appendix, so a reviewer can see what was decided without reading how it works first. * Show what a native TypeScript Dag owns that a stub handler cannot The example declared a bare Dag and bare tasks, which reads as a more verbose way to write the mixed-language case. Carrying a schedule on the Dag and retries on a task shows the reason the native mode exists: the Dag file owns the configuration a Python stub Dag would otherwise own, and a handler for a Python-declared task can carry neither. The fields are recorded as forward-looking. DagSpec and TaskSpec accept only the empty object today, so the record says where an author writes this configuration without claiming to choose the fields themselves. * Rename StubHandler to StubTask in the TypeScript SDK ADRs The name should say what the SDK registers, and what it registers is a task: Python declares it with @task.stub, gives it a task_id, and the SDK supplies only its body. "Handler" described the callable rather than the thing it stands for, leaving these docs using a word no other part of Airflow uses for a task. * Rename StubTask to TaskHandler in the TypeScript SDK ADRs Python declares the task; an SDK supplies only its body. Naming the SDK object after the task overloads a word the native surface already uses for something the user creates with dag.Task, so it names the role the SDK actually plays instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Write the design for TaskFlow and native Dag for the TS-SDK up front, to settle the design early and surface as much context as possible for reviewers on the actual implementation PRs.
Was generative AI tooling used to co-author this PR?