Repository navigation
TS SDK: embed the entrypoint source in the Dag bundle - #73127
Merged
jason810496 merged 1 commit intoSep 19, 2026
Merged
Conversation
jason810496
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jscheffl,
kaxil,
potiuk and
uranusjr
as code owners
September 14, 2026 13:44
1 task done
jason810496
marked this pull request as draft
September 15, 2026 02:20
jason810496
force-pushed
the
feature/ts-sdk/bundle-entrypoint-source
branch
3 times, most recently
from
September 17, 2026 05:22
65b5ebd to
44b821d
Compare
jason810496
marked this pull request as ready for review
September 17, 2026 05:30
jason810496
force-pushed
the
feature/ts-sdk/bundle-entrypoint-source
branch
from
September 17, 2026 06:44
44b821d to
82dc7a8
Compare
jason810496
force-pushed
the
feature/ts-sdk/bundle-entrypoint-source
branch
from
September 18, 2026 03:45
82dc7a8 to
64dbd56
Compare
jason810496
marked this pull request as draft
September 18, 2026 12:57
A packed bundle now ships the entry module as its author wrote it, in a `/*# airflowSource ... #*/` block comment with its own byte range and SHA-256 digest in the layout header. The shipped code region is minified, so it is not the code anyone wrote and there was nothing readable for the Airflow UI to show for a natively authored TypeScript Dag. Only the entry module is embedded, not the module graph behind it; ADR-0006 declined multi-file source display for mixed-language Dags. A block comment, not the line comments the layout and metadata use, because the entrypoint spans the lines it was written on. That makes line terminators legal inside the region and leaves `*/` as the one sequence that must not appear: it would end the comment where Node reads the file, putting the rest of the payload into executable position while the layout still calls those bytes source and both digests still match. The packer inserts a `\` between the two characters, and escapes `*\` the same way so the transformation is reversible; the reader reverses it and rejects a bundle carrying the sequence unescaped rather than vouch for one. The cross-language golden fixture now carries regions shaped like what the packer ships, instead of the raw contents of a `.ts` fixture: a code region nobody could read, and a source region that exercises both escape branches. The escape scheme is a byte-level agreement between the TypeScript encoder and the Python reader, and only an artifact the encoder actually produced can establish it; the Python-side helper that escapes source for other tests is a reimplementation and cannot. The fixture is also excluded from Prettier, which would otherwise reformat it and invalidate its digests. The metadata `dags` mapping is renamed to `task_handlers`, which is what a TypeScript bundle actually provides: handlers for Dags declared elsewhere, not Dag definitions of its own. The mapping stays keyed by Dag ID. airflow_bundle_metadata_version stays at 1.0, redefined in place because the TypeScript packing workflow is unreleased. A source-less bundle still fails closed, on the now-required `source` layout section, before any version check is reached.
jason810496
force-pushed
the
feature/ts-sdk/bundle-entrypoint-source
branch
from
September 18, 2026 13:01
64dbd56 to
50de2c4
Compare
jason810496
marked this pull request as ready for review
September 19, 2026 03:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The shipped code region is minified, so it was nothing readable for Airflow to display for a natively authored TypeScript Dag so we need another section in the artifact to show the entry point raw source.
How
A packed bundle now carries the entry module as its author wrote it, in a region with its own byte range and SHA-256 digest in the layout header:
Only the entry module will be included in the source, not the whole module graph behind it.
A block comment rather than the line comments the layout and metadata use.
The source range's length is declared rather than derivable from a newline, so the reader pins it by checking that the prescribed opener and closer sit where the declared range implies.
Rename the metadata
dagsmapping totask_handlers, which is what a TypeScript bundle currently provides: handlers for Dags declared elsewhere, not Dag definitions of its own.The cross-language golden fixture is reworked. Its code region was the raw contents of a
.tsfixture while itssource region was a trivial
export {};, which is backwards from what the packer emits.so the Python reader is validated against bytes the TypeScript encoder actually produced.
.prettierignore, which would otherwise reformat it and invalidate its digests.Was generative AI tooling used to co-author this PR?