Skip to content

Document Go cross-language argument binding - #71871

Merged
jason810496 merged 2 commits into
apache:mainfrom
jason810496:docs/go-sdk/cross-lang-binding-flow-adr
Sep 18, 2026
Merged

jason810496 merged 2 commits into
apache:mainfrom
jason810496:docs/go-sdk/cross-lang-binding-flow-adr

Conversation

@jason810496

Copy link
Copy Markdown
Member

What

  • Add a short ADR with ASCII diagrams for the wire delivery and Go resolution flows.
  • Document literal, XCom, schema, flat, sole-struct, and process-lifecycle behavior.
  • Link the ADR from the Go SDK README.

Was generative AI tooling used to co-author this PR?

@jason810496 jason810496 self-assigned this Aug 20, 2026
@jason810496
jason810496 requested a review from potiuk August 20, 2026 08:04
@jason810496
jason810496 force-pushed the docs/go-sdk/cross-lang-binding-flow-adr branch from e89a40e to 360cde8 Compare August 21, 2026 03:16
@jason810496
jason810496 requested a review from uranusjr August 21, 2026 06:10
@uros-b

uros-b commented Aug 21, 2026

Copy link
Copy Markdown
Member

+1, LGTM

@uranusjr

Copy link
Copy Markdown
Member

I wonder if we really need to have such different paths to get a literal vs XCom. Since the SDK process for execution is short-lived anyway, maybe it’s better to simply have a GetLiteralArg Execution API endpoint that mirrors GetXCom? This way we don’t need to store any state between steps; the Go task gets everything it needs on Task.Execute.

@jason810496 jason810496 left a comment •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we really need to have such different paths to get a literal vs XCom. Since the SDK process for execution is short-lived anyway, maybe it’s better to simply have a GetLiteralArg Execution API endpoint that mirrors GetXCom? This way we don’t need to store any state between steps; the Go task gets everything it needs on Task.Execute.

Could you elaborate on what you mean by “store any state between steps,” and which steps you have in mind? Do you mean the binding spec persisted between Dag serialization and task execution, or the binding.Plan stored between Go task registration and Task.Execute? Thanks.

Literal arguments are already stored in the serialized Dag as part of #69757. At ti_run endpoint, Airflow reads that binding spec and includes it in TIRunContext.arg_bindings ( https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/api_fastapi/execution_api/services/task_instances.py#L47-L61 )
Each binding contains either the literal value itself or the upstream task reference needed to retrieve the XCom.

Therefore, by the time Task.Execute is called, the Go runtime already has everything needed to resolve and bind the arguments. Only the actual XCom payload requires an additional lookup.

A GetLiteralArg endpoint would make literal and XCom retrieval look more similar, but it would move the literal lookup behind an additional Execution API request rather than eliminate the underlying state. The server would still need to read the literal from the serialized Dag.

Regardless of how the raw value is retrieved, the Go SDK must still analyze the function signature (which is what binding.Analyze does) and bind and decode each value into the corresponding Go parameter.

@jason810496 jason810496 left a comment •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tracked the optional follow-up item in #72612 based on our last offline discussion regarding the dedicated execution API route to stream the literal payload when TI startup.

@jason810496
jason810496 requested review from uranusjr and removed request for uranusjr September 7, 2026 08:18
@jason810496
jason810496 force-pushed the docs/go-sdk/cross-lang-binding-flow-adr branch from 360cde8 to f036721 Compare September 17, 2026 07:39

@henry3260 henry3260 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@jason810496
jason810496 merged commit 1a240f7 into apache:main Sep 18, 2026
64 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants