Skip to content

Fix pod_override serialization in DAG details and executor path#65407

Merged
ephraimbuddy merged 1 commit into
apache:mainfrom
astronomer:fix-pod-override-serialization
Apr 29, 2026
Merged

Fix pod_override serialization in DAG details and executor path#65407
ephraimbuddy merged 1 commit into
apache:mainfrom
astronomer:fix-pod-override-serialization

Conversation

@ephraimbuddy
Copy link
Copy Markdown
Contributor

Sanitize default_args["executor_config"]["pod_override"] in DAG details responses without changing other default_args values, and make V1Pod serialization force the Kubernetes import path so executor config no longer falls back to stringification.

This fixes two related Kubernetes serialization bugs: one where the DAG details API could fail when pod_override was present in default_args, and another where V1Pod objects could be flattened into strings before reaching the Kubernetes executor. The updated tests cover both the API response behavior and the serializer regression with real Kubernetes pod objects.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Generated-by: claude sonnet 4.6

closes: #57743

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:DAG-processing labels Apr 17, 2026
@ephraimbuddy ephraimbuddy added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label Apr 17, 2026
Copy link
Copy Markdown
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix, LGTM overall.

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Outdated
Comment thread airflow-core/src/airflow/serialization/serialized_objects.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Kubernetes V1Pod (pod_override) serialization issues affecting both the DAG details FastAPI response and Airflow’s internal serialization path, preventing API 500s and avoiding accidental stringification before the Kubernetes executor consumes the config.

Changes:

  • Sanitize default_args["executor_config"]["pod_override"] in DAGDetailsResponse to avoid Pydantic serialization failures.
  • Ensure BaseSerialization.serialize() forces a Kubernetes import attempt when encountering V1Pod, so it serializes to a structured dict instead of falling back to str().
  • Add unit tests covering the DAG details API behavior and the V1Pod serialization regression.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Adds default_args sanitizer for executor_config.pod_override to keep DAG details responses JSON-serializable.
airflow-core/src/airflow/serialization/serialized_objects.py Forces _has_kubernetes(attempt_import=True) for V1Pod serialization to avoid stringification.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py Adds a DAG details endpoint test asserting pod_override is sanitized in the response.
airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_dags.py Adds unit tests for _sanitize_pod_override and the default_args validator.
airflow-core/tests/unit/serialization/test_serialized_objects.py Adds regression test ensuring V1Pod serialization triggers _has_kubernetes(attempt_import=True).

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py
@ephraimbuddy ephraimbuddy self-assigned this Apr 20, 2026
@ephraimbuddy ephraimbuddy force-pushed the fix-pod-override-serialization branch 2 times, most recently from 62a4749 to 0943a65 Compare April 21, 2026 09:49
@ephraimbuddy ephraimbuddy requested a review from Copilot April 21, 2026 09:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Outdated
Comment thread airflow-core/src/airflow/serialization/serialized_objects.py
Copy link
Copy Markdown
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're good to merge it after resolving the conflict. thanks!

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py
Comment thread airflow-core/tests/unit/serialization/test_serialized_objects.py Outdated
Sanitize default_args["executor_config"]["pod_override"] in DAG
details responses without changing other default_args values,
and make V1Pod serialization force the Kubernetes import path
so executor config no longer falls back to stringification.

This fixes two related Kubernetes serialization bugs: one where
the DAG details API could fail when pod_override was present in
default_args, and another where V1Pod objects could be flattened
into strings before reaching the Kubernetes executor. The updated
tests cover both the API response behavior and the serializer
regression with real Kubernetes pod objects.
@ephraimbuddy ephraimbuddy force-pushed the fix-pod-override-serialization branch from fc4e41e to f3a666c Compare April 29, 2026 11:39
@ephraimbuddy ephraimbuddy merged commit ff15983 into apache:main Apr 29, 2026
140 checks passed
@ephraimbuddy ephraimbuddy deleted the fix-pod-override-serialization branch April 29, 2026 16:40
@github-actions github-actions Bot added this to the Airflow 3.2.2 milestone Apr 29, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.2 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

@github-actions
Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test Commit Link

You can attempt to backport this manually by running:

cherry_picker ff15983 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

seruman pushed a commit to seruman/airflow that referenced this pull request Apr 30, 2026
…he#65407)

Sanitize default_args["executor_config"]["pod_override"] in DAG
details responses without changing other default_args values,
and make V1Pod serialization force the Kubernetes import path
so executor config no longer falls back to stringification.

This fixes two related Kubernetes serialization bugs: one where
the DAG details API could fail when pod_override was present in
default_args, and another where V1Pod objects could be flattened
into strings before reaching the Kubernetes executor. The updated
tests cover both the API response behavior and the serializer
regression with real Kubernetes pod objects.
vatsrahul1001 pushed a commit that referenced this pull request May 14, 2026
Sanitize default_args["executor_config"]["pod_override"] in DAG
details responses without changing other default_args values,
and make V1Pod serialization force the Kubernetes import path
so executor config no longer falls back to stringification.

This fixes two related Kubernetes serialization bugs: one where
the DAG details API could fail when pod_override was present in
default_args, and another where V1Pod objects could be flattened
into strings before reaching the Kubernetes executor. The updated
tests cover both the API response behavior and the serializer
regression with real Kubernetes pod objects.

(cherry picked from commit ff15983)
vatsrahul1001 added a commit that referenced this pull request May 15, 2026
…) (#66898)

Sanitize default_args["executor_config"]["pod_override"] in DAG
details responses without changing other default_args values,
and make V1Pod serialization force the Kubernetes import path
so executor config no longer falls back to stringification.

This fixes two related Kubernetes serialization bugs: one where
the DAG details API could fail when pod_override was present in
default_args, and another where V1Pod objects could be flattened
into strings before reaching the Kubernetes executor. The updated
tests cover both the API response behavior and the serializer
regression with real Kubernetes pod objects.

(cherry picked from commit ff15983)

Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:DAG-processing backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PydanticSerializationError when using DAG-level pod_override (V1Pod) with KubernetesExecutor

7 participants