Fix double-serialization issue by unwrapping serialized kwargs in encode_trigger by jason810496 · Pull Request #64626 · apache/airflow

jason810496 · 2026-04-02T13:32:15Z

What

Inspired from #64625 but moved the logic into encoder instead of spreading in dag_processing/collection

Verification

The trigger will show up and will not disappear once unpause from Airflow UI
- The trigger show up in Triggerer logging
- Also directly select in DB to verify PGPASSWORD='airflow' psql -h postgres -U postgres -d airflow -c "SELECT * FROM trigger;"
The External-Event Driven Dag with Kafka Queue described in Deserialization error for External-Event Driven Dag #64613 works
- The Dag behave same in 3.1.8

Co-authored-by: Rahul Vats 43964496+vatsrahul1001@users.noreply.github.com

jason810496 · 2026-04-02T13:53:01Z

I'm working on the unit tests right now.

airflow-core/src/airflow/serialization/encoders.py

…ode_trigger

…mitive kwargs

airflow-core/src/airflow/models/trigger.py

airflow-core/tests/unit/dag_processing/test_collection.py

Copilot

Pull request overview

Fixes a trigger double-serialization problem during DAG (re-)serialization that caused asset-watcher trigger hashes to change between the “DAG-parsed” and “DB-loaded” paths (leading to repeated trigger deletion/recreation and disappearing triggers).

Changes:

Update encode_trigger() to unwrap already-serialized kwarg values before re-serializing, preventing double-wrapping.
Add unit + DB tests to validate encode_trigger() idempotency and trigger-hash consistency through DB round-trips.
Adjust Trigger model imports around Callback usage.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
airflow-core/src/airflow/serialization/encoders.py	Adds “unwrap then re-serialize” logic to prevent double-serialization of trigger kwargs.
airflow-core/tests/unit/serialization/test_encoders.py	New tests for `encode_trigger()` idempotency and DAG-vs-DB hash consistency.
airflow-core/tests/unit/dag_processing/test_collection.py	Adds regression tests for asset trigger reference hash consistency and idempotency.
airflow-core/src/airflow/models/trigger.py	Moves `Callback` import usage into methods (query paths involving callbacks).

airflow-core/src/airflow/serialization/encoders.py

Copilot · 2026-04-02T19:18:15Z

airflow-core/tests/unit/serialization/test_encoders.py

+from airflow.serialization.encoders import encode_trigger
+from airflow.serialization.enums import DagAttributeTypes as DAT, Encoding
+from airflow.triggers.base import BaseEventTrigger
+
+pytest.importorskip("airflow.providers.apache.kafka")
+from airflow.providers.apache.kafka.triggers.await_message import AwaitMessageTrigger
+


pytest.importorskip("airflow.providers.apache.kafka") at module import time will skip this entire test module when the Kafka provider isn't installed, including the FileDeleteTrigger coverage (which should be provider-independent) and the DB hash consistency checks.

Consider moving the importorskip/Kafka import into only the Kafka-specific parametrized cases (e.g., conditionally add Kafka params, or mark those params with skip) so the non-Kafka assertions still run in minimal test environments.

airflow-core/src/airflow/models/trigger.py

Copilot · 2026-04-02T19:18:15Z

airflow-core/tests/unit/dag_processing/test_collection.py

+        recreate trigger rows on every heartbeat.
+        """
+        from airflow.models.trigger import Trigger
+        from airflow.serialization.encoders import encode_trigger
+        from airflow.triggers.base import BaseEventTrigger
+


These new inline imports are unnecessary here since the file already has module-level imports, and they make it harder to see dependencies/trigger lint rules. Please move Trigger, encode_trigger, and BaseEventTrigger imports to the top of the module (or otherwise document why they must remain local).

Copilot · 2026-04-02T19:18:15Z

airflow-core/tests/unit/dag_processing/test_collection.py

+        """Calling add_asset_trigger_references twice with the same trigger
+        must not create duplicate rows.
+        """
+        from airflow.models.trigger import Trigger
+
+        trigger = FileDeleteTrigger(filepath="/tmp/test.txt", poke_interval=5.0)


New inline import in the test body; since this module already imports most dependencies at the top, please move Trigger to module scope for consistency/readability (unless there’s a specific reason to keep it local).

github-actions · 2026-04-02T19:24:50Z

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.2.1 based on: backport label targeting v3-2-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

kaxil · 2026-04-02T19:25:15Z

Merged to unblock rc2 but we should follow-up with minor fixes identified about imports

…wargs in `encode_trigger` (#64626) (cherry picked from commit d292e0e) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

github-actions · 2026-04-02T19:25:31Z

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

…wargs in `encode_trigger` (apache#64626) (cherry picked from commit d292e0e) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

…wargs in `encode_trigger` (#64626) (#64642) (cherry picked from commit d292e0e) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>

boring-cyborg bot added the area:DAG-processing label Apr 2, 2026

jason810496 requested review from ashb and bolkedebruin as code owners April 2, 2026 13:32

vatsrahul1001 requested review from kaxil and uranusjr April 2, 2026 13:47

kaxil reviewed Apr 2, 2026

View reviewed changes

airflow-core/src/airflow/serialization/encoders.py Show resolved Hide resolved

airflow-core/src/airflow/serialization/encoders.py Show resolved Hide resolved

jason810496 requested a review from XD-DENG as a code owner April 2, 2026 15:15

jason810496 added 3 commits April 2, 2026 23:56

Fix double-serialization issue by unwrapping serialized kwargs in enc…

26d349c

…ode_trigger

Fix CI failure by lazy import Callback in Trigger model

c33abfa

Add tests for trigger serialization and hash consistency with non-pri…

0084161

…mitive kwargs

jason810496 force-pushed the fix/core-serde/double-encode-trigger branch from 3de7e33 to 0084161 Compare April 2, 2026 15:56

vatsrahul1001 requested a review from kaxil April 2, 2026 17:49

vatsrahul1001 added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label Apr 2, 2026

kaxil reviewed Apr 2, 2026

View reviewed changes

airflow-core/src/airflow/models/trigger.py Show resolved Hide resolved

kaxil reviewed Apr 2, 2026

View reviewed changes

airflow-core/src/airflow/models/trigger.py Show resolved Hide resolved

kaxil reviewed Apr 2, 2026

View reviewed changes

airflow-core/tests/unit/dag_processing/test_collection.py Show resolved Hide resolved

kaxil requested a review from Copilot April 2, 2026 19:12

Copilot started reviewing on behalf of kaxil April 2, 2026 19:13 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

kaxil approved these changes Apr 2, 2026

View reviewed changes

kaxil merged commit d292e0e into apache:main Apr 2, 2026
86 checks passed

github-actions bot added this to the Airflow 3.2.1 milestone Apr 2, 2026

jason810496 mentioned this pull request Apr 7, 2026

Follow-up fixes for double-encode trigger PR #64823

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix double-serialization issue by unwrapping serialized kwargs in encode_trigger#64626

Fix double-serialization issue by unwrapping serialized kwargs in encode_trigger#64626
kaxil merged 3 commits intoapache:mainfrom
jason810496:fix/core-serde/double-encode-trigger

jason810496 commented Apr 2, 2026 •

edited

Loading

Uh oh!

jason810496 commented Apr 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

kaxil commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jason810496 commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Verification

Uh oh!

jason810496 commented Apr 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

kaxil commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Backport successfully created: v3-2-test

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jason810496 commented Apr 2, 2026 •

edited

Loading