Skip to content

Support updating DagRun note from notifier/callback context in Airflow 3 #64890

Closed

Description

@hkc-8010

Description

In Airflow 2, a common notifier/callback pattern was:

  1. Post an alert to an external system such as Slack.
  2. Persist a backlink or correlation identifier on the DagRun note.
  3. Reuse that stored link later to update the original alert if the DagRun is cleared and eventually succeeds.

That migration path is not straightforward in Airflow 3.

In Airflow 3 callback/notifier code, context["dag_run"] is no longer the old SQLAlchemy DagRun contract that many Airflow 2 implementations relied on, and the Task SDK client does not appear to expose a supported helper for updating DagRun.note from callback/notifier runtime code.

There is already public API support for DagRun note patching, so this request is specifically about having a supported callback-safe/runtime-safe way to do that from Airflow 3 notifier code.

Use case/motivation

A concrete use case is a DAG-level failure notifier that:

  1. Fires once when the DagRun fails.
  2. Collects failed task IDs for that run.
  3. Posts a Slack message.
  4. Stores the Slack thread URL in the DagRun note for traceability.
  5. On a later callback (for example after the DagRun is cleared and succeeds), reads that stored link and updates the original Slack message instead of sending a brand new one.

The "list failed tasks" part is portable in Airflow 3 via context["task_instance"].get_task_states(...) in a real callback execution path.

The missing piece is the note update. In Airflow 2 this was commonly done with ORM/session access such as DagRunNote, dag_run.dag_run_note, and provide_session. In Airflow 3 that pattern is not portable, but I could not find a supported replacement in notifier/callback context.

Related issues

Requested outcome

One of the following would unblock Airflow 2 -> 3 notifier migrations cleanly:

  1. Add a supported Task SDK / callback-context helper to update DagRun.note from notifier code.
  2. Expose a documented runtime client operation for patching the current DagRun note from callback code.
  3. If this is intentionally unsupported, document the recommended replacement pattern for storing callback correlation metadata on a DagRun in Airflow 3.

Environment where this came up

  • Apache Airflow 3.1.8
  • DAG-level on_failure_callback
  • Local Astro / Astronomer runtime during migration testing

Minimal callback shape

from airflow.notifications.basenotifier import BaseNotifier


class ExampleNotifier(BaseNotifier):
    def notify(self, context):
        dag_run = context["dag_run"]
        ti = context["task_instance"]

        task_states = ti.get_task_states(
            dag_id=dag_run.dag_id,
            run_ids=[dag_run.run_id],
        )

        failed_tasks = [
            task_id
            for task_id, state in task_states.get(dag_run.run_id, {}).items()
            if state in {"failed", "upstream_failed"}
        ]

        # Supported in Airflow 3: derive failed task list.
        print(failed_tasks)

        # Missing piece: supported way to persist a backlink such as a Slack URL
        # onto the DagRun note from this notifier/callback context.

Why I think this belongs in OSS Airflow

This appears to be a migration and API-surface gap rather than a managed-service issue:

  • Airflow 3 callback context behavior changed relative to many Airflow 2 implementations.
  • Public DagRun note patching exists, so the underlying capability is present.
  • The missing piece is a supported, documented path for callback/notifier code to update the current DagRun note.

Thanks for taking a look.

Activity

  1. eladkal commented on Apr 8, 2026

    @eladkal
    Contributor

    Collects failed task IDs for that run

    This should be given in context starting Airflow 3.2 (fixed in #61274)

    One of the following would unblock Airflow 2 -> 3 notifier migrations cleanly:

    1. Add a supported Task SDK / callback-context helper to update DagRun.note from notifier code.
    2. Expose a documented runtime client operation for patching the current DagRun note from callback code.
    3. If this is intentionally unsupported, document the recommended replacement pattern for storing callback correlation metadata on a DagRun in Airflow 3.

    I think that the initial thought behind notes is that it was designed to be used by on-calls manually but I can understand how the use case is expanded since then. Overall I favor option 1.

  2. ashb commented on Apr 8, 2026

    @ashb
    Member

    This could possibly be part of the State management API (103?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions