Skip to content

MSGraphSensor does not respect timeout parameter #62157

Closed

Description

@morelgeorge

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

  • 12.10.1

Apache Airflow version

3.1.7

Operating System

RHEL 9.6

Deployment

Virtualenv installation

Deployment details

No response

What happened

We are building a DAG file which triggers Dataflow refresh in Power BI Service. To be able to do that, I am using this example.

refresh_dataflow_task = MSGraphAsyncOperator(
        task_id="refresh_dataflow",
        conn_id="PowerBI_Connection",
        url="myorg/groups/{workspaceId}/dataflows/{dataflowId}/refreshes",
        method="POST",
        path_parameters={
            "workspaceId": GROUP_ID,
            "dataflowId": DATAFLOW_ID,
        },
        data={"notifyOption": "NoNotification"}
    )

    dataflow_status = MSGraphSensor(
        task_id="dataflow_refresh_status",
        conn_id="PowerBI_Connection",
        url="myorg/groups/{workspaceId}/dataflows/{dataflowId}/transactions?$top=1",
        path_parameters={
            "workspaceId": GROUP_ID,
            "dataflowId": DATAFLOW_ID
        },
        timeout=10,
        event_processor=lambda response, **context: response["value"][0]["status"] == "Success"  # type: ignore[typeddict-item]
    )

refresh_dataflow_task >> dataflow_status

The sensor works fine when the dataflow refresh is in status "Success". However it does not stop when the timeout value is reached. It is in the loop and I must finish the DAG manually.

What you think should happen instead

When the timeout parameter is set, then the MSGraphSensor should stop when the limit is reached. When the timeout parameter is not set, some default value should be set (Currently, the default value is None)

How to reproduce

  1. Start Power BI object refresh with using MSGraphAsyncOperator.
  2. Cancel the refresh in Power BI service. (The status must be different then is expected by event_processor)
  3. Check if the MSGraphSensor is stopped when the timeout is reached

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. Subham-KRLX commented on Feb 19, 2026

    @Subham-KRLX
    Contributor

    I looked into this. The root cause is that MSGraphSensor.execute_complete() re-defers indefinitely with TimeDeltaTrigger(retry_delay) without ever checking if self.timeout has elapsed. Since
    execute() is fully overridden with defer(), Airflow's built-in sensor timeout enforcement never kicks in.
    Fix: Before re-deferring in execute_complete(), compute elapsed time from context["ti"].start_date and raise AirflowSensorTimeout if self.timeout is set and exceeded. I would like to work on this and want to raise a PR.

  2. eladkal commented on Feb 19, 2026

    @eladkal
    Contributor
  3. jroachgolf84 commented on Feb 20, 2026

    @jroachgolf84
    Collaborator

    @Subham-KRLX - please feel free to create a PR for this issue if you are interested!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions