Skip to content

Airflow 2.2.0 - catchup=False not working #19126

Closed

Description

@andormarkus

Apache Airflow version

2.2.0 (latest released)

Operating System

macOS 11.5.2 or Amazon Linux 2 on EKS

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

What happened

it looks like In Airflow 2.2.0 catchup=False is ignored. The simple_dag is scheduled to run in ever minute.

I did not experience this behivour in Airflow 2.1.3

Airflow 2.2.0

I have paused the DAG at 11:31
Screen Shot 2021-10-21 at 11 31 51

I have unpaused the DAG at 11:35 and it executed all the skipped run despite catchup=False
Screen Shot 2021-10-21 at 11 35 56

Airflow 2.1.2

I have paused the DAG at 11:45
Screen Shot 2021-10-21 at 11 45 22

I have unpaused the DAG at 11:50 and it did not executed all the skipped run
Screen Shot 2021-10-21 at 11 50 27

What you expected to happen

All the DAG run which was scheduled to run under the pause period will not run after I unpause a DAG.

How to reproduce

  1. Use the official docker-compose file - https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml
  2. Add simple_dag.py
    """Sample DAG."""
    
    import time
    from datetime import datetime, timedelta
    
    from airflow import DAG
    from airflow.operators.python import PythonOperator
    
    default_args = {
        "owner": "airflow",
        "depends_on_past": False,
        "start_date": datetime(2020, 1, 1),
        "email": ["support@airflow.com"],
        "email_on_failure": False,
        "email_on_retry": False,
        "retries": 1,
        "retry_delay": timedelta(minutes=5),
    }
    
    
    def sleep() -> bool:
        time.sleep(10)
        return True
    
    
    with DAG("simple_dag", default_args=default_args, schedule_interval="* * * * *", catchup=False) as dag:
        t1 = PythonOperator(task_id="sleep", python_callable=sleep)
  3. Run simple_dag for few minutes
  4. Pause simple_dag
  5. Wait few minutes
  6. Unpause simple_dag

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. lintingbin commented on Oct 23, 2021

    @lintingbin

    I have the same problem.

  2. uranusjr commented on Oct 23, 2021

    @uranusjr
    Member

    Please see #19130.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions