Skip to content

Do not backfill old DagRun.created_at#63825

Merged
vatsrahul1001 merged 1 commit intoapache:mainfrom
astronomer:do-not-backfill-dr-created-at
Mar 18, 2026
Merged

Do not backfill old DagRun.created_at#63825
vatsrahul1001 merged 1 commit intoapache:mainfrom
astronomer:do-not-backfill-dr-created-at

Conversation

@uranusjr
Copy link
Copy Markdown
Member

Filling this field in the migration is much too costly. This field is now left as NULL for rows created prior to the migration.

Close #63550.

@boring-cyborg boring-cyborg bot added the area:db-migrations PRs with DB migration label Mar 17, 2026
@uranusjr
Copy link
Copy Markdown
Member Author

Actually, I couldn’t find any usage of this field. Do we plan to use it in the future for partitions? I want to remove it outright if there’s not a planned usage.

@uranusjr
Copy link
Copy Markdown
Member Author

cc @dstandish

@uranusjr uranusjr force-pushed the do-not-backfill-dr-created-at branch from 5646bc6 to 9fc0919 Compare March 17, 2026 14:13
Filling this field in the migration is much too costly. This field is
now left as NULL for rows created prior to the migration.

I couldn't find this field used anywhere.
@uranusjr uranusjr force-pushed the do-not-backfill-dr-created-at branch from 9fc0919 to 287d856 Compare March 17, 2026 14:16
@ashb
Copy link
Copy Markdown
Member

ashb commented Mar 17, 2026

Please keep the field, it's helpful for ordering and other things I have in mind to add.

@uranusjr
Copy link
Copy Markdown
Member Author

Is it OK it’s nullable (and can contain null values)?

@vatsrahul1001 vatsrahul1001 merged commit 32f3fcc into apache:main Mar 18, 2026
140 of 141 checks passed
@vatsrahul1001 vatsrahul1001 deleted the do-not-backfill-dr-created-at branch March 18, 2026 05:32
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Mar 18, 2026
Do not backfill old DagRun.created_at
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Mar 18, 2026
Do not backfill old DagRun.created_at
fat-catTW pushed a commit to fat-catTW/airflow that referenced this pull request Mar 22, 2026
Do not backfill old DagRun.created_at
techcodie pushed a commit to techcodie/airflow that referenced this pull request Mar 23, 2026
Do not backfill old DagRun.created_at
abhijeets25012-tech pushed a commit to abhijeets25012-tech/airflow that referenced this pull request Apr 9, 2026
Do not backfill old DagRun.created_at
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration 0106 upgrade takes ~6 minutes on large dag_run tables due to full-table UPDATE

4 participants