Skip to content

Add dag_version cleanup indexes for task_instance and dag_run#63407

Draft
shaealh wants to merge 8 commits intoapache:mainfrom
shaealh:shaealh-60145-issue/60145-add-missing-indexes
Draft

Add dag_version cleanup indexes for task_instance and dag_run#63407
shaealh wants to merge 8 commits intoapache:mainfrom
shaealh:shaealh-60145-issue/60145-add-missing-indexes

Conversation

@shaealh
Copy link
Copy Markdown
Contributor

@shaealh shaealh commented Mar 12, 2026


Closes #60145

Adds a migration to speed up airflow db clean -t dag_version by creating missing indexes on:

  • task_instance(dag_version_id) (idx_task_instance_dag_version_id)
  • dag_run(created_dag_version_id) (idx_dag_run_created_dag_version_id)

Downgrade removes both indexes (if_exists=True).

Tests :

  • ruff check airflow-core/src/airflow/migrations/versions/0108_3_2_0_add_dag_version_id_indexes_for_db_cleanup.py
  • pytest airflow-core/tests/unit/utils/test_db.py -k single_head_revision_in_migrations -q --with-db-init
  • pytest airflow-core/tests/unit/migrations/test_no_orm_refs_in_migration_scripts.py -q

Was generative AI tooling used to co-author this PR?

  • Yes (please specify the tool below)
    Codex & ChatGPT

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@shaealh shaealh requested review from XD-DENG and ashb as code owners March 12, 2026 08:00
@shaealh
Copy link
Copy Markdown
Contributor Author

shaealh commented Mar 14, 2026

Please review

@potiuk potiuk marked this pull request as draft April 2, 2026 16:43
@potiuk
Copy link
Copy Markdown
Member

potiuk commented Apr 2, 2026

@shaealh This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

  • Merge conflicts: This PR has merge conflicts with the main branch. Your branch is 515 commits behind main. Please rebase your branch (git fetch origin && git rebase origin/main), resolve the conflicts, and push again. See contributing quick start.

Note: Your branch is 515 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • The comment informs you what you need to do.
  • Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
  • There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates.
  • Maintainers will then proceed with a normal review.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible missing indices on the database tables task_instance and dag_run

2 participants