fix(migrations): move UPDATEs inside disable_sqlite_fkeys in migration 0097#64876
fix(migrations): move UPDATEs inside disable_sqlite_fkeys in migration 0097#64876Lee-W wants to merge 1 commit intoapache:mainfrom
Conversation
|
@vatsrahul1001 It looks like we used the wrong approach for testing, and I applied the wrong constraint during RC1 testing. I didn't have time to test it on RC2 until it was released yesterday 🤦♂️ After running the following command we discussed, I found out there's no dag_run in the DB. breeze start-airflow --backend sqlite --load-example-dags --use-airflow-version 3.1.8
breeze start-airflow --backend sqlite --load-example-dagsSo here's what I do for verifying this PR is correct (how to reproduce was detailed in the description) Step 1breeze start-airflow --backend sqlite --load-example-dags --use-airflow-version 3.1.8 --terminal-multiplexer tmuxStep 2Run a few Dags randomly. We need rows in the Dag run and task instance (or whatever rows that reference Dag using FK) Step 3Run Ctrl-C/Cmd-C on all airflow components in breeze but don't run stop_airflow and don't exit Step 4git chechout to v3.2.0Step 5Go into the breeze terminal again airflow db migrateStep 6Restart all the components in breeze (use the previous command with ↑) |
Dev-iL
left a comment
There was a problem hiding this comment.
Can we modify the migration test (or add a new regression test) to capture this sort of issues?
Why
Migration 0097 failed on SQLite with FOREIGN KEY constraint failed when running DROP TABLE dag. PRAGMA foreign_keys=off (issued inside disable_sqlite_fkeys) is silently ignored when a transaction is already active — SQLite limitation. The two UPDATE statements before disable_sqlite_fkeys triggered SQLAlchemy's autobegin, starting a transaction before the PRAGMA ran.
How to reproduce
Run a few Dags randomly. We need rows in the Dag run and task instance (or whatever rows that references Dag using FK)
Go to the
Shelltab in the breeze terminalRun
What
Move the UPDATE statements inside disable_sqlite_fkeys so PRAGMA foreign_keys=off executes before autobegin opens a transaction.
Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.