Skip to content

Update models to use SQLA2 annotations and synced model nullable constraints#56330

Merged
potiuk merged 2 commits intoapache:mainfrom
aws-mwaa:vincbeck/update_sqla2_null
Oct 15, 2025
Merged

Update models to use SQLA2 annotations and synced model nullable constraints#56330
potiuk merged 2 commits intoapache:mainfrom
aws-mwaa:vincbeck/update_sqla2_null

Conversation

@vincbeck
Copy link
Copy Markdown
Contributor

@vincbeck vincbeck commented Oct 2, 2025

This PR includes 2 changes (hence 2 commits):

If you have already reviewed #55954, you need to review only the 2nd commit of this PR.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@vincbeck vincbeck added the full tests needed We need to run full set of tests for this PR to merge label Oct 2, 2025
@vincbeck vincbeck closed this Oct 2, 2025
@vincbeck vincbeck reopened this Oct 2, 2025
@vincbeck vincbeck force-pushed the vincbeck/update_sqla2_null branch 3 times, most recently from ae7a557 to 98e7933 Compare October 3, 2025 14:05
@vincbeck
Copy link
Copy Markdown
Contributor Author

vincbeck commented Oct 3, 2025

@pierrejeambrun @Dev-iL, I need your help here. In this PR I update all the models to use sqlalchemy 2 annotations. As part of this migration I also had to fix the different mismtaches between the database and models regarding the non-null constraints. But anyway, long story short is, I updated all the models to use sqlalchemy 2 annotations. All tests are passing but 3, only with python 3.13:

FAILED airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_run.py::TestPatchDagRun::test_patch_dag_run[test_dag1-dag_run_2-patch_body5-response_body5-note_data5] - RuntimeError: No response returned.
FAILED airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_run.py::TestPatchDagRun::test_patch_dag_run_with_update_mask[query_params4-patch_body4-response_body4-200-note_data4] - RuntimeError: No response returned.
FAILED airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_task_instances.py::TestPatchTaskInstance::test_update_mask_set_note_should_respond_200[None-ti_note_data1] - RuntimeError: No response returned.

See the full stack trace here.

I do not quite understand the root cause. Surprisingly it fails only with patch APIs. If you have a bit of time, I'd appreciate if you can take a look and maybe you'll find out something I did not.

@vincbeck vincbeck force-pushed the vincbeck/update_sqla2_null branch from 98e7933 to 830e070 Compare October 3, 2025 20:32
@pierrejeambrun pierrejeambrun marked this pull request as ready for review October 15, 2025 14:15
@vincbeck vincbeck force-pushed the vincbeck/update_sqla2_null branch from 830e070 to ba53b14 Compare October 15, 2025 15:25
@potiuk potiuk added the all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs label Oct 15, 2025
@potiuk potiuk closed this Oct 15, 2025
@potiuk potiuk reopened this Oct 15, 2025
@potiuk
Copy link
Copy Markdown
Member

potiuk commented Oct 15, 2025

I also added "all versions" to make sure that all python versions are tested. Closed/reopened to trigger it

@vincbeck
Copy link
Copy Markdown
Contributor Author

Green 🟢

@potiuk potiuk merged commit 5a2e95a into apache:main Oct 15, 2025
588 of 590 checks passed
snreddygopu pushed a commit to Teradata/airflow that referenced this pull request Oct 16, 2025
…traints (apache#56330)

* Update all models to use `sqlalchemy` 2 annotations (apache#55954)

* Fix NULL constraints
TyrellHaywood pushed a commit to TyrellHaywood/airflow that referenced this pull request Oct 22, 2025
…traints (apache#56330)

* Update all models to use `sqlalchemy` 2 annotations (apache#55954)

* Fix NULL constraints
@vincbeck vincbeck deleted the vincbeck/update_sqla2_null branch October 27, 2025 13:58
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Mar 18, 2026
PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Mar 18, 2026
PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Mar 19, 2026
PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Mar 19, 2026
PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.
ephraimbuddy added a commit to astronomer/airflow that referenced this pull request Mar 19, 2026
PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.
ephraimbuddy added a commit that referenced this pull request Mar 19, 2026
…63899)

* Restore nullable ORM fields and drop unreleased corrective migration

PR #56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR #62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR #56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.

* Return some not nullable fixes

* fixup! Return some not nullable fixes
fat-catTW pushed a commit to fat-catTW/airflow that referenced this pull request Mar 22, 2026
…pache#63899)

* Restore nullable ORM fields and drop unreleased corrective migration

PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.

* Return some not nullable fixes

* fixup! Return some not nullable fixes
Suraj-kumar00 pushed a commit to Suraj-kumar00/airflow that referenced this pull request Apr 7, 2026
…pache#63899)

* Restore nullable ORM fields and drop unreleased corrective migration

PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.

* Return some not nullable fixes

* fixup! Return some not nullable fixes
abhijeets25012-tech pushed a commit to abhijeets25012-tech/airflow that referenced this pull request Apr 9, 2026
…pache#63899)

* Restore nullable ORM fields and drop unreleased corrective migration

PR apache#56330 mistakenly changed several ORM-backed columns from nullable to
non-nullable. I later added a corrective migration in PR apache#62234 to
backfill existing NULLs, but after revisiting the issue we found the
migration was only compensating for the nullability mistake introduced by
PR apache#56330 itself, which has not been released.

Instead of shipping a large corrective migration for an unreleased
regression, delete the migration file and restore the affected ORM fields
to nullable so the models stay aligned with the actual schema and
intended pre-release behavior.

* Return some not nullable fixes

* fixup! Return some not nullable fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs area:serialization area:Triggerer full tests needed We need to run full set of tests for this PR to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants