[manerow]

Fixes #37180

Why

Until #37167, the 2.0.3 MySQL migration switched entity_extension.extension to CHARACTER SET ascii COLLATE ascii_bin. PR #37181 stops that for upgrades that have not run 2.0.3 yet. Databases that already ran it (upgraded to 2.0.3 to 2.0.7 or 2.1.0-rc1) still have an ascii column that differs from every other text column in the table: it rejects non-ASCII keys and compares keys case-sensitively. Two states of the same column also mean every future change to it has to handle both.

What changes

A new MySQL data migration step, EntityExtensionCollationAlignment, puts the column back on its table's default character set and collation:

• It only acts when the column differs from its table. Databases that never ran the old statement, or whose column was already converted ahead of the upgrade, do nothing.
• The target is each table's own default, read from information_schema, not a fixed collation. OpenMetadata never set a collation, so tables carry the server default of the day they were created: utf8mb4_0900_ai_ci on MySQL 8, but older databases can have utf8mb4_general_ci. Forcing one collation would copy tables that are already consistent.
• Collision check before copying. The table default usually ignores case, so two keys of one entity that differ only in case (for example values of custom properties Foo and foo) would become duplicates and the copy would fail with ERROR 1062, possibly hours in. The step looks for them first and, if any exist, stops with an error that names them and says what to do, leaving the database unchanged. The check reads only the small secondary index, which also holds id and extension: 1.7 s on a 1M-row, 4 GB table.
• It logs before copying, with the table size, because MySQL copies the whole table and blocks writes to it until the copy ends.

It runs from the 2.0.8 Java migration, and again from 2.1.0 (with runOnce, like the other 2.0.8 steps there), so it also reaches databases already on 2.1 pre-release builds: a changed 2.1.0 migration class is pending again for them.

It runs from Java rather than schemaChanges.sql because it has to stop with a clear message, per ADR:2026-10-22-guarded-index-ddl-runs-from-a-java-data-migration. The 2.0.8 SQL file is not edited to point here, because changing the comment above an existing statement changes its recorded text and makes it run again on databases that already ran 2.0.8. Postgres is unchanged.

Decision record

ADR:2026-11-06-entity-extension-uses-its-tables-collation states the rule: on MySQL, entity_extension.extension uses its table's default collation, and no migration pins it to another one.

Tests

EntityExtensionCollationAlignmentTest runs the step on a real MySQL 8.0 container. An unchanged InnoDB table id means MySQL did not copy the table.

Case	Result
ascii column, table on utf8mb4_0900_ai_ci	Aligned, rows unchanged
Column already matching its table	Untouched, no copy
ascii column, table on utf8mb4_general_ci	Aligned to utf8mb4_general_ci
Foo and foo on the same entity	Stops before copying, names them, column and table unchanged
Foo and foo on different entities	Not a collision, aligned
Second run	No change

Real upgrades with the migration runner, using the 2.1.0-rc1 release image with this step and PR #37181 added, on databases built by running the real migrations:

Database	Upgrade with this change	Table copied	validate
Ran stock 2.0.3 (ascii_bin)	Aligned to utf8mb4_0900_ai_ci; a second run changes nothing	Once	Passes
Ran stock 2.0.3, plus Foo and foo on one entity	Stops before copying with the error below, column unchanged. After deleting one key, the rerun aligns it	Only on the rerun
Already migrated by stock 2.1.0-rc1 (ascii_bin, 2.1.0 recorded)	Aligned on the next upgrade, through the 2.1.0 step	Once	Passes
Ran 2.0.2 only (utf8mb4_0900_ai_ci, 256)	Widened in place by PR #37181, nothing to align	No	Passes

Cannot align entity_extension.extension to utf8mb4_0900_ai_ci: some keys of the same entity differ only in letter case and would become duplicates. They usually come from two custom properties whose names differ only in case. Delete or rename one property of each pair, then run the migration again. Conflicting keys (entity id: keys, up to 20 shown): 11111111-2222-3333-4444-555555555555: table.customProperties.Foo, table.customProperties.foo

[gitar-bot]

Code Review ✅ Approved 1 closed / 1 findings

🔴 High risk · Aligning MySQL entity_extension collation rewrites the table and changes key comparisons, potentially blocking writes or halting upgrades on case collisions.

Adds a MySQL data migration step, EntityExtensionCollationAlignment, that moves entity_extension.extension from ascii_bin back to its table's default collation, with a pre-copy check for case-only key collisions. The open decision-record finding was resolved, and no open issues remain.

✅ 1 closed

✅ Quality: No decision record for the entity_extension.extension collation rule

📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v208/EntityExtensionCollationAlignment.java:21-35 📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v208/EntityExtensionCollationAlignment.java:64-66
This PR decides what character set and collation entity_extension.extension uses on MySQL, and nothing in docs/decisions/ records that rule. No record mentions collation or ascii_bin. Keys that differ only in case now collide, so two custom properties named Foo and foo can't both exist on one entity. A later migration could undo this rule as easily as 2.0.3 did. The description gives the rule in prose ("The target is each table's own default read from information_schema, not a fixed collation"), and since the repo squash-merges, that prose never reaches git log.

Please add a record stating: "On MySQL, entity_extension.extension uses its table's default character set and collation. Migrations never pin it to a fixed collation such as ascii_bin. A migration that changes the column's collation first refuses to run if keys of one entity would become equal." Cite it from EntityExtensionCollationAlignment as ADR:<date>-<slug>.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

_{For more verification details, comment gitar display:verbose.}

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}