[albertlast]

Description

A PostgreSQL forum upgraded from 2.1 to 3.0 arrives without six of the indexes a fresh install has, all of them on columns widened for 2038:

table	column
mail_queue	time_sent
scheduled_tasks	next_time
member_logins	time
log_subscribed	end_time
log_reported	time_started
log_reported_comments	time_sent

The first two are read by the cron on every page load that runs it.

normalize() changes columns first and rebuilds indexes afterwards, and decides which indexes to rebuild partly by asking whether any of the columns they cover are about to change. That question was being asked of the wrong values:

foreach ([$index->columns, $structure['indexes'][$index->name]['columns']] as $cols) {
    if (array_intersect(array_keys($cols), array_keys($columns_to_change)) !== []) {

Both lists hold their column names as values — the definition as ['name' => ...] arrays, the database as plain strings — so array_keys() on either gives 0, 1, 2. $columns_to_change is keyed by column name. The intersection is always empty, and no index has ever been rebuilt for this reason.

On PostgreSQL that loses the index. A change of column type there is carried out by adding a temporary column, copying into it, dropping the original and renaming:

ALTER TABLE smf_mail_queue ADD COLUMN time_sent_tempxx bigint
ALTER TABLE smf_mail_queue DROP COLUMN time_sent      -- takes its indexes with it
ALTER TABLE smf_mail_queue RENAME COLUMN time_sent_tempxx TO time_sent

Nothing then puts the index back.

MySQL is unaffected: it changes a column in place, so an index over that column survives whether or not anything asks for it to be rebuilt.

What makes this hard to notice is that running the upgrade a second time restores them, because by then the columns are already the right type and nothing drops the indexes again. The schema an admin ends up with depends on how many times they ran the upgrader.

The bare names are compared now. MySQL reports a prefix length as part of the column name (body(15)), so that is stripped first.

How this was found, and how it was verified

.docker/rerun-upgrade.sh (#9622) upgrades a 2.1 baseline, upgrades it again, and reports what the second run changed. It should change nothing. On PostgreSQL it reported these six indexes appearing on the second run, which is what led here. The statement order above was then read out of the PostgreSQL log with log_statement='ddl' turned on.

Against the 2.1.7 baseline from the 2.1 development environment:

	before	after
PostgreSQL, the six indexes above	absent after one upgrade	present after one upgrade
PostgreSQL, rerun-upgrade.sh	schema CHANGED	schema unchanged
MySQL, rerun-upgrade.sh	schema unchanged	schema unchanged
MySQL, indexes lost across the upgrade	none	none

Not reachable from the unit suite

Schema\Column::__construct() calls Db::$db->calculate_type(), so the schema classes cannot be instantiated without a connection, and normalize() does nothing but read and alter a live database. Verified by running real upgrades on both engines instead.

Issues References (Fixes|Related|Closes)

1. Related: [3.0][Testing] Check that an upgrade survives being run twice and being interrupted #9622 -- the check that found this
2. Related: [3.0] Recognises a normal index that is only under the wrong name #9624 -- the other half of what the index handling gets wrong on PostgreSQL