Skip to content

[3.0] Keeps the substep count across the migration batches - #9645

Merged
Sesquipedalian merged 1 commit into
SimpleMachines:release-3.0from
albertlast:3.0/migration-batch-substep-offset
Sep 6, 2026
Merged

Sesquipedalian merged 1 commit into
SimpleMachines:release-3.0from
albertlast:3.0/migration-batch-substep-offset

Conversation

@albertlast

Copy link
Copy Markdown
Collaborator

Description

Since #9639, a 2.1 → 3.0 upgrade runs none of the 3.0 migrations. It reports success and stamps the database as 3.0 Alpha 4, but the database is still structurally 2.1.

#9639 gave each version namespace its own performSubsteps() call so that smfVersion could be updated as each batch finished. The substep counter it walks is not per-batch, though. It lives in the query string, performSubsteps() loops while (Maintenance::getCurrentSubStep() < Maintenance::$total_substeps), and nothing resets it between calls. So:

substeps
batch 1, v2_1 81 migrations + 73 normalize() = 154
batch 2, v3_0 22 migrations + 72 normalize() = 94

Batch 1 leaves the counter at 154. Batch 2 sets $total_substeps to 94 and loops while (154 < 94), which never runs. All 22 v3_0 migrations and all 72 v3_0 table normalizations are skipped in silence, performSubsteps() returns true because the loop simply ended, and finalize() then writes the new version number over a 2.1 database.

Against the 2.1.7 baseline from the 2.1 development environment, on release-3.0:

before after
substeps in the migration step 154 248
messages.edit_history missing present
members.spoofdetector_name missing present
calendar.rrule missing present
smf_calendar_holidays, which HolidaysToEvents drops still there gone
smfVersion afterwards 3.0 Alpha 4 3.0 Alpha 4

The last row is the reason this is worth catching quickly: nothing in the outcome says anything went wrong. A forum in that state then fails at runtime in as many ways as there are 3.0 features, and re-running the upgrader does repair it — by then only one namespace is selected, so the counter starts at 0 and the batch runs — which makes it look intermittent.

Why not simply reset the counter

Resetting it between batches fixes the command line and breaks the browser. There, jsonResponse() ends the request, so one substep is one request and the counter is the only thing carrying the position across them. A counter reset to 0 at the end of batch 1 means the next request re-enters migrations() from the top of VERSION_MAP with a number that now points back into batch 1, and the two batches alternate forever.

So the number has to stay unique across the whole step. performSubsteps() takes the count of everything that ran before its batch and finds its own place in it:

while (Maintenance::getCurrentSubStep() - $offset < count($substeps)) {
    $substep = $substeps[Maintenance::getCurrentSubStep() - $offset];

Which means the batches have to be built before any of them runs, since the total is not known until they all are. That total is what Maintenance::$total_substeps is set to, so the progress bar and the template's iTotalSubSteps measure the whole step again rather than whichever batch is in hand.

A batch that has already been done is now skipped by the same arithmetic: its $offset + count() is below the counter, the loop does not run, and it returns true.

How this was verified

Against the 2.1.7 baseline from the 2.1 development environment, with #9644 applied so that the upgrade gets far enough to see any of this:

  • .docker/rerun-upgrade.sh from [3.0][Testing] Check that an upgrade survives being run twice and being interrupted #9622 — the second upgrade changes nothing, on MySQL.
  • A command line upgrade — 248 substeps, every v3_0 migration in the log, the columns above present, smf_calendar_holidays gone.
  • The upgrader loaded in a real browser — iTotalSubSteps reads 248 rather than 154, which is the number the template's JavaScript compares against to decide the step is finished. Under the current code it would stop at 154 believing it was done.

The browser could not be driven to the end here for an unrelated reason: Maintenance::jsonResponse() calls ob_end_clean() whether or not a buffer is open, and this environment runs with output_buffering = 0, so the notice it emits lands in the response before the JSON header and the JavaScript gets HTML. That is reported separately and reproduces without this change applied.

Not reachable from the unit suite

tests/bootstrap.php provides no database and no request, and this is a question about how a request-scoped counter behaves across several calls against a live database. Verified by running real upgrades on both engines instead.

Issues References (Fixes|Related|Closes)

  1. Related: [3.0] Incrementally updates smfVersion setting as migration batches complete #9639, which this repairs rather than reverts — the incremental smfVersion update it added is kept.
  2. Related: [3.0][Testing] Check that an upgrade survives being run twice and being interrupted #9622, the checks this was found with.
  3. Related: [3.0] Imports the schema namespace FixDates now names #9644, which has to be in place before an upgrade gets far enough to see this.

🤖 Generated with Claude Code

Running each version's migrations as its own batch, so that smfVersion can be
written as each one finishes, gave every batch its own performSubsteps() call.
The number those calls walk is not per-batch, though. It lives in the query
string, the loop runs while it is below the number of substeps, and nothing
sets it back between calls.

The v2_1 batch is 81 migrations and 73 table normalizations, and leaves the
count at 154. The v3_0 batch is 22 and 72, sets the total to 94, and asks
whether 154 is below 94. It is not, so all 94 are passed over without a word,
the call reports success because its loop simply ended, and finalize() then
writes the new version number over a database that is still shaped like 2.1.
Nothing in the result says so, which is the worst part: the forum is broken
afterwards in as many ways as there are 3.0 features.

Setting the number back between batches would fix the command line and break
the browser, where jsonResponse() ends the request and one substep is one
request, so that number is the only thing carrying the position between them.
Put back to zero at the end of the first batch, the next request re-enters from
the top of VERSION_MAP with a number that points into the first batch again,
and the two take turns forever.

So it stays unique across the step and each batch is told how many substeps ran
before it, which also means a batch already finished is skipped by the same
arithmetic. The batches are built before any of them runs, since the total is
not known until they all are, and that total is what the progress bar and the
template's iTotalSubSteps measure against.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
@Sesquipedalian
Sesquipedalian merged commit caa5ff6 into SimpleMachines:release-3.0 Sep 6, 2026
9 checks passed
@jdarwood007 jdarwood007 added this to the 3.0 Alpha 5 milestone Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants