Pre-update backup copies free pages: 3 min offline per update on a migrated database that is 59% freelist #32952

Before submitting

• I searched existing issues. Related, but not this: Service updates keep the server offline while copying a large database on filesystems without reflink #21715 (update downtime from the full copy, closed as designed), fix(server): migrated databases can return freed space to disk #30066 (fixes auto_vacuum for new V1 imports only), fix(server): report and cancel pre-update database backups #18633 (backup progress), Interrupted pre-update database backup restarts from zero and leaks staging dirs, leaving the service failed with the update stuck pending #27809 (interrupted backup), statev2.sqlite grows without bound (33 GB in 7 weeks): retention/compaction for orchestration events and turn items #28903 (growth).

Area

apps/server (service launcher, pre-update backup)

What happens

On a database that was migrated from V1, most of the pre-update backup is copying empty pages, and it is done while the server is down.

• statev2.sqlite: 13.55 GB, 3,308,587 pages of 4 KiB.
• freelist_count 1,952,895 (59% of the file), auto_vacuum 0.
• t3 doctor --dry-run: "Database: 12.62 GiB, 7.45 GiB reclaimable."

The launcher's backupDatabaseOnce uses NodeSqlite.backup, a page-level copy, so it copies the freelist too.

Update today, 0.1.8-nightly.20261104.4165 → 0.1.8-nightly.20261108.4367 (UTC+2):

15:21:11.752  Server update prepared; handing off to the service launcher.
15:21:14.289  Relay client stopped
              (nothing logged; server offline)
15:24:27      candidate `t3 serve` process started (ps lstart)
15:24:29.357  Migrations ran successfully  [80_ScheduledTaskThreadMode]
15:24:29.819  Listening on http://0.0.0.0:3773

That is 3 min 13 s offline. A one-step sqlite3 .backup of the same file on the same disk (ext4, NVMe, no reflink) took 173 s, so the backup accounts for almost all of it. The previous 8 updates on this host took between 1 min 20 s and 3 min 45 s from "update prepared" to "Listening", rising as the file grew.

Expected

An update on a database that is mostly free pages should not copy those free pages while the server is down.

Suggestions

1. Back up without the freelist, for example VACUUM INTO for the pre-update snapshot when freelist_count is a large share of page_count. That would be about 5.6 GB here instead of 13.5 GB.
2. Tell already-migrated installs to run the reclaim. fix(server): migrated databases can return freed space to disk #30066 only fixes new imports. Installs that are already on auto_vacuum = 0 keep a growing freelist, and nothing points them to t3 doctor. The server, or the update flow (which is offline anyway), could offer "Reclaim database space" when auto_vacuum = 0 and the freelist is above some threshold.

Side observation (likely #24225)

While the service was down, the macOS desktop's SSH launcher started its own t3 serve --host 127.0.0.1 --port 3773 --base-dir ~/.t3 (the previous runtime, 0.1.8-nightly.20261104.4165) about every 62 s: three times in the gap. A local guard killed them. They did not seem to delay this update, but each one is a second process on the same database while the launcher is mid-backup.

Version / environment

• Server 0.1.8-nightly.20261108.4367; the service unit's launcher is 0.0.46-nightly.20261007.2761 (see [Bug]: Service unit stays pinned to the runtime that installed it, so deleting old runtimes breaks the next restart #17490).
• Linux x64, kernel 7.0.0-38-generic, ext4 on NVMe, systemd user service, macOS desktop connecting over SSH.

Workaround

Running t3 doctor --fix ("Reclaim database space") off-hours. I'll add the before/after sizes and how long it took once it has run.