Repository navigation
fix(db): serialize database init and migrations across processes - #44558
gitRasheed wants to merge 6 commits into
Conversation
Concurrent CLI processes racing one database file crash at startup: the driver enables WAL at connection-open before any busy_timeout is set, so the losers fail instantly with 'database is locked', and the migration path guards its check-then-write with an in-process semaphore only, letting two fresh processes both run schema.up — or both run a pending migration, which for destructive migrations means data loss. Set busy_timeout in both drivers before the first write, and take the cross-process Flock (already used for the models.dev cache race) inside DatabaseMigration.apply, keyed by SQLite's own resolved path so every caller is covered and path aliases collapse. Migration transactions use immediate behavior so a read-to-write upgrade waits on busy_timeout instead of failing at once.
aeffe0b to
7024062
Compare
Overall: this addresses a real multi-process footgun thoroughly — cross-process flock keyed on the resolved DB path,
Minor: |
Default filesystems on Windows and macOS are case-insensitive, so two spellings of one database path must map to one lock. Folding on case-sensitive filesystems too only over-serializes two distinct files for the duration of a migration.
ty mr bot! took point 1 and lowercasing unconditionally. |
(cherry picked from anomalyco#44558)
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
Issue for this PR
Closes #33320
Type of change
What does this PR do?
I reproduced the race by starting six
opencode runprocesses against a fresh global SQLite database. Five failed with "database is locked" within ~15ms.Both drivers enable WAL before setting
busy_timeout, and changing journal mode bypasses the busy handler.DatabaseMigration.applyuses an in-process semaphore around its check and write, so separate processes can run the same migration. Because one shipped migration resets the database, running it twice can silently delete session data. The migration transactions also used DEFERRED mode, which returns SQLITE_BUSY during a read-to-write upgrade without consultingbusy_timeout.I set
busy_timeoutbefore the first write in both drivers and retry WAL conversion within that budget.DatabaseMigration.applynow takes the existing cross-processFlockused bymodels-dev.ts, keyed by the path frompragma_database_listso aliases share a lock. Memory databases skip the lock. Migrations useBEGIN IMMEDIATEand recheck the journal inside the transaction, allowing raced callers to skip completed migrations.#38849 stays open: this covers its startup crashes but not the mid-session ones, which come from a different write path. Its
busy_timeout=0reading is a per-connection artifact; the pragma is set, just bypassed here.How did you verify your code works?
Two cross-process tests in
test/database-migration.test.tsspawn real children on a file barrier: one races four processes at a fresh database, the other rewinds the last migration and asserts the journal ends with exactly one row. They fail unpatched; the manual six-process race went from 1/6 to 6/6 clean. Full core suite and typecheck pass.Screenshots / recordings
Not a UI change.
Checklist