Add pg_dump snapshot cache for software-factory Playwright tests by habdelra · Pull Request #4378 · cardstack/boxel

habdelra · 2026-04-10T13:41:33Z

Summary

Add a pg_dump/pg_restore snapshot cache layer for the software-factory Playwright test template database
Reduces cache:prepare from ~5-10 minutes to < 1 second when the PG template DB is missing but test fixtures haven't changed
Add a lint:snapshot-freshness lint rule that fails when the committed snapshot is stale
Add --update-snapshot flag to cache:prepare for easy snapshot regeneration
Remove duplicative public-software-factory-source fixture — source realm (realm/) is now used directly with a glob filter
Remove obsolete darkfactory.spec.ts tests

The Problem

The cache:prepare step builds a PostgreSQL template database by spinning up a full realm server, indexing all fixtures (~376 files across base realm, source realm, and test fixture directories), and waiting for indexing to complete. This takes 5-10 minutes.

The PG template database doesn't survive machine reboots, PostgreSQL restarts, Docker container rebuilds, or fresh git checkouts. Since test fixtures rarely change, developers frequently sit through a full 10-minute rebuild for content that hasn't changed.

How It Works

Three-tier cache hierarchy

Tier 1 (sub-second): PG template database exists in-process — CREATE DATABASE ... TEMPLATE clone. This is the existing behavior, unchanged.
Tier 2 (< 1 second) — NEW: pg_restore from a committed pg_dump file when the PG template is gone but fixtures haven't changed.
Tier 3 (~5-10 min): Full rebuild — start realm server, index everything, wait for idle. Existing fallback, now also saves a fresh dump after completing.

Snapshot fingerprint

A fingerprint is computed by SHA-256 hashing the full file contents of every file in:

packages/base/ (base realm — most likely to trigger regeneration)
packages/software-factory/realm/ (source realm, filtered through SOURCE_REALM_GLOB — only .gts card definitions and .realm.json)
packages/software-factory/test-fixtures/ (3 fixture directories: darkfactory-adopter, bootstrap-target, test-realm-runner)

Plus the CACHE_VERSION constant (bumped on schema changes). The fingerprint is stored in db-snapshots/fingerprint.json alongside the dump.

Source realm glob filter

The source realm (realm/) contains 205 files but tests only need the core card definitions. A glob filter controls which files are included:

SOURCE_REALM_GLOB = '*.gts .realm.json !document.gts !wiki.gts'

This filter is applied both when computing the fingerprint and when copying the source realm to the temp build directory. Only darkfactory.gts, test-results.gts, and .realm.json make it through. The wiki briefs, document instances, and other content are not necessary for tests.

When the snapshot is created/regenerated

Automatically during cache:prepare after any full build (Tier 3). If the build succeeds and the fixtures match the canonical set, the template is dumped and the fingerprint is written.
Manually via cd packages/software-factory && pnpm cache:prepare --update-snapshot

Dump size optimization

The dump is optimized to ~4 MB by:

Stripping boxel_index_working (rebuilt from boxel_index at clone time)
NULLing last_known_good_deps (14MB of fallback data, rebuilt on next index)
Truncating modules, jobs, job_reservations (rebuilt at startup)
Running VACUUM FULL before dumping
Removing the duplicative public-software-factory-source fixture (source realm indexed once, not twice)

Lint rule: `lint:snapshot-freshness`

A new lint check (auto-included in pnpm lint via the existing concurrently glob) that:

Computes the current fingerprint from source files (~200ms, no PG access needed)
Compares to the committed fingerprint.json
Fails with a clear, developer-friendly error message if they don't match

The error message explains what went wrong, which directories could have caused the change, and gives the exact command to fix it — designed for developers who don't normally work in the software-factory package.

Removed `public-software-factory-source`

The old test-fixtures/public-software-factory-source was a 3-file stub of the source realm that could silently diverge from realm/. It was indexed as both the source realm AND a separate fixture (duplicate). Now the source realm (realm/) is used directly, filtered through SOURCE_REALM_GLOB, and indexed once at /software-factory/.

Files

File	Description
`src/harness/db-snapshot.ts`	Core module: fingerprint computation, pg_dump/restore, glob filter, snapshot check/save
`src/harness/api.ts`	Integrated snapshot check before full build, save after build
`src/harness/isolated-realm-stack.ts`	Filtered source realm copy, symlink resolution
`src/harness/shared.ts`	`dbSnapshotDir` export, `fileFilter` option on `hashRealmFixture`
`src/cli/cache-realm.ts`	`--update-snapshot` flag
`scripts/lint-snapshot-freshness.ts`	Lint check script
`playwright.global-setup.ts`	Removed source realm override and fixture
`tests/fixtures.ts`	Removed source realm override
`tests/darkfactory.spec.ts`	Removed (obsolete)
`package.json`	Added `lint:snapshot-freshness` script
`.gitattributes`	Marks `template.pgdump` as binary
`db-snapshots/template.pgdump`	Compressed pg_dump (~4 MB)
`db-snapshots/fingerprint.json`	Fingerprint + metadata

Test plan

Cold start (no dump): Full build runs with live progress bar, dump + fingerprint generated
Valid dump (fast restore): Restores in < 1s, no indexing, logs "Restoring template from committed snapshot"
Stale dump (wrong fingerprint): Detects mismatch, full build with progress bar, dump regenerated
DB exists, metadata missing: Skips snapshot restore, goes to full build (no CREATE DATABASE error)
Lint fresh: Exits 0, prints "OK"
Lint stale: Exits 1 with clear fix instructions
Source realm glob filter: Only 2 files indexed for source realm (was 204 without filter)
Fingerprint stable across environments: Same fingerprint with and without SOFTWARE_FACTORY_SOURCE_REALM_DIR env var
21 Playwright tests pass in 7.0m (cache:prepare: 0.8s from snapshot)

🤖 Generated with Claude Code

Implements a Tier 2 cache between the existing PG template (Tier 1) and the expensive full rebuild (Tier 3). When the PG template is missing but a committed pg_dump snapshot exists with a matching fingerprint, the template is restored via pg_restore in seconds instead of a ~10 min full rebuild. New files: - db-snapshot.ts: core snapshot logic (fingerprint, dump, restore, check) - lint-snapshot-freshness.ts: lint script to detect stale snapshots - .gitattributes: mark pgdump as binary for git - db-snapshots/.gitkeep: directory placeholder Modified files: - api.ts: integrate snapshot restore/save in both ensureFactoryRealmTemplate and ensureCombinedFactoryRealmTemplate - shared.ts: add dbSnapshotDir export - package.json: add lint:snapshot-freshness script Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Lint error now shows clear instructions for developers unfamiliar with the software-factory package: what went wrong, which directories to check, exact command to fix, and a fallback note about PG requirements - Add --update-snapshot flag to cache:prepare that uses the canonical fixture list (no need to remember which dirs to pass) - Include initial snapshot dump (40MB) and fingerprint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Strip data that is rebuilt at clone/startup time before dumping: - boxel_index_working (exact copy of boxel_index, rebuilt on clone) - boxel_index.last_known_good_deps (14MB of fallback deps, rebuilt on index) - modules (cleared on realm startup) - jobs/job_reservations (cleared on clone) Clone to temp DB, strip, VACUUM FULL, then pg_dump for best compression. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

12MB compressed pg_dump of the Playwright test template database, generated from the current base realm, source realm, and test fixtures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1694bb788c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

packages/software-factory/src/harness/api.ts

Copilot

Pull request overview

This PR adds a committed PostgreSQL pg_dump snapshot + fingerprinting layer to speed up software-factory Playwright template database preparation, with a new lint rule to ensure the committed snapshot stays in sync with fixtures/source.

Changes:

Introduces snapshot creation/restore utilities (pg_dump/pg_restore) and deterministic fingerprinting for fixture/source freshness checks.
Integrates snapshot restore (fast path) and snapshot save (after full rebuild) into the harness template preparation flow.
Adds --update-snapshot support to cache:prepare and a lint:snapshot-freshness script that fails when the committed snapshot is stale.

Reviewed changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/software-factory/src/harness/shared.ts	Exposes `dbSnapshotDir` path for snapshot storage.
packages/software-factory/src/harness/db-snapshot.ts	Implements fingerprinting + snapshot dump/restore helpers.
packages/software-factory/src/harness/api.ts	Adds snapshot restore-before-rebuild and save-after-rebuild hooks for templates.
packages/software-factory/src/cli/cache-realm.ts	Adds `--update-snapshot` to use the canonical fixture set for snapshot regeneration.
packages/software-factory/scripts/lint-snapshot-freshness.ts	Adds lint script to validate committed snapshot freshness via fingerprint.
packages/software-factory/package.json	Registers `lint:snapshot-freshness` script.
packages/software-factory/db-snapshots/fingerprint.json	Adds committed snapshot fingerprint metadata.
packages/software-factory/db-snapshots/.gitkeep	Keeps snapshot directory present in git when empty.
packages/software-factory/.gitattributes	Marks `template.pgdump` as binary to avoid text transforms.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/software-factory/src/harness/db-snapshot.ts

packages/software-factory/scripts/lint-snapshot-freshness.ts

packages/software-factory/src/harness/api.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Gate snapshot writes to canonical fixture set only (prevents single-realm builds from overwriting the committed 4-realm snapshot) - Fix pgEnv() to not set PGPASSWORD to empty string (preserves .pgpass) - Gate snapshot restore on !hasTemplateDatabase (avoids CREATE DATABASE failure when DB exists but metadata is missing) - Fix URL consistency in single-realm restore path (derive both templateRealmURL and templateRealmServerURL from snapshot metadata) - Remove unreachable cacheVersion lint check (already covered by fingerprint) - Add isCanonicalFixtureSet() helper Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The global setup sets SOFTWARE_FACTORY_SOURCE_REALM_DIR to test-fixtures/public-software-factory-source, but the fingerprint was computed using the env-dependent default (realm/). This caused a fingerprint mismatch in CI, bypassing the snapshot restore. Fix: use CANONICAL_SOURCE_REALM_DIR (hardcoded to the Playwright test value) instead of the env-dependent sourceRealmDir, ensuring the fingerprint is consistent across lint, local dev, and CI. Also reduces dump size from 11.7MB to 4.1MB since the canonical source realm is smaller. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The test-fixtures/public-software-factory-source directory was a 3-file stub that could diverge from the real realm/. Replace it with a symlink to realm/ so they can never diverge. Add SOURCE_REALM_GLOB ('*.gts .realm.json !document.gts') that filters which files from the source realm get copied for indexing and included in the snapshot fingerprint. This keeps the test DB small (only card definitions, not instance data) while ensuring the snapshot stays in sync with the actual source realm. Also add realpathSync to copyRealmFixture so symlinks are resolved before copying, and add fileFilter option to hashRealmFixture. All 25 Playwright tests pass. cache:prepare restores from snapshot in 4.4s (vs ~10min full build). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

wiki.gts is not needed for the Playwright test suite. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The public-software-factory-source fixture was a duplicate of the source realm (realm/) — the same content was being indexed at two different URL paths. Remove it and let the source realm be indexed once at /software-factory/, using the glob filter to copy only card definitions (darkfactory.gts, test-results.gts) to the temp build dir. Also remove darkfactory.spec.ts which tested card rendering against the adopter fixture — these schema tests are covered by runtime-schema.spec.ts and the bootstrap/tool-executor tests. Results: - 3 fixture realms instead of 4 - Dump size: 4.1 MB (down from 11.7 MB) - cache:prepare from snapshot: 0.8s - 21 Playwright tests pass in 7.0m Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

backspace

I’m approving but I only superficially understand and haven’t exercised locally

habdelra and others added 5 commits April 10, 2026 09:10

Fix prettier formatting in snapshot cache files

1174430

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add initial snapshot for current main

6a66ef6

12MB compressed pg_dump of the Playwright test template database, generated from the current base realm, source realm, and test fixtures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

habdelra requested a review from Copilot April 10, 2026 13:46

chatgpt-codex-connector bot reviewed Apr 10, 2026

View reviewed changes

packages/software-factory/src/harness/api.ts Outdated Show resolved Hide resolved

Copilot started reviewing on behalf of habdelra April 10, 2026 13:46 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

habdelra and others added 7 commits April 10, 2026 09:53

Fix fingerprint.json trailing newline for prettier

e740011

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add comment clarifying SOURCE_REALM_GLOB intent

3794270

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Exclude wiki.gts from source realm glob

92400f9

wiki.gts is not needed for the Playwright test suite. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

habdelra requested a review from a team April 10, 2026 16:25

backspace approved these changes Apr 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pg_dump snapshot cache for software-factory Playwright tests#4378

Add pg_dump snapshot cache for software-factory Playwright tests#4378
habdelra wants to merge 12 commits intomainfrom
cs-snapshot-cache

habdelra commented Apr 10, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

backspace left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

habdelra commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

The Problem

How It Works

Three-tier cache hierarchy

Snapshot fingerprint

Source realm glob filter

When the snapshot is created/regenerated

Dump size optimization

Lint rule: lint:snapshot-freshness

Removed public-software-factory-source

Files

Test plan

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

backspace left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

habdelra commented Apr 10, 2026 •

edited

Loading

Lint rule: `lint:snapshot-freshness`

Removed `public-software-factory-source`