Skip to content

Run checks for files that still name deleted tests (#3536) - #4158

Open
szsunyuan wants to merge 1 commit into
garrytan:mainfrom
szsunyuan:fix/check-files-that-name-deleted-tests
Open

szsunyuan wants to merge 1 commit into
garrytan:mainfrom
szsunyuan:fix/check-files-that-name-deleted-tests

Conversation

@szsunyuan

@szsunyuan szsunyuan commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Removing or renaming a test now runs checks for files that still name its old path, even when another test was already selected. This catches stale test lists before they slip through the local checks.

Why (in your own words)

Pregate, the local pre-push test check, can report success after a test is deleted while skipping a remaining curation-list test that would fail on the stale filename. This happens when the deleted test already maps to another test, either through dependencies or a surviving test's path reference: selectLanes takes its mapped-test early return before checking other files that still name the removed path.

This addresses #3536. Merged parent #3534 explicitly left that early-return case as a known limit; this is the deferred follow-up, not a repeat of the parent fix.

Validation status: Full free CI passed on head 45701d4c4. The 165 focused local checks and both typechecks also pass. Required selected paid coverage and remaining shipping reviews are still pending; skipped fork jobs are not passes.

Why this shape

Reuse the existing remaining-file coverage logic through a private helper. Call it before the mapped-test early return, without moving the entire removal-waiver path or changing its import and ownership checks.

The policy's own dependencies keys intentionally map removed paths, so treating those keys as stale uses would reject otherwise valid mappings. Only those keys are excluded from that policy-file scan. Dependency test targets and other policy fields still count, with regressions against exempting the whole policy or all dependency contents.

Five files, one commit, no new public exports, settings or dependencies. The diff stays bounded for maintainer integration. It does not include #4150's separate array-spread cleanup, even though that work touches the same module.

What it does

  • Applies remaining-file coverage to both mapped deletions and rename old names.
  • Preserves the originally selected tests and adds the tests for remaining non-Markdown files that name the removed test.
  • Refuses an uncovered reference instead of silently passing; preserves intentional empty dependency declarations.
  • Keeps valid dependency-map keys while checking dependency targets and other policy references.
  • Adds eight regression/control cases in the existing test/pregate-removed-paths.test.ts owner file.
  • Updates docs/pregate.md and docs/troubleshooting.md to distinguish reference checks from unmapped-removal waivers.

What it deliberately does not do

Live evidence

Bun 1.4.2 and Node 20.20.2 on Linux, with umask 022, LC_ALL=C, private HOME/state/TMPDIR and local bare Git origins. No browser, cloud VM or live model was used.

Actual CLI before and after

The fixture uses the existing makeFixtureRepo, commit and runBin helpers. It invokes the real command with --base main --allow-repo-commands --stage tests, requires an actual lanes record, and extracts the following fields. Both sides intentionally retain the same stale curation entry.

Before: unchanged main production code at d479c5bf1:

{
  "source": "base-d479c5bf1",
  "exit": 0,
  "laneStatus": "pass",
  "code": null,
  "detail": "1 tests ran in 1 file(s), exit 0",
  "ran": "ran: node --test 'test/retry.test.js'"
}

After: the production code committed as 45701d4c4:

{
  "source": "working-fix",
  "exit": 1,
  "laneStatus": "fail",
  "code": "PREGATE_LANE_FAILED",
  "detail": "local lane exited 1 (1 failed of 2 ran; 2 files; pipefail)",
  "ran": "ran: node --test 'test/curation.test.js' 'test/retry.test.js'"
}

The repaired gate now runs the missing curation check and rejects the stale entry. It does not repair the fixture's list or drop the originally mapped retry test.

Failing-first regressions and controls

The final eight new cases were also copied into an isolated detached worktree at d479c5bf1, without changing its production code. That control produced 7 failures and 1 passing control, with 47 unrelated cases filtered out:

  • Four stale-curation failures: dependency mapping or surviving-test reference, each with deletion or rename.
  • Two policy-reference failures: runner command and dependency test target.
  • One uncovered-namer failure.
  • The intentional no-test declaration remains the passing control.

On the repaired implementation, the complete owner file passes all 55 cases, including those eight and the existing removal/import/source/owning-suite controls.

Focused checks

Commands ran with the isolated environment described above. Counts below do not double-count the separate new-case runs or the failing base control.

Command Pass Fail
bun --no-env-file --config=/dev/null test test/pregate-removed-paths.test.ts --timeout=30000 55 0
bun --no-env-file --config=/dev/null test test/pregate-selection.test.ts test/pregate-policy-lane-verdicts.test.ts --timeout=30000 73 0
bun --no-env-file --config=/dev/null test test/test-of-test-ratchet.test.ts test/spawnsync-timeout-tripwire.test.ts test/module-size-ratchet.test.ts test/egress-receipt-wiring.test.ts --timeout=30000 22 0
bun --no-env-file --config=/dev/null test test/changelog-fragments.test.ts --timeout=30000 15 0
Total 165 0

No skipped or filtered cases in those four runs. Product bun run typecheck exits 0. bun run typecheck:test reports 812 known diagnostics, none new. The committed source/test bytes match the passing inputs. Runtime docs were updated after those runs, and the later release fragment passed its own suite. Plain-title and changelog prose checks, whitespace checks, and exact staged/committed patch redaction scans pass with zero redaction findings.

Scope

  • Changed: five files, +106/-17 against base d479c5bf1: lib/pregate/lane-selection.ts, the existing removed-path owner test, two runtime docs, and one branch fragment. One commit: 45701d4c47acae31d1bebdd0eca1f95bd43a4ab3.
  • Verified live by: actual CLI before/after above, the unchanged-base regression control, 165 focused passing checks, both typechecks, full free CI on this head, and exact-byte publication checks.
  • Did NOT test: the full bun run test suite locally, required selected paid coverage, or a complete /review and /ship certification. Local reproduction and owner checks ran on Linux. Full free CI passed remotely; this is not a claim of running the owner suite natively on every platform.

The free paid-coverage planner (bun run eval:select --base upstream/main --json) reaches 252 registered cases through this shared module. The PR profile selects 108 behaviors, no dedicated quality judges, and defers 144 broad behaviors to scheduled/release coverage. None was run locally; none receives pass credit here.

Liveness proof (required for external contributors)

image

Checklist

Test plan

  • Reproduce the mapped deletion falsely passing while the curation test is skipped.
  • Prove the new regression cases fail against unchanged main production code.
  • Cover dependency and surviving-test mappings, deletion and rename, uncovered references and intentional empty declarations.
  • Keep policy runner and dependency-target references checked rather than exempting the whole policy.
  • Pass the complete owner suite, adjacent selection/policy controls, source/test guards and fragment checks.
  • Pass strict product types and the unchanged test-type debt limit.
  • Check the plain title, release-fragment prose, whitespace and exact outgoing patch for sensitive content.
  • Complete full free acceptance on this PR head.
  • Complete required selected paid coverage, with maintainer-run fork-PR coverage where needed.
  • Complete remaining shipping reviews and verification before merge.

Fork PRs do not receive upstream eval secrets. Skipped paid jobs are not passes; required maintainer-run coverage remains necessary before merge. Broad-profile deferrals are scheduled/release coverage, not PR passes.

Fixes #3536.

Apply remaining-file coverage to mapped deletions and renames without
changing removal-waiver eligibility or dropping the original test selection.
Keep intentional dependency keys valid, but retain checks for dependency
targets and other policy references. Add regressions and update runtime docs.

Fixes: garrytan#3536

Co-Authored-By: Claude Code <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Nov 8, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@capy-ai

capy-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Thanks for the follow-up to #3534. This is on the crew's pick-up list. Pregate changes are paused until 2026-11-21, so after that a crew lane will rebuild it on current main and credit you as co-author when it lands, then close this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pregate: a removed test that maps to a test skips the files-naming-it check (pre-existing on main)

1 participant