feat: experimental server persistent caching

[Akryum]

Description

Fixes #1309

I have seen loading speed increase of up to 5x faster in subsequent warm server starts on my computer. It's basically almost as fast as a page reload.

Try it

You can install it with a special release at @akryum/vite@server-cache:

npm i -D vite@npm:@akryum/vite@server-cache
yarn add -D vite@npm:@akryum/vite@server-cache
pnpm i -D vite@npm:@akryum/vite@server-cache

It can be enabled with the new experimental.serverPersistentCache config.

import { defineConfig } from 'vite'

export default defineConfig({
  experimental: {
    serverPersistentCache: true
  }
})

You can further customize the feature with an object:

import { defineConfig } from 'vite'

export default defineConfig({
  experimental: {
    serverPersistentCache: {
      enabled: true,
      cacheVersion: 'MeowV1',
      cacheVersionFromFiles: [
        'some-config-file.js',
      ],
      exclude: (url) => url.includes('@imagetools') || url.includes('node_modules') || !!url.match(/\.(jpg|webp)/),
      cacheDir: 'cache-vite-server-1',
    }
  }
})

If you encounter issues try clearing the cache:

rm -rf ./node_modules/.vite/server-cache

Known issues

• Page load might fail if the optimizeDeps cache (node_modules/.vite) is empty (workaround is to clear persistent cache and try again)
• Cached module HMR Now working!

Demo

Screencast.from.28-10-2022.00.09.52.webm

On the app I'm testing this against, the page load goes from ~10s (cold start) to ~2s (cache hit).

Additional context

This is a PoC about using persistent caching in development.

The idea is to cache as close as possible to the slowest part of the module processing: await pluginContainer.transform(), in order to avoid breaking stuff as much as possible.

Implementation notes:

• Cache writes code and source map content directly in separate files to avoid JSON parsing as much as possible
• Cache writes and reads are lazy on a per-module basis.
• There is a manifest storing useful data about the persistent cache.
• The cache has an automatic busting logic based on the config cacheVersion and cacheVersionFromFiles (using content hashing). By default it also adds the vite version, the vite config file, the define values, the env values, the package lock file and the tsconfig.json file.
• There is a special logic to automatically cache await pluginContainer.load() logic in case side effects from await pluginContainer.transform() would affect it. For example, the Svelte plugin adds CSS subrequests in an internal cache as a side-effect of transform().
• HMR on cached modules works by restoring a few properties to the Module Graph Node such as importedModules.
• Imports of optimized deps are rewritten using the final browserHash so that they are correct for the next server start.
• I have added a few logs that could help debugging issue while the feature is experimental. Most could then be removed.

---

What is the purpose of this pull request?

• Bug fix
• New Feature
• Documentation update
• Other

Before submitting the PR, please make sure you do the following

• Read the Contributing Guidelines.
• Read the Pull Request Guidelines and follow the Commit Convention.
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
• Ideally, include relevant tests that fail without this PR but pass with it.

[Akryum]

Will probably not work with Svelte as they use a custom cache for the .svelte file CSS requests:
https://github.com/sveltejs/vite-plugin-svelte/blob/49f80e75264343032fccb5e8939168174e65bc91/packages/vite-plugin-svelte/src/index.ts#L111

[Akryum]

Found a solution: cache the loadResult and restore it in case pluginContainer.load() didn't yield a result.

In the Svelte plugin:

• This is what happens when loading the .svelte file CSS: load('SystemDialog.svelte?svelte&type=style&lang.css') => internalCache.getCSS(id)

Solution:

• Cache miss
  • Persist loadResult and save the relation with the SystemDialog.svelte file into the cache manifest
• Cache hit
  • internalCache.getCSS('SystemDialog.svelte?svelte&type=style&lang.css') will return null since transform('SystemDialog.svelte') was never called (it's where the .svelte file is split in multiple parts into the plugin internalCache)
  • Restore the previously cached loadResult to get the correct code (only the <style> part)

I tested on this svelte app:
https://github.com/PuruVJ/macos-web/

Cold start (no cache):

Warm start (cache hit):

[Akryum]

Tested with https://github.com/martpie/museeks (electron + React)

Cold start:

Warm start:

[Akryum]

Tested with https://github.com/yyx990803/vite-vs-next-turbo-hmr (React)

Cold start:

Warm start:

[Akryum]

Test on https://github.com/noeldemartin/umai (Vue)

Cold start:

Warm start:

[Akryum]

Tested with yyx990803/vite-vs-next-turbo-hmr (React)

So I have tested again but on Chrome as I had a file descriptor limit issue.

Cold start:

Warm start:

[ArnaudBarre]

Thanks a lot for this work. The api with custom exclude and config files is really what was missing in my implementation. And the lazy loading of cache values is a lot more coherent with the vite architecture.

I think what is missing is a check for environment variables. A simple first solution could be to exclude from the cache files that contains pattern like process.env or import.meta.env

[ArnaudBarre]

Can you explain a bit more the goal of this whole block? (maybe with an example of two ids for svelte)

I don't understand why the override of await pluginContainer.load(id, { ssr }) is only done when idWithoutHmrFlag !== file

Maybe a first version could only focus on the transform part and see if some plugins can migrate the heavy work from load to transform?

[Akryum]

The goal is to restore the result of plugin.load in case it needed side effects from plugin.transform. For example (not real requests):

• transform: Foo.svelte => put <style> part into an internal cache as side-effect
• load: Foo.svelte?type=css => retrieve the content of <style> from the internal cache.

[ArnaudBarre]

I will bootstrap the a simple svelte template with the inspect plugin to better understand how it works and comeback here after.

[aleclarson]

As I understand it, the load hook will return nothing for Foo.svelte?type=css when Foo.svelte was already cached (and thus, never processed by transform pipeline). By caching the load result, Vite can still access the Foo.svelte?type=css contents for the cached Foo.svelte module.

Since the load hook is always called (even for cached modules), this is a necessary workaround.

[aleclarson]

Ideally, there would be a way for a load result to declare a "dependency" on another file's contents. This way, Vite wouldn't have to cache the load result of every file.

edit: Oh I just remembered that we have addWatchFile from the Rollup API. That could probably work. 🤔

[benmccann]

I wonder about the option name actually. It's sort of an implementation detail that the cache is on the server object. Perhaps just persistentCache or devPersistentCache? The former would allow a dev vs build option in case we ever wanted to add build caching in the future

[benmccann]

this line could use a comment as to either when it would occur or why we want to exclude it

[Akryum]

Well it excludes the vite client

[aleclarson]

Well it excludes the vite client

That doesn't explain why 😆

[Akryum]

I wonder about the option name actually. It's sort of an implementation detail that the cache is on the server object. Perhaps just persistentCache or devPersistentCache? The former would allow a dev vs build option in case we ever wanted to add build caching in the future

Yes it's server as opposed to build. I guess it could be under server if moved out of experimental.

[bluwy]

In the last meeting while discussing #10943, we thought it would be nice if all the caching logic is handled as an internal Vite plugin. It's not completely necessary for this PR but it would make it easier to maintain. It might involve adding hooks/options/metadata for the Vite plugin lifecycle, but they should be kept internal only.

[ethernet8023]

Hey @Akryum - we've been testing your branch on our codebase, and have run into an issue; refreshes after code has changed break the dev server if your code has circular imports.
I wish my codebase didn't have circular imports, but we have thousands, so fixing them quickly sn't very viable 😓

Here's a minimal reproduction! https://github.com/arilotter/vite-caching-circular-imports

Let me know if I can help 😁

[patak-cat]

Leaving a comment for reference, this PR has proven that server persistent caching could bring a big perf boost to some projects. It wasn't merged as part of Vite 4 because we were unsure of the current caching invalidation logic being correct for all cases.

I think we have two options moving forward (I think we should pursue both):

• Re-evaluate the idea from @aleclarson of providing a caching hooks API so this PR or other experimentation can be implemented by plugins (with caching logic that may work only for certain kinds of projects, that is totally fine for a plugin)
• Keep working on identifying perf bottlenecks and resolving them instead of relying on caching. Experimentation from @ArnaudBarre points in this direction as a way to improve cold start times. Some big improvements could come from:
  • Replace magic string with a Rust alternative (and in general help Lukas to port parts of rollup to native).
  • Replace postCSS with Lighting CSS.
  • Implement the SSR transformation on native.

[rtsao]

IMHO persistent caching in core might be preferable, even if it is an opt-in experiment and disabled by default initially.

• A core value proposition of Vite is fast performance and currently SSR startup is a major bottleneck at the moment
• It may be more difficult to implement caching as a plugin instead and ultimately expanding the plugin API surface area has costs as well.

Support for a basic level of caching that works for every project seems far more valuable than supporting niche caching logic for specific applications and circumstances.

If there's still desire to have caching in core, I can work on adapting the bug reported by @arilotter as a failing test and try fixing it.

[luxaritas]

Not knowing the details of the situation here, I also want to ask if being "unsure of the current caching invalidation logic being correct for all cases" is the issue, why isn't one of the potential avenues to pursue making it sufficiently robust? Or has there been some situation identified where it'd be impossible for Vite to cover all its bases (presumably even with configuration)?

[patak-cat]

@Akryum would it be possible to re-test some of the examples you tried here using vite@4.3.0-beta.2? It would be great to know if the latest performance improvements reduce the need for caching, or how far we are from the ideal of a cached warm start to keep digging (or re-evaluate caching once more if we can't find a way to further speed up the dev server).

[luxaritas]

or re-evaluate caching once more if we can't find a way to further speed up the dev server

Apologies if I'm misreading, but if it isn't being considered, please also keep in mind the potential impact of caching on full builds, not just the dev server. This has the potential to be important eg for monorepo tools that rely on full rebuilds of dependency packages on change (eg, nx watch in a package-based monorepo)

[caghand]

@Akryum would it be possible to re-test some of the examples you tried here using vite@4.3.0-beta.2? It would be great to know if the latest performance improvements reduce the need for caching, or how far we are from the ideal of a cached warm start to keep digging (or re-evaluate caching once more if we can't find a way to further speed up the dev server).

@patak-dev, as someone who is fetching ~6000 modules per page, I can say that the speed increase in Vite 4.3.x is absolutely incredible. It's around 5 times faster! I could not believe it at first, but it's all real. It will make a huge difference to our productivity.
Personally, I had been eagerly waiting for this "persistent caching" functionality, but now I don't need it. :)

[patak-cat]

@caghand thanks for getting back with metrics for your app! Happy to see 4.3 has such a big effect in your case. Would it be possible to share CPU profiles of cold and warm start with us? We should continue to check if there are other perf gains out there. You can use pnpm run dev --force --open --profile then p, and the same without --force for warm start. A CPU profile will be generated at root. You could also use https://github.com/antfu/vite-plugin-inspect which will show you times for each plugin you use. You may be able to identify bottlenecks in some community plugins (and we can work together with them to improve them)

[Akryum]

Continued in #14333