V4 cache handler

[nico-martin]

Improved Download Progress Tracking

Problem

Transformers.js couldn't reliably track total download progress because:

• File lists weren't known before downloads started
• File sizes were inconsistent (compressed vs uncompressed)
• No cache awareness before initiating downloads

Solution

New Exported Functions

• get_files(): Determines required files before downloading
• get_model_files() / get_tokenizer_files() / get_processor_files(): Helper functions to identify files for each component
• get_file_metadata(): Fetches file metadata using Range requests without downloading full content
  • Returns fromCache boolean to identify cached files
  • Ensures consistent uncompressed file sizes
• is_cached(): Checks if all files from a model are already in cache

Enhanced Progress Tracking

• readResponse() with expectedSize: Falls back to metadata when content-length header is missing
• total_progress callback: Provides aggregate progress across all files

Review

One thing I am not super confident is the get_model_files function. I tried to test it with different model architectures, but maybe I missed some that load files that are not in that function. @xenova, could you smoke-test some models and write mie the models that fail?

Easiest way to do that is:

import {
  get_files,
  pipeline,
} from "@huggingface/transformers";

const expectedFiles = await get_files(
  "onnx-community/gemma-3-270m-it-ONNX",
  {
    dtype: "fp32",
    device: "webgpu",
  }
);
const loadedFiles = new Set();
const pipe = await pipeline(
  "text-generation",
  "onnx-community/gemma-3-270m-it-ONNX",
  {
    dtype: "fp32",
    device: "webgpu",
    progress_callback: (e) => {
      if (e.file) loadedFiles.add(e.file);
    },
  }
);

console.log(
  "SAME FILES:",
  expectedFiles.sort().join(",") === Array.from(loadedFiles).sort().join(",")
);

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

Very exciting PR! 🙌 Just a quick review from scanning the PR briefly.

[xenova]

can we find a way to export a single Object/static class, which encapsulates all these methods?

e.g.,

import { ClassName } from '@huggingface/transformers'; // no idea what to call it yet haha

const x = await ClassName.get_files(...);

I feel like this could be more future proof.

No idea what to call this wrapper class, but something along the lines of CacheRegistry, FileRegistry, HubRegistry could work? Maybe we can find a similar abstraction in the transformers/huggingface_hub library?

[nico-martin]

Good point. I think the perfect solution would be to have a different pattern for the pipeline:

const textGenerator = new Pipeline({
  "text-generation",
  "onnx-community/gemma-3-270m-it-ONNX",
  {
    dtype: "fp32",
    device: "webgpu",
  }
});

const isCached = await textGenerator.isCached();
const files = await textGenerator.getFiles();

await textGenerator.load({ progress_callback: console.log })

const output = await textGenerator.generate([
  {
    role: "system",
    content: "You are a helpful assistant",
  },
  {
    role: "user",
    content: "How are you?",
  },
], {
  max_new_tokens: 1024,
})

Then those methods would be on the pipe. But I definitely see your point. The whole feature should be encapsuled in one export. I'll think of a cleaner solution.

[nico-martin]

How about this? https://github.com/huggingface/transformers.js/blob/v4-cache-handler/packages/transformers/src/utils/cache/ModelRegistry.js

[xenova]

can we check which models are not correctly set? Ideally, we shouldn't need manual checking here.

[nico-martin]

Yes, we can check! I added logging to detect models not in MODEL_TYPE_MAPPING:

console.warn(                                                                   
  `[get_model_files] Architecture(s) not found in MODEL_TYPE_MAPPING: [${archList}] ` + 
  `for model type '${config.model_type}'. Using heuristic detection. ` +
  `Consider adding to packages/transformers/src/models/registry.js`
);                                                                              

I also improved the lookup to check model_type before falling back to heuristics. This fixed models like modnet (no architectures array) and briaai/RMBG-1.4 which were triggering the fallback unnecessarily.

[xenova]

if generation_config.json isn't found, do we throw an error, or gracefully ignore it? 👀

Luckily, as you said, users very rarely load the non-generation variants of these models.

[nico-martin]

Good question. I mean I wouldn't throw an error. If there is no generation_config.json I don't see a problem. It would expect a file, but the expected fize will be 0. So the total would still be correct. On the progress callback it will then load all the other files up to 100%, then its done. I think its fine.

[nico-martin]

In fact, its better to add a file if we are not sure. The only downside could be a 404 on the metadata check. But I dont think thats a problem.