Fix a integer overflow error caused by the small zstd decompression buffer size

[qilme]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @qilme, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical integer overflow error that could occur during Zstandard decompression. The issue stemmed from an undersized static buffer, which has now been replaced with a dynamically calculated size based on Zstandard's specifications. This change enhances the stability and reliability of the decompression process, preventing potential data corruption or application crashes.

Highlights

• Zstd Decompression Buffer Fix: The buffer size used for Zstandard decompression has been increased to prevent an integer overflow error.
• Robust Buffer Sizing: The buffer is now sized dynamically using std.compress.zstd.default_window_len + std.compress.zstd.block_size_max to ensure it is always sufficient for decompression.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request fixes an issue in zstd decompression by increasing the buffer size, correctly using constants from the zstd library. However, the new buffer is over 8MB and is allocated on the stack, which will likely cause a stack overflow. My review includes a critical suggestion to allocate this buffer on the heap instead, preventing the crash.

[gemini-code-assist]

The new buffer size is over 8MB. Allocating this on the stack will likely cause a stack overflow. It should be allocated on the heap using the allocator passed to this function.

My suggestion heap-allocates the buffer and defers freeing it to prevent a memory leak.

                var buf = try allocator.alloc(u8, std.compress.zstd.default_window_len + std.compress.zstd.block_size_max); defer allocator.free(buf);
                var dcp = std.compress.zstd.Decompress.init(&reader, buf, .{});

[jiacai2050]

how will the issue happen?

[qilme]

When i fetch a tar.zst dep, zigfetch paniced with a stack trace below:

thread 26732 panic: integer overflow
X:\scoop\apps\zig\0.15.2\lib\std\compress\zstd\Decompress.zig:128:37: 0x7ff67799378e in rebase (zigfetch_zcu.obj)
    assert(capacity <= r.buffer.len - d.window_len);
                                    ^
X:\scoop\apps\zig\0.15.2\lib\std\compress\zstd\Decompress.zig:191:59: 0x7ff677993de9 in streamIndirectInner (zigfetch_zcu.obj)
    if (r.buffer.len - r.end < zstd.block_size_max) rebase(r, zstd.block_size_max);
                                                          ^
X:\scoop\apps\zig\0.15.2\lib\std\compress\zstd\Decompress.zig:186:31: 0x7ff677993cb7 in readVec (zigfetch_zcu.obj)
    return streamIndirectInner(d);
                              ^
X:\scoop\apps\zig\0.15.2\lib\std\Io\Reader.zig:372:37: 0x7ff6778b579b in readVec (zigfetch_zcu.obj)
        return n + (r.vtable.readVec(r, data[i..]) catch |err| switch (err) {
                                    ^
X:\scoop\apps\zig\0.15.2\lib\std\Io\Reader.zig:631:21: 0x7ff677896af1 in readSliceShort (zigfetch_zcu.obj)
        i += readVec(r, &data) catch |err| switch (err) {
                    ^
X:\scoop\apps\zig\0.15.2\lib\std\tar.zig:352:49: 0x7ff677896196 in readHeader (zigfetch_zcu.obj)
        const n = try self.reader.readSliceShort(&self.header_buffer);
                                                ^
X:\scoop\apps\zig\0.15.2\lib\std\tar.zig:394:35: 0x7ff677899876 in next (zigfetch_zcu.obj)
        while (try self.readHeader()) |header| {
                                  ^
X:\scoop\apps\zig\0.15.2\lib\std\tar.zig:592:23: 0x7ff67789d0fe in pipeToFileSystem (zigfetch_zcu.obj)
    while (try it.next()) |file| {
                      ^
X:\src\zigcli\src\bin\zigfetch.zig:525:29: 0x7ff67789df41 in unpackTarball (zigfetch_zcu.obj)
    std.tar.pipeToFileSystem(out_dir, reader, .{
                            ^
X:\src\zigcli\src\bin\zigfetch.zig:459:41: 0x7ff6778a6a1a in fetchPackage (zigfetch_zcu.obj)
                return try unpackTarball(allocator, out_dir, &dcp.reader);
                                        ^
X:\src\zigcli\src\bin\zigfetch.zig:226:41: 0x7ff677885798 in cachePackageFromUrl (zigfetch_zcu.obj)
    const sub_dirname = try fetchPackage(allocator, url, out_dir);
                                        ^
X:\src\zigcli\src\bin\zigfetch.zig:151:56: 0x7ff6778a916b in calcHash (zigfetch_zcu.obj)
                            _ = try cachePackageFromUrl(allocator, u, hash);
                                                       ^
X:\src\zigcli\src\bin\zigfetch.zig:186:30: 0x7ff67788b2e3 in cachePackageFromLocal (zigfetch_zcu.obj)
    const hash = try calcHash(allocator, dir, "", false);
                             ^
X:\src\zigcli\src\bin\zigfetch.zig:178:43: 0x7ff67788afd9 in handleDir (zigfetch_zcu.obj)
    const hash = try cachePackageFromLocal(allocator, dir);
                                          ^
X:\src\zigcli\src\bin\zigfetch.zig:109:22: 0x7ff67788c09e in main (zigfetch_zcu.obj)
        try handleDir(allocator, path);
                     ^
X:\scoop\apps\zig\0.15.2\lib\std\start.zig:602:28: 0x7ff67788cabd in main (zigfetch_zcu.obj)
    return callMainWithArgs(@as(usize, @intCast(c_argc)), @as([*][*:0]u8, @ptrCast(c_argv)), envp);
                           ^
X:\scoop\apps\zig\0.15.2\lib\libc\mingw\crt\crtexe.c:259:0: 0x7ff6779c818b in __tmainCRTStartup (crt2.obj)
    mainret = _tmain (argc, argv, envp);

X:\scoop\apps\zig\0.15.2\lib\libc\mingw\crt\crtexe.c:179:0: 0x7ff6779c81eb in mainCRTStartup (crt2.obj)
  ret = __tmainCRTStartup ();

???:?:?: 0x7ff9c47e5c06 in ??? (KERNEL32.DLL)
???:?:?: 0x7ff9c5259cbf in ??? (ntdll.dll)

[qilme]

Here capacity is std.compress.zstd.block_size_max, d.window_len is std.compress.zstd.default_window_len by default.

X:\scoop\apps\zig\0.15.2\lib\std\compress\zstd\Decompress.zig:128:37: 0x7ff67799378e in rebase (zigfetch_zcu.obj)
    assert(capacity <= r.buffer.len - d.window_len);

r.buffer.len is introdued from here:

X:\src\zigcli\src\bin\zigfetch.zig:459:41: 0x7ff6778a6a1a in fetchPackage (zigfetch_zcu.obj)
                return try unpackTarball(allocator, out_dir, &dcp.reader);

which is what i modified

[qilme]

Zig repo's is written like this:

        .@"tar.zst" => {
            const window_len = std.compress.zstd.default_window_len;
            const window_buffer = try f.arena.allocator().alloc(u8, window_len + std.compress.zstd.block_size_max);
            var decompress: std.compress.zstd.Decompress = .init(resource.reader(), window_buffer, .{
                .verify_checksum = false,
                .window_len = window_len,
            });
            return try unpackTarball(f, tmp_directory.handle, &decompress.reader);
        },