Skip to content

LibCompress: Decode short Huffman symbols at end of stream - #27098

Open
Arthur031221 wants to merge 1 commit into
SerenityOS:masterfrom
Arthur031221:fix-deflate-short-symbol-s17
Open

Arthur031221 wants to merge 1 commit into
SerenityOS:masterfrom
Arthur031221:fix-deflate-short-symbol-s17

Conversation

@Arthur031221

Copy link
Copy Markdown

SerenityOS unzip aborts on valid source archives. In a ZIP made with git archive --format=zip 80baeb29708a, 460 entries fail, starting at Base/home/anon/Source/js/object-expression.js.

The Huffman decoder requires the full lookup width at EOF even when fewer bits contain a complete symbol. This change uses an explicit relaxed peek, then requires the matched code length. Strict streams still reject truncated symbols and propagate read errors.

Fixes #25005 and #22130.

Added tests for short symbols, truncated codes, and stream errors. Tested with ctest --test-dir Build/lagom/Tests/LibCompress --output-on-failure.

Assisted by Claude/Codex.

Allow lookahead to return the bits left at end of stream, then require
the matched code's full bit length before consuming a symbol. This
keeps truncated symbols and stream errors from being accepted.
@github-actions github-actions Bot added the 👀 pr-needs-review PR needs review from a maintainer or community member label Oct 11, 2026
@ADKaster

Copy link
Copy Markdown
Member

How does this targeted fix compare to the comprehensive refactor by @DanShaders in #24567 ?

Nico and Tim seem to have discussed using the method you are using in this patch in the referenced issues, so it might be reasonable to use it -- I didn't look very closely

@Arthur031221

Copy link
Copy Markdown
Author

This fixes the synchronous path unzip uses. Both peek past EOF, then read the matched symbol's actual bit length. #24567 adds async DEFLATE/zlib decoders and integrates async zlib into HTTP, but leaves the existing synchronous Huffman overload unchanged. On this PR's current head, TestDeflate and TestBitStream pass (21 and 12 cases), including the short-tail input 4b4c4a4ec48600, truncated-symbol cases, and read-error propagation.

@Arthur031221

Copy link
Copy Markdown
Author

Is there anything else you need from me on this one?

@DanShaders

Copy link
Copy Markdown
Contributor

You should probably advertise this on #code-review in Discord.

(Not sure what exactly Andrew wanted me to look at here)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

👀 pr-needs-review PR needs review from a maintainer or community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CanonicalCode::read_symbol()'s peeking doesn't work with UnsatisfiableReadBehavior::Reject

3 participants