feature: Masked() and MaskedToBuf() — apply bitmask to keys and merge colliding containers#32
Open
aliszka wants to merge 2 commits intooptimizations_vol4from
Open
feature: Masked() and MaskedToBuf() — apply bitmask to keys and merge colliding containers#32aliszka wants to merge 2 commits intooptimizations_vol4from
aliszka wants to merge 2 commits intooptimizations_vol4from
Conversation
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
52b58bb to
4a51c7d
Compare
Creates a bitmap backed by a caller-supplied byte slice instead of a heap allocation. The _ptr field retains a GC reference to the original []byte so the underlying memory is not collected while the bitmap is live. Uses clear() to zero the buffer before initialisation — consistent with the rest of the codebase after Memclr was removed. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Masked() applies a bitmask to every key and returns a new bitmap where keys that collapse to the same masked value have their containers merged via container-level OR operations. MaskedToBuf() is the same but uses a caller-provided byte slice as the underlying buffer, avoiding heap allocation when the buffer is large enough. Optimizations: - Pre-size key space upfront to avoid repeated key expansion and container memmoves during the merge loop. - When an inline merge fails and the replaced container is at the end of b.data, trim and regrow in place to eliminate dead container space. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Secrets | View in Orca |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Bitmap keys encode dimensions in the upper bits. Callers need to project a bitmap onto a sub-dimension by masking out the upper bits and merging containers whose keys collide under the mask — without materialising an intermediate bitmap per dimension.
Approach
Masked(mask uint64)iterates over all source containers, applies the mask to each key, and OR-merges containers that collapse to the same masked key into the result. The lowest 16 bits of the mask are forced to zero since keys never use them.Two optimisations in the merge loop keep allocations minimal:
expandConditionally(an, 0)) so the loop never shifts container data to make room for new keys.b.data, the slice is trimmed and regrown in place, avoiding a dead container fragment.MaskedToBuf(mask, buf)is the same operation but writes into a caller-supplied byte slice viaNewBitmapToBuf, avoiding heap allocation when the caller can reuse a pre-allocated buffer.Key areas for review
bitmap_opt.go:maskedInto— the trim-and-regrow path (b.data = b.data[:boff]) is only safe whenboff+len(bc) == len(b.data); verify the condition is tightbitmap_opt.go:maskedInto—expandConditionally(an, 0)pre-sizes for the worst case (no key collisions); with heavy collisions the key space is over-allocated, but this is bounded by the source key countRisks and mitigations
Testing
Tests cover nil input, no key overlap, key collision with OR merge (including multi-key collapse under zero mask), mask boundary conditions (zero mask, identity mask, low-16-bits ignored), and the buffer-swap logic in the OR accumulation loop for non-overlapping arrays, overlapping arrays, array-to-bitmap conversion, and three-container chains.