Add roaring bitmap to data structures in storage

[Ulimo]

This data structure allows saving bitmaps in a smaller size on disk which will be utilized in the new persistent storage solution when required to save boolean values.

[Copilot]

Pull request overview

This PR introduces a new Roaring Bitmap data structure under FlowtideDotNet.Storage.DataStructures, intended to provide a compact on-disk representation for boolean/bitset-like data in the upcoming persistent storage solution.

Changes:

• Added RoaringBitmap with backing RoaringArray and container types (ArrayContainer, BitmapContainer) plus enumerators.
• Added initial serialization helpers (Utils.WriteInt/WriteUshort) and a draft RoaringArray.Serialize API.
• Added basic unit tests for Add/Contains.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
tests/FlowtideDotNet.Storage.Tests/DataStructures/RoaringBitmapTests.cs	Adds initial unit tests for roaring bitmap behavior.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/Utils.cs	Adds bit helpers + buffer writer primitives; includes bitmap-to-array helper.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/RoaringBitmapEnumerator.cs	Adds an IEnumerator<int> implementation over roaring bitmap containers.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/RoaringBitmap.cs	Adds the main RoaringBitmap type (Add/Contains/Remove/Count) and a stub Serialize.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/RoaringArray.cs	Adds the high/low container array structure and a draft serialization routine.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/IContainerEnumerator.cs	Adds a small internal enumerator interface for container iteration.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/Container.cs	Adds base container abstraction for roaring bitmap containers.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/BitmapContainerEnumerator.cs	Adds iteration over bitmap containers.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/BitmapContainer.cs	Adds the bitmap-backed container implementation.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/ArrayContainerEnumerator.cs	Adds iteration over array containers.
src/FlowtideDotNet.Storage/DataStructures/RoaringBitMap/ArrayContainer.cs	Adds the array-backed container implementation and conversion to bitmap container.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Benchmark

Details

Benchmark suite	Current: c2aea78	Previous: 859330d	Ratio
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.InnerJoin	274064480 ns (± 7155024.177620459)	307785430 ns (± 9897277.576973714)	0.89
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.LeftJoin	427001090 ns (± 24359664.94056426)	461337240 ns (± 19940786.10567074)	0.93
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ProjectionAndNormalization	124922140 ns (± 13333853.194132095)	125533570 ns (± 10748358.533159066)	1.00
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.SumAggregation	136669610 ns (± 10065425.344155993)	138528520 ns (± 8580649.062577441)	0.99
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithMapAggregation	1946848370 ns (± 183256715.87583873)	1823440280 ns (± 56615189.59691922)	1.07
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.WindowSum	307038710 ns (± 28375509.419115867)	359591925 ns (± 5946342.432057349)	0.85
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithStructAggregation	1258771420 ns (± 78035210.68275383)	1507727890 ns (± 61734781.22500134)	0.83

This comment was automatically generated by workflow using github-action-benchmark.