Preparing version v5.0.0

[richox]

New Feature

• Supports UDAF falling back.
• Supports native round-robin partitioner.
• Supports native range partitioner.
• Supports native WindowGroupLimitExec introduced in Spark-3.5.
• Supports SHJ falling back to SMJ when built side is too big.
• Fully supports to Apache Celeborn shuffle service.
• Initial supports to Apache Uniffle shuffle service.
• Initial supports to Apache Paimon datasource.

Improvement

Improved memory management in AggExec/SortMergeJoinExec, reducing number of OOMs.
Imptoved metric statistics.

Bug fixes

• Fixed inconsistent string to data casting.
• Fixed inconsistent bloom filter join when bloom filter is generated by Spark.
• Fixed incorrect sort ordering when writing tables with dynamic partitions.
• Fixed inconsistent sha2x functions.
• Fixed a lot of bugs those might lead to query failure, see What's Changed.

What's Changed

• (celeborn shuffle read) force disable decompression because compressi… by @richox in (celeborn shuffle read) force disable decompression because compressi… #897
• .gitignore file add target-docker folder ignore by @wsk1314zwr in .gitignore file add target-docker folder ignore #862
• Add Blaze icon and issue navigation In IDEA by @cxzl25 in Add Blaze icon and issue navigation In IDEA #699
• Add comma to line in README file by @xleoken in Add comma to line in README file #778
• Add support of Apache Uniffle for remote shuffle service by @zuston in Add support of Apache Uniffle for remote shuffle service #796
• Add support of building native with --features by @zuston in Add support of building native with --features #797
• Add support of mac aarch64 for tpcds data generator by @zuston in Add support of mac aarch64 for tpcds data generator #792
• Add support of memory profile by @zuston in Add support of memory profile #798
• Add support of pprof dump for rust execution by @zuston in Add support of pprof dump for rust execution #793
• Apply scalafix removeUnusedImports by @cxzl25 in Apply scalafix removeUnusedImports #960
• Apply spotless by @cxzl25 in Apply spotless #728
• Automatic cancel previous CI tests when newly commit comes for per PR by @zuston in Automatic cancel previous CI tests when newly commit comes for per PR #794
• Automatically use the protoc version downloaded by the maven plugin by @cxzl25 in Automatically use the protoc version downloaded by the maven plugin #702
• Avoid warning log No such type of ValidateSparkPlan by @cxzl25 in Avoid warning log No such type of ValidateSparkPlan #948
• Bump Paimon version from 0.9.0 to 1.0.0 by @harveyyue in Bump Paimon version from 0.9.0 to 1.0.0 #751
• Bump async-trait from 0.1.83 to 0.1.84 by @dependabot in Bump async-trait from 0.1.83 to 0.1.84 #740
• Bump async-trait from 0.1.84 to 0.1.85 by @dependabot in Bump async-trait from 0.1.84 to 0.1.85 #746
• Bump async-trait from 0.1.85 to 0.1.86 by @dependabot in Bump async-trait from 0.1.85 to 0.1.86 #810
• Bump async-trait from 0.1.86 to 0.1.87 by @dependabot in Bump async-trait from 0.1.86 to 0.1.87 #865
• Bump async-trait from 0.1.87 to 0.1.88 by @dependabot in Bump async-trait from 0.1.87 to 0.1.88 #887
• Bump bigdecimal from 0.4.7 to 0.4.8 by @dependabot in Bump bigdecimal from 0.4.7 to 0.4.8 #931
• Bump bytes from 1.10.0 to 1.10.1 by @dependabot in Bump bytes from 1.10.0 to 1.10.1 #864
• Bump bytes from 1.9.0 to 1.10.0 by @dependabot in Bump bytes from 1.9.0 to 1.10.0 #811
• Bump bytesize from 1.3.0 to 1.3.2 by @dependabot in Bump bytesize from 1.3.0 to 1.3.2 #819
• Bump bytesize from 1.3.2 to 2.0.0 by @dependabot in Bump bytesize from 1.3.2 to 2.0.0 #847
• Bump bytesize from 2.0.0 to 2.0.1 by @dependabot in Bump bytesize from 2.0.0 to 2.0.1 #859
• Bump foldhash from 0.1.3 to 0.1.4 by @dependabot in Bump foldhash from 0.1.3 to 0.1.4 #710
• Bump foldhash from 0.1.4 to 0.1.5 by @dependabot in Bump foldhash from 0.1.4 to 0.1.5 #886
• Bump itertools from 0.13.0 to 0.14.0 by @dependabot in Bump itertools from 0.13.0 to 0.14.0 #733
• Bump jemalloc_pprof from 0.6.0 to 0.7.0 by @dependabot in Bump jemalloc_pprof from 0.6.0 to 0.7.0 #863
• Bump log from 0.4.22 to 0.4.25 by @dependabot in Bump log from 0.4.22 to 0.4.25 #764
• Bump log from 0.4.25 to 0.4.26 by @dependabot in Bump log from 0.4.25 to 0.4.26 #846
• Bump log from 0.4.26 to 0.4.27 by @dependabot in Bump log from 0.4.26 to 0.4.27 #909
• Bump once_cell from 1.20.2 to 1.20.3 by @dependabot in Bump once_cell from 1.20.2 to 1.20.3 #816
• Bump once_cell from 1.20.3 to 1.21.0 by @dependabot in Bump once_cell from 1.20.3 to 1.21.0 #874
• Bump once_cell from 1.21.0 to 1.21.1 by @dependabot in Bump once_cell from 1.21.0 to 1.21.1 #882
• Bump once_cell from 1.21.1 to 1.21.2 by @dependabot in Bump once_cell from 1.21.1 to 1.21.2 #924
• Bump once_cell from 1.21.2 to 1.21.3 by @dependabot in Bump once_cell from 1.21.2 to 1.21.3 #929
• Bump poem from 1.3.59 to 3.1.6 by @dependabot in Bump poem from 1.3.59 to 3.1.6 #799
• Bump poem from 3.1.6 to 3.1.7 by @dependabot in Bump poem from 3.1.6 to 3.1.7 #845
• Bump poem from 3.1.7 to 3.1.8 by @dependabot in Bump poem from 3.1.7 to 3.1.8 #918
• Bump poem from 3.1.8 to 3.1.9 by @dependabot in Bump poem from 3.1.8 to 3.1.9 #946
• Bump prost from 0.13.3 to 0.13.4 by @dependabot in Bump prost from 0.13.3 to 0.13.4 #688
• Bump prost from 0.13.4 to 0.13.5 by @dependabot in Bump prost from 0.13.4 to 0.13.5 #823
• Bump rand from 0.8.5 to 0.9.0 by @dependabot in Bump rand from 0.8.5 to 0.9.0 #801
• Bump rand from 0.9.0 to 0.9.1 by @dependabot in Bump rand from 0.9.0 to 0.9.1 #956
• Bump serde from 1.0.216 to 1.0.217 by @dependabot in Bump serde from 1.0.216 to 1.0.217 #800
• Bump serde from 1.0.217 to 1.0.218 by @dependabot in Bump serde from 1.0.217 to 1.0.218 #842
• Bump serde from 1.0.217 to 1.0.219 by @dependabot in Bump serde from 1.0.217 to 1.0.219 #893
• Bump serde from 1.0.218 to 1.0.219 by @dependabot in Bump serde from 1.0.218 to 1.0.219 #867
• Bump smallvec from 2.0.0-alpha.10 to 2.0.0-alpha.11 by @dependabot in Bump smallvec from 2.0.0-alpha.10 to 2.0.0-alpha.11 #900
• Bump smallvec from 2.0.0-alpha.7 to 2.0.0-alpha.8 by @dependabot in Bump smallvec from 2.0.0-alpha.7 to 2.0.0-alpha.8 #692
• Bump smallvec from 2.0.0-alpha.8 to 2.0.0-alpha.9 by @dependabot in Bump smallvec from 2.0.0-alpha.8 to 2.0.0-alpha.9 #698
• Bump smallvec from 2.0.0-alpha.9 to 2.0.0-alpha.10 by @dependabot in Bump smallvec from 2.0.0-alpha.9 to 2.0.0-alpha.10 #770
• Bump sonic-rs from 0.3.16 to 0.3.17 by @dependabot in Bump sonic-rs from 0.3.16 to 0.3.17 #694
• Bump sonic-rs from 0.3.17 to 0.4.0 by @dependabot in Bump sonic-rs from 0.3.17 to 0.4.0 #875
• Bump sonic-rs from 0.4.0 to 0.4.1 by @dependabot in Bump sonic-rs from 0.4.0 to 0.4.1 #932
• Bump sonic-rs from 0.4.1 to 0.5.0 by @dependabot in Bump sonic-rs from 0.4.1 to 0.5.0 #943
• Bump tempfile from 3.14.0 to 3.15.0 by @dependabot in Bump tempfile from 3.14.0 to 3.15.0 #741
• Bump tempfile from 3.15.0 to 3.16.0 by @dependabot in Bump tempfile from 3.15.0 to 3.16.0 #802
• Bump tempfile from 3.16.0 to 3.17.1 by @dependabot in Bump tempfile from 3.16.0 to 3.17.1 #828
• Bump tempfile from 3.17.1 to 3.18.0 by @dependabot in Bump tempfile from 3.17.1 to 3.18.0 #866
• Bump tempfile from 3.18.0 to 3.19.0 by @dependabot in Bump tempfile from 3.18.0 to 3.19.0 #884
• Bump tempfile from 3.19.0 to 3.19.1 by @dependabot in Bump tempfile from 3.19.0 to 3.19.1 #894
• Bump tokio from 1.42.0 to 1.43.0 by @dependabot in Bump tokio from 1.42.0 to 1.43.0 #755
• Bump tokio from 1.43.0 to 1.44.1 by @dependabot in Bump tokio from 1.43.0 to 1.44.1 #883
• Bump tokio from 1.44.1 to 1.44.2 by @dependabot in Bump tokio from 1.44.1 to 1.44.2 #939
• Bump tonic-build from 0.12.3 to 0.13.0 by @dependabot in Bump tonic-build from 0.12.3 to 0.13.0 #917
• Bump uuid from 1.11.0 to 1.11.1 by @dependabot in Bump uuid from 1.11.0 to 1.11.1 #754
• Bump uuid from 1.11.1 to 1.12.0 by @dependabot in Bump uuid from 1.11.1 to 1.12.0 #765
• Bump uuid from 1.12.0 to 1.12.1 by @dependabot in Bump uuid from 1.12.0 to 1.12.1 #783
• Bump uuid from 1.12.1 to 1.13.1 by @dependabot in Bump uuid from 1.12.1 to 1.13.1 #814
• Bump uuid from 1.13.1 to 1.14.0 by @dependabot in Bump uuid from 1.13.1 to 1.14.0 #840
• Bump uuid from 1.14.0 to 1.15.1 by @dependabot in Bump uuid from 1.14.0 to 1.15.1 #853
• Bump zstd from 0.13.2 to 0.13.3 by @dependabot in Bump zstd from 0.13.2 to 0.13.3 #841
• Cast function should convert scientific notation to correct decimal value by @harveyyue in Cast function should convert scientific notation to correct decimal value #844
• Dev repartitioning by @gy11233 in Dev repartitioning #693
• Dev repartitioning by @gy11233 in Dev repartitioning #734
• Duplicated project schema will cause index out of bounds exception in orc_exec by @harveyyue in Duplicated project schema will cause index out of bounds exception in orc_exec #723
• Expect sha2 function result will be consistent with spark by @harveyyue in Expect sha2 function result will be consistent with spark #966
• Fallback cast date type to SparkUDFWrapper function by @harveyyue in Fallback cast date type to SparkUDFWrapper function #838
• Fix MacOS compile by @cxzl25 in Fix MacOS compile #724
• Fix OrcScan reads missing data column by @ASiegeLion in Fix OrcScan reads missing data column #716
• Fix orc map type entries field naming issue by @harveyyue in Fix orc map type entries field naming issue #732
• Fix result is empty when bloom filter is built by spark side for some situation by @xm0830 in Fix result is empty when bloom filter is built by spark side for some situation #911
• Fix some incorrect module name mapping in docker compose file by @harveyyue in Fix some incorrect module name mapping in docker compose file #709
• Fix the rootless-docker action failure when building the jar in github action by @zuston in Fix the rootless-docker action failure when building the jar in github action #813
• Fix udaf, add udaf enable conf by @gy11233 in Fix udaf, add udaf enable conf #851
• Move scala object HiveClientHelper from java to scala folder by @harveyyue in Move scala object HiveClientHelper from java to scala folder #822
• NativeConverters adds aggregate function return type by @richox in NativeConverters adds aggregate function return type #930
• ProjectExec adds cast automatically when data types not matched by @richox in ProjectExec adds cast automatically when data types not matched #916
• Remove bug_report unnecessary information by @cxzl25 in Remove bug_report unnecessary information #727
• Scan parquet/orc config by @cxzl25 in Scan parquet/orc config #949
• Support cast decimal data type with different precision and sale by @harveyyue in Support cast decimal data type with different precision and sale #839
• Support long type for ceil function by @harveyyue in Support long type for ceil function #825
• Support more native parquet scan metrics by @harveyyue in Support more native parquet scan metrics #876
• Support native scan hive paimon cow table by @harveyyue in Support native scan hive paimon cow table #708
• Support shuffle read records and total time metrics by @Flyangz in Support shuffle read records and total time metrics #873
• Supports UDAF and other aggregate functions not implemented by @gy11233 in Supports UDAF and other aggregate functions not implemented #848
• Update rust toolchain to latest nightly by @wForget in Update rust toolchain to latest nightly #861
• [BLAZE-287][FOLLOWUP] BlazeCelebornShuffleWriter should use mapped shuffle id for rerunning stage of fetch failure by @SteNicholas in [BLAZE-287][FOLLOWUP] BlazeCelebornShuffleWriter should use mapped shuffle id for rerunning stage of fetch failure #712
• [BLAZE-700] Minor nit fix for hyperlink by @merrily01 in [BLAZE-700] Minor nit fix for hyperlink #701
• [BLAZE-704] Specify name for spark ext function by @wForget in [BLAZE-704] Specify name for spark ext function #705
• [BLAZE-706] Fix year/month/day functions data type by @wForget in [BLAZE-706] Fix year/month/day functions data type #703
• [BLAZE-707][FOLLOWUP] NativePaimonTableScanExec should use shimed PartitionedFile and min partition number by @SteNicholas in [BLAZE-707][FOLLOWUP] NativePaimonTableScanExec should use shimed PartitionedFile and min partition number #713
• [BLAZE-725] Bump Spark from 3.5.3 to 3.5.4 by @SteNicholas in [BLAZE-725] Bump Spark from 3.5.3 to 3.5.4 #726
• [BLAZE-729] Fix a typo in the Shebang line of the shell script by @merrily01 in [BLAZE-729] Fix a typo in the Shebang line of the shell script #730
• [BLAZE-736] Write time should increment for mapperEnd in CelebornPart… by @HYBG-1126 in [BLAZE-736] Write time should increment for mapperEnd in CelebornPart… #739
• [BLAZE-744] Bump Celeborn version from 0.5.2 to 0.5.3 by @SteNicholas in [BLAZE-744] Bump Celeborn version from 0.5.2 to 0.5.3 #745
• [BLAZE-747] Enhance the ArrowFFIExporter.exportNextBatch method to execute conditionally by @merrily01 in [BLAZE-747] Enhance the ArrowFFIExporter.exportNextBatch method to execute conditionally #748
• [BLAZE-747][FOLLOW-UP] Fix user changed in FFI NextBatch by @Flyangz in [BLAZE-747][FOLLOW-UP] Fix user changed in FFI NextBatch #769
• [BLAZE-760] Fallback shuffle exchange when range partitioning with unsupported type by @wForget in [BLAZE-760] Fallback shuffle exchange when range partitioning with unsupported type #761
• [BLAZE-762] Return null when log function input is negative by @wForget in [BLAZE-762] Return null when log function input is negative #763
• [BLAZE-773] Support long type for floor function by @wForget in [BLAZE-773] Support long type for floor function #774
• [BLAZE-775] Support float type for sum function by @wForget in [BLAZE-775] Support float type for sum function #776
• [BLAZE-786] Mark big decimal value convertion as unsupported by @wForget in [BLAZE-786] Mark big decimal value convertion as unsupported #787
• [BLAZE-790] Support LZ4_RAW compression codec for parquet by @SteNicholas in [BLAZE-790] Support LZ4_RAW compression codec for parquet #791
• [BLAZE-803] Support statistics of ExecutionPlan for LimitExec by @SteNicholas in [BLAZE-803] Support statistics of ExecutionPlan for LimitExec #804
• [BLAZE-805] Support statistics of ExecutionPlan for SortExec by @SteNicholas in [BLAZE-805] Support statistics of ExecutionPlan for SortExec #807
• [BLAZE-808] Support statistics of ExecutionPlan for WindowExec by @SteNicholas in [BLAZE-808] Support statistics of ExecutionPlan for WindowExec #809
• [BLAZE-877] Bump Celeborn version from 0.5.3 to 0.5.4 by @SteNicholas in [BLAZE-877] Bump Celeborn version from 0.5.3 to 0.5.4 #878
• [BLAZE-879] Bump Spark from 3.5.4 to 3.5.5 by @SteNicholas in [BLAZE-879] Bump Spark from 3.5.4 to 3.5.5 #881
• [BLAZE-891] Remove stop interface of RssPartitionWriterBase by @SteNicholas in [BLAZE-891] Remove stop interface of RssPartitionWriterBase #892
• [BLAZE-895] Bump Paimon from 1.0.0 to 1.0.1 by @SteNicholas in [BLAZE-895] Bump Paimon from 1.0.0 to 1.0.1 #896
• [BLAZE-902] Fix UnifflePartitionWriter invoke ShuffleWriteMetricsReporter#incWriteTime with nano seconds by @SteNicholas in [BLAZE-902] Fix UnifflePartitionWriter invoke ShuffleWriteMetricsReporter#incWriteTime with nano seconds #903
• [BLAZE-905] Bytes written should increment in UnifflePartitionWriter#write by @SteNicholas in [BLAZE-905] Bytes written should increment in UnifflePartitionWriter#write #906
• [BLAZE-941] BlazeCelebornShuffleReader should add batch open stream time to fetch wait time by @SteNicholas in [BLAZE-941] BlazeCelebornShuffleReader should add batch open stream time to fetch wait time #942
• add childOrderingRequired tag to DataWritingCommandExec by @richox in add childOrderingRequired tag to DataWritingCommandExec #963
• add conf: spark.blaze.enable.scan.parquet/orc by @richox in add conf: spark.blaze.enable.scan.parquet/orc #944
• add expr string to SparkUDFWrapper by @lihao712 in add expr string to SparkUDFWrapper #967
• add malloc_conf for memory profiling by @Flyangz in add malloc_conf for memory profiling #836
• assert_eq key_rows sorted_row_indices by @cxzl25 in assert_eq key_rows sorted_row_indices #954
• bug fixes by @richox in bug fixes #717
• bug fixes by @richox in bug fixes #777
• clean rss shuffle writer api by @richox in clean rss shuffle writer api #820
• close inactive issues by @richox in close inactive issues #738
• code refactoring and bug fixes by @richox in https://github..com/kwai/blaze/pull/952
• complete UDAF fallback implementation by @richox in complete UDAF fallback implementation #888
• convert scalar value using arrow ipc by @richox in convert scalar value using arrow ipc #938
• feat(spill): Align with the multi IO compression codec in spill by @zuston in feat(spill): Align with the multi IO compression codec in spill #657
• feat: Activate symbolize feature for heap profile by @zuston in feat: Activate symbolize feature for heap profile #937
• fix NPE while getting spill buf metrics by @richox in fix NPE while getting spill buf metrics #904
• fix SortExec error when sort key exprs are empty by @richox in fix SortExec error when sort key exprs are empty #958
• fix UDAF fallbacking with literal params by @richox in fix UDAF fallbacking with literal params #922
• fix agg failure: index out of bounds by @richox in fix agg failure: index out of bounds #899
• fix bloom_filter_might_contain + literal params by @richox in fix bloom_filter_might_contain + literal params #914
• fix build error and code style by @wForget in fix build error and code style #781
• fix celeborn shuffle writer memory leaking by @richox in fix celeborn shuffle writer memory leaking #889
• fix ci: add --all-opens for supporting jdk17 by @richox in fix ci: add --all-opens for supporting jdk17 #758
• fix ci: cache spark binaries by @richox in fix ci: cache spark binaries #696
• fix ci: trigger ci when opening PR by @richox in fix ci: trigger ci when opening PR #711
• fix ci: trigger ci when opening/changing PR by @richox in fix ci: trigger ci when opening/changing PR #714
• fix ci: update to actions/upload-artifact@v4 by @richox in fix ci: update to actions/upload-artifact@v4 #756
• fix ci: update to actions/upload-artifact@v4 by @richox in fix ci: update to actions/upload-artifact@v4 #757
• fix ci: use cached spark-bin directory to walk around permission denied issue by @richox in fix ci: use cached spark-bin directory to walk around permission denied issue #766
• fix ci: use huaweicloud mirror to download spark binaries by @richox in fix ci: use huaweicloud mirror to download spark binaries #742
• fix error when bloom filter is null by @richox in fix error when bloom filter is null #925
• fix error when copying BlazeColumnarArray by @richox in fix error when copying BlazeColumnarArray #926
• fix error when subquery is not finished by @richox in fix error when subquery is not finished #928
• fix get_indexed_field nullable error by @lihao712 in fix get_indexed_field nullable error #968
• fix hanging in corner case: ArrowFFIExporter implements AutoCloseable by @richox in fix hanging in corner case: ArrowFFIExporter implements AutoCloseable #831
• fix http server startup by @Flyangz in fix http server startup #834
• fix imprecise ScalarValue memory size by @richox in fix imprecise ScalarValue memory size #962
• fix inconsistent string to date casting by @richox in fix inconsistent string to date casting #912
• fix incorrect WindowGroupLimit conversion by @lihao712 in fix incorrect WindowGroupLimit conversion #964
• fix incorrect expression conversion: Days should be DayOfMonth by @richox in fix incorrect expression conversion: Days should be DayOfMonth #691
• fix missing ReturnType in convertMoreAggregateExpr by @richox in fix missing ReturnType in convertMoreAggregateExpr #934
• fix mvn build helper by @richox in fix mvn build helper #735
• fix native shuffle reader with HeapByteBuffer by @richox in fix native shuffle reader with HeapByteBuffer #935
• fix performance issues by @richox in fix performance issues #743
• fix possible hanging in ffi reader by @richox in fix possible hanging in ffi reader #860
• fix possible panic in spawn_worker_thread_on_stream by @richox in fix possible panic in spawn_worker_thread_on_stream #849
• fix range repartitioning proto issue by @gy11233 in fix range repartitioning proto issue #752
• fix rss bug: forced spilling an unspillable memory consumer by @richox in fix rss bug: forced spilling an unspillable memory consumer #898
• fix some issues causing 137 oom by @richox in fix some issues causing 137 oom #815
• fix spark_xxhash64 + literal error by @richox in fix spark_xxhash64 + literal error #920
• fix test failures by @richox in fix test failures #720
• fix union error with empty inputs by @lihao712 in fix union error with empty inputs #959
• fix-ci: adjust memory configuration by @richox in fix-ci: adjust memory configuration #768
• fix-ci: pull_request_target -> pull_request by @richox in fix-ci: pull_request_target -> pull_request #782
• fix-ci: remote incorrect cache by @richox in fix-ci: remote incorrect cache #779
• fix-ci: rust fmt by @richox in fix-ci: rust fmt #780
• fix-ci: use specified jdk version by @richox in fix-ci: use specified jdk version #767
• get_array_mem_size() prefers capacity to len by @lihao712 in get_array_mem_size() prefers capacity to len #969
• get_json_object support blank space after '.' in path by @xm0830 in get_json_object support blank space after '.' in path #915
• improve MetricNode: fix metric missing in Union children by @richox in improve MetricNode: fix metric missing in Union children #832
• introduce spark version control with spark-version-annotation-macros,… by @richox in introduce spark version control with spark-version-annotation-macros,… #908
• keep same algorithm between put_long/put_binary and might_contain_long/might_contain_binary by @xm0830 in keep same algorithm between put_long/put_binary and might_contain_long/might_contain_binary #913
• minor fixes of OOM cases by @richox in minor fixes of OOM cases #817
• normalize shuffle write time to output io time by @richox in normalize shuffle write time to output io time #953
• optimize sort merge join and avoid oom by @lihao712 in optimize sort merge join and avoid oom #970
• refactor aggregate unfreeze_from_rows() and fix UDAF fallbacking error by @richox in refactor aggregate unfreeze_from_rows() and fix UDAF fallbacking error #940
• refactor rss shuffle writer, fix incorrect map status by @richox in refactor rss shuffle writer, fix incorrect map status #901
• remove eager shuffle reading by @richox in remove eager shuffle reading #858
• rewrite UnionExec and support auto type casting by @richox in rewrite UnionExec and support auto type casting #927
• set arrows default struct conflict policy to APPEND by @richox in set arrows default struct conflict policy to APPEND #933
• style check and reformat by @cxzl25 in style check and reformat #961
• support orc reading based on index by @Flyangz in support orc reading based on index #871
• support orc scan bytes metric by @Flyangz in support orc scan bytes metric #821
• supports WindowGroupLimitExec by @richox in supports WindowGroupLimitExec #957
• supports falling back hash join to sort merge join when hash table is too big by @richox in supports falling back hash join to sort merge join when hash table is too big #753
• use better aggregate OwnedKey construction by @richox in use better aggregate OwnedKey construction #784
• use separated thread in ffi exporter by @richox in use separated thread in ffi exporter #788

New Contributors

• @ASiegeLion made their first contribution in Fix OrcScan reads missing data column #716
• @Flyangz made their first contribution in [BLAZE-747][FOLLOW-UP] Fix user changed in FFI NextBatch #769
• @HYBG-1126 made their first contribution in [BLAZE-736] Write time should increment for mapperEnd in CelebornPart… #739
• @cxzl25 made their first contribution in Add Blaze icon and issue navigation In IDEA #699
• @gy11233 made their first contribution in Dev repartitioning #693
• @merrily01 made their first contribution in [BLAZE-700] Minor nit fix for hyperlink #701
• @wsk1314zwr made their first contribution in .gitignore file add target-docker folder ignore #862
• @xleoken made their first contribution in Add comma to line in README file #778
• @xm0830 made their first contribution in Fix result is empty when bloom filter is built by spark side for some situation #911

Full Changelog: v4.0.1...v5.0.0-preview1