Skip to content

fix some issues causing 137 oom#815

Merged
richox merged 1 commit intomasterfrom
dev-minox-fix
Feb 8, 2025
Merged

fix some issues causing 137 oom#815
richox merged 1 commit intomasterfrom
dev-minox-fix

Conversation

@richox
Copy link
Contributor

@richox richox commented Feb 6, 2025

remove arrow child allocator

introduce spark.blaze.onHeapSpill.memoryFraction

do not send empty batches

set ArrowWriter vector initial capacity to zero

coalesce scan input

fix ffi reader hanging

simplify logging

introduce spark.blaze.onHeapSpill.memoryFraction

do not send empty batches

set ArrowWriter vector initial capacity to zero

coalesce scan input

fix ffi reader hanging

simplify logging
@richox richox merged commit 5aa407e into master Feb 8, 2025
618 checks passed
@richox richox mentioned this pull request Apr 27, 2025
@cxzl25 cxzl25 deleted the dev-minox-fix branch September 5, 2025 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant