feat: add closure support to the object compactor#13194
Draft
Kha wants to merge 4 commits intoleanprover:masterfrom
Draft
feat: add closure support to the object compactor#13194Kha wants to merge 4 commits intoleanprover:masterfrom
Kha wants to merge 4 commits intoleanprover:masterfrom
Conversation
This PR adds new incremental module serialization functions that save/load a single module at a time with explicit sharing via dep regions and compactor state, alongside the existing batch saveModuleDataParts API. Two sharing mechanisms: - CompactedRegion dep regions for import sharing (address range check with rc==0 fast-reject + binary search) - CompactorState for same-process chaining (pre-loaded obj_table)
Member
Author
|
!bench |
|
Benchmark results for 4e305f7 against 4786e08 are in. Significant changes detected! @Kha
Large changes (2🟥)
|
github-actions
bot
added
the
toolchain-available
Collaborator
|
Reference manual CI status:
|
mathlib-nightly-testing bot
pushed a commit
to leanprover-community/batteries
that referenced
this pull request
Mar 30, 2026
github-actions
bot
added
the
mathlib4-nightly-available
mathlib-nightly-testing bot
pushed a commit
to leanprover-community/mathlib4-nightly-testing
that referenced
this pull request
Mar 30, 2026
mathlib-lean-pr-testing
bot
added
the
builds-mathlib
|
Mathlib CI status (docs):
|
This adds the ability to compact and deserialize closures, which was previously rejected with `closures cannot be compacted`. Closure function pointers are preserved as raw pointers at save time, with a per-library relocation table in the v3 olean header extension for cross-invocation ASLR fixup. The library table is built once per compactor via platform-specific APIs (`dl_iterate_phdr` on Linux, `_dyld_*` on macOS, `EnumProcessModules` on Windows). When mmap succeeds and all relocation deltas are zero (same process), the fast path skips all fixups. When actual relocation is needed on mmap`d data, the region is copied to a writable buffer first.
Kha
added
the
release-ci
mathlib-nightly-testing bot
pushed a commit
to leanprover-community/batteries
that referenced
this pull request
Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the ability to compact and deserialize closures, which was previously rejected with "closures cannot be compacted". Closure function pointers are preserved as raw pointers at save time, with a per-library relocation table stored as a file trailer for cross-invocation ASLR fixup. The library table is built once per compactor via platform-specific APIs (
dl_iterate_phdron Linux,_dyld_*on macOS,EnumProcessModuleson Windows). Files without closures are unchanged (backward compatible via a reserved flag bit).For now it is assumed that serialization of closures is a special case affecting only smaller files, so when actual relocation is needed on
mmap'd data, the region is copied to a writable buffer first.