Remove usages of weak symbols#57523
Merged
topolarity merged 1 commit intoJuliaLang:masterfrom Feb 27, 2025
Merged
Conversation
topolarity
changed the title
Keno
requested changes
Feb 25, 2025
Member
Keno
left a comment
Keno
left a comment
There was a problem hiding this comment.
As discussed, NAK. This is for linking everything statically into one image and for when the dynamic linker is unavailable.
topolarity
force-pushed
the
ct/rm-weak-symbol
branch
2 times, most recently
from
661d228 to
39d55dc
Compare
Member
Author
|
Re-worked to support statically linking sysimage again (untested) We don't have official support for this in the Makefile, but it should be just a matter of filtering out the |
Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac. We should be able to lookup these symbols just fine in our `jl_exe_handle` as long as it was linked / compiled with `-rdynamic` Migrating away from weak symbols seems a good idea, given their uneven implementation across platforms.
topolarity
force-pushed
the
ct/rm-weak-symbol
branch
from
39d55dc to
c3dd2af
Compare
Keno
approved these changes
Feb 27, 2025
Member
Keno
left a comment
Keno
left a comment
There was a problem hiding this comment.
I think this is fine. It makes the build process a little less automagic, but this is deep arcana anyway, and I don't see any major downside to having to be explicit.
Keno
pushed a commit
that referenced
this pull request
Feb 28, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on #57523.
KristofferC
pushed a commit
that referenced
this pull request
May 5, 2025
Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac, since I believe we should be able to lookup these symbols just fine in our `jl_exe_handle` as long as it was linked / compiled with `-rdynamic` Migrating away from weak symbols seems a good idea, given their uneven implementation across platforms. (cherry picked from commit 4a6ada6)
53 tasks
gbaraldi
added a commit
that referenced
this pull request
May 5, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on #57523.
xal-0
pushed a commit
to xal-0/julia
that referenced
this pull request
Aug 18, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
xal-0
added a commit
that referenced
this pull request
Aug 20, 2025
commit d062060 Author: Sam Schweigel <sam.schweigel@juliahub.com> Date: Wed Aug 6 16:35:34 2025 -0700 System image compression with zstd Co-authored-by: Gabriel Baraldi <baraldigabriel@gmail.com> madvise with MADV_WILLNEED before accessing sysimage pages MADV_WILLNEED sysimage pages before checksumming or decompressing Even without compression, this gives about an 8% improvement in load times. Fix using jl_page_size before it is initialized Add zstd to ANALYSIS_DEPS now that it is used in staticdata.c Fix jl_page_size being uninitialized in jl_prefetch_system_image Add --compress-sysimage flag; use zstd-15 when enabled commit 56a6065 Author: Sam Schweigel <sam.schweigel@juliahub.com> Date: Mon Aug 18 13:27:13 2025 -0700 Janky zstd backport commit 4b731bb Author: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Date: Thu Feb 27 22:20:02 2025 -0500 staticdata: Refactor sysimage loading (#57542) Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on #57523.
xal-0
added a commit
to xal-0/julia
that referenced
this pull request
Aug 20, 2025
commit d062060 Author: Sam Schweigel <sam.schweigel@juliahub.com> Date: Wed Aug 6 16:35:34 2025 -0700 System image compression with zstd Co-authored-by: Gabriel Baraldi <baraldigabriel@gmail.com> madvise with MADV_WILLNEED before accessing sysimage pages MADV_WILLNEED sysimage pages before checksumming or decompressing Even without compression, this gives about an 8% improvement in load times. Fix using jl_page_size before it is initialized Add zstd to ANALYSIS_DEPS now that it is used in staticdata.c Fix jl_page_size being uninitialized in jl_prefetch_system_image Add --compress-sysimage flag; use zstd-15 when enabled commit 56a6065 Author: Sam Schweigel <sam.schweigel@juliahub.com> Date: Mon Aug 18 13:27:13 2025 -0700 Janky zstd backport commit 4b731bb Author: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Date: Thu Feb 27 22:20:02 2025 -0500 staticdata: Refactor sysimage loading (JuliaLang#57542) Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
xal-0
pushed a commit
to xal-0/julia
that referenced
this pull request
Aug 20, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
topolarity
added a commit
to topolarity/julia
that referenced
this pull request
Aug 20, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
topolarity
added a commit
to xal-0/julia
that referenced
this pull request
Aug 20, 2025
Introduce `jl_image_buf_t` to represent the in-memory details of an unparsed system image and clean-up several global variables and split loading logic in staticdata.c so that we get (almost) everything we need from the sysimage handle at once. This allows sysimage loading to be separated into three phases: 1. Lookup the raw sysimage buffer as a `jl_image_buf_t` 2. For .so sysimages, parse the sysimage and initialize JIT targets, etc. 3. Finally load the sysimage into a `jl_image_t` Care was taken to preserve the existing behavior of calling `jl_set_sysimg_so` to configure the sysimage before initializing Julia, although this is likely to be next on the chopping block in a follow-up. Dependent on JuliaLang#57523.
gbaraldi
pushed a commit
that referenced
this pull request
Sep 9, 2025
…ls (#59470) > Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac, since I believe we should be able to lookup these symbols just fine in our `jl_exe_handle` as long as it was linked / compiled with `-rdynamic` > > Migrating away from weak symbols seems a good idea, given their uneven implementation across platforms. (cherry picked from commit 4a6ada6) @gbaraldi asked if this could be backported separately from #59227. Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
59 tasks
DilumAluthge
removed
the
backport 1.11
DilumAluthge
added a commit
that referenced
this pull request
Nov 3, 2025
Backported PRs: - [x] #57523 <!-- Remove usages of weak symbols --> - [x] #58127 <!-- [DOC] Update installation docs: /downloads/ => /install/ --> - [x] #58202 <!-- [release-1.11] malloc: use jl_get_current_task to fix null check --> - [x] #58554 <!-- remove workaround for controlling terminal behavior in repl_cmd --> - [x] #58584 <!-- Make `Ptr` values static-show w/ type-information --> - [x] #59062 <!-- remove a testset from MMAP that might cause CI to now fail on Windows --> - [x] #59300 <!-- Update the developer docs to reflect the use of JuliaSyntax.jl --> - [x] #57604 <!-- `@nospecialize` for `string_index_err` --> - [x] #59329 <!-- aotcompile: destroy LLVM context after serializing combined module --> - [x] #59418 <!-- Fix startup when history file is bad --> - [x] #56890 <!-- Enable getting non-boxed LLVM type from Julia Type --> - [x] #59559 <!-- codegen: mark write barrier field load as volatile --> - [x] #59572 <!-- Only apply Base.Sort.SubArrayOptimization when iszero(v.offset1) --> Need manual backport: - [ ] #56329 <!-- loading: clean up more concurrency issues --> - [ ] #56956 <!-- Add "mea culpa" to foreign module assignment error. --> - [ ] #57035 <!-- linux: workaround to avoid deadlock inside dl_iterate_phdr in glibc --> - [ ] #57089 <!-- Block thread from receiving profile signal with stackwalk lock --> - [ ] #57249 <!-- restore non-freebsd-unix fix for profiling --> - [ ] #58011 <!-- Remove try-finally scope from `@time_imports` `@trace_compile` `@trace_dispatch` --> - [ ] #58062 <!-- remove unnecessary edge from `exp_impl` to `pow` --> - [ ] #58157 <!-- add showing a string to REPL precompile workload --> - [ ] #58209 <!-- Specialize `one` for the `SizedArray` test helper --> - [ ] #58356 <!-- codegen: remove readonly from abstract type calling convention --> - [ ] #58415 <!-- [REPL] more reliable extension loading --> - [ ] #58510 <!-- Don't filter `Core` methods from newly-inferred list --> - [ ] #58110 <!-- relax dispatch for the `IteratorSize` method for `Generator` --> - [ ] #58965 <!-- Fix `hygienic-scope`s in inner macro expansions --> - [ ] #58971 <!-- Fix alignment of failed precompile jobs on CI --> - [ ] #59066 <!-- build: Also pass -fno-strict-aliasing for C++ --> - [ ] #59428 <!-- Correctly set the variant bits of uuid1 --> Contains multiple commits, manual intervention needed: - [ ] #55877 <!-- fix FileWatching designs and add workaround for a stat bug on Apple --> - [ ] #56755 <!-- docs: fix scope type of a `struct` to hard --> - [ ] #57809 <!-- Fix fptrunc Float64 -> Float16 rounding through Float32 --> - [ ] #57398 <!-- Make remaining float intrinsics require float arguments --> - [ ] #56351 <!-- Fix `--project=@script` when outside script directory --> - [ ] #57129 <!-- clarify that time_ns is monotonic --> - [ ] #58134 <!-- Note annotated string API is experimental in Julia 1.11 in HISTORY.md --> - [ ] #58401 <!-- check that hashing of types does not foreigncall (`jl_type_hash` is concrete evaluated) --> - [ ] #58435 <!-- Fix layout flags for types that have oddly sized primitive type fields --> - [ ] #58483 <!-- Fix tbaa usage when storing into heap allocated immutable structs --> - [ ] #58512 <!-- Make more types jl_static_show readably --> - [ ] #58012 <!-- Re-enable tab completion of kwargs for large method tables --> - [ ] #58683 <!-- Add 0 predecessor to entry basic block and handle it in inlining --> - [ ] #59112 <!-- Add builtin function name to add methods error --> - [ ] #56823 <!-- Make version of opaque closure constructor in world --> - [ ] #59467 <!-- `CoreLogging`: prevent some `Annotated*`-related instability --> Non-merged PRs with backport label: - [ ] #59450 <!-- Propagate Addrspaces: fix lift of memset --> - [ ] #58848 <!-- Set array size only when safe to do so --> - [ ] #55958 <!-- also redirect JL_STDERR etc. when redirecting to devnull --> - [ ] #55956 <!-- Make threadcall gc safe --> - [ ] #55534 <!-- Set stdlib sources as read-only during installation --> - [ ] #55499 <!-- propagate the terminal's `displaysize` to the `IOContext` used by the REPL --> - [ ] #55458 <!-- Allow for generically extracting unannotated string --> - [ ] #55457 <!-- Make AnnotateChar equality consider annotations --> - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` --> --------- Co-authored-by: Dilum Aluthge <dilum@aluthge.com> Co-authored-by: Sam Schweigel <sam.schweigel@juliahub.com> Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Co-authored-by: Jameson Nash <vtjnash@gmail.com> Co-authored-by: Adam Wheeler <adamwhlr@gmail.com> Co-authored-by: Neven Sajko <4944410+nsajko@users.noreply.github.com> Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com> Co-authored-by: William Moses <gh@wsmoses.com> Co-authored-by: Valentin Churavy <v.churavy@gmail.com> Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
Merged
Merged
57 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inspired by a bug I hit on MinGW recently with weak symbol resolution. I'm not sure why we started using these in 70000ac, since I believe we should be able to lookup these symbols just fine in our
jl_exe_handleas long as it was linked / compiled with-rdynamicMigrating away from weak symbols seems a good idea, given their uneven implementation across platforms.