libunwind warning when compiling a wasmtime project against musl

[pimeys]

Test Case

Load any wasm component (wasip2) with wasmtime 21.0.1, if you compile the rust host against musl, a warning is printed to the terminal:

libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE

The component and the whole system works correctly, but the log is confusing and there's no way to tell if something is actually broken.

Expected Results

It should work as with builds against glibc, or builds on macos with no libunwind warning printed to the terminal.

Actual Results

I can't seem to find anything being broken really...

Versions and Environment

Wasmtime version or commit: 21.0.1

Operating system: nixos linux unstable

Architecture: x86_64-unknown-linux-musl for the host and wasm32-wasip1 for the guest.

Extra Info

I found issues from the past, such as

wasmerio/wasmer#2150
#1904

But nothing more recent...

[fitzgen]

Seems like a regression of #1904 and #1914.

FWIW, we do not rely on the system's libunwind for correctness, we only emit .eh_frame and call __register_frame for the benefit of tools like profilers. You can also turn this off via https://docs.rs/wasmtime/latest/wasmtime/struct.Config.html#method.native_unwind_info

IIRC, different libc implementations have different signatures for __register_frame, and this might be what we are running into here. If so, that isn't really something we can fix...

[fitzgen]

IIRC, different libc implementations have different signatures for __register_frame, and this might be what we are running into here. If so, that isn't really something we can fix...

Context:

wasmtime/crates/wasmtime/src/runtime/vm/sys/unix/unwind.rs

Lines 19 to 34 in 58b4d09

	/// There are two primary unwinders on Unix platforms: libunwind and libgcc.
	///
	/// Unfortunately their interface to `__register_frame` is different. The
	/// libunwind library takes a pointer to an individual FDE while libgcc takes a
	/// null-terminated list of FDEs. This means we need to know what unwinder
	/// is being used at runtime.
	///
	/// This detection is done currently by looking for a libunwind-specific symbol.
	/// This specific symbol was somewhat recommended by LLVM's
	/// "RTDyldMemoryManager.cpp" file which says:
	///
	/// > We use the presence of __unw_add_dynamic_fde to detect libunwind.
	///
	/// I'll note that there's also a different libunwind project at
	/// https://www.nongnu.org/libunwind/ but that doesn't appear to have
	/// `__register_frame` so I don't think that interacts with this.

[pimeys]

Is there any way to hide this message or will it break any functionality?

[fitzgen]

I believe it is the libunwind implementation printing the message, not Wasmtime.

You can turn off Wasmtime's generation of unwind info via https://docs.rs/wasmtime/latest/wasmtime/struct.Config.html#method.native_unwind_info which should make it so that the message stops printing.

[pimeys]

Yeah, this solved it for us. I don't really know should I close this issue or not. It's kind of still an issue if wanting to use the native unwind info, but also wanting to target musl.

[alexcrichton]

This cropping up again is a consequence of #8028 and how dlsym looks like it always returns NULL in a static build. This means that if you build a static binary, which I believe you're doing here, then it'll always think it's using libgcc which is incorrect.

So effectively this boils down to the mechanism used to detect libgcc-vs-libunwind. Another possible option is to use weak symbols but that's not possible in stable Rust so would require some C trickery to do that. I don't know of other options myself.

[bjorn3]

You can use weak symbols from inline asm, right? Would require a separate implementation for each architecture though.

[alexcrichton]

Oh? I had no idea! If that works that might be a reasonable way to go here

[kate-goldenring]

We are encountering this in Spin with Wasmtime v25.0.0: spinframework/spin#2889