Skip to content

Commit bd2ea90

Browse files
authored
Define garbage collection rooting APIs (#8011)
* Define garbage collection rooting APIs Rooting prevents GC objects from being collected while they are actively being used. We have a few sometimes-conflicting goals with our GC rooting APIs: 1. Safety: It should never be possible to get a use-after-free bug because the user misused the rooting APIs, the collector "mistakenly" determined an object was unreachable and collected it, and then the user tried to access the object. This is our highest priority. 2. Moving GC: Our rooting APIs should moving collectors (such as generational and compacting collectors) where an object might get relocated after a collection and we need to update the GC root's pointer to the moved object. This means we either need cooperation and internal mutability from individual GC roots as well as the ability to enumerate all GC roots on the native Rust stack, or we need a level of indirection. 3. Performance: Our rooting APIs should generally be as low-overhead as possible. They definitely shouldn't require synchronization and locking to create, access, and drop GC roots. 4. Ergonomics: Our rooting APIs should be, if not a pleasure, then at least not a burden for users. Additionally, the API's types should be `Sync` and `Send` so that they work well with async Rust. For example, goals (3) and (4) are in conflict when we think about how to support (2). Ideally, for ergonomics, a root would automatically unroot itself when dropped. But in the general case that requires holding a reference to the store's root set, and that root set needs to be held simultaneously by all GC roots, and they each need to mutate the set to unroot themselves. That implies `Rc<RefCell<...>>` or `Arc<Mutex<...>>`! The former makes the store and GC root types not `Send` and not `Sync`. The latter imposes synchronization and locking overhead. So we instead make GC roots indirect and require passing in a store context explicitly to unroot in the general case. This trades worse ergonomics for better performance and support for moving GC and async Rust. Okay, with that out of the way, this module provides two flavors of rooting API. One for the common, scoped lifetime case, and another for the rare case where we really need a GC root with an arbitrary, non-LIFO/non-scoped lifetime: 1. `RootScope` and `Rooted<T>`: These are used for temporarily rooting GC objects for the duration of a scope. Upon exiting the scope, they are automatically unrooted. The internal implementation takes advantage of the LIFO property inherent in scopes, making creating and dropping `Rooted<T>`s and `RootScope`s super fast and roughly equivalent to bump allocation. This type is vaguely similar to V8's [`HandleScope`]. [`HandleScope`]: https://v8.github.io/api/head/classv8_1_1HandleScope.html Note that `Rooted<T>` can't be statically tied to its context scope via a lifetime parameter, unfortunately, as that would allow the creation and use of only one `Rooted<T>` at a time, since the `Rooted<T>` would take a borrow of the whole context. This supports the common use case for rooting and provides good ergonomics. 2. `ManuallyRooted<T>`: This is the fully general rooting API used for holding onto non-LIFO GC roots with arbitrary lifetimes. However, users must manually unroot them. Failure to manually unroot a `ManuallyRooted<T>` before it is dropped will result in the GC object (and everything it transitively references) leaking for the duration of the `Store`'s lifetime. This type is roughly similar to SpiderMonkey's [`PersistentRooted<T>`], although they avoid the manual-unrooting with internal mutation and shared references. (Our constraints mean we can't do those things, as mentioned explained above.) [`PersistentRooted<T>`]: http://devdoc.net/web/developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS::PersistentRooted.html At the end of the day, both `Rooted<T>` and `ManuallyRooted<T>` are just tagged indices into the store's `RootSet`. This indirection allows working with Rust's borrowing discipline (we use `&mut Store` to represent mutable access to the GC heap) while still allowing rooted references to be moved around without tying up the whole store in borrows. Additionally, and crucially, this indirection allows us to update the *actual* GC pointers in the `RootSet` and support moving GCs (again, as mentioned above). * Reorganize GC-related submodules in `wasmtime-runtime` * Reorganize GC-related submodules in `wasmtime` * Use `Into<StoreContext[Mut]<'a, T>` for `Externref::data[_mut]` methods * Run rooting tests under MIRI * Make `into_abi` take an `AutoAssertNoGc` * Don't use atomics to update externref ref counts anymore * Try to make lifetimes/safety more-obviously correct Remove some transmute methods, assert that `VMExternRef`s are the only valid `VMGcRef`, etc. * Update extenref constructor examples * Make `GcRefImpl::transmute_ref` a non-default trait method * Make inline fast paths for GC LIFO scopes * Make `RootSet::unroot_gc_ref` an `unsafe` function * Move Hash and Eq for Rooted, move to impl methods * Remove type parameter from `AutoAssertNoGc` Just wrap a `&mut StoreOpaque` directly. * Make a bunch of internal `ExternRef` methods that deal with raw `VMGcRef`s take `AutoAssertNoGc` instead of `StoreOpaque` * Fix compile after rebase * rustfmt * revert unrelated egraph changes * Fix non-gc build * Mark `AutoAssertNoGc` methods inline * review feedback * Temporarily remove externref support from the C API Until we can add proper GC rooting. * Remove doxygen reference to temp deleted function * Remove need to `allow(private_interfaces)` * Fix call benchmark compilation
1 parent ebbfc90 commit bd2ea90

50 files changed

Lines changed: 3617 additions & 1250 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

benches/call.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ fn bench_host_to_wasm<Params, Results>(
189189
let mut space = vec![ValRaw::i32(0); params.len().max(results.len())];
190190
b.iter(|| unsafe {
191191
for (i, param) in params.iter().enumerate() {
192-
space[i] = param.to_raw(&mut *store);
192+
space[i] = param.to_raw(&mut *store).unwrap();
193193
}
194194
untyped
195195
.call_unchecked(&mut *store, space.as_mut_ptr(), space.len())
@@ -348,7 +348,7 @@ fn wasm_to_host(c: &mut Criterion) {
348348
Val::I64(0) => {}
349349
_ => unreachable!(),
350350
}
351-
space[0] = Val::F32(0).to_raw(&mut caller);
351+
space[0] = Val::F32(0).to_raw(&mut caller).unwrap();
352352
Ok(())
353353
})
354354
.unwrap();

crates/c-api/include/wasmtime.h

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -152,9 +152,7 @@
152152
* provided access to it. For example in a host function created with
153153
* #wasmtime_func_new you can use #wasmtime_context_t in the host function
154154
* callback. This is because an argument, a #wasmtime_caller_t, provides access
155-
* to #wasmtime_context_t. On the other hand a destructor passed to
156-
* #wasmtime_externref_new, however, cannot use a #wasmtime_context_t because
157-
* it was not provided access to one. Doing so may lead to memory unsafety.
155+
* to #wasmtime_context_t.
158156
*
159157
* ### Stores
160158
*

crates/c-api/include/wasmtime/val.h

Lines changed: 7 additions & 98 deletions
Original file line numberDiff line numberDiff line change
@@ -14,81 +14,6 @@
1414
extern "C" {
1515
#endif
1616

17-
/**
18-
* \typedef wasmtime_externref_t
19-
* \brief Convenience alias for #wasmtime_externref
20-
*
21-
* \struct wasmtime_externref
22-
* \brief A host-defined un-forgeable reference to pass into WebAssembly.
23-
*
24-
* This structure represents an `externref` that can be passed to WebAssembly.
25-
* It cannot be forged by WebAssembly itself and is guaranteed to have been
26-
* created by the host.
27-
*/
28-
typedef struct wasmtime_externref wasmtime_externref_t;
29-
30-
/**
31-
* \brief Create a new `externref` value.
32-
*
33-
* Creates a new `externref` value wrapping the provided data, returning the
34-
* pointer to the externref.
35-
*
36-
* \param data the host-specific data to wrap
37-
* \param finalizer an optional finalizer for `data`
38-
*
39-
* When the reference is reclaimed, the wrapped data is cleaned up with the
40-
* provided `finalizer`.
41-
*
42-
* The returned value must be deleted with #wasmtime_externref_delete
43-
*/
44-
WASM_API_EXTERN wasmtime_externref_t *
45-
wasmtime_externref_new(void *data, void (*finalizer)(void *));
46-
47-
/**
48-
* \brief Get an `externref`'s wrapped data
49-
*
50-
* Returns the original `data` passed to #wasmtime_externref_new. It is required
51-
* that `data` is not `NULL`.
52-
*/
53-
WASM_API_EXTERN void *wasmtime_externref_data(wasmtime_externref_t *data);
54-
55-
/**
56-
* \brief Creates a shallow copy of the `externref` argument, returning a
57-
* separately owned pointer (increases the reference count).
58-
*/
59-
WASM_API_EXTERN wasmtime_externref_t *
60-
wasmtime_externref_clone(wasmtime_externref_t *ref);
61-
62-
/**
63-
* \brief Decrements the reference count of the `ref`, deleting it if it's the
64-
* last reference.
65-
*/
66-
WASM_API_EXTERN void wasmtime_externref_delete(wasmtime_externref_t *ref);
67-
68-
/**
69-
* \brief Converts a raw `externref` value coming from #wasmtime_val_raw_t into
70-
* a #wasmtime_externref_t.
71-
*
72-
* Note that the returned #wasmtime_externref_t is an owned value that must be
73-
* deleted via #wasmtime_externref_delete by the caller if it is non-null.
74-
*/
75-
WASM_API_EXTERN wasmtime_externref_t *
76-
wasmtime_externref_from_raw(wasmtime_context_t *context, void *raw);
77-
78-
/**
79-
* \brief Converts a #wasmtime_externref_t to a raw value suitable for storing
80-
* into a #wasmtime_val_raw_t.
81-
*
82-
* Note that the returned underlying value is not tracked by Wasmtime's garbage
83-
* collector until it enters WebAssembly. This means that a GC may release the
84-
* context's reference to the raw value, making the raw value invalid within the
85-
* context of the store. Do not perform a GC between calling this function and
86-
* passing it to WebAssembly.
87-
*/
88-
WASM_API_EXTERN void *
89-
wasmtime_externref_to_raw(wasmtime_context_t *context,
90-
const wasmtime_externref_t *ref);
91-
9217
/// \brief Discriminant stored in #wasmtime_val::kind
9318
typedef uint8_t wasmtime_valkind_t;
9419
/// \brief Value of #wasmtime_valkind_t meaning that #wasmtime_val_t is an i32
@@ -104,9 +29,6 @@ typedef uint8_t wasmtime_valkind_t;
10429
/// \brief Value of #wasmtime_valkind_t meaning that #wasmtime_val_t is a
10530
/// funcref
10631
#define WASMTIME_FUNCREF 5
107-
/// \brief Value of #wasmtime_valkind_t meaning that #wasmtime_val_t is an
108-
/// externref
109-
#define WASMTIME_EXTERNREF 6
11032

11133
/// \brief A 128-bit value representing the WebAssembly `v128` type. Bytes are
11234
/// stored in little-endian order.
@@ -136,11 +58,6 @@ typedef union wasmtime_valunion {
13658
/// If this value represents a `ref.null func` value then the `store_id` field
13759
/// is set to zero.
13860
wasmtime_func_t funcref;
139-
/// Field used if #wasmtime_val_t::kind is #WASMTIME_EXTERNREF
140-
///
141-
/// If this value represents a `ref.null extern` value then this pointer will
142-
/// be `NULL`.
143-
wasmtime_externref_t *externref;
14461
/// Field used if #wasmtime_val_t::kind is #WASMTIME_V128
14562
wasmtime_v128 v128;
14663
} wasmtime_valunion_t;
@@ -186,14 +103,6 @@ typedef union wasmtime_val_raw {
186103
///
187104
/// Note that this field is always stored in a little-endian format.
188105
void *funcref;
189-
/// Field for when this val is a WebAssembly `externref` value.
190-
///
191-
/// If this is set to 0 then it's a null externref, otherwise this must be
192-
/// passed to `wasmtime_externref_from_raw` to determine the
193-
/// `wasmtime_externref_t`.
194-
///
195-
/// Note that this field is always stored in a little-endian format.
196-
void *externref;
197106
} wasmtime_val_raw_t;
198107

199108
/**
@@ -203,11 +112,9 @@ typedef union wasmtime_val_raw {
203112
* \union wasmtime_val
204113
* \brief Container for different kinds of wasm values.
205114
*
206-
* Note that this structure may contain an owned value, namely
207-
* #wasmtime_externref_t, depending on the context in which this is used. APIs
208-
* which consume a #wasmtime_val_t do not take ownership, but APIs that return
209-
* #wasmtime_val_t require that #wasmtime_val_delete is called to deallocate
210-
* the value.
115+
* APIs which consume a #wasmtime_val_t do not take ownership, but APIs that
116+
* return #wasmtime_val_t require that #wasmtime_val_delete is called to
117+
* deallocate the value.
211118
*/
212119
typedef struct wasmtime_val {
213120
/// Discriminant of which field of #of is valid.
@@ -222,12 +129,14 @@ typedef struct wasmtime_val {
222129
* Note that this only deletes the contents, not the memory that `val` points to
223130
* itself (which is owned by the caller).
224131
*/
225-
WASM_API_EXTERN void wasmtime_val_delete(wasmtime_val_t *val);
132+
WASM_API_EXTERN void wasmtime_val_delete(wasmtime_context_t *context,
133+
wasmtime_val_t *val);
226134

227135
/**
228136
* \brief Copies `src` into `dst`.
229137
*/
230-
WASM_API_EXTERN void wasmtime_val_copy(wasmtime_val_t *dst,
138+
WASM_API_EXTERN void wasmtime_val_copy(wasmtime_context_t *context,
139+
wasmtime_val_t *dst,
231140
const wasmtime_val_t *src);
232141

233142
#ifdef __cplusplus

crates/c-api/src/async.rs

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,12 @@ async fn invoke_c_async_callback<'a>(
113113
let mut hostcall_val_storage = mem::take(&mut caller.data_mut().hostcall_val_storage);
114114
debug_assert!(hostcall_val_storage.is_empty());
115115
hostcall_val_storage.reserve(params.len() + results.len());
116-
hostcall_val_storage.extend(params.iter().cloned().map(|p| wasmtime_val_t::from_val(p)));
116+
hostcall_val_storage.extend(
117+
params
118+
.iter()
119+
.cloned()
120+
.map(|p| wasmtime_val_t::from_val(&mut caller, p)),
121+
);
117122
hostcall_val_storage.extend((0..results.len()).map(|_| wasmtime_val_t {
118123
kind: WASMTIME_I32,
119124
of: wasmtime_val_union { i32: 0 },
@@ -151,7 +156,7 @@ async fn invoke_c_async_callback<'a>(
151156
// Translate the `wasmtime_val_t` results into the `results` space
152157
for (i, result) in out_results.iter().enumerate() {
153158
unsafe {
154-
results[i] = result.to_val();
159+
results[i] = result.to_val(&mut caller.caller);
155160
}
156161
}
157162
// Move our `vals` storage back into the store now that we no longer
@@ -229,7 +234,7 @@ async fn do_func_call_async(
229234
match result {
230235
Ok(()) => {
231236
for (slot, val) in results.iter_mut().zip(wt_results.iter()) {
232-
crate::initialize(slot, wasmtime_val_t::from_val(val.clone()));
237+
crate::initialize(slot, wasmtime_val_t::from_val(&mut store, val.clone()));
233238
}
234239
params.truncate(0);
235240
store.data_mut().wasm_val_storage = params;
@@ -240,7 +245,7 @@ async fn do_func_call_async(
240245

241246
#[no_mangle]
242247
pub unsafe extern "C" fn wasmtime_func_call_async<'a>(
243-
store: CStoreContextMut<'a>,
248+
mut store: CStoreContextMut<'a>,
244249
func: &'a Func,
245250
args: *const wasmtime_val_t,
246251
nargs: usize,
@@ -251,10 +256,16 @@ pub unsafe extern "C" fn wasmtime_func_call_async<'a>(
251256
) -> Box<wasmtime_call_future_t<'a>> {
252257
let args = crate::slice_from_raw_parts(args, nargs)
253258
.iter()
254-
.map(|i| i.to_val());
259+
.map(|i| i.to_val(&mut store))
260+
.collect::<Vec<_>>();
255261
let results = crate::slice_from_raw_parts_mut(results, nresults);
256262
let fut = Box::pin(do_func_call_async(
257-
store, func, args, results, trap_ret, err_ret,
263+
store,
264+
func,
265+
args.into_iter(),
266+
results,
267+
trap_ret,
268+
err_ret,
258269
));
259270
Box::new(wasmtime_call_future_t { underlying: fut })
260271
}

crates/c-api/src/func.rs

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -249,7 +249,12 @@ pub(crate) unsafe fn c_callback_to_rust_fn(
249249
let mut vals = mem::take(&mut caller.data_mut().hostcall_val_storage);
250250
debug_assert!(vals.is_empty());
251251
vals.reserve(params.len() + results.len());
252-
vals.extend(params.iter().cloned().map(|p| wasmtime_val_t::from_val(p)));
252+
vals.extend(
253+
params
254+
.iter()
255+
.cloned()
256+
.map(|p| wasmtime_val_t::from_val(&mut caller, p)),
257+
);
253258
vals.extend((0..results.len()).map(|_| wasmtime_val_t {
254259
kind: crate::WASMTIME_I32,
255260
of: wasmtime_val_union { i32: 0 },
@@ -272,7 +277,7 @@ pub(crate) unsafe fn c_callback_to_rust_fn(
272277

273278
// Translate the `wasmtime_val_t` results into the `results` space
274279
for (i, result) in out_results.iter().enumerate() {
275-
results[i] = result.to_val();
280+
results[i] = result.to_val(&mut caller.caller);
276281
}
277282

278283
// Move our `vals` storage back into the store now that we no longer
@@ -330,7 +335,7 @@ pub unsafe extern "C" fn wasmtime_func_call(
330335
&mut params,
331336
crate::slice_from_raw_parts(args, nargs)
332337
.iter()
333-
.map(|i| i.to_val()),
338+
.map(|i| i.to_val(&mut store)),
334339
nresults,
335340
);
336341

@@ -345,7 +350,7 @@ pub unsafe extern "C" fn wasmtime_func_call(
345350
Ok(Ok(())) => {
346351
let results = crate::slice_from_raw_parts_mut(results, nresults);
347352
for (slot, val) in results.iter_mut().zip(wt_results.iter()) {
348-
crate::initialize(slot, wasmtime_val_t::from_val(val.clone()));
353+
crate::initialize(slot, wasmtime_val_t::from_val(&mut store, val.clone()));
349354
}
350355
params.truncate(0);
351356
store.data_mut().wasm_val_storage = params;

crates/c-api/src/global.rs

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -79,12 +79,13 @@ pub unsafe extern "C" fn wasm_global_set(g: &mut wasm_global_t, val: &wasm_val_t
7979

8080
#[no_mangle]
8181
pub unsafe extern "C" fn wasmtime_global_new(
82-
store: CStoreContextMut<'_>,
82+
mut store: CStoreContextMut<'_>,
8383
gt: &wasm_globaltype_t,
8484
val: &wasmtime_val_t,
8585
ret: &mut Global,
8686
) -> Option<Box<wasmtime_error_t>> {
87-
let global = Global::new(store, gt.ty().ty.clone(), val.to_val());
87+
let val = val.to_val(&mut store);
88+
let global = Global::new(store, gt.ty().ty.clone(), val);
8889
handle_result(global, |global| {
8990
*ret = global;
9091
})
@@ -100,18 +101,20 @@ pub extern "C" fn wasmtime_global_type(
100101

101102
#[no_mangle]
102103
pub extern "C" fn wasmtime_global_get(
103-
store: CStoreContextMut<'_>,
104+
mut store: CStoreContextMut<'_>,
104105
global: &Global,
105106
val: &mut MaybeUninit<wasmtime_val_t>,
106107
) {
107-
crate::initialize(val, wasmtime_val_t::from_val(global.get(store)))
108+
let gval = global.get(&mut store);
109+
crate::initialize(val, wasmtime_val_t::from_val(store, gval))
108110
}
109111

110112
#[no_mangle]
111113
pub unsafe extern "C" fn wasmtime_global_set(
112-
store: CStoreContextMut<'_>,
114+
mut store: CStoreContextMut<'_>,
113115
global: &Global,
114116
val: &wasmtime_val_t,
115117
) -> Option<Box<wasmtime_error_t>> {
116-
handle_result(global.set(store, val.to_val()), |()| {})
118+
let val = val.to_val(&mut store);
119+
handle_result(global.set(store, val), |()| {})
117120
}

crates/c-api/src/ref.rs

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -41,14 +41,9 @@ pub extern "C" fn wasm_ref_copy(r: Option<&wasm_ref_t>) -> Option<Box<wasm_ref_t
4141
}
4242

4343
#[no_mangle]
44-
pub extern "C" fn wasm_ref_same(a: Option<&wasm_ref_t>, b: Option<&wasm_ref_t>) -> bool {
45-
match (a.map(|a| &a.r), b.map(|b| &b.r)) {
46-
(Some(Ref::Extern(Some(a))), Some(Ref::Extern(Some(b)))) => a.ptr_eq(b),
47-
(None, None) => true,
48-
// Note: we don't support equality for `Func`, so we always return
49-
// `false` for `funcref`s.
50-
_ => false,
51-
}
44+
pub extern "C" fn wasm_ref_same(_a: Option<&wasm_ref_t>, _b: Option<&wasm_ref_t>) -> bool {
45+
// We need a store to determine whether these are the same reference or not.
46+
abort("wasm_ref_same")
5247
}
5348

5449
#[no_mangle]

0 commit comments

Comments
 (0)