[fuzz] Configure the differential target#4773
Conversation
This change is a follow-on from bytecodealliance#4515 to add the ability to configure the `differential` fuzz target by limiting which engines and modules are used for fuzzing. This is incredibly useful when troubleshooting, e.g., when an engine is more prone to failure, we can target that engine exclusively. The effect of this configuration is visible in the statistics now printed out from bytecodealliance#4739. Engines are configured using the `ALLOWED_ENGINES` environment variable. We can either subtract from the set of allowed engines (e.g., `ALLOWED_ENGINES=-v8`) or build up a set of allowed engines (e.g., `ALLOWED_ENGINES=wasmi,spec`), but not both at the same time. `ALLOWED_ENGINES` only configures the left-hand side engine; the right-hand side is always Wasmtime. When omitted, `ALLOWED_ENGINES` defaults to [`wasmtime`, `wasmi`, `spec`, `v8`]. The generated WebAssembly modules are configured using `ALLOWED_MODULES`. This environment variables works the same as above but the available options are: [`wasm-smith`, `single-inst`].
|
Here is an example from running the target locally: $ ALLOWED_MODULES=-wasm-smith cargo +nightly fuzz run differentia
...
=== Execution rate (384 successes / 2000 attempted modules): 19.20% ===
wasmi: 0.96%, spec: 8.85%, wasmtime: 83.97%, v8: 6.22%
wasm-smith: 0.00%, single-inst: 100.00% |
|
@jameysharp, you've had some opinions on this in the past so I tagged you for review (?): I really would have preferred to use |
jameysharp
left a comment
There was a problem hiding this comment.
It's true: I have, at times, had opinions! 😆
I don't feel very strongly about any of these review comments. This PR is fine as-is, so I'm marking this approved. But if you want to take a little more time on it I think these suggestions are improvements.
Subscribe to Label Actioncc @fitzgen DetailsThis issue or pull request has been labeled: "fuzzing"Thus the following users have been cc'd because of the following labels:
To subscribe or unsubscribe from this label, edit the |
jameysharp
left a comment
There was a problem hiding this comment.
I think you've addressed all my suggestions. Thanks! I hope you found these suggestions helpful.
Yeah, they were great! |
This change is a follow-on from #4515 to add the ability to configure
the
differentialfuzz target by limiting which engines and modules areused for fuzzing. This is incredibly useful when troubleshooting, e.g.,
when an engine is more prone to failure, we can target that engine
exclusively. The effect of this configuration is visible in the
statistics now printed out from #4739.
Engines are configured using the
ALLOWED_ENGINESenvironment variable.We can either subtract from the set of allowed engines (e.g.,
ALLOWED_ENGINES=-v8) or build up a set of allowed engines (e.g.,ALLOWED_ENGINES=wasmi,spec), but not both at the same time.ALLOWED_ENGINESonly configures the left-hand side engine; theright-hand side is always Wasmtime. When omitted,
ALLOWED_ENGINESdefaults to [
wasmtime,wasmi,spec,v8].The generated WebAssembly modules are configured using
ALLOWED_MODULES. This environment variables works the same as abovebut the available options are: [
wasm-smith,single-inst].