Skip to content

Support executing Pulley in Wasmtime #9744

Merged
alexcrichton merged 12 commits into
bytecodealliance:mainfrom
alexcrichton:pulley-in-wasmtime
Dec 6, 2024
Merged

Support executing Pulley in Wasmtime #9744
alexcrichton merged 12 commits into
bytecodealliance:mainfrom
alexcrichton:pulley-in-wasmtime

Conversation

@alexcrichton
Copy link
Copy Markdown
Member

This commit is the initial implementation of executing the Pulley
interpreter from the wasmtime crate. This gives access to all of the
wasmtime crate's runtime APIs backed by execution of bytecode in
Pulley. This builds on the previous PRs I've been making for support in
Pulley to culminate in testing on CI in this PR. This PR handles some
final tidbits related to producing a runnable image that can be
interpreted by the wasmtime crate such as:

  • Pulley compilation artifacts are no longer marked as natively
    executable, just read-only.
  • Pulley compilation artifacts include wasm-to-array trampolines like
    normal platforms (removes a pulley special-case).
  • Dispatch of host calls from Pulley to the Wasmtime runtime are
    implemented.
  • Pulley's list of panicking wasm features is slimmed down as most are
    covered by "this lowering isn't supported" errors.
  • Execution of wasm code now has an if to see whether Pulley is
    enabled within a Store or not.
  • Traps and basic "unwinding" of the pulley stack are implemented (e.g.
    a "pulley version" of setjmp and longjmp, sort of)
  • Halting the interpreter has been refactored to help shrink the size of
    ControlFlow<Done> and handle metadata with each done state.

Some minor refactorings are also included here and there along with a
few fixes here and there necessary to get tests passing.

The next major part of this commit is updates to our wast test suite
and executing all *.wast files. Pulley is now executed by default for
all files as a new execution engine. This means that all platforms in CI
will start executing Pulley tests. At this time almost all tests are
flagged as "expected to fail" but there are a small handful of
allow-listed tests which are expected to pass. This exact list will
change over time as CLIF lowerings are implemented and the interpreter
is extended.

Follow-up PRs will extend the testing strategy further such as:

  • Extending #[wasmtime_test] to include Pulley in addition to Winch.
  • Getting testing set up on CI for 32-bit platforms.

@alexcrichton alexcrichton requested a review from fitzgen December 5, 2024 19:54
@alexcrichton
Copy link
Copy Markdown
Member Author

I'll note that this is in draft form as it currently is based on #9743. I'm going ahead and opening this up to see what CI has to say about this.

@github-actions github-actions Bot added cranelift Issues related to the Cranelift code generator wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime labels Dec 5, 2024
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Dec 5, 2024

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

  • If you added a new Config method, you wrote extensive documentation for
    it.

    Details

    Our documentation should be of the following form:

    Short, simple summary sentence.
    
    More details. These details can be multiple paragraphs. There should be
    information about not just the method, but its parameters and results as
    well.
    
    Is this method fallible? If so, when can it return an error?
    
    Can this method panic? If so, when does it panic?
    
    # Example
    
    Optional example here.
    
  • If you added a new Config method, or modified an existing one, you
    ensured that this configuration is exercised by the fuzz targets.

    Details

    For example, if you expose a new strategy for allocating the next instance
    slot inside the pooling allocator, you should ensure that at least one of our
    fuzz targets exercises that new strategy.

    Often, all that is required of you is to ensure that there is a knob for this
    configuration option in wasmtime_fuzzing::Config (or one
    of its nested structs).

    Rarely, this may require authoring a new fuzz target to specifically test this
    configuration. See our docs on fuzzing for more details.

  • If you are enabling a configuration option by default, make sure that it
    has been fuzzed for at least two weeks before turning it on by default.


Details

To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

This commit is the initial implementation of executing the Pulley
interpreter from the `wasmtime` crate. This gives access to all of the
`wasmtime` crate's runtime APIs backed by execution of bytecode in
Pulley. This builds on the previous PRs I've been making for support in
Pulley to culminate in testing on CI in this PR. This PR handles some
final tidbits related to producing a runnable image that can be
interpreted by the `wasmtime` crate such as:

* Pulley compilation artifacts are no longer marked as natively
  executable, just read-only.
* Pulley compilation artifacts include wasm-to-array trampolines like
  normal platforms (removes a pulley special-case).
* Dispatch of host calls from Pulley to the Wasmtime runtime are
  implemented.
* Pulley's list of panicking wasm features is slimmed down as most are
  covered by "this lowering isn't supported" errors.
* Execution of wasm code now has an `if` to see whether Pulley is
  enabled within a `Store` or not.
* Traps and basic "unwinding" of the pulley stack are implemented (e.g.
  a "pulley version" of `setjmp` and `longjmp`, sort of)
* Halting the interpreter has been refactored to help shrink the size of
  `ControlFlow<Done>` and handle metadata with each done state.

Some minor refactorings are also included here and there along with a
few fixes here and there necessary to get tests passing.

The next major part of this commit is updates to our `wast` test suite
and executing all `*.wast` files. Pulley is now executed by default for
all files as a new execution engine. This means that all platforms in CI
will start executing Pulley tests. At this time almost all tests are
flagged as "expected to fail" but there are a small handful of
allow-listed tests which are expected to pass. This exact list will
change over time as CLIF lowerings are implemented and the interpreter
is extended.

Follow-up PRs will extend the testing strategy further such as:

* Extending `#[wasmtime_test]` to include Pulley in addition to Winch.
* Getting testing set up on CI for 32-bit platforms.

prtest:full
Enable pulley for wasmtime/wasmtime-cli and also enable all features for
wasmtime-environ
@alexcrichton alexcrichton marked this pull request as ready for review December 5, 2024 22:24
@alexcrichton alexcrichton requested review from a team as code owners December 5, 2024 22:24
@alexcrichton
Copy link
Copy Markdown
Member Author

@fitzgen ok should be rebased and updated, I believe s390x CI is failing but I think most of the rest should be passing. Once results come in I'll figure out how to best ignore s390x for now.

Comment thread crates/wasmtime/src/runtime/vm/interpreter.rs
Copy link
Copy Markdown
Member

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Super excited to see this progress!

A few nitpicks and suggestions below, but nothing major

;; Fold `(trap[n]z (icmp ...))` together.

(rule 1 (lower (trapz (icmp cc a b @ (value_type (fits_in_64 ty))) code))
(rule 1 (lower (trapz (icmp cc a b @ (value_type (ty_32_or_64 ty))) code))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious: why was this change needed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is due to some other rule only accepting 32 or 64-bit inputs which causes ISLE to panic as something not a partial constructor. Basically tests panic unless this is here, so a future refactoring will need to flesh out more lowering rules for 8/16-bit types.

Comment thread crates/cranelift/src/obj.rs Outdated
Comment thread crates/wasmtime/src/runtime/store.rs
Comment thread crates/wasmtime/src/runtime/vm.rs
Comment thread crates/wasmtime/src/runtime/vm/interpreter.rs Outdated
Comment thread crates/wast-util/src/lib.rs Outdated
Comment thread pulley/src/interp.rs
Comment thread pulley/src/interp.rs
Comment thread pulley/src/interp.rs
Comment thread pulley/src/interp.rs Outdated
@alexcrichton alexcrichton requested a review from a team as a code owner December 6, 2024 20:06
@alexcrichton alexcrichton added this pull request to the merge queue Dec 6, 2024
Merged via the queue into bytecodealliance:main with commit 70a3793 Dec 6, 2024
@alexcrichton alexcrichton deleted the pulley-in-wasmtime branch December 6, 2024 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cranelift Issues related to the Cranelift code generator wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants