Skip to content

update to wasmtime 3.0.0#65

Merged
JakeChampion merged 9 commits into
mainfrom
jake/wasmtime
Apr 15, 2023
Merged

update to wasmtime 3.0.0#65
JakeChampion merged 9 commits into
mainfrom
jake/wasmtime

Conversation

@JakeChampion
Copy link
Copy Markdown
Collaborator

@JakeChampion JakeChampion commented Jan 4, 2023

wasmtime 0.38.0 removed it's module linking implementation to make room for the upcoming support for the component model. bytecodealliance/wasmtime#3958

This PR removes all the module linking functionality from wizer and updates to wasmtime 0.38.0

This PR also changes from using Trap::new to using the anyhow! macro, Trap::new was removed in wasmtime 3.0.0 bytecodealliance/wasmtime#5149

I've not worked much (if at all) in the wasmtime or wizer codebases, I may have done many things which are not correct here, hopefully it's along the right tracks 🤞

@JakeChampion JakeChampion requested a review from fitzgen January 4, 2023 17:37
@JakeChampion JakeChampion changed the title update to wasmtime 0.38.0 update to wasmtime 3.0.0 Jan 4, 2023
@JakeChampion JakeChampion force-pushed the jake/wasmtime branch 3 times, most recently from 62af22b to 40d59ea Compare January 4, 2023 18:04
Copy link
Copy Markdown
Member

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Just one test that was deleted that shouldn't be.

Comment thread tests/tests.rs
Comment on lines -140 to -164
#[test]
fn multi_memory() -> Result<()> {
run_wat(
&[],
42,
r#"
(module
(memory $m1 1)
(memory $m2 1)
(func (export "wizer.initialize")
i32.const 0
i32.const 41
i32.store (memory $m1) offset=1337
i32.const 0
i32.const 1
i32.store (memory $m2) offset=1337)
(func (export "run") (result i32)
i32.const 0
i32.load (memory $m1) offset=1337
i32.const 0
i32.load (memory $m2) offset=1337
i32.add))
"#,
)
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test shouldn't be deleted, we should still support multi-memory Wasm.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've re-added it now, the test fails but I unfortunately don't know how to make it pass.

failures:

---- multi_memory stdout ----
Error: unknown operator or unexpected token
     --> <anon>:8:16
      |
    8 |     i32.store (memory $m1) offset=1337
      |                ^


failures:
    multi_memory

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the WAT syntax for multi-memory has changed so that should just be

i32.store $m1 offset=1337

now.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I made that change to both the store calls and both the load calls however I believe it fails to generate valid wasm. The error that is generated is

Error: unknown memory 1337 (at offset 75)

This error comes from store call for the $m2 memory.

If I change the calls to use different offsets from each other then the test passes but I'm not sure if that is just avoiding a potential bug that I've introduced within this PR by updating wasmtime.

If useful, this is the wat which fails to validate:

(module
  (memory $m1 1)
  (memory $m2 1)
  (func (export "wizer.initialize")
    i32.const 0
    i32.const 41
    i32.store $m1 offset=1337
    i32.const 0
    i32.const 1
    i32.store $m2 offset=1337)
  (func (export "run") (result i32)
    i32.const 0
    i32.load $m1 offset=1337
    i32.const 0
    i32.load $m2 offset=1337
    i32.add))

And this is the wat which passes the test:

(module
  (memory $m1 1)
  (memory $m2 1)
  (func (export "wizer.initialize")
    i32.const 0
    i32.const 41
    i32.store $m1 offset=1337
    i32.const 0
    i32.const 1
    i32.store $m2 offset=1)
  (func (export "run") (result i32)
    i32.const 0
    i32.load $m1 offset=1337
    i32.const 0
    i32.load $m2 offset=1
    i32.add))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh. That looks correct by my reading of the parsing code but it looks like it is trying to treat the offset (static index into the memory) as a memory index (thing that decides which memory we are accessing).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually IDs should work too.

In fact, if I copy/paste the failing example and run our validator on it, it succeeds:

$ cat ~/scratch/multi-memory.wat 
(module
 (memory $m1 1)
 (memory $m2 1)
 (func (export "wizer.initialize")
       i32.const 0
       i32.const 41
       i32.store $m1 offset=1337
       i32.const 0
       i32.const 1
       i32.store $m2 offset=1337)
 (func (export "run") (result i32)
       i32.const 0
       i32.load $m1 offset=1337
       i32.const 0
       i32.load $m2 offset=1337
       i32.add))
$ wasm-tools validate --features multi-memory ~/scratch/multi-memory.wat 
$ echo $?
0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure that you tested that exact version of the WAT and not accidentally some in between version while editing and poking at the test case?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested that exact version, yep, I've pushed it to GH now, here is the failing CI logs -- https://github.com/bytecodealliance/wizer/actions/runs/3876213375/jobs/6609775823

To get more information from that failing test, I ran this locally:
❯ RUST_LOG=debug cargo test --package wizer --test tests -- multi_memory --exact --nocapture
Which then output the message:

Error: unknown memory 1337 (at offset 75)

I'm not sure why it is failing though

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm maybe wasm-tools have changed since wasmtime 3.0 and 4.0? If you rebase your 4.0 work on the latest version of this branch, do you get the same failure?

Copy link
Copy Markdown
Collaborator Author

@JakeChampion JakeChampion Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rebased on wasmtime 4 branch and the same test failure happens.

I went to see what the prewizened wasm looked like after parsing the wat 56bb567 (#65) , and this is what it is:

 === PreWizened Wasm ==========================================================
    (module
      (type (;0;) (func))
      (type (;1;) (func (result i32)))
      (func (;0;) (type 0)
        i32.const 0
        i32.const 41
        i32.store offset=1337
        i32.const 0
        i32.const 1
        i32.store (memory 1337) offset=1)
      (func (;1;) (type 1) (result i32)
        i32.const 0
        i32.load offset=1337
        i32.const 0
        i32.load (memory 1337) offset=1
        i32.add)
      (memory $m1 1)
      (memory $m2 1)
      (export "wizer.initialize" (func 0))
      (export "run" (func 1)))
    ===========================================================================

Comment thread tests/tests.rs
@alexcrichton
Copy link
Copy Markdown
Member

I've sent #75 which is a PR to this PR which I think should get the tests working.

Jake Champion and others added 9 commits April 15, 2023 01:01
wasmtime 0.36.0 removed it's module linking implementation to make room for the upcoming support for the component model. bytecodealliance/wasmtime#3958

This commit removes all the module linking functionality from wizer and updates to wasmtime 0.38.0
this update required changes from the removed `Trap::new` to anyhow! macro, Trap::new was removed in bytecodealliance/wasmtime#5149
Bring these up-to-date with the latest.
These dependencies are far behind the latest versions so this
fast-forwards them up to versions where larger changes are required.
This updates them enough to perform some minor rewrites here and there
but nothing major is changing.
This involved deleting quite a bit more module-linking related code in
`wizer`.
…mory

This commit further updates the `wasmparser` dependency across some
versions with minor renamings and structuring of the API. This pulls in
enough support to get the `multi_memory` test passing.
@JakeChampion JakeChampion merged commit 8425b0b into main Apr 15, 2023
@JakeChampion JakeChampion deleted the jake/wasmtime branch April 15, 2023 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants