cranelift: Add VEX and EVEX memory encodings

[afonso360]

👋 Hey,

Feature

We should add the memory encodings for VEX and EVEX instructions.

Benefit

Better codegen.

Implementation

• Add the encodings in the VEX and EVEX encoders
• Fix lower.isle as needed to produce the new instructions

Alternatives

We could keep doing what we do now and have worse codegen.

cc: @abrown

[afonso360]

We discussed this a little bit in today's cranelift meeting. While it would be nice for cranelift, @cfallin pointed out that it would be slightly less relevant for wasmtime since the instructions that use these memory encodings have alignment restrictions, and WebAssembly has alignment hints but no guarantees, so we can't use it there.

[abrown]

I discussed this with @cfallin a bit and wanted to correct some misconceptions. As I did some digging in the x86 manual, the SSE documentation (section 10.3) does indicate that memory operands must be aligned:

The address of a 128-bit packed memory operand must be aligned on a 16-byte boundary, except in the following
cases:
• The MOVUPS instruction supports unaligned accesses.
• Scalar instructions that use a 4-byte memory operand that is not subject to alignment requirements.

But the same restriction does not apply for VEX (read AVX) and EVEX (AVX-512) instructions:

Memory alignment requirements on EVEX-encoded SIMD instructions are similar to VEX-encoded SIMD instructions. Memory alignment applies to EVEX-encoded SIMD instructions in three categories:
• Explicitly-aligned SIMD load and store instructions accessing 64 bytes of memory with EVEX prefix encoded
vector length of 512 bits (e.g., VMOVAPD, VMOVAPS, VMOVDQA, etc.). These instructions always require the
memory address to be aligned on a 64-byte boundary.15-14 Vol. 1
PROGRAMMING WITH INTEL® AVX-512
• Explicitly-unaligned SIMD load and store instructions accessing 64 bytes or less of data from memory (e.g.,
VMOVUPD, VMOVUPS, VMOVDQU, VMOVQ, VMOVD, etc.). These instructions do not require the memory
address to be aligned on a natural vector-length byte boundary.
• Most arithmetic and data processing instructions encoded using EVEX support memory access semantics.
When these instructions access from memory, there are no alignment restrictions.

There are additional benefits to using VEX and EVEX encodings (e.g., using 3 registers) that @cfallin pointed out so I think this issue should still be considered. In fact, this is something I would be interested in helping out with (cc: @alexcrichton, who may have something already in progress).

[alexcrichton]

I ended up giving this a stab in #5795, but it could very well be wrong! (I also didn't handle EVEX yet)

[alexcrichton]

The final bit of EVEX was done in #6416, so I think this is done now.