Skip to content

[WebGPU] Add int4 and uint4 QuantizeLinear and Reshape - #33792

Merged
Edward Chen (edgchen1) merged 5 commits into
mainfrom
adityar-microsoft-webgpu-int4-quantize-linear
Nov 5, 2026
Merged

Edward Chen (edgchen1) merged 5 commits into
mainfrom
adityar-microsoft-webgpu-int4-quantize-linear

Conversation

@adrastogi

Copy link
Copy Markdown
Contributor

Description

Adds int4/uint4 support to the WebGPU EP for the W3C WebNN quantizeLinear/dequantizeLinear conformance (WPT) cases. Every gap was found by WPT running on the WebGPU EP with CPU fallback disabled (Edge Canary, RTX/D3D12).

Commits:

  1. Support int4 and uint4 in Reshape.
    • This is why the DQ cases failed. Chromium's WebNN ORT backend builds opset-21 models. It reshapes a broadcastable scale and zero point to the scalar or 1-D shape that (De)QuantizeLinear takes: [1] becomes a scalar, and [1,1,2,1] becomes [2] for per-axis.
    • The CPU Reshape kernels register only IRv9 types, which don't include int4, so constant folding can't fold a Reshape of an int4/uint4 zero point. WebGPU Reshape didn't claim the 4-bit types either, so the graph failed to build.
    • int8 zero points pass only because CPU folds their Reshape. The DequantizeLinear kernel already handled int4/uint4, including odd sizes, so it is unchanged.
    • Reshape is a pure copy/view, so int4/uint4 are added from opset 21, the first opset whose schema allows them.
  2. Add int4 and uint4 QuantizeLinear output.
    • int4/uint4 y/y_zero_point for opsets 21-22, 23, 24, 25-27 and 28.
    • Each invocation packs 8 nibbles into one u32 word, with the first element in the low nibble. Odd element counts work, and packed output bytes are never shared between invocations.
    • Results saturate to [-8, 7] / [0, 15]. The 4-bit zero point is read from packed u32 storage and sign-extended for int4.
    • Unchanged from [WebGPU] Add QuantizeLinear and int8 Cast #33683: the exact-quotient rounding, the precision filter in both GetCapability paths, and the int8/uint8 WGSL. The plugin EP looks kernels up in the same registry, so it claims the new types without changes to ep/ep.cc.

Semantics and packing (two 4-bit values per byte, sign extension, blocked quantization, saturation ranges) follow Edward Chen (@edgchen1)'s unmerged branch edgchen1/webgpu_quantizelinear. That branch refactors QL/DQ into a WGSL template plus quantization_utils, but it predates #33683. This PR applies the same semantics to main's current kernel instead of porting the refactor.

Tests are pinned to WebGPU with CPU EP fallback disabled:

  • onnxruntime/test/providers/webgpu/tensor/quantize_linear_test.cc:
    • The WPT int4/uint4 QL and DQ vectors (f32 and f16), both directly and with the Reshape nodes Chromium inserts.
    • Rounding and saturation, multi-word per-axis and blocked zero points with odd sizes, all opsets, output_dtype without a zero point, and a Q -> DQ round trip.
  • onnxruntime/test/providers/webgpu/tensor/reshape_op_test.cc: int4/uint4 Reshape, including to a scalar, from both an input and an initializer.

The existing CPU QuantizeLinearOpTest.Int4/UInt4/OddLarge_* tests now also run on WebGPU.

Validation (Windows ARM64, NVIDIA RTX Spark N1X / D3D12, at 50323bf):

  • onnxruntime_provider_test --gtest_filter="WebGpuQuantizeLinearTest.*:WebGpuDequantizeLinearTest.*:TensorOpTest.Reshape*:QuantizeLinearOpTest.*:DequantizeLinearOpTest.*:QuantizeLinearOp21BlockedTest.*:QuantizeLinearOpMLFloat16Test.*:QuantizeLinearPrecisionTest.*": 188 tests, 184 passed, 0 failed, 4 skipped. The skips are DML-only.
  • Sensitivity checks, both applied and rebuilt:
    • With the Reshape 4-bit types disabled (since_version >= 99), TensorOpTest.Reshape_int4_webgpu and WebGpuDequantizeLinearTest.Int4WebNNConformance_Float/Float16 fail at session initialization, because the node lands on CPU. Int4PerAxisMultiWordZeroPoint, which has no Reshape, still passes.
    • With the nibble order swapped in the QuantizeLinear shader, every WebGpuQuantizeLinearTest.Int4* test fails on values.
  • WPT, using the plugin EP (--use_webgpu shared_lib) built from 50323bf with Edge Canary 159.0.4372.0 and ORT runtime 1.28.3:
    • quantizeLinear: 29/29 pass (was 15/29)
    • dequantizeLinear: 33/33 pass (was 27/33)
    • reshape: 67/67 pass
    • qdq_subgraph: 43/43 pass

Motivation and Context

This fixes the WebNN conformance failures on the WebGPU EP:

  • quantizeLinear.https.any.js, 14 subtests (f32 and f16 each):
    • int4 zeroPoint of odd size and of even size
    • 2D with int4 zeroPoint of even size
    • 2D with int4 zeroPoint and block_size = [3, 2]
    • uint4 zeroPoint of odd size and of even size
    • 1D with uint4 zeroPoint and block_size = 3
  • dequantizeLinear.https.any.js, 6 subtests:
    • uint4 and int4 1D tensors of odd size, with f32 and f16 1D scale
    • uint4 4D tensor with broadcasting f32/f16 4D scale and uint4 4D zeroPoint

WebNN disables CPU fallback, so an unclaimed node makes the graph fail to build.

Aditya Rastogi and others added 2 commits October 31, 2026 15:42
Chromium's WebNN backend (ONNX opset 21) reshapes broadcastable
(De)QuantizeLinear scale and zero point tensors to the scalar or 1-D shape
the ONNX ops take. The CPU Reshape kernels only register IRv9 types (no
int4/uint4), so constant folding cannot fold a Reshape of an int4/uint4 zero
point, and with CPU EP fallback disabled the WebGPU EP must run it. Reshape
is a pure copy/view, so the 4-bit types are safe to add from opset 21, the
first opset whose schema allows them.

This is why the WebNN dequantizeLinear "int4/uint4 1D tensor of odd size"
and "uint4 4D tensor with broadcasting 4D scale and uint4 4D zeroPoint"
conformance tests failed even though the DequantizeLinear kernel already
supports int4/uint4.

Co-authored-by: edgchen1 <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds int4/uint4 output (and zero point) to the WebGPU QuantizeLinear kernel
for opsets 21-22, 23, 24, 25-27 and 28. Each invocation packs 8 4-bit
results into one u32 word, first element in the low nibble, so odd element
counts and packed output bytes are never shared between invocations.
Results saturate to [-8, 7] / [0, 15]. The 4-bit zero point is read from
packed u32 storage and sign-extended for int4. Main's exact rounding,
precision filter and int8/uint8 WGSL are unchanged.

Semantics and packing follow Edward Chen's unmerged branch
edgchen1/webgpu_quantizelinear, applied to main's current kernel.

Tests mirror the WebNN quantizeLinear/dequantizeLinear int4/uint4
conformance vectors, including the Reshape nodes Chromium inserts before
the op, plus rounding/saturation, multi-word per-axis and blocked zero
points, all opsets, output_dtype without a zero point and a Q -> DQ round
trip, all pinned to WebGPU with CPU fallback disabled.

Co-authored-by: edgchen1 <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the WebGPU int4/uint4 QuantizeLinear and Reshape changes.

No open issues 🎉

What changed in this PR

Extends the WebGPU execution provider with int4/uint4 QuantizeLinear output and Reshape support, addressing WebNN conformance gaps when CPU fallback is disabled.

Changes:

  • Adds packed 4-bit quantization with signed zero-point handling, saturation, and odd-sized output support.
  • Enables int4/uint4 Reshape from opset 21.
  • Adds WebGPU-pinned tests for conformance vectors, rounding, per-axis and blocked quantization, reshaped parameters, and round trips.
File Description
onnxruntime/​test/​providers/​webgpu/​tensor/​reshape_op_test.cc Tests 4-bit Reshape with inputs, initializers, and scalar outputs.
onnxruntime/​test/​providers/​webgpu/​tensor/​quantize_linear_test.cc Adds 4-bit quantization and dequantization regression coverage.
onnxruntime/​core/​providers/​webgpu/​tensor/​reshape.cc Registers 4-bit Reshape types from opset 21.
onnxruntime/​core/​providers/​webgpu/​quantization/​quantize_linear.h Adds the 4-bit shader mode and updates packing comments.
onnxruntime/​core/​providers/​webgpu/​quantization/​quantize_linear.cc Implements 4-bit packing, zero-point extraction, saturation, and type registration.

Review effort: Balanced

@adrastogi
adrastogi marked this pull request as ready for review November 2, 2026 23:19
Aditya Rastogi and others added 2 commits November 4, 2026 15:27
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@edgchen1
Edward Chen (edgchen1) merged commit b512932 into main Nov 5, 2026
@edgchen1
Edward Chen (edgchen1) deleted the adityar-microsoft-webgpu-int4-quantize-linear branch November 5, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants