Repository navigation
Update Eigen to 5.0.1 + SSE2 fix (broken Clip & Sign) - #32475
Conversation
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
🟡 Changes recommended
The newly added Python regression tests are currently model-invalid/fragile (domain/opset mismatch, Constant attribute usage, NumPy clip call) and may not exercise the CPU/Eigen path unless CPUExecutionProvider is forced.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Updates the pinned Eigen dependency to the 5.0.1 release and introduces a new Eigen patch to fix an SSE2 int64 compare bug that can break CPU Sign and Clip behavior for specific values (e.g., 2147483649).
Changes:
- Bump Eigen source pin in
cmake/deps.txtto a 5.0.1 commit archive/hash. - Update Eigen patch application to drop the old s390x build patch and add a new SSE2 int64-compare backport patch.
- Add Python regression tests for
SignandClipon int64 inputs.
File summaries
| File | Description |
|---|---|
| onnxruntime/test/python/test_sign.py | Adds a minimal ONNX model + regression test for Sign with problematic int64 values. |
| onnxruntime/test/python/test_clip.py | Adds a minimal ONNX model + regression test for Clip with problematic int64 values. |
| cmake/patches/eigen/sse2-pcmp-lt-int64-sign.patch | New Eigen backport patch to fix SSE2 Packet2l less-than compare behavior. |
| cmake/patches/eigen/s390x-build.patch | Removes an older s390x-focused patch that is no longer applied. |
| cmake/external/eigen.cmake | Adjusts which Eigen patches are applied during FetchContent. |
| cmake/deps.txt | Updates Eigen URL/hash and refreshes the pinning commentary. |
Review details
Suppressed comments (3)
onnxruntime/test/python/test_sign.py:30
- This test is intended to validate the CPU/Eigen implementation, but InferenceSession without an explicit providers list may run the model on CUDA/other EPs when available and miss the regression. Force CPUExecutionProvider to ensure the SSE2 code path is exercised on capable machines.
sess = ort.InferenceSession(make_sign_model().SerializeToString())
x = np.asarray([2147483649, 2147483649], np.int64)
(res,) = sess.run(None, {"x": x})
onnxruntime/test/python/test_clip.py:15
- The Clip node is set to domain="ai.onnx" but the model imports the default opset domain (""). To keep the model unambiguous and consistent with other tests, omit the domain field for standard ONNX ops.
clip_node = helper.make_node(
op_type="Clip",
domain="ai.onnx",
inputs=["x", "", "max"],
outputs=["y"],
)
onnxruntime/test/python/test_clip.py:33
- Use np.clip’s standard argument names/ordering for portability across NumPy versions.
np.clip(x, None, 0)(ora_min=None, a_max=0) avoids relying onmin/maxkeyword aliases.
np.testing.assert_array_equal(res, np.clip(x, min=None, max=0))
- Files reviewed: 6/6 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
🟡 Changes recommended
The new Python regression tests should be aligned with the repo’s test conventions (headers and unittest-style discovery) to ensure they reliably execute in CI.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
onnxruntime/test/python/test_clip.py:36
- Tests in this folder are typically implemented as unittest.TestCase methods (see test_pep561_py_typed_marker.py), which works with both unittest discovery and pytest runners. As a free function, this test may be skipped depending on how the suite is executed.
def test_clip():
sess = ort.InferenceSession(make_clip_model().SerializeToString(), providers=["CPUExecutionProvider"])
x = np.asarray([2147483649, 2147483649], np.int64)
(res,) = sess.run(None, {"x": x})
onnxruntime/test/python/test_sign.py:30
- Tests in this folder are typically implemented as unittest.TestCase methods (see test_pep561_py_typed_marker.py), which works with both unittest discovery and pytest runners. As a free function, this test may be skipped depending on how the suite is executed.
def test_sign():
sess = ort.InferenceSession(make_sign_model().SerializeToString(), providers=["CPUExecutionProvider"])
x = np.asarray([2147483649, 2147483649], np.int64)
(res,) = sess.run(None, {"x": x})
- Files reviewed: 6/6 changed files
- Comments generated: 2
- Review effort level: Lite
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
d82d856 to
ef00c33
Compare
|
I moved the tests to c++ |
There was a problem hiding this comment.
🟡 Changes recommended
The new Clip int64 regression test likely needs the existing DirectML skip guard (as used by Clip_Default_int64) to avoid known DML failures for int64 Clip until #41968513 is resolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 1
- Review effort level: Lite
|
Hariharan Seshadri (@hariharans29) Ti-Tai Wang (@titaiwangms), maybe you could take a look since you touched (or reviewed changes to) deps.txt not too long ago? |
|
The vcpkg port was not updated: cmake/vcpkg-ports/eigen3/portfile.cmake:10 still pins the old commit with no pcmp patch, and Windows CI builds with --use_vcpkg, so those builds keep the buggy SSE2 compare and the new tests would fail there. |
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
|
mirounga Thanks for taking a look. Here are the latest updates.
|
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
8cf5e8a to
f3ee44c
Compare
|
The QNN test failure is expected since QNN narrows int64 to int32, which produces wrong results for the test in question. I excluded QNN for that test. The other failures are due to the new eigen commit not being available on the vcpkg mirror. Changming Sun (@snnn) , could you maybe take a look? |
|
Thanks, Christian. I have left Microsoft. Someone else will help review this PR. |
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
…egression test
The CUDA plugin EP took Eigen headers from ${Eigen3_SOURCE_DIR}, which is
empty in vcpkg builds. It only worked because the previous eigen3 vcpkg
port copied headers into include/.
WebGPU computes int64 Clip in i32, so inputs above INT32_MAX wrap.
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
|
Thanks eserscor ! I think the remaining issues should be fixed with the latest changes. Could you trigger the CI again, please? |
|
eserscor, I'm afraid I don't have permission to merge. Would be nice to have this in the bag for the next release. |
|
Sorry for the spam! I hope you don't mind me pinging this again. Would be great to have this in the bag for the next release. eserscor Ti-Tai Wang (@titaiwangms) mirounga Hariharan Seshadri (@hariharans29) |
|
Apologies I missed the spam, merged. |
|
I'm afraid we missed the |
This cherry-picks the following commits for the release: | Commit ID | PR Number | Commit Title | |-----------|-----------|-------------| | 85dcb54 | #32475 | Update Eigen to 5.0.1 + SSE2 fix (broken Clip & Sign) | | 1ea552d | #33976 | [CUDA/CPU] Add hybrid GPU/CPU expert offload for MoEPacked block-packed weights | | 3a8b8f2 | #34066 | [CUDA] Optimize multi-token MatMulPacked GGUF GEMV (M=2..8) and eliminate ReadU32Dev warp divergence | | 64506ea | #34098 | [CUDA] Guard tests requiring SM80 on older GPUs | | 74f861c | #34067 | [MoE] Optimize multi-token (M=4, 8) MoEPacked CPU hybrid cold routes and GPU FC2 decode | | 6aa9b27 | #33963 | [CUDA] Extend MoEPacked for top-k, dispatch and fused MoE calculation | | 8fb1b27 | #33969 | [CUDA] Add Mamba2Verify CUDA Kernel | | cedae42 | #33994 | [CUDA] Update `MoEGroupRouter` for routing to `MoEPacked` | | 158d0c0 | #34002 | [CUDA] Speed up native Bolt packed projections and IQ2 experts | | 38c6dce | #34003 | [CUDA] Reduce native packed decode tile and FC2 loader overhead | | d5704db | #34147 | [CUDA] Optimize MoEPacked, MoEGroupRouter, and MatMulPacked for Nemotron Bolt | | 7cca0e5 | #34126 | Fix Windows junction detection in model_package for non-MSVC standard libraries | | 2bf400a | #34099 | Fix external data validation for Hugging Face Hub shared blob store | | d2f77cd | #34164 | [CUDA] Accelerate Nemotron Bolt large-chunk prefill across contrib ops | | ca7ea48 | #28457 | Add set_default_logger_callback Python API for custom ORT logging | | fca9447 | #34117 | [CUDA] Optimize CUDA prefill and quantized paged attention | | 1804c6f | #34114| [CUDA] Optimize MoEPacked GGUF offload 3-stage prefill pipeline and F16C/Batch4/40-core cold decode | --------- Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com> Signed-off-by: rui-ren <ruiren1225@gmail.com> Co-authored-by: Christian Bourjau <cbourjau@users.noreply.github.com> Co-authored-by: BBaditi <bbaditi@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Rui Ren <ruiren1225@gmail.com> Co-authored-by: Mustapha Jaber <mustjab@users.noreply.github.com> Co-authored-by: Mustapha Jaber <mustjab+microsoft@microsoft.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com> Copilot-Session: f4d0e1a3-a389-4cc8-9c85-0fd0e3ffce5b Copilot-Session: c8741c5a-3e9e-4860-b846-7d6fb0025040

Description
onnxruntime is currently using an Eigen commit from somewhere in between Eigen 3.4.1 and Eigen 5.0.1 (despite the comment in deps.txt). This PR updates Eigen to the latest 5.0.1 release. Note that Eigen is a headers-only library, which should make this process smooth.
One of the currently applied patches is no longer needed (it is part of the 5.0.1 release), and the other is simply disabling some warnings (still present after this PR). The new patch backports an upstream fix for a bug which I had attempted to fix here somewhat incorrectly a long time ago. My proposed fix merely avoided the broken SSE2 code path.
Motivation and Context
The main motivation is to properly fix the Sign and Clip operators on installations that use the SSE2 path. Onnxruntime is currently using an outdated and odd Eigen version, which makes backporting awkward. It seems much easier and generally beneficial to update Eigen to a proper release, plus some well-defined patches.