Repository navigation
Bump Microsoft.ML.OnnxRuntime from 1.29.0 to 1.32.0 - #30
Open
dependabot[bot] wants to merge 1 commit into
Open
dependabot[bot] wants to merge 1 commit into
dependabot[bot] wants to merge 1 commit into
Conversation
--- updated-dependencies: - dependency-name: Microsoft.ML.OnnxRuntime dependency-version: 1.32.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updated Microsoft.ML.OnnxRuntime from 1.29.0 to 1.32.0.
Release notes
Sourced from Microsoft.ML.OnnxRuntime's releases.
1.32.0
ONNX Runtime 1.32.0 reintegrates ONNX 1.23.2 with opset 28 support, introduces CPU/GPU hybrid MoE offloading and shared-layout CPU QMoE kernels, expands device-memory, weightless-compilation, and Python model-package APIs, dramatically accelerates session initialization on large graphs, and strengthens runtime and optimizer reliability. These notes cover changes since ONNX Runtime 1.31.1. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases.
Highlights
SpaceToDepth,Mod, signedBitShift, andBFloat16extensions) and updated DLPack to v1.3 across all implemented ORT numeric and boolean tensor types (#33134, #32362, #33848).QMoE(NVFP4, INT4, INT3, and INT2) and block-packedMoEPackedmodels, backed by shared-layout AVX-512/AVX2 CPUQMoEkernels and memory-mapped expert storage (#33450, #33451, #33452, #33973).OrtApi::EpDevice_GetMemoryCapacity, explicit multi-mode weightless compilation (ModelCompilationOptions_SetWeightlessModeandep.enable_weightless_mode), Python bindings for the Model Package APIs, and zero-copy CPU buffer borrowing inAddExternalInitializers(#33447, #32899, #33640, #33877).GatedDeltaNetper-key-dimension decay), right-padded chunked prefill inGroupQueryAttention, native GGUF/block-packed operators (GatherPacked,MatMulPacked,MoEPacked), and sparse-attention index sharing (#33817, #33503, #33953, #33224).Announcements & Compatibility
ACLExecutionProvider) execution provider, its build options, and its standalone C registration API have been removed. Python'sget_all_providers()no longer listsACLExecutionProvider; applications should remove it from explicit provider lists (#33851).NvTensorRTRTXExecutionProviderremoved: In-tree provider-bridgeNvTensorRTRTXExecutionProvidersources have been removed in favor of the standalone NVIDIA TensorRT RTX plugin EP registered viaOrtApi::RegisterExecutionProviderLibrary.NvTensorRtRtxProviderOptionsis retained as a deprecated declaration for source compatibility, and legacyAppendExecutionProvidercalls now report a deprecation error directing callers to the plugin EP (#32276).ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT6E2M3andONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT6E3M2public element-type mappings andCastconversions (#33134).kDLBoolmetadata and bidirectional support for all implemented ORT numeric and boolean types, including all fivefloat8formats, packedfloat4,int4/uint4, andint2/uint2(float6remains unsupported in DLPack). The low-levelis_bool_tensoroverride was removed, anduint8DLPack metadata now always imports asuint8(#33848).OrtCompileApi::ModelCompilationOptions_SetWeightlessEnabledand theep.enable_weightless/ep.enable_weightless_ep_context_nodessession options are deprecated in favor ofModelCompilationOptions_SetWeightlessModeandep.enable_weightless_mode("0"=NONE,"1"=EXTERNAL_ONLY,"2"=ALL). EPs can now advertise multi-mode capability viaOrtWeightlessSupport_ALL_OR_EXTERNAL_ONLYandweightless_supported_modes(#32899).New Features
Core APIs & Runtime
OrtApi::EpDevice_GetMemoryCapacity(C++EpDevice::GetMemoryCapacity()) andOrtEpFactory::GetDeviceMemoryCapacityto query device-global total memory and estimated currently available memory (DEFAULTorHOST_ACCESSIBLE), with built-in and plugin CUDA EP support (#33447).weightless_supported_modes), applications request an explicit mode (ModelCompilationOptions_SetWeightlessMode/ep.enable_weightless_mode), and compiled EPContext models recordweightless_modeandexternal_dependencies(source_model,external_weights,ep_context_binary) in model metadata (#32899).session.use_external_initializer_file_buffers_directly="1"toAddExternalInitializersfor zero-copy borrowing of naturally aligned CPU buffers larger than 127 bytes on little-endian hosts, and documented the read-only guarantee for external initializers added viaOrtModelEditorApi::AddInitializerToGraphwithdata_is_external=true(#33877, #33717).session.external_data_loader.*) to accelerate external initializer loading into GPU memory across built-in and plugin CUDA EPs (#33839).session.moe.offload.*), genericsession.kernel_dispatch_policy(auto,latency,memory,safe) configuration plumbing, Level-1 workspace/prepack memory accounting fallback for nodes absent from memory-profile CSVs, and internal capability-planning protocol types (#33451, #33042, #33311, #33841).<profile_file>_compile) inonnxruntime_perf_testwhen-pis used with--compile_ep_context(#33053, #33694).onnxruntime_DISABLE_EXCEPTIONS(--disable_exceptions) usable withoutonnxruntime_MINIMAL_BUILDand added theonnxruntime_DISABLE_STREAM=ONCMake option to disable device-stream support in non-minimal builds (#33120, #33482).ONNX Opset 28 & Shared Contrib Operators
These updates affect core ONNX and contributed operator schemas and cross-provider behavior, not only the separately released GPU plugins.
SpaceToDepth(DCRandCRD) on CPU and CUDA with version-split registrations across DML, JS, WebGPU, and QNN; opset-28 floating floor-modulusModon CPU, CUDA, and DML; opset-28BFloat16support forCompress,OneHot,ReverseSequence,Unique,Celu, andEinsumon CPU (plusCompress,OneHot, andReverseSequenceon CUDA); andBFloat16Pad(opsets 21–25) on CPU and CUDA (#32362, #32369, #32378, #34062).GatedDeltaNetwith Kimi Delta Attention (KDA) per-key-dimension decay across CPU, CUDA, and WebGPU, and extendedGatedDeltaNetandCausalConvWithStatestate/gate handling for multi-token speculative decoding (#33817, #33471).query_seqlensinput toGroupQueryAttentionto skip right-padded prefill tokens and idle batch entries (query_seqlens[b] = 0) on CPU and CUDA, and addedq_norm_epsilon/k_norm_epsilonattributes to preserve distinct Q and K normalization epsilons during GQA pre-norm fusion (#33503, #33505, #33832, #33363).QMoEv1 packed-code contract, added opt-in post-SiLU clipping (swiglu_limit_after_activation) and tanh-GEGLU(activation_type="geglu") support to MoE/QMoE schemas, and addedGatherPacked(CPU/CUDA),MatMulPacked(CUDA, including native Boltglobal_scale), andMoEPacked(CUDA) to consume native GGUF/block-packed quantized weights directly (#33394, #33954, #33633, #33953, #33962).PackedSparseAttentionIndexerMergefor multi-token prediction index sharing and optionalstate_status/ mid-state outputs onPackedSparseAttentionIndexeracross CUDA and WebGPU; extendedGatedAddwith an optional projectedgate_weightacross CPU, CUDA, and WebGPU; and supported empty unpacked weights in CPU and WebGPUMatMulNBits(#33224, #33383, #33472, #33470, #33559).SwiGLU,RMSNormRotary,HyperConnectionLowRankMix,RouterGateUpProj,SpeculativePrefixCommit,GatedDeltaNetStateReplay,ConvStateReplay, andMatMulNBitsQLoRA(#32126, #33469, #33384, #33650, #33474, #33213, #33493, #33370).Plugin Execution Provider Infrastructure
OrtMemoryInfoABI mismatch across the ORT/plugin EP boundary by convertingOrtMemoryInfoinstances exclusively through the C API (#33571).OrtEpFactorystructs in provider-bridge EP factories so unimplemented optional callbacks are null, and prevented double DLL initialization and handle leaks when probing and registering plugin EP libraries (#33117, #33701).shared_assetspath confinement and symlink validation (#32790, #33826).Execution Provider Updates
... (truncated)
1.31.1
ONNX Runtime 1.31.1 is a patch release that fixes an OpenVINO-enabled Python build regression and corrects WASM Debug external-data tests. These notes cover changes since ONNX Runtime 1.31.0.
Highlights
Contributors
Thanks to our 3 human contributors for this release!
@eserscor, @mustjab, @tianleiwu
Full Changelog: rel-1.31.0...rel-1.31.1
Release highlights were prepared with AI assistance.
1.31.0
ONNX Runtime 1.31.0 stabilizes model-package and EPContext data APIs, improves CPU model loading and quantized MoE inference, expands device-based execution-provider selection, and strengthens model-loading and runtime reliability. These notes cover changes since ONNX Runtime 1.30.0. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases.
Highlights
PREFER_NPU(#31975).Announcements & Compatibility
GatherBlockQuantizedbehavior change: out-of-range indices now produce zero output slices across CPU, CUDA, and WebGPU for both integer and floating-point quantized formats. This deliberately differs from ONNXGathererror semantics (#32480).OrtApi::GetModelPackageApiand the stableOrtModelPackageApitable. EPContext callback registration now uses stable APIs; the corresponding experimental lookup entries were removed. The stable native callback contract carries callback/state pointers, with payload limits enforced by application callbacks, bindings, or EPs rather than a native read-options policy (#33166, #32265).--use_aclnow receive a deprecation warning (#32564).--use_windows_telemetryoronnxruntime_USE_WINDOWS_TELEMETRY=ON; telemetry can be disabled with--no_telemetry(#32384).New Features
Core APIs & Runtime
session.strict_workspace_verification=1to fail initialization on workspace declaration overruns or nonzero orphaned reservations; strict verification is not supported for ORT-format model loads (#31962, #32189).onnxruntime_DISABLE_DEVICE_DISCOVERY=ONto skip platform accelerator probing and report only the CPU device (#32592).Shared Contrib Operators
These updates affect core operator schemas and cross-provider behavior, not only the separately released GPU plugins.
EngramGateandNGramHashMappingfor Qwen4-Exp, and addedVarlenNGramHashMappingfor packed DeepSeek Engram batching with request-local history. Packed hashing also supports Qwen4-Exp head offsets, EOS resets, and segment boundaries (#32285, #32358, #32467).activationattribute toGatedRMSNormfor SiLU, its Swish alias, and Sigmoid gating across CPU, CUDA, and WebGPU. SiLU remains the default (#32512).BranchwiseRMSNorm,ScaledSiLU,HyperConnectionPreMix, andHyperConnectionPostMix, with CPU, CUDA, and WebGPU implementations (#32687).GatherBlockQuantizedto FP8/FP4 data, full-axis blocks, and scale broadcasting, and improved CPU UINT8 dequantization. See the out-of-range-index behavior change above (#32480, #32621).Plugin Execution Provider Infrastructure
Execution Provider Updates
CUDA EP
CUDA updates include sparse attention and packed continuous batching, expanded paged speculative decoding, INT2 and mixed-width QMoE execution, opt-in MatMul auto-tuning, improved MXFP4 support on Blackwell consumer GPUs, and reliability fixes.
... (truncated)
1.30.1
This is a patch release on top of v1.30.0, containing targeted model-validation and memory-safety fixes, improved Plugin EP data transfers, and build and packaging fixes.
Highlights
Bug Fixes
CausalConvWithState,GatedDeltaNet,GroupQueryAttention, andSparseAttention(#32607).EmbedLayerNormalizationshape inference with scalar beta,ConvTransposeWithDynamicPads, and shape-inference input-data lookup (#32610, #32679, #32681).LooporScanbody input is also an initializer, rejecting the invalid graph instead of crashing (#32609).Build and Packaging
Contributors
Thanks to our 8 contributors for this release!
@adrastogi, @bbernhar, @eserscor, @miaobin, @shiyi9801, @tianleiwu, @wangw-1991, @xiaofeihan1
Full Changelog: v1.30.0...v1.30.1
These release notes were drafted with AI assistance.
1.30.0
ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1.
Highlights
Announcements & Compatibility
-Donnxruntime_USE_FP4_QMOE=OFF(#32096, #32163).block_size=32. Set-Donnxruntime_USE_FPA_INTB_GEMM_FULL=ONwhen building from source to retain the full kernel set, including BF16, zero-point, bias, larger-block-size, and native Hopper variants (#32324).GemmandMatMulexecution is gated on hardware acceleration. CPU-assigned FP16 nodes without a matching kernel now fall back to FP32 (#32301, #32197).Security & Reliability
Model Loading, Memory, and Input Validation
Split,Scan,GatherND,ScatterND,SpaceToDepth/DepthToSpace,Crop,Conv,Normalizer, and pooling (#29461, #31668, #32034, #32039, #32076, #32157, #32160, #32161, #32345, #32349).BifurcationDetectorinputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage (#31648, #31701, #32009, #32078, #32144).TreeEnsemblenode references and bounded subtree comparison, rejected non-finite CPURoiAligncoordinates, and requiredImageScalerbias to match the channel count (#32031, #32043, #32011, #32002).MatMulFpQ4shape inputs, and checked MLAS blockwise quantization/dequantization index ranges (#31682, #32032, #32007).GPU Bounds and Resource Lifetimes
MatMulNBits,RemovePadding,RotaryEmbedding,SparseAttention, Whisper beam search, NMS, QDQ, andGatherElements(#31643, #31994, #31995, #31996, #31998, #32014, #32029, #32030).CudaAsyncBufferstaging storage alive across CUDA graph replay (#31968, #32121).Dependencies and Tooling
js-yaml,joi,fast-uri, and the Next.js end-to-end fixture (#32397, #32486, #32488, #32505, #32508).New Features
Core APIs & Runtime
KernelContext::GetPreallocatedOutput(#29726, #32089).EngramGateandNGramHashMapping, and expanded kernel coverage for Qwen-3.5 operators (#32268, #32106).Plugin Execution Providers
... (truncated)
1.29.1
This is a patch release on top of v1.29.0, containing GroupQueryAttention capability and KV-cache layout improvements, plugin Execution Provider performance tooling updates, and targeted graph and optimizer fixes.
GroupQueryAttention
causalattribute, with explicit handling for unsupported execution paths (#31704)attention_biaswith a sliding-window KV cache, including explicit position IDs and post-eviction bias indexing (#32302)Runtime and Performance Tools
onnxruntime_perf_testto use plugin Execution Provider device allocators for generated inputs, loaded test data, and pre-allocated outputs, avoiding unnecessary per-run host/device copies (#32244)Bug Fixes and Documentation
MulandPowpatterns (#32016)Contributors
Thanks to our 7 contributors for this release!
@adrastogi, @apsonawane, @edgchen1, @javier-intel, @jnagi-intel, @tianleiwu, @Wayne-Ch
Release highlights were drafted with AI assistance and are subject to release-team review.
Full Changelog: v1.29.0...v1.29.1
Commits viewable in compare view.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)