Repository navigation
Support read-only allocator for use with initializers - #25348
Merged
George Wu (jywu-mysoft) merged 5 commits intoJul 24, 2025
Merged
George Wu (jywu-mysoft) merged 5 commits into
George Wu (jywu-mysoft) merged 5 commits into
Conversation
…separate allocator that is only used for initializers. Update the SessionState logic to support this allocator type being provided, and use it when doing device allocations for initializers.
Scott McKay (skottmckay)
marked this pull request as ready for review
July 20, 2025 00:38
Contributor
|
does it also support |
Contributor
Author
Yes. The ORT implementation deals in IExecutionProvider. If it's a plugin EP the OrtEp gets wrapped by an IExecutionProvider based class. If the IExecutionProvider's CreatePreferredAllocators returns an allocator with type or OrtReadOnlyAllocator in the SessionState ctor it will be used for initializers. |
Yulong Wang (fs-eire)
approved these changes
Jul 23, 2025
George Wu (jywu-mysoft)
deleted the
skottmckay/AddSupportForReadOnlyAllocatorToBeUsedWithInitializers
branch
July 24, 2025 15:10
Ryan Metcalfe (RyanMetcalfeInt8)
pushed a commit
to RyanMetcalfeInt8/onnxruntime
that referenced
this pull request
Jul 29, 2025
### Description <!-- Describe your changes. --> Add new allocator type of OrtReadOnlyAllocator to enable providing a separate allocator that is only used for initializers. Update the SessionState logic to support this allocator type being provided, and use it when doing device allocations for initializers. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Performance.
Sanket Kale (sanketkaleoss)
pushed a commit
to sanketkaleoss/onnxruntime
that referenced
this pull request
Aug 11, 2025
### Description <!-- Describe your changes. --> Add new allocator type of OrtReadOnlyAllocator to enable providing a separate allocator that is only used for initializers. Update the SessionState logic to support this allocator type being provided, and use it when doing device allocations for initializers. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Performance.
Yulong Wang (fs-eire)
added a commit
that referenced
this pull request
Aug 11, 2025
### Description add usage of `ReadonlyAllocator` for WebGPU EP ### Motivation and Context `ReadonlyAllocator` is added in #25348 to allow an EP to register a separated allocator only use for initializers. WebGPU EP already has different handling to initializers and non-initializers, and this change makes WebGPU EP to use the preferred way to deal with it. Now the allocator depends on the `OrtAllocatorType` instead of the session initialization status.
Maximilian Müller (gedoensmax)
pushed a commit
to gedoensmax/onnxruntime
that referenced
this pull request
Sep 2, 2025
### Description add usage of `ReadonlyAllocator` for WebGPU EP ### Motivation and Context `ReadonlyAllocator` is added in microsoft#25348 to allow an EP to register a separated allocator only use for initializers. WebGPU EP already has different handling to initializers and non-initializers, and this change makes WebGPU EP to use the preferred way to deal with it. Now the allocator depends on the `OrtAllocatorType` instead of the session initialization status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add new allocator type of OrtReadOnlyAllocator to enable providing a separate allocator that is only used for initializers.
Update the SessionState logic to support this allocator type being provided, and use it when doing device allocations for initializers.
Motivation and Context
Performance.