Repository navigation
Add support for CUDA architecture family codes - #27278
Merged
Tianlei Wu (tianleiwu) merged 1 commit intoFeb 8, 2026
Merged
Conversation
This change extends CUDA architecture handling to support family-specific codes (suffix 'f') introduced in CUDA 12.9, aligning with updates made to Triton Inference Server repositories (backend and onnxruntime_backend). Changes: 1. Added CUDAARCHS environment variable support (standard CMake variable) - Allows users to override architecture list via environment variable - Takes precedence when CMAKE_CUDA_ARCHITECTURES is not set 2. Extended regex patterns to recognize family code suffix 'f' - Supports codes like 100f, 110f, 120f for CC 10.x, 11.x, 12.x families - Preserves 'f' suffix during parsing phase 3. Updated normalization logic to handle family codes - Family codes (ending with 'f') preserved without adding -real suffix - Traditional codes continue to receive -real or -a-real suffixes - Architecture-specific codes (with 'a') remain unchanged 4. Extended architecture support lists - Added SM 110 to ARCHITECTURES_WITH_KERNELS - Added SM 110 to ARCHITECTURES_WITH_ACCEL Family-specific codes (introduced in CUDA 12.9/Blackwell) enable forward compatibility within a GPU family. For example, 100f runs on CC 10.0, 10.3, and future 10.x devices, using features common across the family. Usage examples: - CUDAARCHS="75;80;90;100f;110f;120f" cmake .. - cmake -DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;90-real;100f;120f" .. - python build.py --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES="100f;110f" The implementation supports mixed formats in the same list: - Traditional: 75-real, 80-real, 90-real - Architecture-specific: 90a-real (CC 9.0 only) - Family-specific: 100f, 110f, 120f (entire family) Note: Current defaults still use bare numbers (75;80;90;100;120) which normalize to architecture-specific codes with 'a' suffix. Users who want family-specific behavior should explicitly use the 'f' suffix via CUDAARCHS environment variable or CMAKE_CUDA_ARCHITECTURES. References: - NVIDIA Blackwell and CUDA 12.9 Family-Specific Architecture Features: https://developer.nvidia.com/blog/nvidia-blackwell-and-nvidia-cuda-12-9-introduce-family-specific-architecture-features/ - Triton Inference Server backend updates (commit f5e901f)
Misha Chornyi (mc-nv)
force-pushed
the
mc-nv/TRI-624/update-arch-codes
branch
from
February 7, 2026 02:14
44f9e9f to
319e132
Compare
Contributor
Author
Dmitry Tokarev (dmitry-tokarev-nv)
approved these changes
Feb 7, 2026
Contributor
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
Tianlei Wu (tianleiwu)
approved these changes
Feb 8, 2026
Tianlei Wu (tianleiwu)
pushed a commit
that referenced
this pull request
Feb 12, 2026
This change extends CUDA architecture handling to support family-specific codes (suffix 'f') introduced in CUDA 12.9, aligning with updates made to Triton Inference Server repositories (backend and onnxruntime_backend). Changes: 1. Added CUDAARCHS environment variable support (standard CMake variable) - Allows users to override architecture list via environment variable - Takes precedence when CMAKE_CUDA_ARCHITECTURES is not set 2. Extended regex patterns to recognize family code suffix 'f' - Supports codes like 100f, 110f, 120f for CC 10.x, 11.x, 12.x families - Preserves 'f' suffix during parsing phase 3. Updated normalization logic to handle family codes - Family codes (ending with 'f') preserved without adding -real suffix - Traditional codes continue to receive -real or -a-real suffixes - Architecture-specific codes (with 'a') remain unchanged 4. Extended architecture support lists - Added SM 110 to ARCHITECTURES_WITH_KERNELS - Added SM 110 to ARCHITECTURES_WITH_ACCEL Family-specific codes (introduced in CUDA 12.9/Blackwell) enable forward compatibility within a GPU family. For example, 100f runs on CC 10.0, 10.3, and future 10.x devices, using features common across the family. Usage examples: - CUDAARCHS="75;80;90;100f;110f;120f" cmake .. - cmake -DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;90-real;100f;120f" .. - python build.py --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES="100f;110f" The implementation supports mixed formats in the same list: - Traditional: 75-real, 80-real, 90-real - Architecture-specific: 90a-real (CC 9.0 only) - Family-specific: 100f, 110f, 120f (entire family) Note: Current defaults still use bare numbers (75;80;90;100;120) which normalize to architecture-specific codes with 'a' suffix. Users who want family-specific behavior should explicitly use the 'f' suffix via CUDAARCHS environment variable or CMAKE_CUDA_ARCHITECTURES. References: - NVIDIA Blackwell and CUDA 12.9 Family-Specific Architecture Features: https://developer.nvidia.com/blog/nvidia-blackwell-and-nvidia-cuda-12-9-introduce-family-specific-architecture-features/ - Triton Inference Server backend updates (commit f5e901f) ### Description <!-- Describe your changes. --> ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
Tianlei Wu (tianleiwu)
added a commit
that referenced
this pull request
Feb 13, 2026
This cherry-picks the following commits for the 1.24.2 release: - #27096 - #27077 - #26677 - #27238 - #27213 - #27256 - #27278 - #27275 - #27276 - #27216 - #27271 - #27299 - #27294 - #27266 - #27176 - #27126 - #27252 --------- Co-authored-by: Xiaofei Han <xiaofeihan@microsoft.com> Co-authored-by: Jiajia Qin <jiajiaqin@microsoft.com> Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com> Co-authored-by: qti-monumeen <monumeen@qti.qualcomm.com> Co-authored-by: Ankit Maheshkar <ankit.maheshkar@intel.com> Co-authored-by: Eric Crawford <eric.r.crawford@intel.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: guschmue <22941064+guschmue@users.noreply.github.com> Co-authored-by: Guenther Schmuelling <guschmue@microsoft.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: angelser <32746004+angelser@users.noreply.github.com> Co-authored-by: Angela Serrano Brummett <angelser@microsoft.com> Co-authored-by: Misha Chornyi <99709299+mc-nv@users.noreply.github.com> Co-authored-by: hariharans29 <9969784+hariharans29@users.noreply.github.com> Co-authored-by: eserscor <erscor@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Baiju Meswani <bmeswani@microsoft.com> Co-authored-by: Adrian Lizarraga <adlizarraga@microsoft.com> Co-authored-by: Ti-Tai Wang <titaiwang@microsoft.com> Co-authored-by: bmehta001 <bmehta001@users.noreply.github.com>
This was referenced Feb 23, 2026
deps(nuget): Bump the microsoft-packages group with 2 updates
Ellerbach/azure-ai-search-simulator#53
Merged
This was referenced May 4, 2026
This was referenced Jul 1, 2026
This was referenced Jul 8, 2026
This was referenced Jul 22, 2026
This was referenced Aug 1, 2026
This was referenced Aug 8, 2026
This was referenced Aug 17, 2026
This was referenced Aug 27, 2026
This was referenced Sep 17, 2026
Connor Baker (ConnorBaker)
added a commit
to ConnorBaker/onnxruntime
that referenced
this pull request
Oct 14, 2026
setup_cuda_architectures() dropped the suffix of every "-real" entry
("^(([1-9])([0-9])+)[af]?-real$" keeps only the number) and then
re-added "a" for the numbers in ARCHITECTURES_WITH_ACCEL. That has
two effects:
* "120f-real" became "120a-real", and "100f-real" became "100a-real".
Per the CUDA C++ Programming Guide (Compute Capabilities appendix),
compute_120a/sm_120a code runs only on compute capability 12.0,
while the family target compute_120f covers 12.0 and 12.1, and
compute_100f covers 10.0, 10.3 and 10.7. So a build asking for
portable 12.x SASS got code that does not run on 12.1 (GB10/DGX
Spark), and a 10.x one got code that does not run on 10.3 (B300).
Bare "120f" was already kept verbatim (family codes were added in
microsoft#27278, 6625856, which extended the -real regex to [af]? but
still discarded the match).
* "107a-real" (any "a" entry whose number is not in
ARCHITECTURES_WITH_ACCEL) silently became "107-real".
Keep the suffix: "<cc>f-real" is kept verbatim (the "-real" tells it
apart from "<cc>f", which also embeds PTX), and "<cc>a-real" keeps its
"a". The normalization loop passes "<cc>f-real" through like "<cc>f".
_ORIG now holds "120f-real" for such an entry. Its consumers
(HAS_SM*_OR_LATER, which matches the leading number, and the
onnxruntime_parse_cuda_arch-based gates) handle that. Since "121" and
"121a-real" now normalize to the same entry, the normalized list is
de-duplicated.
Coordination with the open microsoft#29601 ("Follow
standard CMake semantics for CUDA architecture suffixes"): that PR
splits the parsing into real and virtual lists, keeps every -virtual
entry, and gives suffix-less entries PTX as well. It uses the same
"-real" regex as the second branch here, "^(([1-9])([0-9])+[af]?)-real$"
(suffix captured), and adds the same
list(REMOVE_DUPLICATES CMAKE_CUDA_ARCHITECTURES_NORMALIZED), so the two
changes overlap textually without contradicting each other. One
difference: microsoft#29601 still emits family entries without a kind suffix
("120f"), so "120f-real" gains PTX and "120f-virtual" gains SASS. On
top of microsoft#29601 the equivalent fix is to emit "<cc>f-real" for real-list
and "<cc>f-virtual" for virtual-list family entries.
cmake -P harness (independent model of which GPUs each entry
generates SASS for, plus "an explicit a/f suffix survives"): before, 7
violations on 5 of 30 lists ("120f-real", "121f-real", "100f-real",
"107a-real", "120f-real;121a-real"). After, none. Only lists that
contain a "-real" entry normalize differently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Connor Baker (ConnorBaker)
added a commit
to ConnorBaker/onnxruntime
that referenced
this pull request
Oct 14, 2026
setup_cuda_architectures() dropped the suffix of every "-real" entry
("^(([1-9])([0-9])+)[af]?-real$" keeps only the number) and then
re-added "a" for the numbers in ARCHITECTURES_WITH_ACCEL. That has
two effects:
* "120f-real" became "120a-real", and "100f-real" became "100a-real".
Per the CUDA C++ Programming Guide (Compute Capabilities appendix),
compute_120a/sm_120a code runs only on compute capability 12.0,
while the family target compute_120f covers 12.0 and 12.1, and
compute_100f covers 10.0, 10.3 and 10.7. So a build asking for
portable 12.x SASS got code that does not run on 12.1 (GB10/DGX
Spark), and a 10.x one got code that does not run on 10.3 (B300).
Bare "120f" was already kept verbatim (family codes were added in
microsoft#27278, 6625856, which extended the -real regex to [af]? but
still discarded the match).
* "107a-real" (any "a" entry whose number is not in
ARCHITECTURES_WITH_ACCEL) silently became "107-real".
Keep the suffix: "<cc>f-real" is kept verbatim (the "-real" tells it
apart from "<cc>f", which also embeds PTX), and "<cc>a-real" keeps its
"a". The normalization loop passes "<cc>f-real" through like "<cc>f".
_ORIG now holds "120f-real" for such an entry. Its consumers
(HAS_SM*_OR_LATER, which matches the leading number, and the
onnxruntime_parse_cuda_arch-based gates) handle that. Since "121" and
"121a-real" now normalize to the same entry, the normalized list is
de-duplicated.
Coordination with the open microsoft#29601 ("Follow
standard CMake semantics for CUDA architecture suffixes"): that PR
splits the parsing into real and virtual lists, keeps every -virtual
entry, and gives suffix-less entries PTX as well. It uses the same
"-real" regex as the second branch here, "^(([1-9])([0-9])+[af]?)-real$"
(suffix captured), and adds the same
list(REMOVE_DUPLICATES CMAKE_CUDA_ARCHITECTURES_NORMALIZED), so the two
changes overlap textually without contradicting each other. One
difference: microsoft#29601 still emits family entries without a kind suffix
("120f"), so "120f-real" gains PTX and "120f-virtual" gains SASS. On
top of microsoft#29601 the equivalent fix is to emit "<cc>f-real" for real-list
and "<cc>f-virtual" for virtual-list family entries.
cmake -P harness (independent model of which GPUs each entry
generates SASS for, plus "an explicit a/f suffix survives"): before, 7
violations on 5 of 30 lists ("120f-real", "121f-real", "100f-real",
"107a-real", "120f-real;121a-real"). After, none. Only lists that
contain a "-real" entry normalize differently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Connor Baker (ConnorBaker)
added a commit
to ConnorBaker/onnxruntime
that referenced
this pull request
Nov 3, 2026
setup_cuda_architectures() dropped the suffix of every "-real" entry
("^(([1-9])([0-9])+)[af]?-real$" keeps only the number) and then
re-added "a" for the numbers in ARCHITECTURES_WITH_ACCEL. That has
two effects:
* "120f-real" became "120a-real", and "100f-real" became "100a-real".
Per the CUDA C++ Programming Guide (Compute Capabilities appendix),
compute_120a/sm_120a code runs only on compute capability 12.0,
while the family target compute_120f covers 12.0 and 12.1, and
compute_100f covers 10.0, 10.3 and 10.7. So a build asking for
portable 12.x SASS got code that does not run on 12.1 (GB10/DGX
Spark), and a 10.x one got code that does not run on 10.3 (B300).
Bare "120f" was already kept verbatim (family codes were added in
microsoft#27278, 6625856, which extended the -real regex to [af]? but
still discarded the match).
* "107a-real" (any "a" entry whose number is not in
ARCHITECTURES_WITH_ACCEL) silently became "107-real".
Keep the suffix: "<cc>f-real" is kept verbatim (the "-real" tells it
apart from "<cc>f", which also embeds PTX), and "<cc>a-real" keeps its
"a". The normalization loop passes "<cc>f-real" through like "<cc>f".
_ORIG now holds "120f-real" for such an entry. Its consumers
(HAS_SM*_OR_LATER, which matches the leading number, and the
onnxruntime_parse_cuda_arch-based gates) handle that. Since "121" and
"121a-real" now normalize to the same entry, the normalized list is
de-duplicated.
Coordination with the open microsoft#29601 ("Follow
standard CMake semantics for CUDA architecture suffixes"): that PR
splits the parsing into real and virtual lists, keeps every -virtual
entry, and gives suffix-less entries PTX as well. It uses the same
"-real" regex as the second branch here, "^(([1-9])([0-9])+[af]?)-real$"
(suffix captured), and adds the same
list(REMOVE_DUPLICATES CMAKE_CUDA_ARCHITECTURES_NORMALIZED), so the two
changes overlap textually without contradicting each other. One
difference: microsoft#29601 still emits family entries without a kind suffix
("120f"), so "120f-real" gains PTX and "120f-virtual" gains SASS. On
top of microsoft#29601 the equivalent fix is to emit "<cc>f-real" for real-list
and "<cc>f-virtual" for virtual-list family entries.
cmake -P harness (independent model of which GPUs each entry
generates SASS for, plus "an explicit a/f suffix survives"): before, 7
violations on 5 of 30 lists ("120f-real", "121f-real", "100f-real",
"107a-real", "120f-real;121a-real"). After, none. Only lists that
contain a "-real" entry normalize differently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change extends CUDA architecture handling to support family-specific codes (suffix 'f') introduced in CUDA 12.9, aligning with updates made to Triton Inference Server repositories (backend and onnxruntime_backend).
Changes:
Added CUDAARCHS environment variable support (standard CMake variable)
Extended regex patterns to recognize family code suffix 'f'
Updated normalization logic to handle family codes
Extended architecture support lists
Family-specific codes (introduced in CUDA 12.9/Blackwell) enable forward compatibility within a GPU family. For example, 100f runs on CC 10.0, 10.3, and future 10.x devices, using features common across the family.
Usage examples:
The implementation supports mixed formats in the same list:
Note: Current defaults still use bare numbers (75;80;90;100;120) which normalize to architecture-specific codes with 'a' suffix. Users who want family-specific behavior should explicitly use the 'f' suffix via CUDAARCHS environment variable or CMAKE_CUDA_ARCHITECTURES.
References:
Description
Motivation and Context