[Justin Chu (justinchuby)]

What this enables

The NEON f16↔f32 conversion kernel (cast_kernel_neon.cpp) is excluded on all Apple targets today: mlas.h gates MLAS_F16VEC_INTRINSICS_SUPPORTED behind #if !defined(__APPLE__), and cmake/onnxruntime_mlas.cmake excludes the source with if (NOT APPLE). macOS ARM64 therefore falls back to scalar conversion loops.

This enables only that cast kernel on macOS arm64, via a new MLAS_CAST_F16_NEON_SUPPORTED macro kept separate from MLAS_F16VEC_INTRINSICS_SUPPORTED, so fp16 arithmetic, i8mm, and bf16 remain disabled on Apple.

Scope

The C++ path is gated on TARGET_OS_OSX plus MLAS_TARGET_ARM64; CMake uses CMAKE_SYSTEM_NAME STREQUAL "Darwin" within the ARM64 source branch. This excludes iOS, tvOS, and visionOS.

Universal2 is supported: ONNXRUNTIME_MLAS_MULTI_ARCH compiles this source only into the onnxruntime_mlas_arm64 slice. The x86_64 slice remains unchanged and does not define MLAS_CAST_F16_NEON_SUPPORTED.

No -march flag is needed

vcvt_f32_f16 / vcvt_f16_f32 are baseline AArch64 conversion instructions (FCVTL/FCVTN). arm_neon.h guards them under #if (__ARM_FP & 2), not __ARM_FEATURE_FP16_VECTOR_ARITHMETIC. FEAT_FP16 governs fp16 arithmetic, which this kernel does not perform.

NaN contract

For NaN results the tests assert NaN-ness and the sign bit; payload bits are deliberately not compared. Hardware conversion can preserve or quiet/truncate a NaN payload while the scalar float-to-half reference canonicalizes it to 0x7E00. Raw-bit equality remains required for every non-NaN result.

Test coverage

Test	What it establishes
CastFp16ShortExecuteTest	Bit-exact non-NaN conversion at 18 boundary-focused lengths: 1, 2, 3, 5, 7, 9, 15, 16, 17, 31, 32, 63, 64, 128, 255, 256, 1024, and 65536
CastFp16UnalignedSourcesTest	All three non-zero source-alignment offsets for both conversion directions, exercising the NEON alignment prologues plus vector and tail paths
CastFp16SpecialValuesTest	±0, ±Inf, positive/negative qNaN and sNaN, positive/negative half subnormals, subnormal underflow ties, and a normal round-to-nearest-even tie
CastFp16KernelDispatchTest	Both MLAS cast dispatch pointers are non-null on macOS arm64; its Apple platform guard is independent of MLAS_CAST_F16_NEON_SUPPORTED, preventing a missing feature macro from making the test a false-green no-op

Validation

• Targeted Windows x64 build of onnxruntime_mlas_test succeeds after syncing current main; all 39 CastFp16* tests pass. This validates test integration and cross-platform behavior but does not substitute for Apple ARM64 execution.
• The prior PR head passed native Apple Silicon CPU build-and-test lanes and the universal2 webgpu / build-and-test (arm64, x86_64, Release) lane. Fresh CI for the newly pushed, main-synced head is intentionally not awaited here.
• An earlier kernel-only AArch64 cross-build executed under QEMU validated the baseline NEON conversion instruction semantics, including the expected NaN payload difference. This was Linux emulation, not Apple Silicon hardware validation.

No performance claims

Nothing was measured. This replaces scalar conversion loops with the existing vectorized kernel on macOS arm64; whether and where that matters is not established here.

[Copilot]

Pull request overview

Enables NEON FP16↔FP32 conversion kernels on macOS ARM64 while keeping other FP16 arithmetic disabled.

Changes:

• Adds a macOS ARM64-specific capability macro and CMake source inclusion.
• Enables NEON cast-kernel dispatch.
• Expands conversion, special-value, and dispatch tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
cmake/onnxruntime_mlas.cmake	Includes the NEON cast kernel for Darwin ARM64 builds.
onnxruntime/core/mlas/inc/mlas.h	Defines macOS ARM64 cast support.
onnxruntime/core/mlas/lib/cast_kernel_neon.cpp	Updates kernel support documentation.
onnxruntime/core/mlas/lib/mlasi.h	Exposes NEON cast declarations under the new capability.
onnxruntime/core/mlas/lib/platform.cpp	Enables runtime dispatch under the new capability.
onnxruntime/test/mlas/unittest/test_cast_fp16.cpp	Expands conversion and dispatch tests.

Suppressed comments (1)

onnxruntime/test/mlas/unittest/test_cast_fp16.cpp:146

• This F32→F16 case set has only positive NaNs and no inputs whose half result is denormal. Therefore it does not exercise the NaN sign contract or the advertised denormal coverage in this conversion direction. Add negative NaNs and exact positive/negative half-denormal equivalents.

    std::vector<float> f32_input = {
        0.0f, -0.0f,
        std::numeric_limits<float>::infinity(),
        -std::numeric_limits<float>::infinity(),
        std::numeric_limits<float>::quiet_NaN(),
        std::numeric_limits<float>::signaling_NaN(),
        1.00048828125f};

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Hariharan Seshadri (hariharans29)]

Review — PR #31993: Enable MLAS f16↔f32 cast kernel on Apple macOS arm64

Summary of changes (6 files, targeted +100/−~10)

• mlas.h: adds a new MLAS_CAST_F16_NEON_SUPPORTED macro, gated on __APPLE__ && MLAS_TARGET_ARM64 && TARGET_OS_OSX (via <TargetConditionals.h>). Deliberately kept separate from the existing MLAS_F16VEC_INTRINSICS_SUPPORTED so fp16 arithmetic, i8mm, and bf16 stay disabled on Apple — this only opens the conversion path.
• cast_kernel_neon.cpp: header comment updated to reflect the two-macro condition. Kernel body untouched.
• mlasi.h and platform.cpp: declaration and dispatch-table assignment guards widened from MLAS_F16VEC_INTRINSICS_SUPPORTED && MLAS_TARGET_ARM64 to (MLAS_F16VEC_INTRINSICS_SUPPORTED || MLAS_CAST_F16_NEON_SUPPORTED) && MLAS_TARGET_ARM64 — symmetric.
• cmake/onnxruntime_mlas.cmake: adds cast_kernel_neon.cpp to mlas_platform_srcs on CMAKE_SYSTEM_NAME STREQUAL "Darwin" (deliberately not the CMake APPLE which is true on iOS/tvOS/visionOS as well). No compile flag added.
• test_cast_fp16.cpp: adds TestSpecialValues (IEEE 754 corner cases + RNE tie) and TestKernelIsDispatched (dispatch-table pointer non-null), plus more length values in the bulk test.

The technical claim is correct — I verified it

vcvt_f32_f16 and vcvt_f16_f32 are baseline AArch64 (FCVTL/FCVTN in the F16C-in-AArch64 sense — arm_neon.h guards them under #if (__ARM_FP & 2), not __ARM_FEATURE_FP16_VECTOR_ARITHMETIC). FEAT_FP16 governs float16x8_t arithmetic (fadd/fmla on packed halves), which this kernel does not perform. The kernel body (cast_kernel_neon.cpp lines 47–177) uses only baseline conversions plus load/store/reinterpret intrinsics — no packed fp16 arithmetic anywhere. The prior "hardware specific compilation flag" comment in mlas.h was correct in aggregate for the fp16 kernel family, but overbroad for the cast path specifically. This PR punches a precise, minimal hole in that gate.

Scope isolation is well-done

I traced every other consumer of MLAS_F16VEC_INTRINSICS_SUPPORTED:

• activate_fp16.cpp, dwconv.cpp, pooling_fp16.cpp, halfgemm.cpp, halfgemm.h (declaration and MlasHalfGemmGetDispatch), hgemm_kernel_neon.cpp, gelu_neon_fp16.cpp, erf_neon_fp16.cpp, eltwise_kernel_neon.cpp, softmax_kernel_neon.cpp, rotary_embedding_kernel_neon.cpp, qnbitgemm_kernel_neon.cpp, qnbitgemm_kernel_neon.h, fp16_common.h — all use float16x8_t arithmetic and are correctly not widened by this PR. On macOS arm64, MlasHalfGemmGetDispatch() still returns &MlasHalfGemmDispatchDefault (scalar), which is what you want. ✓
• The platform.cpp block that assigns ErfFP16KernelRoutine / GeluFP16KernelRoutine / TanhFP16KernelRoutine is gated on MLAS_F16VEC_INTRINSICS_SUPPORTED (not the widened guard), so those stay nullptr on Apple. ✓

Only the two cast-dispatch slots (CastF16ToF32Kernel, CastF32ToF16Kernel) get wired up, which is exactly the stated scope.

NaN test contract is right

I initially raised an eyebrow at the asymmetric NaN assertion (std::isnan(dispatch) + sign only, vs. bit-exact for non-NaN). The description walks through why: hardware FCVTN preserves the payload bits (truncated to fit in 10 mantissa bits) while the software reference MLAS_Float2Half canonicalizes every NaN to 0x7E00. The two examples in the description (0x7FA00000 → 0x7F00 hardware vs. 0x7E00 software) match FCVTN's spec (it copies bits [22:13] into the fp16 mantissa, which for 0x7FA00000 = binary(exp=all-ones, mant=010 0000 0000 0000 0000 0000) gives fp16 mantissa 1 0000 0000 0 = 0x300, so the result is 0x7C00 | 0x300 = 0x7F00). ✓ Bit-exact NaN assertion here would break the test on every real AArch64 target, so this contract is required, not stylistic.

Universal2 clarification worth making

The PR description says "No universal2", but I traced the CMake and that's misleading — the description means "we don't add any x86_64-Apple dispatch", not "universal2 arm64 slices don't get the kernel". Concretely:

• When OSX_ARCHITECTURES = "arm64;x86_64", ONNXRUNTIME_MLAS_MULTI_ARCH = TRUE (onnxruntime_mlas.cmake line 486).
• The ARM64 branch (line ~641) runs and appends cast_kernel_neon.cpp to mlas_platform_srcs.
• onnxruntime_add_static_library(onnxruntime_mlas_arm64 ${mlas_platform_srcs}) builds those sources with OSX_ARCHITECTURES "arm64", so __aarch64__ is defined, MLAS_TARGET_ARM64 is set, TARGET_OS_OSX is 1 → MLAS_CAST_F16_NEON_SUPPORTED is defined for the arm64 slice.
• The x86_64 slice's compilation of platform.cpp runs with MLAS_TARGET_ARM64 undefined, so the whole widened guard is dead code there. ✓

That's actually the correct behavior (arm64 slice gets NEON, x86_64 slice stays scalar). The PR description should be tweaked to say "no new x86_64-Apple dispatch; universal2 arm64 slice inherits the macOS-arm64 behavior naturally" so the Copilot review comment (medium severity) doesn't get read as "code is wrong". Either fix the description or add a one-line CMake comment explaining the interaction with ONNXRUNTIME_MLAS_MULTI_ARCH. I'd take the latter — it's the more discoverable place.

Comments / suggestions

1. Stale comment above MLAS_F16VEC_INTRINSICS_SUPPORTED in mlas.h. Lines 92–95 still say:

   // Had to temporary disable fp16 under APPLE ARM64, as compiling
   // the source files require a hardware specific compilation flag.
   // When building an universial binary for APPLE, this flag would
   // cause trouble for x64 target.
   

   This is now imprecise — the "compiling requires a hardware specific compilation flag" claim is not true for cast_kernel_neon.cpp specifically (per the whole point of this PR). Tighten to something like "fp16 arithmetic kernels need -march=armv8.2-a+fp16 and are disabled on Apple; the f16↔f32 cast kernel is baseline AArch64 and is enabled separately below via MLAS_CAST_F16_NEON_SUPPORTED." Prevents the next contributor from staring at the same "temporary" hedge for another five years.

2. Copilot low-priority Set up CI with Azure Pipelines #1 (negative NaN coverage) — take it. It's four lines, and it converts the sign-bit assertion from vacuous to load-bearing. Trivially:

   const uint16_t kNegQNaN = 0xFE00;
   const uint16_t kNegSNaN = 0xFC01;

   and add to special_bits. On the F32→F16 direction, similarly add -std::numeric_limits<float>::quiet_NaN() / -std::numeric_limits<float>::signaling_NaN(). Right now if a kernel silently zeroed the sign bit on negative NaNs, this test would not catch it.

3. Copilot low-priority Remove vsts test runner in cmake file #2 (test-length range wording) — the description says "lengths 1–259" but the test iterates 18 discrete values in {1, 2, 3, 5, 7, 9, 15, 16, 17, 31, 32, 63, 64, 128, 255, 256, 1024, 65536}. The test is fine as a boundary set; just fix the description to "18 boundary-focused lengths from 1 to 65 536, including all non-multiples of the 4/8-lane vector widths." Precision matters here since the CI story rests on this test running natively.

4. Consistency nit in the widened guard. The condition (defined(MLAS_F16VEC_INTRINSICS_SUPPORTED) || defined(MLAS_CAST_F16_NEON_SUPPORTED)) && defined(MLAS_TARGET_ARM64) appears in two places (mlasi.h, platform.cpp). Given it will likely repeat if a follow-up enables another Apple-arm64 kernel, consider centralizing:

   // in mlas.h or a shared internal header
   #if defined(MLAS_F16VEC_INTRINSICS_SUPPORTED) || defined(MLAS_CAST_F16_NEON_SUPPORTED)
   #define MLAS_HAS_NEON_F16_CAST
   #endif

   then both guards become #if defined(MLAS_HAS_NEON_F16_CAST) && defined(MLAS_TARGET_ARM64). Not a blocker.

5. TestKernelIsDispatched compiles to nothing on x86 CI hosts. That's intentional and documented in the comment (This test compiles to nothing on platforms without a vectorised kernel). But RegisterTests() unconditionally returns 1 and registers the test, meaning on x86 you'll see a CastFp16/KernelDispatched gtest listing that runs an empty body. Fine, but if you want the test to be visible only on platforms it can meaningfully assert on, wrap the RegisterTest call itself in the same #if. Cosmetic.

Non-issues I re-verified

• <TargetConditionals.h> include in a public MLAS header is safe: the include is nested inside #if defined(__APPLE__), and on Apple SDKs the header is always available (part of the base SDK, not a specific framework). No -framework dependency added. ✓
• Kernel body uses float16_t via float16x4_t / vreinterpret_*_f16. That type is baseline on AArch64 (as opposed to _Float16 arithmetic, which is FEAT_FP16-gated). Compiles clean on any AArch64 target with arm_neon.h. ✓
• No regressions to non-Apple ARM64 paths: on Linux arm64 (or Windows ARM64), MLAS_F16VEC_INTRINSICS_SUPPORTED is still set the same way, so the widened guard's LHS is true, and the RHS MLAS_CAST_F16_NEON_SUPPORTED is undefined but harmless. Existing behavior unchanged. ✓
• The #include "core/mlas/lib/mlasi.h" in test_cast_fp16.cpp is needed to reach GetMlasPlatform(). Existing tests in that directory already reach into internal headers (platform.cpp grep shows several such patterns). Consistent. ✓
• CI: description accurately describes the two transient infra flakes (Build Minimal (Globally Allowed Types) queue starvation, iOS Detox hang). Both are documented pre-existing issues and are unrelated to this diff. ✓

Recommendation

Approve after (1) the mlas.h "temporary" comment is tightened and (2) the negative-NaN coverage is added. Both are small, don't warrant another round. (3) is a description edit. (4) and (5) are follow-up-worthy nits.

This is a clean, well-scoped, correctly-reasoned enabling PR — the author has already done the hard part of separating conversion (baseline) from arithmetic (FEAT_FP16), and the test contract for NaN handling is exactly right for hardware-vs-software round-tripping.