Skip to content

[Bug]: MIGraphX EP crashes in bundled amdhip64_7.dll (access violation, 0xc0000005) compiling a large model on RX 7900 XTX #15

Open

Description

@Looong01

Which sample?

N/A — this is a Windows ML runtime crash in the MIGraphX execution provider itself (package MicrosoftCorporationII.WinML.AMD.GPU.EP.1.8, EP version 1.8.57.0), not tied to any sample code in this repo. Filing here per the maintainers' guidance in case triage wants to redirect to microsoft/WindowsAppSDK; happy to move it there if preferred.

Steps to reproduce

  1. On a Windows machine whose only GPU is an AMD Radeon RX 7900 XTX (gfx1100), build a C++
    application using the ONNX Runtime C++ API (via Windows ML's EP catalog) that:
    • Discovers and registers the MIGraphX EP via WinMLEpCatalog (EnsureReady + GetLibraryPath),
      then Ort::Env::RegisterExecutionProviderLibrary("MIGraphX", ).
    • Selects the MIGraphX device via Ort::Env::GetEpDevices(), filtering for EP name
      "MIGraphX", and appends it with SessionOptions::AppendExecutionProvider_V2.
    • Before session creation, pins the ONNX graph's "batch" free dimension to a fixed value
      (e.g. 8) via SessionOptions::AddFreeDimensionOverrideByName("batch", 8), on a ~930MB
      ONNX graph (converted from a KataGo Go-playing neural network; the exact model isn't
      important, any moderately large graph with a pinned static batch dimension should do).
  2. Construct the Ort::Session (first-time compile, no on-disk compile cache present yet for
    this model/shape).
  3. Observe: CPU spikes to 100% for several minutes, host RAM climbs to several GB, VRAM usage
    stays at 0% throughout, then the process crashes with no exception and no error output.

This reproduced identically twice, from two separate process launches (once via an interactive
stdio/GTP-protocol subprocess, once via a one-shot batch-mode subcommand) — same crash, same
module, same offset both times.

Minimal repro code is not attached since this was found via an existing open-source project
(a fork of KataGo with a Windows ML backend); happy to extract a minimal standalone repro if
that would help triage — let me know.

Expected behavior

The model compiles successfully and inference proceeds, OR the failure surfaces as a catchable
ONNX Runtime exception (e.g. Ort::Exception / EP_FAIL) rather than a native access violation.

For reference, the identical model/code/EP-package-version compiles successfully (in ~9 minutes)
on a different machine with an integrated AMD GPU + NPU, so this appears specific to this
discrete-GPU (RX 7900 XTX) configuration.

Actual behavior

The process hard-crashes with an unhandled access violation. Windows Event Viewer (Application
log, Event ID 1000, source "Application Error") shows:

Faulting application name: katago.exe, version: 0.0.0.0, time stamp: 0x6a4fb5f2
Faulting module name: amdhip64_7.dll, version: 7.2.2606.20, time stamp: 0x69e9440d
Exception code: 0xc0000005
Fault offset: 0x0000000000465007
Faulting module path: C:\Program Files\WindowsApps
MicrosoftCorporationII.WinML.AMD.GPU.EP.1.8_1.8.57.0_x64__8wekyb3d8bbwe
ExecutionProvider\amdhip64_7.dll

Reproduced twice (different PIDs, different process launches) with the IDENTICAL faulting
offset both times — this rules out a random memory-pressure or race-condition explanation.
System RAM was 64GB with 55%+ free at time of testing, so this is not a simple OOM.

No exception is thrown into the calling application before the crash — the crash happens
entirely inside the EP's own bundled amdhip64_7.dll, downstream of the Ort::Session
constructor, so it cannot be caught by the caller's try/catch around session creation.

Windows version

Windows 11 25H2 OS Internal version: 26200.8737 ( Win32NT 10.0.26200.0 Microsoft Windows NT 10.0.26200.0)

Architecture

x64

Hardware (CPU / GPU / NPU)

Intel Core i7-9700, AMD Radeon RX 7900XTX

Execution provider

MIGraphX (explicitly selected via winmlProvider config option in the calling application; not using automatic EP-selection policy). EP name as reported by the catalog: MIGraphXExecutionProvider. Package: MicrosoftCorporationII.WinML.AMD.GPU.EP.1.8_8wekyb3d8bbwe, version 1.8.57.0.

Windows ML / Windows App SDK package version

Microsoft.Windows.AI.MachineLearning 2.0.297-preview

Diagnostic logs (optional)

No response

Activity

  1. Looong01 commented on Jul 10, 2026

    @Looong01
    Author
    日志名称:          Application
    来源:            Application Error
    日期:            2026/7/10 8:52:49
    事件 ID:         1000
    任务类别:          应用程序崩溃事件
    级别:            错误
    关键字:           
    用户:            DESKTOP-6RS03CK\lizel
    计算机:           DESKTOP-6RS03CK
    描述:
    出错应用程序名称: katago.exe,版本: 0.0.0.0,时间戳: 0x6a4fb5f2
    出错模块名称: amdhip64_7.dll, 版本: 7.2.2606.20,时间戳: 0x69e9440d
    异常代码: 0xc0000005
    错误偏移: 0x0000000000465007
    出错进程 ID: 0x97A8
    出错应用程序开始时间: 0x1DD1005E69941F1
    Faulting 应用程序路径: C:\Envs\katago-winml\katago.exe
    Faulting 模块路径: C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.AMD.GPU.EP.1.8_1.8.57.0_x64__8wekyb3d8bbwe\ExecutionProvider\amdhip64_7.dll
    Report ID: a195a9c9-5a63-4e9a-bee0-c9e25a7fb69d
    Faulting 包全名: 
    Faulting 程序包相对应用程序 ID:  个
    事件 Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Application Error" Guid="{a0e9b465-b939-57d7-b27d-95d8e925ff57}" />
        <EventID>1000</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>100</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2026-07-10T00:52:49.0190159Z" />
        <EventRecordID>37793</EventRecordID>
        <Correlation />
        <Execution ProcessID="42228" ThreadID="34764" />
        <Channel>Application</Channel>
        <Computer>DESKTOP-6RS03CK</Computer>
        <Security UserID="S-1-5-21-1426860120-2675591737-2411372617-1012" />
      </System>
      <EventData>
        <Data Name="AppName">katago.exe</Data>
        <Data Name="AppVersion">0.0.0.0</Data>
        <Data Name="AppTimeStamp">6a4fb5f2</Data>
        <Data Name="ModuleName">amdhip64_7.dll</Data>
        <Data Name="ModuleVersion">7.2.2606.20</Data>
        <Data Name="ModuleTimeStamp">69e9440d</Data>
        <Data Name="ExceptionCode">c0000005</Data>
        <Data Name="FaultingOffset">0000000000465007</Data>
        <Data Name="ProcessId">0x97a8</Data>
        <Data Name="ProcessCreationTime">0x1dd1005e69941f1</Data>
        <Data Name="AppPath">C:\Envs\katago-winml\katago.exe</Data>
        <Data Name="ModulePath">C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.AMD.GPU.EP.1.8_1.8.57.0_x64__8wekyb3d8bbwe\ExecutionProvider\amdhip64_7.dll</Data>
        <Data Name="IntegratorReportId">a195a9c9-5a63-4e9a-bee0-c9e25a7fb69d</Data>
        <Data Name="PackageFullName">
        </Data>
        <Data Name="PackageRelativeAppId">
        </Data>
      </EventData>
    </Event>
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions