[Bugfix][XPU] Fall back to torch query when getMemoryInfo reports zero free memory

[CySpiegel]

Purpose

On client Intel Arc GPUs (2x Arc Pro B70 / Battlemage G31, Ubuntu 26.04, compute-runtime 26.05.37020, torch 2.13+xpu, Level Zero 1.28) the custom op torch.ops._C_cache_ops.getMemoryInfo used by get_mem_info_wrapper in vllm/platforms/xpu.py returns free = 0 with a correct total:

device 0: torch.ops._C_cache_ops.getMemoryInfo -> free=0.00 GiB, total=30.30 GiB | torch.xpu.mem_get_info -> free=0.74, total=30.30
device 1: torch.ops._C_cache_ops.getMemoryInfo -> free=0.00 GiB, total=30.30 GiB | torch.xpu.mem_get_info -> free=1.31, total=30.30

(captured with another vLLM instance holding most of the VRAM; on an idle device mem_get_info reports ~30 GiB free while the custom op still reports 0.)

Because get_mem_info_wrapper is monkey-patched over torch.accelerator.get_memory_info, the XPU worker's gpu_memory_utilization free-memory check on startup always fails and vLLM cannot start on these devices at all.

This PR adds a defensive fallback: when the custom op reports an impossible free == 0 with total > 0, use torch.xpu.mem_get_info, which returns correct values on the same stack. On platforms where the op works, nothing changes. If the maintainers prefer fixing the op in vllm-xpu-kernels instead, this fallback is still cheap insurance for the sysman/driver combinations where the op misreports.

No existing issue or PR covers this (searched "xpu mem_get_info", "getMemoryInfo", "free memory 0 xpu").

Test Plan

Repro of the op misreport (run on the affected machine):

import torch, vllm_xpu_kernels._C
for d in (0, 1):
    print(torch.ops._C_cache_ops.getMemoryInfo(d), torch.xpu.mem_get_info(d))

End-to-end: vllm serve on 2x Arc Pro B70 with --tensor-parallel-size 2 --gpu-memory-utilization 0.90, both without and with the patch; the full serving stack (Qwen3.8-27B FP8 and INT4, TP=2, XPU graphs, MTP) has run with this fallback on this hardware continuously since 2026-08-23, including throughput/latency benchmarks and HumanEval/GSM8K evals.

pre-commit run (ruff check/format, mypy) passes.

Test Result

• Without the patch: worker startup fails the free-memory check on every launch (free reported as 0).
• With the patch: worker logs Free memory on device (30.27/30.3 GiB) on startup. Desired GPU memory utilization is (0.9, 27.27 GiB) and serving proceeds normally; KV cache sizing is correct (e.g. 495,794 tokens for Qwen3.8-27B INT4 at TP=2).

---

AI assistance: this change was developed with Claude Code (Claude Fable 5); the human submitter reviewed every line, ran the tests above on the hardware, and defends the change.

Update 2026-08-27: re-verified on this PR's base commit

Fork synced to main @ ca90b9e7d (this PR's base); three vllm serve launches on 2x Arc Pro B70 (TP=2) with this fallback in place — Qwen3.8-27B INT4+MTP, Gemma 4 31B INT4, Qwen3.8-27B online FP8 — all started normally and reported correct free memory (Free memory on device (30.27/30.3 GiB) on startup); without the fallback the same stack cannot start (op reports free=0 on both devices, see repro above). Full speed/correctness legs on all three were unchanged vs the previous base.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀