[Bug]: GPTQ fused-shard check rejects Gemma 4 qkv_proj (global-attention layers have no v_proj) when modules_in_block_to_quantize is derived from the checkpoint #53992

Your current environment

vLLM main @ ca90b9e7d (2026-08-27; the affected code — vllm/model_executor/layers/quantization/auto_gptq.py, vllm/model_executor/layers/quantization/utils/gptq_utils.py — is identical on current main), Python 3.12, torch 2.13.0+xpu, transformers 5.15.1, 2x Intel Arc Pro B70 (Battlemage), TP=2. The failure is in platform-independent config/loader code; it does not depend on the XPU backend.

🐛 Describe the bug

Serving Intel/gemma-4-31B-it-int4-AutoRound (AutoRound export, packing_format: auto_round:auto_gptq, W4 g128 sym) through the GPTQ config path fails at model construction:

vllm serve <checkpoint-with-quant_method-gptq> --language-model-only --tensor-parallel-size 2 ...
...
  File "vllm/model_executor/layers/linear.py", line 266, in __init__
  File "vllm/model_executor/layers/quantization/auto_gptq.py", line 264, in get_quant_method
  File "vllm/model_executor/layers/quantization/utils/gptq_utils.py", line 144, in get_linear_quant_method
  File "vllm/model_executor/layers/quantization/utils/gptq_utils.py", line 121, in is_layer_gptq_quantized
ValueError: Detected some but not all shards of language_model.model.layers.5.self_attn.qkv_proj are quantized. All shards of fused layers to have the same precision.

Layer 5 is Gemma 4's first global-attention layer. Gemma 4's global-attention layers have no v_proj at all (k_eq_v: V is taken from K), so the checkpoint contains q_proj/k_proj qweights for that layer and no v_proj tensor of any dtype.

Mechanism:

• When the config carries no modules_in_block_to_quantize, AutoGPTQConfig.maybe_update_config derives it by scanning safetensors metadata for non-float parameters. That yields the quantized module names only.
• is_layer_gptq_quantized expands the fused qkv_proj through packed_modules_mapping into [q_proj, k_proj, v_proj], sees v_proj "not quantized" — it is simply absent — and raises.

Any architecture whose fused layer legitimately lacks a shard hits the same check. Gemma 4 is the current example (its 26B/31B AutoRound/GPTQ exports).

Repro (no GPU needed to reach the error): take the Intel checkpoint, rewrite quantization_config to {"quant_method": "gptq", "bits": 4, "group_size": 128, "sym": true, "desc_act": false} (no module list), vllm serve it. Adding "modules_in_block_to_quantize": ["mlp.down_proj","mlp.gate_proj","mlp.up_proj","self_attn.k_proj","self_attn.o_proj","self_attn.q_proj","self_attn.v_proj"] (per-block suffixes) avoids the scan and the model loads and serves correctly (33 tok/s single-stream / 148 tok/s batched on 2x B70, outputs coherent) — which is the workaround we ship in our image's converter.

Proposed fix (PR being prepared; happy to adjust): during the shard scan also record the set of module prefixes present in the checkpoint, and let is_layer_gptq_quantized skip shards that do not exist in the checkpoint at all, so only present-but-unquantized shards trigger the mismatch error. Behaviour is unchanged when modules_in_block_to_quantize comes from the config.

Tracking on our side: CySpiegel#4

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.