[Bug]: MTP speculative decoding cannot start under pipeline parallelism — SupportsPP demanded of the draft model

[nickus]

Your current environment

vLLM 0.26.1rc1.dev693+g7f7a32cfe (nightly), 8× RTX 3090 (sm_86), CUDA 12.x,
VLLM_USE_V2_MODEL_RUNNER=1. Also reproduced on dev678 and dev686.
Code references below are against main @ 98f86b9c0232.

🐛 Describe the bug

Speculative decoding with an MTP head cannot start at all under pipeline
parallelism. It fails during create_engine_config, before any weight is loaded:

vllm serve <model-with-mtp-head> --pipeline-parallel-size 4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

NotImplementedError: Pipeline parallelism is not supported for this model.
Supported models implement the `SupportsPP` interface.

Traceback path:

vllm/config/speculative.py  _verify_args
    -> self.draft_model_config.verify_with_parallel_config(self.draft_parallel_config)
vllm/config/model.py        verify_with_parallel_config

speculative.py L1411-1414 on main:

        if self.draft_model_config:
            self.draft_model_config.verify_with_parallel_config(
                self.draft_parallel_config
            )

Why the check does not apply to a draft model

create_draft_parallel_config copies the target's pipeline_parallel_size into
the draft's parallel config, so verification then demands SupportsPP of the
draft. But the drafter is only ever built on one rank — v1/worker/gpu/model_runner.py:

if self.speculative_config is not None:
    if self.is_last_pp_rank:
        self.speculator = init_speculator(self.vllm_config, self.device)

A draft model is therefore never split across pipeline stages, so whether it
could be split is not a question that needs answering about it.

Scope

This is not specific to one model. Glm4MoeMTP and DeepSeekMTP are declared as
(nn.Module, <...>MixtureOfExperts) — neither inherits SupportsPP, and neither
do the other ~20 entries of MTPModelTypes (deepseek_mtp, glm4_moe_mtp,
ernie_mtp, qwen3_next_mtp, minimax_m3_mtp, longcat_flash_mtp, …). Every
MTP head is blocked under pipeline_parallel_size > 1 by this check.

Suggested fix

Verify the draft as the single-stage model it actually is, and restore the config
afterwards so rank and world-size bookkeeping downstream is untouched:

if self.draft_model_config:
    _dpc = self.draft_parallel_config
    if _dpc.pipeline_parallel_size > 1:
        _pp = _dpc.pipeline_parallel_size
        object.__setattr__(_dpc, "pipeline_parallel_size", 1)
        try:
            self.draft_model_config.verify_with_parallel_config(_dpc)
        finally:
            object.__setattr__(_dpc, "pipeline_parallel_size", _pp)
    else:
        self.draft_model_config.verify_with_parallel_config(_dpc)

Note on what comes next

With this relaxed, MTP under PP gets further but does not yet work: the last rank
then dies inside compile_or_warm_up_model →
v1/worker/gpu/warmup.py::warmup_kernels → _run_decode_step →
get_pp_group().irecv_tensor_dict(...), and after that the output is wrong
unless async scheduling is enabled. The scheduling half is filed separately; I am
happy to open a third issue for the warmup receive if that is useful rather than
noise.

Before submitting a new issue...

• I searched the tracker (several phrasings) and did not find this reported.

---

Assisted-By: Claude

[cr-zhao]

This looks like it may already be covered by #46994, which is adding MTP speculative decoding support under pipeline parallelism and includes the SupportsPP / draft-model PP validation issue. Might be worth linking these.

[nickus]

PR raised: #52117.

@cr-zhao thanks — I checked #46994 before filing: it touches the MTP model classes, MLA backends and pp_utils tests (14 files), but not vllm/config/speculative.py, so the config-time gate this issue is about is not covered there. The two are complementary: #46994 makes MTP under PP work, this PR makes it startable. Linked both ways now.

One note for whoever reviews both: with the gate relaxed, spec+PP has a second, independent failure — it silently corrupts output unless async scheduling is on. That is #52071, fix in a separate PR.

---

Assisted-By: Claude