DoS caused by sending `/v1/completions` with prompt embeds payload with models that use M-RoPE

Summary

Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server.

Sending a pure prompt embeds payload in a /v1/completions request with a model using M-RoPE causes the EngineCore to fail an assertion and fatally crash, shutting down the entire server application.

Any remote user who is authorized to make a /v1/completions endpoint can trivially make such a request and induce a crash.

Details

Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer.

In commit 56669c1, a simple assert intended to be a type-narrowing assert was added to the _init_mrope_positions method in GPUModelRunner (the offending line on main at the time of writing:

assert req_state.prompt_token_ids is not None, (
            "M-RoPE requires prompt_token_ids to be available."
        )

This type narrowing assert is to prevent mypy errors later in the function because None is not a valid type for mrope_model.get_mrope_input_positions. Unfortunately, this assertion is not always true. /v1/completions requests that specify prompt=None and prompt_embeds=<not none> will indeed create a CachedRequestState where prompt_token_ids is None. This triggers the assertion, which in turn crashes the EngineCore and the Server application.

(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3997, in execute_model
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]     deferred_state_corrections_fn = self._update_states(scheduler_output)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1239, in _update_states
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]     self._init_mrope_positions(req_state)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1582, in _init_mrope_positions
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]     assert req_state.prompt_token_ids is not None, (
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] AssertionError: M-RoPE requires prompt_token_ids to be available.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] AsyncLLM output_handler failed.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] Traceback (most recent call last):
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704]     outputs = await engine_core.get_output_async()
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1030, in get_output_async
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704]     raise self._format_exception(outputs) from None
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.

All requests using the /v1/chat/completions endpoints will have text/prompt_token_ids parts (corresponding to the chat template), and prompt_embeds parts are handled as mm_features. This method (rightly) filters out those prompt_embeds content parts as they are treated as text positions.

I believe a sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:

def _init_mrope_positions(self, req_state: CachedRequestState):
    model = self.get_model()
    assert supports_mrope(model), "M-RoPE support is not implemented."
    mrope_model = cast(SupportsMRoPE, model)

    # Filter out prompt_embeds modality (text-only position info)
    mrope_features = [
        f for f in req_state.mm_features if f.modality != "prompt_embeds"
    ]
    
    # Handle both token_ids and embeddings-only inputs
    if req_state.prompt_token_ids is not None:
        input_tokens = req_state.prompt_token_ids
    elif req_state.prompt_embeds is not None:
        # For text-only embeddings, dummy token IDs are safe since
        # get_mrope_input_positions only uses len(input_tokens) when mm_features is empty
        seq_len = req_state.prompt_embeds.shape[0]
        input_tokens = list(range(seq_len))
        # Verify no mm_features remain (should be true after prompt_embeds filter)
        assert len(mrope_features) == 0, (
            "M-RoPE with prompt_embeds-only input should have no multimodal features"
        )
    else:
        raise ValueError(
            "M-RoPE requires either prompt_token_ids or prompt_embeds."
        )

    req_state.mrope_positions, req_state.mrope_position_delta = (
        mrope_model.get_mrope_input_positions(
            input_tokens,
            mrope_features,
        )
    )

Technically, in isolation, this method still crashes in the case where req_state.prompt_token_ids is None and req_state.mm_features, so the solution above still leaves that potential vector open. Best I can tell, however, such a req_state is impossible in the first place in online mode, because it would require a /v1/completions request with prompt_embeds AND multimodal features, but the /v1/completions request schema does not expose multimodal inputs in any way that I can see. Today, those are the only two endpoints with prompt_embeds support.

When in offline mode, I believe it is technically possible to directly create an EngineCoreRequest that has prompt_embeds and not prompt_token_ids and mm_features, and pass that to LLM.generate. I believe that would trigger this same assertion, and no validation would prevent that combination. I strongly suspect, though, that this combination would be undefined in any model that support M-RoPE, because it would not be possible to determine which token positions correspond to mm_features. My proposed solution above would end up not setting req_state.mrope_positions and req_state.mrope_position_delta in this scenario, which could result in undefined behavior.

I am far more familiar with prompt_embeds than I am with M-RoPE, and I am aware that each model that supports it is responsible for defining its own get_mrope_input_positions which have varying implementations. I don't know enough to be prescriptive in how the two features should interact in the offline case, other than possibly raising a validation error earlier on preventing that combination (which would emulate the current assertion behavior). Regardless, in offline mode, the chances of a remote user being able to exploit this are slim-to-nil compared to the online case which is incredibly straightforward.

Impact

What kind of vulnerability is it? Who is impacted?

• Denial of Service caused by an incorrect assertion inside of the GPUModelRunner which causes a fatal EngineCore exception
• Any configuration with --enable-prompt-embeds and M-RoPE-supported model is vulnerable
• The attack is extremely easy from the remote attacker's perspective (copying the official prompt_embeds online mode docs examples almost-verbatim, accounting for model-name and connection details, of course, will induce a guaranteed shutdown)