Summary

In vLLM v0.26.0, deepstream is not classified as a GPU backend and its decode path omits VLLM_MAX_IMAGE_PIXELS. An unauthenticated client can therefore activate DeepStream at request time, initialize the process-wide GPU decode pool, and submit video that other backends reject. Concurrent exploitation caused 50% of lightweight canary requests to time out, resulting in a partial denial of service.

Details

Background

Two relevant security controls existed before the DeepStream implementation was merged:

1. PR #47010 / commit 364ee36 introduced VLLM_MAX_IMAGE_PIXELS and video frame-dimension checks to prevent compressed media from causing memory-exhaustion denial of service.
2. PR #47259 / commit ba22152 blocked request-level selection of GPU video backends that were not configured and included in GPU-memory reservation at startup.

The DeepStream RFC #41843 subsequently proposed an NVDEC/GStreamer backend to improve video-decoding performance. The implementation was merged in commit e23b193 / PR #42424.

DeepStream was not introduced to fix either earlier vulnerability. Instead, the later performance feature did not integrate with the two existing security boundaries.

Root cause 1: backend identity confusion

VideoMediaIO.merge_kwargs() blocks request-level GPU backend overrides using the following check:

for key in ("video_backend", "backend"):
    requested = runtime_kwargs.get(key)
    if requested and VIDEO_LOADER_REGISTRY.backend_requires_gpu(requested):
        static_val = (default_kwargs or {}).get(key)
        if static_val != requested:
            runtime_kwargs = {
                k: v for k, v in runtime_kwargs.items() if k != key
            }

However, backend_requires_gpu() treats every unregistered name as non-GPU:

def backend_requires_gpu(self, name: str) -> bool:
    return self._requires_gpu.get(name, False)

PyNvVideoCodec is independently registered with requires_gpu=True. DeepStream is not. Instead, it is implemented as an inner codec of the loader registered under the name opencv:

@VIDEO_LOADER_REGISTRY.register("opencv")
class VideoBackend(
    VideoLoader,
    OpenCVVideoBackendMixin,
    PyAVVideoBackendMixin,
    TorchCodecVideoBackendMixin,
    PyNvVideoCodecVideoBackendMixin,
    DeepStreamVideoBackendMixin,
):
    ...

Consequently, the following request path is allowed:

media_io_kwargs.video.backend = "deepstream"
  -> VideoMediaIO.merge_kwargs()
  -> backend_requires_gpu("deepstream") == False
  -> runtime override is retained
  -> VideoBackend.load_bytes(backend="deepstream")
  -> DeepStream/NVDEC GPU decoding is activated

A remote request can therefore activate a GPU video backend that was not selected or included in GPU-memory accounting at startup, contrary to the security invariant introduced by PR #47259.

Root cause 2: missing pixel-limit enforcement

The OpenCV, PyAV, TorchCodec, and PyNvVideoCodec paths call _check_frame_pixel_limit() before decoding. The DeepStream branch obtains width and height from probe_metadata() but discards them:

total_frames, original_fps, duration, _w, _h, codec = probe_metadata(data)

It proceeds to decode_indices() without calling:

_check_frame_pixel_limit(_w, _h)

As a result, the same video can be rejected by OpenCV but accepted when a request changes the backend to DeepStream.

Root cause 3: request-controlled process-wide state

DeepStream uses a lazy process-wide singleton. Its size is obtained from request-level media kwargs and limited to 1–16:

pool_size = max(1, min(int(pool_size), 16))
cls._pool = DecodePool(num_workers=pool_size)

The first successful request determines the pool size for the remainder of the process lifetime. Clean-start testing with request values 1, 2, 4, 8, and 16 produced matching initialization logs:

[DeepStream] initializing decode pool with 1 workers
[DeepStream] initializing decode pool with 2 workers
[DeepStream] initializing decode pool with 4 workers
[DeepStream] initializing decode pool with 8 workers
[DeepStream] initializing decode pool with 16 workers

Suggested remediation

• Register deepstream as a GPU-backed codec and apply the same GPU classification to loader and codec selection.
• Reject request-level DeepStream selection unless it was statically configured and included in startup GPU-memory reservation.
• Fail closed for unknown request-level backend names.
• Call _check_frame_pixel_limit(width, height) immediately after probe_metadata().
• Make pool_size startup-only rather than request-controlled.
• Add a total decoded-work limit based on selected frames × width × height.
• Add regression tests for request-level DeepStream selection and OpenCV/DeepStream pixel-limit parity.

Impact

This is a CWE-400 resource-consumption and partial denial-of-service vulnerability caused by GPU backend identity confusion and inconsistent enforcement of resource controls.

An unauthenticated remote client can:

• activate DeepStream/NVDEC without startup configuration or GPU-memory reservation;
• choose the initial size of the process-wide DecodePool;
• bypass VLLM_MAX_IMAGE_PIXELS for DeepStream video requests; and
• degrade availability for unrelated requests through concurrent GPU video workloads.

The demonstrated impact is partial and recoverable availability loss. During testing, 50% of canary requests in the measured attack window timed out, while some requests continued to succeed and the service later recovered.

No confidentiality or integrity impact, process crash, GPU OOM, or persistent complete outage was observed.

Why only one CVE is needed and all the root causes are dependant?

These three root causes are not independently exploitable. Root causes 2 and 3 have no security impact without root cause 1, because only root cause 1 allows an unauthenticated remote client to activate the DeepStream code path. They share a single origin (commit e23b193), a single attacker profile, a single prerequisite (request-level backend override), and a single impact (partial DoS). Fixing root cause 1 alone eliminates the entire remote attack surface. Root causes 2 and 3 represent defense-in-depth hardening for legitimately configured deployments, not independently exploitable vulnerabilities.