Skip to content

[BugFix] [KVConnector] Fix KV events for LMCache connector - #32169

Merged
NickLucche merged 5 commits into
vllm-project:mainfrom
hickeyma:fix-lmcache-connector
Jan 13, 2026
Merged

[BugFix] [KVConnector] Fix KV events for LMCache connector#32169
NickLucche merged 5 commits into
vllm-project:mainfrom
hickeyma:fix-lmcache-connector

Conversation

@hickeyma

@hickeyma hickeyma commented Jan 12, 2026

Copy link
Copy Markdown
Contributor

Purpose

A small fix for the LMCache connector as LMCache does not yet support lora_name as an events property. Bug introduced in #27577.

The error occurs when try to prompt a model with LMCache connector and KV events enabled:

[...]
(EngineCore_DP0 pid=223048) [2026-01-12 11:02:52,233] LMCache INFO: Reqid: cmpl-a310c95f98e4f404-0-8bf3d3d5, Total tokens 32, LMCache hit tokens: 0, need to load: 0 (vllm_v1_adapter.py:1602:lmcache.integration.vllm.vllm_v1_adapter)
(Worker_TP1 pid=223151) [2026-01-12 11:02:52,271] LMCache INFO: Storing KV cache for 32 out of 32 tokens (skip_leading_tokens=0) for request cmpl-a310c95f98e4f404-0-8bf3d3d5 (vllm_v1_adapter.py:1374:lmcache.integration.vllm.vllm_v1_adapter)
(Worker_TP0 pid=223150) [2026-01-12 11:02:52,271] LMCache INFO: Storing KV cache for 32 out of 32 tokens (skip_leading_tokens=0) for request cmpl-a310c95f98e4f404-0-8bf3d3d5 (vllm_v1_adapter.py:1374:lmcache.integration.vllm.vllm_v1_adapter)
(Worker_TP1 pid=223151) [2026-01-12 11:02:52,273] LMCache INFO: Stored 32 out of total 32 tokens. size: 0.0024 gb, cost 1.8921 ms, throughput: 1.2903 GB/s; offload_time: 1.8005 ms, put_time: 0.0916 ms (cache_engine.py:446:lmcache.v1.cache_engine)
(Worker_TP0 pid=223150) [2026-01-12 11:02:52,274] LMCache INFO: Stored 32 out of total 32 tokens. size: 0.0024 gb, cost 2.0154 ms, throughput: 1.2113 GB/s; offload_time: 1.9142 ms, put_time: 0.1013 ms (cache_engine.py:446:lmcache.v1.cache_engine)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822] WorkerProc hit an exception.
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822] Traceback (most recent call last):
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/v1/executor/multiproc_executor.py", line 817, in worker_busy_loop
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     output = func(*args, **kwargs)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]              ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/v1/worker/worker_base.py", line 365, in execute_model
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     return self.worker.execute_model(scheduler_output)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/conda-envs/vllm-test/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     return func(*args, **kwargs)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/v1/worker/gpu_worker.py", line 629, in execute_model
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     output = self.model_runner.execute_model(
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/conda-envs/vllm-test/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     return func(*args, **kwargs)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/v1/worker/gpu_model_runner.py", line 3277, in execute_model
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     self.maybe_get_kv_connector_output(scheduler_output) as kv_connector_output,
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/conda-envs/vllm-test/lib/python3.12/contextlib.py", line 144, in __exit__
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     next(self.gen)
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/v1/worker/kv_connector_model_runner_mixin.py", line 139, in _get_kv_connector_output
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     output.kv_cache_events = kv_connector.get_kv_connector_kv_cache_events()
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]   File "/data/dev/github.com/vllm-project/vllm/vllm/distributed/kv_transfer/kv_connector/v1/lmcache_connector.py", line 237, in get_kv_connector_kv_cache_events
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]     lora_name=e.lora_name,
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822]               ^^^^^^^^^^^
(Worker_TP1 pid=223151) ERROR 01-12 11:02:52 [multiproc_executor.py:822] AttributeError: 'CacheStoreEvent' object has no attribute 'lora_name'

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Note

Prevents crashes when using LMCache KV events by aligning with current LMCache event schema.

  • In lmcache_connector.py, set lora_name=None when creating BlockStored from LMCache events (LMCache does not provide lora_name yet).

Written by Cursor Bugbot for commit 609dec1. This will update automatically on new commits. Configure here.


Note

Prevents crashes when LMCache KV events omit lora_name.

  • In lmcache_connector.py, construct BlockStored with lora_name=getattr(e, "lora_name", None) to align with current LMCache event schema and maintain compatibility

Written by Cursor Bugbot for commit ecaab07. This will update automatically on new commits. Configure here.


Note

Cursor Bugbot is generating a summary for commit 0233a46. Configure here.


Note

Fixes LMCache KV events compatibility by making lora_name optional when constructing BlockStored.

  • In lmcache_connector.py, set lora_name=getattr(e, "lora_name", None) to align with LMCache events that omit lora_name, preventing AttributeError crashes during KV event processing

Written by Cursor Bugbot for commit 0233a46. This will update automatically on new commits. Configure here.

LMCache does not yet support `lora_name` as an events property.
This PR fixes the LMcache connector for now until the property is
supported.

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
@mergify mergify Bot added the kv-connector label Jan 12, 2026
@hickeyma hickeyma changed the title [BugFix] Fix KV events for LMCache connector [BugFix] [KVConnector] Fix KV events for LMCache connector Jan 12, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a critical AttributeError that occurs when using the LMCache connector with KV events enabled, due to the lora_name property not being available in the current version of LMCache. The fix correctly resolves the crash by setting lora_name to None. I've provided one suggestion to make the fix more robust and future-proof by using getattr, which will automatically adapt when LMCache adds support for lora_name in a future release.

Comment thread vllm/distributed/kv_transfer/kv_connector/v1/lmcache_connector.py Outdated
hickeyma and others added 2 commits January 12, 2026 11:22
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>

@NickLucche NickLucche left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hickeyma !

@NickLucche
NickLucche enabled auto-merge (squash) January 13, 2026 14:04
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 13, 2026
@NickLucche
NickLucche merged commit 5102654 into vllm-project:main Jan 13, 2026
53 checks passed
@hickeyma
hickeyma deleted the fix-lmcache-connector branch January 13, 2026 15:54
@njhill njhill added this to the v0.14.0 milestone Jan 13, 2026
simon-mo pushed a commit that referenced this pull request Jan 13, 2026
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
sammysun0711 pushed a commit to sammysun0711/vllm that referenced this pull request Jan 16, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
akh64bit pushed a commit to akh64bit/vllm that referenced this pull request Jan 16, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
mystous pushed a commit to mystous/vllm_hybrid that referenced this pull request May 10, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
0826joyce pushed a commit to 0826joyce/vllm-serving-optimization that referenced this pull request May 19, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
…ect#32169)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kv-connector ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants