Weight sync refactor + move sparse nccl engine by hao-aaron · Pull Request #44353 · vllm-project/vllm

hao-aaron · 2026-06-02T19:33:11Z

Purpose

Current state of the weight sync path requires non weight transfer engine components to assume too much about weight sync internals. Gpu workers should not need to worry about whether to run layerwise reload, or how to construct weight loaders for weight transfer engines. These concepts are weight transfer engine specific, and should be taken care of by the engine. Also, NCCL sparse weight update does not share enough with full nccl update and should be moved out to a separate engine for cleanliness.

1. Remove the checkpoint-vs-kernel format distinction

start_weight_update(is_checkpoint_format=...) existed only to toggle whether
layerwise reload ran. In practice every dense flow used checkpoint format, and
the single kernel-format user (sparse updates) is now its own engine (see #3).
The flag was therefore dead weight that leaked an engine-internal decision into
the public API. It is removed everywhere; start_weight_update() is now
parameterless.

2. Make the weight-update lifecycle explicit and per-engine

start_weight_update / finish_weight_update are now abstract on
WeightTransferEngine, so every backend must declare how it prepares and
finalizes an update. Layerwise reload is no longer special-cased in the worker —
each engine opts in by running it inside its own start/finish (dense engines do;
in-place engines are no-ops). This puts the "how do my weights get loaded"
decision next to the transport that produced them.

3. Extract sparse NCCL into a standalone engine

Sparse (flat-index, in-place patch) updates shared nothing with dense NCCL
except process-group initialization, yet lived inside the same class behind
update_kind branching. They are now a separate sparse_nccl backend
(SparseNCCLWeightTransferEngine) that is not a subclass of the dense
engine; the only shared code (rendezvous / process-group setup) lives in a small
nccl_common module both import. This removes cross-contamination between two
unrelated transfer strategies and lets each evolve independently.

4. Reduce the worker to a pass-through

GPUWorker no longer knows anything weight-transfer-specific: no layerwise
calls, no loader construction, no update_kind dispatch, no format state. It
only forwards start/update/finish to the engine and keeps a single
_weight_update_active guard (deliberately left in the worker so engines need no
super() chaining). Engines now receive vllm_config and device at
construction and load directly into self.model, so receive_weights no longer
takes a load_weights callback.

Behavior changes

start_weight_update / the /start_weight_update HTTP endpoint take no
arguments. Kernel-format weight sync is no longer supported via the weight
transfer path.
New weight transfer backend: WeightTransferConfig(backend="sparse_nccl").
WeightTransferEngineFactory.create_engine and the engine constructor now take
(config, vllm_config, device, model).
The mid-level reload_weights / update_config APIs are intentionally
untouched.

Test Plan

Ensured the following tests pass:
(test_weight_transfer 42 incl. NCCL/sparse/IPC cross-process integration,
test_weight_transfer_llm 5, test_gpu_worker_weight_transfer 6,
test_gpu_model_runner 31).

End-to-end examples (exit 0): rlhf_nccl.py, rlhf_ipc.py, and
rlhf_sparse_nccl.py — the latter confirms dense vs sparse produce identical
post-update outputs while the sparse payload is ~0.16 MB vs ~942 MB dense.

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: hao-aaron <ahao@anyscale.com>

# Conflicts: # docs/training/weight_transfer/base.md # tests/distributed/test_weight_transfer.py Signed-off-by: hao-aaron <ahao@anyscale.com>

mergify · 2026-06-02T22:50:26Z

Documentation preview: https://vllm--44353.org.readthedocs.build/en/44353/

bedeks · 2026-06-03T23:57:33Z

-            load_weights: Callable that loads weights into the model. Called
-                         incrementally for each weight to avoid OOM.
        """
        device_index = torch.accelerator.current_device_index()


This reads the ambient current CUDA device but after this refactor worker.update_weights() no longer wraps the receive path in with torch.device(self.device). So this line can no longer guarantee that current_device_index() would match the worker’s assigned self.device. Couldn't this break for multi-gpu?

good catch, thanks for review

init_device actually sets this up before hand:

vllm/vllm/v1/worker/gpu_worker.py

Lines 313 to 314 in 3cc871a

self.device = torch.device(f"cuda:{visible_device_index}")

torch.accelerator.set_device_index(self.device)

We implicitly use the current device index even at init_weight_transfer_engine right now:

vllm/vllm/distributed/weight_transfer/nccl_engine.py

Lines 155 to 156 in d86c66c

# Create stateless process group

device = torch.accelerator.current_device_index()

Signed-off-by: hao-aaron <ahao@anyscale.com>

mergify · 2026-06-08T18:50:59Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hao-aaron.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2026-06-30T07:15:43Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hao-aaron.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

# Conflicts: # tests/distributed/test_weight_transfer.py

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com>

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: Spicy-Stick <873805887@qq.com>

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Signed-off-by: xqchen7 <chenxueqing7@huawei.com>

Signed-off-by: hao-aaron <ahao@anyscale.com> Signed-off-by: haoaaron <ahao@anyscale.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com>

### What this PR does / why we need it? Ports the vLLM weight sync lifecycle refactor from [vllm-project/vllm#44353](vllm-project/vllm#44353) to the Ascend HCCL and NPU IPC backends. The upstream refactor moves weight-loading decisions out of the worker and into each weight transfer engine. This PR applies the same ownership model on Ascend: - delegates `start_weight_update`, `update_weights`, and `finish_weight_update` from `NPUWorker` to the configured transfer engine; - updates HCCL and NPU IPC engines to load weights through their engine-owned model; - adds checkpoint-format layerwise reload setup and finalization to NPU IPC; - uses the engine-owned device for NPU IPC handle matching and tensor rebuilding; - resets the worker lifecycle state when an update raises, allowing a subsequent update attempt; and - removes legacy lifecycle request fallbacks from the HCCL/NPU IPC examples and E2E tests. ### Does this PR introduce _any_ user-facing change? Yes. Weight updates now use the explicit lifecycle: `start_weight_update -> update_weights -> finish_weight_update` `start_weight_update` no longer accepts `is_checkpoint_format`; checkpoint-format handling belongs to each transfer engine. RL clients should pause generation before starting an update and resume it after finishing. ### How was this patch tested? - `ruff format --check` and `ruff check` on all nine changed Python files with Ruff 0.14.0. - `python -m compileall` on all nine changed Python files. - `python -m pytest -s -v tests/ut/worker/a2/test_worker_v1.py -k TestNPUWorkerWeightUpdate` - `python -m pytest -s -v tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py` - `python -m pytest -s -v tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py` - `python -m pytest -s -v tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py` The unit tests and both NPU IPC/HCCL E2E tests passed on an Ascend self-hosted runner with vLLM 0.25.1. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@d02df74 --------- Signed-off-by: Ulrica111 <ulrica111@users.noreply.github.com> Co-authored-by: Ulrica111 <ulrica111@users.noreply.github.com>

) ### What this PR does / why we need it? Ports the vLLM weight sync lifecycle refactor from [vllm-project/vllm#44353](vllm-project/vllm#44353) to the Ascend HCCL and NPU IPC backends. The upstream refactor moves weight-loading decisions out of the worker and into each weight transfer engine. This PR applies the same ownership model on Ascend: - delegates `start_weight_update`, `update_weights`, and `finish_weight_update` from `NPUWorker` to the configured transfer engine; - updates HCCL and NPU IPC engines to load weights through their engine-owned model; - adds checkpoint-format layerwise reload setup and finalization to NPU IPC; - uses the engine-owned device for NPU IPC handle matching and tensor rebuilding; - resets the worker lifecycle state when an update raises, allowing a subsequent update attempt; and - removes legacy lifecycle request fallbacks from the HCCL/NPU IPC examples and E2E tests. ### Does this PR introduce _any_ user-facing change? Yes. Weight updates now use the explicit lifecycle: `start_weight_update -> update_weights -> finish_weight_update` `start_weight_update` no longer accepts `is_checkpoint_format`; checkpoint-format handling belongs to each transfer engine. RL clients should pause generation before starting an update and resume it after finishing. ### How was this patch tested? - `ruff format --check` and `ruff check` on all nine changed Python files with Ruff 0.14.0. - `python -m compileall` on all nine changed Python files. - `python -m pytest -s -v tests/ut/worker/a2/test_worker_v1.py -k TestNPUWorkerWeightUpdate` - `python -m pytest -s -v tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py` - `python -m pytest -s -v tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py` - `python -m pytest -s -v tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py` The unit tests and both NPU IPC/HCCL E2E tests passed on an Ascend self-hosted runner with vLLM 0.25.1. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@d02df74 --------- Signed-off-by: Ulrica111 <ulrica111@users.noreply.github.com> Co-authored-by: Ulrica111 <ulrica111@users.noreply.github.com>

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com>

) ### What this PR does / why we need it? Ports the vLLM weight sync lifecycle refactor from [vllm-project/vllm#44353](vllm-project/vllm#44353) to the Ascend HCCL and NPU IPC backends. The upstream refactor moves weight-loading decisions out of the worker and into each weight transfer engine. This PR applies the same ownership model on Ascend: - delegates `start_weight_update`, `update_weights`, and `finish_weight_update` from `NPUWorker` to the configured transfer engine; - updates HCCL and NPU IPC engines to load weights through their engine-owned model; - adds checkpoint-format layerwise reload setup and finalization to NPU IPC; - uses the engine-owned device for NPU IPC handle matching and tensor rebuilding; - resets the worker lifecycle state when an update raises, allowing a subsequent update attempt; and - removes legacy lifecycle request fallbacks from the HCCL/NPU IPC examples and E2E tests. ### Does this PR introduce _any_ user-facing change? Yes. Weight updates now use the explicit lifecycle: `start_weight_update -> update_weights -> finish_weight_update` `start_weight_update` no longer accepts `is_checkpoint_format`; checkpoint-format handling belongs to each transfer engine. RL clients should pause generation before starting an update and resume it after finishing. ### How was this patch tested? - `ruff format --check` and `ruff check` on all nine changed Python files with Ruff 0.14.0. - `python -m compileall` on all nine changed Python files. - `python -m pytest -s -v tests/ut/worker/a2/test_worker_v1.py -k TestNPUWorkerWeightUpdate` - `python -m pytest -s -v tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py` - `python -m pytest -s -v tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py` - `python -m pytest -s -v tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py` The unit tests and both NPU IPC/HCCL E2E tests passed on an Ascend self-hosted runner with vLLM 0.25.1. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@d02df74 --------- Signed-off-by: Ulrica111 <ulrica111@users.noreply.github.com> Co-authored-by: Ulrica111 <ulrica111@users.noreply.github.com>

### What this PR does / why we need it? #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py - Conditionally implement start_weight_update() and finish_weight_update() as no-op methods for non-0.23.0 releases. - Keep the NPU IPC weight transfer engine compatible with the updated WeightTransferEngine interface. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/platform/patch_torch_accelerator.py - Redirect torch.accelerator.get_memory_info() to torch.npu.mem_get_info() on non-0.23.0. - Avoid crashes caused by the unsupported C10 DeviceAllocator path when constructing MemorySnapshot. - Align with the existing NPU-specific memory API patches. - Upstream source: commit 747b068 (v0.24.0+ MemorySnapshot(device=device) path). --- #### vllm_ascend/patch/worker/patch_qwen3_dflash.py - Wrap DFlashQwen3ForCausalLM._read_mask_embedding() to ignore optional mask embedding download failures. - Preserve the expected "mask embedding not present" behavior when the file is unavailable. - Upstream source: vllm#46104 (vllm-project/vllm#46104). --- #### vllm_ascend/worker/v2/model_runner.py #### vllm_ascend/patch/worker/patch_v2/patch_input_batch.py - Forward is_padding and prompt_lens when constructing AscendInputBatch. - Match the updated upstream InputBatch interface and avoid initialization failures on newer releases. - Upstream source: vllm#40654 (vllm-project/vllm#40654). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Add the reduce_results argument to DeepseekV2MLAAttention. - Forward the parameter to RowParallelLinear to stay compatible with the updated upstream attention initialization. - Keep the implementation compatible across all supported vLLM versions. - Upstream source: vllm#45895 (vllm-project/vllm#45895). --- #### vllm_ascend/distributed/device_communicators/npu_communicator.py - Register a no-op all2all_manager for NPUCommunicator. - Bypass the upstream MoE fault-tolerance check (which queries all2all_manager when data_parallel_size > 1 and is_moe) while preserving the existing MC2 communication path. - Keep compatibility with the updated distributed initialization. - Related upstream changes: - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Share routed expert parameters through direct nn.Parameter aliasing instead of creating wrapper parameters. - Ensure both legacy and routed_experts parameter paths reference the same underlying weights. - Apply the aliasing strategy to all routed-expert MoE models on newer vLLM releases. - Related upstream changes: - vllm#40996 (vllm-project/vllm#40996) - vllm#46892 (vllm-project/vllm#46892) --- #### vllm_ascend/worker/worker.py #### vllm_ascend/distributed/weight_transfer/npu_ipc_engine.py #### vllm_ascend/distributed/weight_transfer/hccl_engine.py #### vllm_ascend/patch/platform/patch_weight_transfer_engine.py - Adapt WeightTransferEngineFactory.create_engine() and WeightTransferEngine.__init__() to support both legacy and current upstream signatures. - Keep weight transfer compatible across v0.23.0, v0.24.0, and newer upstream releases. - Upstream source: vllm#44353 (vllm-project/vllm#44353). --- #### vllm_ascend/patch/worker/patch_deepseek_v2.py - Remove the upstream model-level all-gather path for DeepSeek-V2 on non-0.23.0. - Keep the implementation compatible with the Ascend MC2 dispatch flow. - Avoid tensor shape mismatches and residual concatenation failures introduced by the upstream refactor. - Related upstream changes: - vllm#41184 (vllm-project/vllm#41184) --- #### vllm_ascend/ops/fused_moe/fused_moe.py - Remove the unnecessary .contiguous() call after weight transposition on non-0.23.0. - Reduce transient NPU peak memory during MoE weight loading. - Prevent OOM caused by duplicate temporary tensor allocations. - Related upstream changes: - vllm#44589 (vllm-project/vllm#44589) - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@ee0da84 --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com>

) ### What this PR does / why we need it? Ports the vLLM weight sync lifecycle refactor from [vllm-project/vllm#44353](vllm-project/vllm#44353) to the Ascend HCCL and NPU IPC backends. The upstream refactor moves weight-loading decisions out of the worker and into each weight transfer engine. This PR applies the same ownership model on Ascend: - delegates `start_weight_update`, `update_weights`, and `finish_weight_update` from `NPUWorker` to the configured transfer engine; - updates HCCL and NPU IPC engines to load weights through their engine-owned model; - adds checkpoint-format layerwise reload setup and finalization to NPU IPC; - uses the engine-owned device for NPU IPC handle matching and tensor rebuilding; - resets the worker lifecycle state when an update raises, allowing a subsequent update attempt; and - removes legacy lifecycle request fallbacks from the HCCL/NPU IPC examples and E2E tests. ### Does this PR introduce _any_ user-facing change? Yes. Weight updates now use the explicit lifecycle: `start_weight_update -> update_weights -> finish_weight_update` `start_weight_update` no longer accepts `is_checkpoint_format`; checkpoint-format handling belongs to each transfer engine. RL clients should pause generation before starting an update and resume it after finishing. ### How was this patch tested? - `ruff format --check` and `ruff check` on all nine changed Python files with Ruff 0.14.0. - `python -m compileall` on all nine changed Python files. - `python -m pytest -s -v tests/ut/worker/a2/test_worker_v1.py -k TestNPUWorkerWeightUpdate` - `python -m pytest -s -v tests/ut/distributed/weight_transfer/test_npu_ipc_engine.py` - `python -m pytest -s -v tests/e2e/pull_request/one_card/test_npu_ipc_weight_transfer.py` - `python -m pytest -s -v tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py` The unit tests and both NPU IPC/HCCL E2E tests passed on an Ascend self-hosted runner with vLLM 0.25.1. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@d02df74 --------- Signed-off-by: Ulrica111 <ulrica111@users.noreply.github.com> Co-authored-by: Ulrica111 <ulrica111@users.noreply.github.com>

initial refactor

49aae63

Signed-off-by: hao-aaron <ahao@anyscale.com>

hao-aaron marked this pull request as ready for review June 2, 2026 20:21

hao-aaron requested review from AndreasKaratzas, DarkLight1337, NickLucche, ProExpertProg, WoosukKwon, aarnphm, hmellor, houseroad, mgoin, njhill, robertgshaw2-redhat, tlrmchlsmth, yewentao256 and youkaichao as code owners June 2, 2026 20:21

hao-aaron added 2 commits June 2, 2026 20:57

Merge remote-tracking branch 'origin/main' into weight-sync-refactor

f92e1c1

Merge remote-tracking branch 'origin/main' into weight-sync-refactor

5382d0d

# Conflicts: # docs/training/weight_transfer/base.md # tests/distributed/test_weight_transfer.py Signed-off-by: hao-aaron <ahao@anyscale.com>

hao-aaron force-pushed the weight-sync-refactor branch from a049651 to 5382d0d Compare June 2, 2026 22:25

mergify Bot added documentation Improvements or additions to documentation frontend v1 labels Jun 2, 2026

kouroshHakha added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 2, 2026

bedeks reviewed Jun 3, 2026

View reviewed changes

hao-aaron added 2 commits June 4, 2026 12:54

Merge branch 'main' into weight-sync-refactor

9258b4d

x

1c8c1f6

Signed-off-by: hao-aaron <ahao@anyscale.com>

mergify Bot added the needs-rebase label Jun 8, 2026

aoshen02 mentioned this pull request Jun 14, 2026

[RFC] RL CI Matrix for vLLM: Behavioral + Physical + Protocol Coverage #45585

Open

15 tasks

mergify Bot added the needs-rebase label Jun 30, 2026

Merge remote-tracking branch 'origin/main' into weight-sync-refactor

1c938aa

# Conflicts: # tests/distributed/test_weight_transfer.py

mergify Bot removed the needs-rebase label Jul 1, 2026

Merge branch 'main' into weight-sync-refactor

cc4d3ab

ywang96 merged commit 77a9c5a into vllm-project:main Jul 1, 2026
113 of 118 checks passed

vllm-agent mentioned this pull request Jul 2, 2026

Revert "Weight sync refactor + move sparse nccl engine" (#44353) #47371

Draft

zhangxinyuehfad mentioned this pull request Jul 5, 2026

[CI] main2main 0703 commit: 1f486d9 vllm-project/vllm-ascend#10454

Merged

hmellor mentioned this pull request Jul 6, 2026

[CI] Fix some errors on main #47726

Merged

aoshen02 mentioned this pull request Jul 11, 2026

[RFC]: Native Weight Syncing APIs #31848

Open

26 tasks

zhao-stack mentioned this pull request Jul 14, 2026

[CI] main2mainv0.24 vllm-project/vllm-ascend#11875

Closed

Ulrica111 mentioned this pull request Jul 25, 2026

[Refactor] Align weight transfer lifecycle with vLLM vllm-project/vllm-ascend#12876

Merged

Navigation Menu

Uh oh!

Uh oh!

Weight sync refactor + move sparse nccl engine - #44353

Weight sync refactor + move sparse nccl engine#44353
ywang96 merged 14 commits into
vllm-project:mainfrom
hao-aaron:weight-sync-refactor

hao-aaron commented Jun 2, 2026 •

edited

Loading

Uh oh!

mergify Bot commented Jun 2, 2026

Uh oh!

bedeks Jun 3, 2026

Uh oh!

hao-aaron Jun 4, 2026

Uh oh!

SumanthRH Jun 23, 2026

Uh oh!

mergify Bot commented Jun 8, 2026

Uh oh!

mergify Bot commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

	self.device = torch.device(f"cuda:{visible_device_index}")
	torch.accelerator.set_device_index(self.device)

	# Create stateless process group
	device = torch.accelerator.current_device_index()

Uh oh!

Uh oh!

Conversation

hao-aaron commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

1. Remove the checkpoint-vs-kernel format distinction

2. Make the weight-update lifecycle explicit and per-engine

3. Extract sparse NCCL into a standalone engine

4. Reduce the worker to a pass-through

Behavior changes

Test Plan

Uh oh!

mergify Bot commented Jun 2, 2026

Uh oh!

bedeks Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

hao-aaron Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

SumanthRH Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Jun 8, 2026

Uh oh!

mergify Bot commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

hao-aaron commented Jun 2, 2026 •

edited

Loading