[v0.26.0] Upgrade GPU and NPU backends

[jiangkuaixue123]

Purpose

Upgrade the AFD GPU and NPU backends together from the vLLM 0.19.1 contract to vLLM 0.26.0.

This PR promotes the completed update/v0.26.0 integration branch to main as one combined change. It brings together the GPU remote-experts refactor from #182 and the NPU runtime, PCP-removal, and DBO compatibility work from #183, #184, and #185.

Issue

• Related PRs: [v0.26.0][GPU] Integrate DeepSeek remote-experts refactor #182, feat(npu): support vLLM 0.26 #183, refactor(npu): remove PCP support from model runner v1 #184, fix(npu): restore DBO compatibility for vLLM 0.26 #185
• No issue is closed by this PR.

Scope

• In scope:
  • Pin the plugin to vLLM 0.26.0 and refresh the lockfile.
  • Move the GPU DeepSeek MoE split to the remote-experts boundary while preserving the native model hierarchy.
  • Support both Attention-side and FFN-side gate placement, ordered P2P custom ops, CUDA Graph execution, and DBO paths on GPU.
  • Align the Ascend/NPU attention and FFN runners, forward context, fused-MoE selection, W8A8 force-load-balance path, and compatibility patches with vLLM/vLLM-Ascend 0.26 APIs.
  • Remove PCP-only model-runner-v1 paths that vLLM-Ascend 0.26 no longer supports.
  • Restore AFD NPU DBO and two-ubatch behavior while retaining the runtime communication backend.
  • Update focused unit coverage and design documentation for the new runtime contract.
• Out of scope:
  • Compatibility with vLLM 0.19.1; this is an intentional single-version upgrade.
  • GPU ModelRunner V2.
  • Quantized GPU MoE and EPLB validation.
  • NPU CAMAsync validation and a dedicated real-W8A8 force-load-balance E2E model.

Implementation Notes

The GPU path keeps native Model.forward, DecoderLayer.forward, and DeepseekV2MoE.forward, and installs a parameter-free Attention experts proxy at the remote-experts boundary. Routed and shared experts remain owned by the FFN model runners, and the existing connector protocol is reused.

The NPU path rebases plugin-owned runners and isolated compatibility patches onto the v0.26 APIs. It removes stale PCP fields and hooks, refreshes the DeepSeek FusedMoE factory after plugin initialization, captures force-load-balance configuration during method construction, and preserves AFD DBO settings across Ascend platform validation.

Test Plan

• Run the repository CPU-only CI on the combined PR.
• Retain the completed GPU eager, CUDA Graph, TP/EP, DP/EP, and DBO validation from [v0.26.0][GPU] Integrate DeepSeek remote-experts refactor #182.
• Retain the completed NPU focused unit, marker-based E2E, accuracy, PCP-removal, and DBO/two-ubatch validation from feat(npu): support vLLM 0.26 #183, refactor(npu): remove PCP support from model runner v1 #184, and fix(npu): restore DBO compatibility for vLLM 0.26 #185.

Test Result

• Documentation refresh: targeted pre-commit hooks, local Markdown link validation, git diff --check, and the pinned vLLM-Ascend installation link check passed.
• Combined diff: git diff --check upstream/main...upstream/update/v0.26.0 passed.
• GPU integration ([v0.26.0][GPU] Integrate DeepSeek remote-experts refactor #182): focused unit suite 92 passed; complete unit suite 353 passed and 29 skipped; CPU-only CI passed on Python 3.10-3.13; GPU eager, CUDA Graph, TP2/EP2, DP2/EP2, and representative 2A2F DBO validation passed.
• NPU v0.26 integration (feat(npu): support vLLM 0.26 #183): 32 focused unit tests passed; marker-based non-accuracy E2E 14 passed and 4 expected DBO skips; GSM8K eager/graph limit-200 and full eager 1319-sample validation completed.
• NPU PCP removal (refactor(npu): remove PCP support from model runner v1 #184): lint/format, profiler unit tests, compileall, and a four-A3-NPU 2A2F eager E2E passed.
• NPU DBO restoration (fix(npu): restore DBO compatibility for vLLM 0.26 #185): 22 targeted unit tests and four live-execution/auxiliary-output tests passed; DeepSeek-V2-Lite E2E reported 6 passed and 4 expected skips; DeepSeek-V3.2 Attention DP8/FFN DP4 DBO captured two ubatches on all eight Attention ranks and served eight concurrent completions successfully.

Docs Impact

• Refreshed the root README and recipe index for the vLLM 0.26 support boundary.
• Updated GPU and synchronous NPU guides and recipes for CUDA remote experts, both GPU gate placements, NPU DBO, and vLLM-Ascend source commit 80d8c194f.
• Marked CAM async/PCP8 material as a historical v0.19.1 experiment because it was not revalidated for v0.26 model runner v1.
• Updated module design documents and GitHub issue/PR templates to match the new compatibility contract.

---

Essential PR Checklist

• Purpose is clear and linked to public context when possible.
• Scope is bounded.
• Compatibility with vLLM v0.19.1 is considered; this PR intentionally upgrades the supported contract to v0.26.0.
• No changes are made to the vLLM source checkout.
• Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
• Compatibility patches are isolated, documented with AFD patch boundaries, and covered by focused tests.
• Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
• Validation evidence is included, including expected skips and hardware-gated coverage.
• Documentation impact is stated.

[jiangkuaixue123]

Scan:

Category	Result
Tests / verification	1 finding(s) below
Security	no finding reported
Docs / comments	3 finding(s) below
Behavior / compatibility	1 finding(s) below
Correctness	no finding reported

Validated:

• [sweep] config_validation.py:56-91 — all branches in create_engine_config can occur; worker_cls_was_auto captured before backend relaxation; Ascend patch applied before upstream config creation; backend restores are ordered correctly
• [sweep] config_validation.py:99-116 — __post_init__ now only handles backend relaxation; worker selection moved to create_engine_config; no merge damage
• [sweep] config_validation.py:148-165 — _should_relax_engine_args_backend replaced getattr with direct attribute access; nixl_ep added to exclusion set; enable_dbo/ubatch_size guard matches EngineArgs-level semantics
• [sweep] config_validation.py:168-181 — _should_relax_vllm_config_backend uses use_ubatching vs EngineArgs-level enable_dbo/ubatch_size — intentional divergence between first-validation and re-validation paths
• [sweep] ascend_platform.py:29-71 — apply_afd_ascend_dbo_config_patch now patches check_and_update_config (not _fix_incompatible_config) and returns bool; snapshot/restore wraps whole method
• [sweep] ascend_platform.py:74-100 — _AFDDBOConfigSnapshot no longer captures use_ubatching; _restore_afd_dbo_config guard changed from 3-condition OR to 2-condition AND; see finding below
• [validated] ascend_platform.py:99 — guard not saved.enable_dbo and saved.ubatch_size == 0 is equivalent to old guard minus use_ubatching; early return when nothing to restore is correct
• [sweep] force_load_balance.py:207-243 — __init__ on AscendW8A8DynamicFusedMoEMethod captures AFD config only; does NOT call super().__init__() and does NOT set self.quant_type; see finding below

Reviewed 12 candidates. Dropped 4 (i0 misread the patched init, i1/i9 speculation about use_ubatching without evidence, i3 misinterpreted the guard logic). Kept 8: one minor confirmation request for the unconditional output queue task, one nit about the engine_id comment, one stale artifact, two documentation updates, two verification asks (add a comment about router_logits on NPU and a test for closed-connector error path), and one test-assertion suggestion.

Verdict: COMMENT

Findings without a valid current diff anchor

These findings are preserved in the review body because their file/line could not be mapped to the fetched PR diff.

• vllm_afd_plugin.egg-info/PKG-INFO:46 [minor] — vllm_afd_plugin.egg-info/PKG-INFO still references vLLM 0.19.1 (grep returned 6 matches). Either regenerate the metadata during the build or add vllm_afd_plugin.egg-info/ to .gitignore to prevent stale artifacts. Evidence: grep for '0.19.1' in vllm_afd_plugin.egg-info/PKG-INFO returned 6 matches all with old version references. The README.md and other source files in the diff correctly updated their references.
• afd_plugin/v1/worker/npu/ffn_model_runner.py:246 [minor] — The NPU FFN runner (_ffn_forward at lines 235-265) receives AFDA2FTransferPayload but never accesses payload.router_logits (unlike the GPU runner). Add a comment or assertion that the synchronous CAMP2P path does not use router_logits (GPU remote-experts only). Evidence: Read afd_plugin/v1/worker/npu/ffn_model_runner.py lines 235-265 (the _ffn_forward method). The payload destructuring at line 253-255 assigns hidden_states, context, metadata, states but never reads router_logits. Compare with GPU FFN runner at ffn_model_runner.py line 218-222 which conditionally reads payload.router_logits.

[jiangkuaixue123]

[minor] The unified add_request_async now calls self._ensure_output_queue_task() unconditionally (line 358), while the previous AFD async-DP path (deleted lines ~329-341) explicitly omitted it. Confirm whether this behavioral change is intentional or if the AFD path should still skip the output queue task.

Evidence: Read async_dp_engine.py: the old code had two separate branches; the AFD branch (after # PATCH START) omitted _ensure_output_queue_task(). The new unified code calls it unconditionally at line 358.

[jiangkuaixue123]

[minor] The patch comment at line 244 says 'adds layer_idx and gate_placement' but the actual parameter is named compute_gate_on_attention (line 251). Update the comment to match the actual signature.

Evidence: File afd_plugin/model_executor/models/deepseek_v2.py: patch comment at line 244 reads # Signature: AFD-owned; adds layer_idx and gate_placement and omits while the init signature at line 251 defines compute_gate_on_attention: bool. Grep for gate_placement across the repo confirms this string appears nowhere else — it was renamed but the comment was not updated.

[jiangkuaixue123]

[minor] The docstring for apply_afd_ascend_dbo_config_patch (lines 27-37) only mentions DBO/ubatching fields, but the snapshot now includes all2all_backend. Update the docstring to also mention that all2all_backend is preserved, consistent with the inline comment at line 56.

Evidence: File afd_plugin/compat/patches/npu/ascend_platform.py: docstring at lines 27-37 describes the snapshot/restore flow but only mentions DBO/ubatching fields. _AFDDBOConfigSnapshot at line 24 includes all2all_backend: str. _restore_afd_dbo_config at line 105 restores parallel_config.all2all_backend = saved.all2all_backend. The inline patch comment at lines 55-60 already correctly mentions both DBO fields and the temporary ubatching backend.

[hsliuustc0106]

please update the copilot knowledge/rules as well after this upgrade

[hsliuustc0106]

please create a release for v0.19.1 before this PR merged. the whl is enough

[hsliuustc0106]

Three inline findings from the vLLM 0.26 compatibility review.

[hsliuustc0106]

[P1] Aggregate FFN DP sizes before the external-router MoE call

The newly supported CUDA external-router branch enters native MoE after P2P fan-in, but forward_context.dp_metadata still carries raw per-Attention counts. In documented-valid 4A2F, counts [3, 5, 7, 11] make F0 receive 8 concatenated tokens while vLLM’s AgRs path reads local size 3 and asserts sizes[rank] == hidden_states.shape[0]; the FFN rank aborts while Attention waits for its response. The raw-metadata defect predates this PR for gate-on-FFN, but this PR newly routes supported CUDA compute_gate_on_attention=true through it. Please derive FFN-side subgroup totals ([8, 18] here) after TP expansion while retaining raw counts for P2P wire allocation, or reject A != F. Add a native AgRs 4A2F test for both gate placements.

[hsliuustc0106]

[P2] Keep the declared patch signature exact

This new comment says the signature matches pinned vLLM, but upstream UBatchWrapper._create_sm_control_context(vllm_config: VllmConfig) has no return annotation while this override declares -> AbstractContextManager[None]. AGENTS.md explicitly requires patched signatures—including return types—to match exactly. Please remove the annotation or document it as an intentional signature delta instead of claiming an exact match.

[hsliuustc0106]

[P2] Update the force-load-balance patch inventory

This row still says the plugin patches AscendFusedMoE.__init__ and uses a layer-owned switch. The refreshed implementation now globally assigns both AscendW8A8DynamicFusedMoEMethod.__init__ and .apply (force_load_balance.py:442-443), with method-owned state; there is no remaining AscendFusedMoE assignment. Please update the symbol inventory and ownership wording so future vLLM-Ascend upgrade/removal audits inspect the actual patch.

[jiangkuaixue123]

please update the copilot knowledge/rules as well after this upgrade

ok

[jiangkuaixue123]

please create a release for v0.19.1 before this PR merged. the whl is enough

ok