[v0.26.0] Upgrade GPU and NPU backends

.github/ISSUE_TEMPLATE/100-bug-report.yml

@@ -8,7 +8,7 @@ body:
     attributes:
       value: >
         Before submitting, please search existing issues to avoid duplicates.
-        afd-plugin targets vLLM v0.19.1 unless an issue explicitly says otherwise.
+        afd-plugin targets vLLM v0.26.0 unless an issue explicitly says otherwise.
 
   - type: markdown
     attributes:
@@ -76,5 +76,5 @@ body:
       options:
         - label: I searched existing issues for related reports.
           required: true
-        - label: I confirmed whether this reproduces against vLLM v0.19.1 or explained why not.
+        - label: I confirmed whether this reproduces against vLLM v0.26.0 or explained why not.
           required: true

.github/ISSUE_TEMPLATE/200-feature-request.yml

@@ -8,7 +8,7 @@ body:
     attributes:
       value: >
         Before submitting, please search existing issues and RFCs. Keep proposals
-        tied to vLLM v0.19.1 compatibility unless the issue explicitly proposes a
+        tied to vLLM v0.26.0 compatibility unless the issue explicitly proposes a
         version expansion.
 
   - type: textarea
@@ -31,7 +31,7 @@ body:
     id: compatibility
     attributes:
       label: vLLM compatibility and extension points
-      description: Explain how this should work without modifying the vLLM v0.19.1 source tree.
+      description: Explain how this should work without modifying the vLLM v0.26.0 source tree.
       placeholder: |
         Preferred extension point:
         Compat shim needed:

.github/PULL_REQUEST_TEMPLATE.md

@@ -42,7 +42,7 @@ or any behavior that intentionally differs from the original AFD commit.
 
 - [ ] Purpose is clear and linked to public context when possible.
 - [ ] Scope is bounded.
-- [ ] Compatibility with vLLM v0.19.1 is considered.
+- [ ] Compatibility with vLLM v0.26.0 is considered.
 - [ ] No changes are made to the vLLM source checkout.
 - [ ] Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
 - [ ] Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.

README.md

@@ -16,7 +16,7 @@ tests for GPU and Ascend NPU deployments.
 > This project is still experimental and needs more large-scale testing across
 > different hardware backends.
 
-The target runtime is **vLLM `v0.19.1`**. The plugin does not modify the vLLM
+The target runtime is **vLLM `v0.26.0`**. The plugin does not modify the vLLM
 source tree. AFD behavior is installed through the `vllm.general_plugins` entry
 point, `--additional-config`, automatically selected role workers, plugin-owned
 model wrappers, and narrow version-scoped compatibility shims.
@@ -34,6 +34,9 @@ Core runtime support:
   execution for CUDA and Ascend NPU.
 - Eager and `FULL_DECODE_ONLY` graph execution, plus backend-specific profiling
   support.
+- Native DBO with exactly two ubatches on CUDA and the synchronous Ascend path.
+- DeepSeek MoE handoff at the remote-experts boundary on CUDA, with the gate
+  placed on either Attention or FFN.
 
 Model support:
 
@@ -49,19 +52,21 @@ See the [recipe index](recipe/README.md) for deployment and benchmark examples.
 | --- | --- | --- | --- | --- | --- |
 | `P2pNcclAFDConnector` | CUDA | Decode | Sync | `FULL_DECODE_ONLY` CUDA graph | FFN ranks are ordered before Attention ranks. `num_attention_ranks` must be greater than or equal to `num_ffn_ranks` and divisible by it. See the [DeepSeek V2 Lite recipe](recipe/gpu/P2pNcclAFDConnector/deepseek_v2_lite/README.md). |
 | `CAMP2pAFDConnector` | Ascend NPU | Decode | Sync | `FULL_DECODE_ONLY` ACL graph | Uses HCCL/CAMP2P custom ops. Ascend ops build by default on NPU platforms. See the [synchronous DeepSeek V3.2 recipe](recipe/npu/CAMP2pAFDConnector/deepseek_v3_2/README.md). |
-| `CAMAsyncAFDConnector` | Ascend NPU | Prefill | Async | Not supported | Uses CAM async-DP custom ops and requires `async=true` with the Ascend NPU workers. See the [asynchronous DeepSeek V3.2 recipe](recipe/npu/CAMAsyncAFDConnector/deepseek_v3_2/README.md). |
+| `CAMAsyncAFDConnector` | Ascend NPU | Prefill | Async | Not supported | Experimental. The v0.26 upgrade did not revalidate this path; the checked-in [PCP8 recipe](recipe/npu/CAMAsyncAFDConnector/deepseek_v3_2/README.md) records the earlier v0.19.1 experiment only. |
 
 Connector implementations are grouped by backend package:
 `afd_plugin.connectors.gpu` for GPU-only connectors,
 `afd_plugin.connectors.npu` for NPU-only connectors.
 
 Known gaps:
 
-- vLLM versions other than `0.19.1` are not claimed as supported.
+- vLLM versions other than `0.26.0` are not claimed as supported.
 - vLLM/vLLM-Ascend model runner v2 is not supported.
 - GPU and NPU E2E tests are opt-in and require real hardware plus model weights.
 - GPU CUDA graph support is limited to `FULL_DECODE_ONLY`.
-- GPU DBO plus CUDA graph is limited to exactly two ubatches.
+- Native DBO is limited to exactly two ubatches.
+- CAM async and PCP-based NPU model-runner-v1 deployments are not part of the
+  v0.26 validated runtime matrix.
 
 ## Install
 
@@ -85,36 +90,28 @@ command:
 uv sync --group dev --extra vllm
 ```
 
-The optional extra pins `vllm==0.19.1`.
+The optional extra pins `vllm==0.26.0`.
 
 ### Ascend NPU installation
 
-AFD's Ascend path is validated on openEuler 22.03 (aarch64) with Ascend 910C /
-Atlas A3. Install a compatible driver and firmware, and confirm the devices
-with `npu-smi info`. Use this compatible release baseline:
+AFD's synchronous Ascend path is validated on openEuler 22.03 (aarch64) with
+Ascend 910C / Atlas A3. Install a compatible driver and firmware, and confirm
+the devices with `npu-smi info`. Use this source baseline:
 
 | Component | Version |
 | --- | --- |
 | Python | `3.10` or `3.11` |
-| vLLM | `0.19.1` |
-| vLLM-Ascend | `0.19.1rc1` |
-| CANN / NNAL | `8.5.1` |
-| torch | `2.9.0` |
-| torch-npu | `2.9.0` |
+| vLLM | `0.26.0` |
+| vLLM-Ascend | commit [`80d8c194f`](https://github.com/vllm-project/vllm-ascend/commit/80d8c194f7584b17fe08065ea99a130916f6b0e7) |
+| CANN / torch / torch-npu | Use the mutually compatible versions required by that vLLM-Ascend source snapshot. |
 
 #### Environment
 
-The following A3/openEuler environment has been validated:
-
-```bash
-docker pull quay.io/ascend/vllm-ascend:v0.19.1rc1-a3-openeuler
-```
-
-Use the fixed
-[vLLM-Ascend installation guide](https://github.com/vllm-project/vllm-ascend/blob/v0.19.1rc1/docs/source/installation.md)
-to start the image with the device and driver configuration for your host. The
-image includes the matched CANN and NNAL environment. Run the remaining commands
-inside the container from the AFD repository root.
+The v0.26 integration was refreshed against vLLM-Ascend commit `80d8c194f`;
+the repository does not currently claim a released v0.26 container tag. Use the
+[installation guide at that source snapshot](https://github.com/vllm-project/vllm-ascend/blob/80d8c194f7584b17fe08065ea99a130916f6b0e7/docs/source/installation.md)
+to prepare a matching A3/openEuler environment, then install AFD from the
+repository root. Do not reuse the former v0.19.1rc1 image as a v0.26 runtime.
 
 #### Install AFD
 

afd_plugin/compat/npu/runtime.py

@@ -29,8 +29,8 @@ def apply_afd_ascend_patches_if_needed() -> None:
         apply_afd_ascend_dbo_config_patch,
     )
 
-    apply_afd_ascend_dbo_config_patch()
-    _PATCHES_APPLIED = True
+    if apply_afd_ascend_dbo_config_patch():
+        _PATCHES_APPLIED = True
 
 
 __all__ = [

afd_plugin/compat/patches/async_dp_engine.py

@@ -9,7 +9,7 @@
 4. ``vllm.v1.engine.core_client.DPAsyncMPClient.add_request_async``
 
 Why:
-    vLLM 0.19.1's native MoE DP path uses ``DPEngineCoreProc`` and DP wave
+    vLLM 0.26.0's native MoE DP path uses ``DPEngineCoreProc`` and DP wave
     notifications. AFD async-DP Attention ranks are connector-driven and must
     step independently while keeping the original DP/EP topology for expert
     placement and weight loading.
@@ -30,7 +30,7 @@
 
 from collections.abc import Iterator
 from contextlib import contextmanager
-from typing import TYPE_CHECKING, Any, TypeAlias
+from typing import TYPE_CHECKING
 
 import vllm.v1.engine.core as engine_core_module
 import vllm.v1.engine.core_client as core_client_module
@@ -55,24 +55,17 @@
     )
     from vllm.v1.executor import Executor
 
-    EngineLaunchResult: TypeAlias = tuple[
-        CoreEngineProcManager | CoreEngineActorManager | None,
-        DPCoordinator | None,
-        EngineZmqAddresses,
-        Queue | None,
-    ]
-
 
 # Patch reason: vLLM's MoE DP engine process uses DPEngineCoreProc, but AFD
 # async Attention ranks are connector-driven and must not run DP wave logic.
 # Patch functionality: keep upstream startup flow while selecting EngineCoreProc
 # for AFD async Attention configs.
 # Signature: matches upstream; no added parameters.
 def run_engine_core(
-    *args: Any,
+    *args,
     dp_rank: int = 0,
     local_dp_rank: int = 0,
-    **kwargs: Any,
+    **kwargs,
 ):
     """Replace MoE DP proc selection for AFD async Attention engines."""
 
@@ -96,10 +89,12 @@ def run_engine_core(
             process_title,
         )
         engine_core_module.decorate_logs()
+        if parallel_config.numa_bind:
+            engine_core_module.numa_utils.log_current_affinity_state(process_title)
 
         if data_parallel and vllm_config.kv_transfer_config is not None:
             vllm_config.kv_transfer_config.engine_id = (
-                f"{vllm_config.kv_transfer_config.engine_id}_dp{local_dp_rank}"
+                f"{vllm_config.kv_transfer_config.engine_id}_dp{dp_rank}"
             )
             engine_core_module.logger.debug(
                 "Setting kv_transfer_config.engine_id to %s",
@@ -124,13 +119,22 @@ def run_engine_core(
             parallel_config.data_parallel_rank = 0
             engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
 
+        assert engine_core is not None
+
         def wakeup_engine() -> None:
+            # Wakes up idle engine via input_queue when shutdown is requested
+            # Not safe in a signal handler - we may interrupt the main thread
+            # while it is holding the non-reentrant input_queue.mutex
             engine_core.input_queue.put_nowait((EngineCoreRequestType.WAKEUP, None))
 
         signal_callback = engine_core_module.SignalCallback(wakeup_engine)
 
-        def signal_handler(signum: int, frame: object) -> None:
-            del signum, frame
+        def signal_handler(signum, frame):
+            signal_name = engine_core_module.signal.Signals(signum).name
+            engine_core_module.logger.info(
+                "[shutdown] EngineCore: trigger received signal=%s",
+                signal_name,
+            )
             engine_core.shutdown_state = (
                 engine_core_module.EngineShutdownState.REQUESTED
             )
@@ -148,7 +152,7 @@ def signal_handler(signum: int, frame: object) -> None:
         engine_core.run_busy_loop()
 
     except SystemExit:
-        engine_core_module.logger.debug("EngineCore exiting.")
+        engine_core_module.logger.info_once("[shutdown] EngineCore: exiting busy loop")
         raise
     except Exception as exc:
         if engine_core is None:
@@ -184,7 +188,14 @@ def launch_core_engines(
     log_stats: bool,
     addresses: EngineZmqAddresses,
     num_api_servers: int = 1,
-) -> Iterator[EngineLaunchResult]:
+) -> Iterator[
+    tuple[
+        CoreEngineProcManager | CoreEngineActorManager | None,
+        DPCoordinator | None,
+        EngineZmqAddresses,
+        Queue | None,
+    ]
+]:
     """Disable coordinator wave mode while launching AFD async-DP engines."""
 
     parallel_config = vllm_config.parallel_config
@@ -197,7 +208,7 @@ def launch_core_engines(
 
     offline_mode = local_start_index is not None
 
-    tensor_queue = None
+    tensor_queue: Queue | None = None
     multimodal_config = vllm_config.model_config.multimodal_config
     if multimodal_config is not None and multimodal_config.mm_tensor_ipc == "torch_shm":
         tensor_queue = engine_utils_module.get_mp_context().Queue()
@@ -269,10 +280,13 @@ def launch_core_engines(
     if parallel_config.enable_elastic_ep:
         handshake_local_only = False
 
+    rpc_port = (
+        parallel_config.data_parallel_rpc_port or engine_utils_module.get_open_port()
+    )
     handshake_address = engine_utils_module.get_engine_client_zmq_addr(
         handshake_local_only,
         host,
-        parallel_config.data_parallel_rpc_port,
+        rpc_port,
     )
 
     if local_engines_only and dp_rank > 0:
@@ -329,39 +343,26 @@ async def add_request_async(
 ) -> None:
     """Skip the DP wave ``FIRST_REQ`` notification for AFD async-DP."""
 
-    if not is_afd_async_dp(self.vllm_config):
-        self._ensure_stats_update_task()
-
-        request.current_wave = self.current_wave
-        request.client_index = self.client_index
-
-        chosen_engine = self.get_core_engine_for_request(request)
-        to_await = self._send_input(EngineCoreRequestType.ADD, request, chosen_engine)
-        if not self.engines_running:
-            req_msg = core_client_module.msgspec.msgpack.encode(
-                ("FIRST_REQ", chosen_engine),
-            )
-            await self.first_req_send_socket.send(req_msg)
-
-        await to_await
-
-        self._ensure_output_queue_task()
-        return None
-
-    # ### PATCH START: AFD async-DP request wakeup
-    # Async-DP engines step independently, so skip the coordinator FIRST_REQ
-    # wakeup while preserving normal routing.
     self._ensure_stats_update_task()
 
     request.current_wave = self.current_wave
     request.client_index = self.client_index
 
     chosen_engine = self.get_core_engine_for_request(request)
     to_await = self._send_input(EngineCoreRequestType.ADD, request, chosen_engine)
+    # ### PATCH START: AFD async-DP request wakeup
+    # Async-DP engines step independently, so skip the coordinator FIRST_REQ
+    # wakeup while preserving normal routing.
+    if not self.engines_running and not is_afd_async_dp(self.vllm_config):
+        req_msg = core_client_module.msgspec.msgpack.encode(
+            ("FIRST_REQ", chosen_engine),
+        )
+        await self.first_req_send_socket.send(req_msg)
+    # ### PATCH END: AFD async-DP request wakeup
+
     await to_await
 
     self._ensure_output_queue_task()
-    # ### PATCH END: AFD async-DP request wakeup
 
 
 def _is_afd_async_attention_config(vllm_config: VllmConfig) -> bool: