Skip to content

[v0.19.1rc1] fix(npu): preserve AFD DBO config during auto worker selection - #190

Closed
jiaran-king wants to merge 1 commit into
release/v0.19.1rc1from
codex/issue188-release-v0191rc1
Closed

[v0.19.1rc1] fix(npu): preserve AFD DBO config during auto worker selection#190
jiaran-king wants to merge 1 commit into
release/v0.19.1rc1from
codex/issue188-release-v0191rc1

Conversation

@jiaran-king

@jiaran-king jiaran-king commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

This is the release/v0.19.1rc1 backport of the same focused Issue #188 fix proposed in #189.

  • base: release/v0.19.1rc1@921ef6fc725011b4662ed2c3f4eb6942df1f2115
  • source fix commit: 50a42b73083a54294c34751905b74a3c3de6302d
  • backport commit: 3fde850
  • the five resulting source/test files are byte-identical to the main-branch fix
  • no main-only commits were merged into the release branch

Root cause and fix

With AFD NPU DBO enabled and worker_cls="auto", vLLM-Ascend v0.19.1rc1 rewrites the temporary validation backend from deepep_low_latency to flashinfer_all2allv. vLLM 0.19.1 then rejects that backend during native DBO validation before the AFD worker can be created.

The backport:

  • installs the Ascend patch before engine config construction
  • wraps the complete NPUPlatform.check_and_update_config normalization pass
  • restores enable_dbo, ubatch_size, and all2all_backend in finally
  • performs role-specific AFD worker selection after upstream resolves the automatic worker
  • keeps repeated VllmConfig.__post_init__() validation safe
  • retries patch installation after an early vLLM-Ascend import failure

Users should continue to omit --worker-cls. Explicit Attention/FFN workers remain a compatibility path, not a requirement. Validation temporarily uses deepep_low_latency; the existing worker initialization helper keeps the actual Ascend runtime backend at flashinfer_all2allv.

Validation on the release base

Environment: macOS arm64, Python 3.12.13. The release checkout was installed editable without runtime dependencies for CPU-safe tests.

  • python -m pytest -q tests/unit/compat/patches/test_config_validation.py tests/unit/compat/test_runtime.py
    • 29 passed
  • python -m pytest tests/unit -o addopts=
    • 148 passed, 23 skipped
  • python -m ruff check .
    • passed
  • python -m ruff format --check .
    • 128 files already formatted
  • git diff --check upstream/release/v0.19.1rc1..HEAD
    • passed
  • content comparison against fix(npu): preserve AFD DBO config during auto worker selection #189's commit for all five changed files
    • identical

The regression suite models the actual platform ordering and covers Attention/FFN automatic workers, DBO state preservation, repeated validation, exception restoration, explicit workers, non-AFD NPU behavior, DBO off, GPU behavior, 310P/Xlite restrictions, runtime backend selection, and ImportError retry.

Real NPU validation pending

This task did not access an NPU server or run hardware E2E. External validation should confirm:

  • no explicit --worker-cls on either role
  • no flashinfer_all2allv DBO assertion during startup
  • correct automatic Attention/FFN worker selection
  • DBO remains enabled and both ubatches execute
  • FULL_DECODE_ONLY graph capture/replay succeeds
  • /v1/models and completion requests succeed
  • actual worker runtime backend is flashinfer_all2allv
  • auto-worker output matches the explicit-worker compatibility control

No DBO performance or throughput claim is made.

Related to #188 and #189. This PR must not close Issue #188 by itself.

@jiaran-king
jiaran-king marked this pull request as ready for review August 3, 2026 10:04

jiaran-king commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

@yujuancao07, this release/v0.19.1rc1 backport is ready for real NPU acceptance testing of #188. This task intentionally did not run NPU hardware E2E. Could you please validate commit 3fde8500fd51a63f444ce046252a2166f79ccc74 against the checklist in the PR body—especially startup without --worker-cls, Attention/FFN automatic worker selection, preserved DBO with two executed ubatches, graph/completion success, and runtime flashinfer_all2allv—and post the key results or logs here? The issue should remain open until both target PRs are merged and hardware validation is recorded.

@jiaran-king
jiaran-king removed the request for review from hsliuustc0106 August 3, 2026 10:20
Signed-off-by: zzh <jiaranran2@gmail.com>
@jiaran-king
jiaran-king force-pushed the codex/issue188-release-v0191rc1 branch from bf7a525 to 3fde850 Compare August 3, 2026 10:39
@jiangkuaixue123

Copy link
Copy Markdown
Collaborator

Please add test result.

@yujuancao07

Copy link
Copy Markdown
Contributor
================================================================================================== FAILURES ===================================================================================================
________________________________________________________________________ test_npu_afd_config_patch_retries_after_initial_import_error _________________________________________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0xfffeee256c50>

    def test_npu_afd_config_patch_retries_after_initial_import_error(monkeypatch):
        fake_package = ModuleType("vllm_ascend")
        fake_package.__path__ = []
        monkeypatch.setitem(sys.modules, "vllm_ascend", fake_package)
        monkeypatch.delitem(sys.modules, "vllm_ascend.platform", raising=False)
        monkeypatch.setattr(ascend_runtime, "_PATCHES_APPLIED", False)

        ascend_runtime.apply_afd_ascend_patches_if_needed()

>       assert ascend_runtime._PATCHES_APPLIED is False
E       assert True is False
E        +  where True = ascend_runtime._PATCHES_APPLIED

tests/unit/compat/test_runtime.py:216: AssertionError

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants