Skip to content

fix(npu): preserve AFD DBO config during auto worker selection - #189

Closed
jiaran-king wants to merge 1 commit into
mainfrom
codex/issue188-main
Closed

fix(npu): preserve AFD DBO config during auto worker selection#189
jiaran-king wants to merge 1 commit into
mainfrom
codex/issue188-main

Conversation

@jiaran-king

Copy link
Copy Markdown
Collaborator

Summary

  • install the AFD Ascend compatibility patch before upstream constructs the NPU VllmConfig
  • preserve enable_dbo, ubatch_size, and the validation backend across the complete vLLM-Ascend platform normalization pass
  • select the role-specific AFD NPU worker only after upstream resolves worker_cls="auto"
  • keep repeated VllmConfig.__post_init__() validation safe and allow a failed early vLLM-Ascend import to be retried

Root cause

For an AFD NPU config with DBO enabled, AFD temporarily presents deepep_low_latency so vLLM 0.19.1's native DBO backend assertion can run. vLLM-Ascend v0.19.1rc1 then normalizes an automatic, non-SP worker by replacing that backend with flashinfer_all2allv. The native assertion consequently fails before the AFD worker is created.

The previous compatibility patch wrapped only NPUPlatform._fix_incompatible_config. It did not cover the later backend rewrite in the complete NPUPlatform.check_and_update_config pass.

Behavior after this change

Users continue to omit --worker-cls:

  • Attention resolves to afd_plugin.v1.worker.npu.AFDNPUAttentionWorker
  • FFN resolves to afd_plugin.v1.worker.npu.AFDNPUFFNWorker
  • DBO and two-ubatch settings survive platform normalization
  • deepep_low_latency is only the validation-safe backend
  • the existing worker initialization helper still selects the actual Ascend runtime backend, flashinfer_all2allv

Explicit role-specific workers remain compatible, but are not required by this fix. Non-AFD NPU and GPU platform behavior is unchanged. Existing 310P/Xlite automatic-selection restrictions remain in place.

Validation performed

Environment: macOS arm64, Python 3.12.13. The package was installed editable without runtime dependencies for CPU-safe tests.

  • python -m pytest -q tests/unit/compat/patches/test_config_validation.py tests/unit/compat/test_runtime.py
    • 29 passed
    • proves the real normalization order is modeled: DBO reset, auto-worker backend rewrite, native backend assertion, state restoration, and role-specific worker selection
  • python -m pytest tests/unit -o addopts=
    • 148 passed, 23 skipped
  • python -m ruff check .
    • passed
  • python -m ruff format --check .
    • 128 files already formatted
  • git diff --check
    • passed

The regression coverage also verifies repeated __post_init__, exception-safe restoration, explicit workers, non-AFD NPU behavior, DBO-off behavior, GPU automatic workers, 310P/Xlite restrictions, the runtime flashinfer_all2allv helper, and retry after an initial vLLM-Ascend import failure.

Real NPU validation pending

This task did not log in to an NPU server or execute hardware E2E. External validation should confirm:

  • Attention and FFN commands omit --worker-cls
  • startup no longer raises the flashinfer_all2allv DBO assertion
  • both processes select the correct role-specific AFD NPU workers
  • DBO remains enabled with two ubatches and both ubatches execute
  • FULL_DECODE_ONLY graph capture/replay completes
  • /v1/models and a completion request succeed
  • the actual worker runtime backend is flashinfer_all2allv
  • auto-worker output matches the explicit-worker compatibility control

This change makes no DBO performance or throughput claim.

Related to #188. The issue should remain open until both target branches are merged and hardware validation is recorded.

Signed-off-by: zzh <jiaranran2@gmail.com>
@jiaran-king
jiaran-king force-pushed the codex/issue188-main branch from 9e9ff3a to 50a42b7 Compare August 3, 2026 10:39
@jiangkuaixue123
jiangkuaixue123 deleted the codex/issue188-main branch August 6, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants