Skip to content

Fix pure-text AFD model config identity (release/v0.19.1rc1) - #175

Merged
jiangkuaixue123 merged 2 commits into
vllm-project:release/v0.19.1rc1from
ShwStone:fix/issue-172-model-config-identity
Jul 29, 2026
Merged

Fix pure-text AFD model config identity (release/v0.19.1rc1)#175
jiangkuaixue123 merged 2 commits into
vllm-project:release/v0.19.1rc1from
ShwStone:fix/issue-172-model-config-identity

Conversation

@ShwStone

Copy link
Copy Markdown
Contributor

Purpose

Backport of #174 to release/v0.19.1rc1.

Fix the model configuration regression introduced by #144 and reported in
#172.

get_afd_model_config() copied and replaced hf_config without updating
hf_text_config. For pure-text models, this broke the original identity:

model_config.hf_text_config is model_config.hf_config

vLLM-Ascend consequently treated DeepSeekV2-Lite as a model with a distinct
nested text configuration. Under Attention TP2 with FlashComm1/SP, MLA skipped
the required query/KV gather and passed TP-local x together with full-batch
RoPE cos/sin, causing npu_interleave_rope to fail.

Issue

Scope

  • In scope:
    • Convert the model config with deepcopy, which preserves aliasing within
      the copied object graph, so the pure-text identity
      hf_text_config is hf_config is retained automatically.
    • Keep a genuinely distinct nested hf_text_config distinct (as a private
      copy).
    • Unit coverage for both configuration layouts.
  • Out of scope:
    • Changes to vLLM or vLLM-Ascend.
    • Changes to FlashComm1/SP scheduling or MLA implementation.
    • Changes to Async CAM communication or MoE ubatching.
    • Changes to multimodal model configuration semantics.

Implementation Notes

get_afd_model_config() now deep-copies the ModelConfig and updates the
architecture on the copied hf_config. deepcopy retains internal aliasing,
so for pure-text configs hf_text_config is hf_config still holds on the copy;
a genuinely distinct nested hf_text_config stays a separate object.

Compared with the shallow-copy variant initially proposed in #174, this avoids
hand-maintaining identity relationships if ModelConfig evolves, at the cost
of privatizing all fields of the copied config.

Test Plan

  • pytest tests/unit/package/test_package.py (CPU-only; vLLM-gated tests run
    where vLLM is installed).
  • GPU/NPU validation: DeepSeekV2-Lite, Attention TP2 with FlashComm1/SP.

Test Result

  • Unit tests pass for both configuration layouts (pure-text identity preserved,
    nested text config kept distinct).
  • Real-hardware validation passed: the npu_interleave_rope failure under
    Attention TP2 with FlashComm1/SP is resolved and the model runs correctly.

Docs Impact

  • Files updated: none.
  • If none, reason: behavior-only bug fix; documented via code comments and
    unit tests.

Essential PR Checklist
  • Purpose is clear and linked to public context when possible.
  • Scope is bounded.
  • Compatibility with vLLM v0.19.1 is considered.
  • No changes are made to the vLLM source checkout.
  • Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
  • Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
  • Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
  • Validation evidence is included, including skipped GPU tests when applicable.
  • Documentation impact is stated.

ShwStone added 2 commits July 29, 2026 14:12
Signed-off-by: ShwStone <haowenshi@outlook.com>
Signed-off-by: ShwStone <haowenshi@outlook.com>
Copilot AI review requested due to automatic review settings July 29, 2026 07:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR backports the fix for a regression in get_afd_model_config() where converting the model config to an AFD architecture could break the pure-text invariant hf_text_config is hf_config, leading to downstream vLLM-Ascend behavior differences and NPU RoPE failures in certain TP2 + FlashComm1/SP paths.

Changes:

  • Switch get_afd_model_config() to deepcopy(model_config) before rewriting hf_config.architectures, preserving aliasing inside the copied object graph.
  • Add unit coverage for both pure-text configs (identity preserved) and configs with a genuinely distinct nested hf_text_config (kept distinct via private copy).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
afd_plugin/model_executor/models/model_utils.py Use deepcopy to preserve hf_config/hf_text_config aliasing when producing the AFD-targeting model config.
tests/unit/package/test_package.py Extend unit tests to assert correct identity/aliasing behavior for pure-text vs. nested text-config layouts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/unit/package/test_package.py
@jiangkuaixue123
jiangkuaixue123 merged commit 921ef6f into vllm-project:release/v0.19.1rc1 Jul 29, 2026
6 checks passed
@ShwStone
ShwStone deleted the fix/issue-172-model-config-identity branch July 31, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants