Fix pure-text AFD model config identity - #174
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a regression in get_afd_model_config() where converting a pure-text ModelConfig to an AFD architecture broke the invariant hf_text_config is hf_config, which downstream vLLM-Ascend logic relies on to distinguish pure-text vs nested-text/multimodal configurations.
Changes:
- Update
get_afd_model_config()to create a single privatehf_configcopy and (when applicable) re-pointhf_text_configto preserve pure-text object identity. - Add unit tests covering both layouts: pure-text identity preservation and nested
hf_text_configpreservation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
afd_plugin/model_executor/models/model_utils.py |
Preserves hf_config/hf_text_config identity for pure-text configs while keeping truly distinct nested text configs unchanged. |
tests/unit/package/test_package.py |
Adds targeted unit coverage for both configuration layouts to prevent regressions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
52233c2 to
3c99fdb
Compare
Signed-off-by: ShwStone <haowenshi@outlook.com>
jiangkuaixue123
left a comment
There was a problem hiding this comment.
Would it be safer to use deepcopy(model_config) here and then update afd_model_config.hf_config.architectures? deepcopy preserves aliasing within the copied object graph, so the pure-text invariant hf_text_config is hf_config would be retained automatically, and it would reduce the risk of missing other identity relationships if ModelConfig evolves.
The semantic difference is that a genuinely distinct/nested hf_text_config would also become a private copy, whereas this patch intentionally keeps it shared with the source config. Is that sharing required? If not, deepcopy may provide stronger isolation and a simpler implementation. Otherwise, could we document why only hf_config should be copied so this choice is explicit?
3c99fdb to
a6e6bbc
Compare
|
I confused GPG signing with DCO; I always thought signing was enough to pass DCO. I've already recommitted. |
Signed-off-by: ShwStone <haowenshi@outlook.com>
|
@jiangkuaixue123 Agreed — the deepcopy approach is cleaner and preserves the aliasing invariants automatically. I've switched the implementation to |

Purpose
Fix the model configuration regression introduced by #144 and reported in
#172.
get_afd_model_config()copied and replacedhf_configwithout updatinghf_text_config. For pure-text models, this broke the original identity:vLLM-Ascend consequently treated DeepSeekV2-Lite as a model with a distinct
nested text configuration. Under Attention TP2 with FlashComm1/SP, MLA skipped
the required query/KV gather and passed TP-local
xtogether with full-batchRoPE
cos/sin, causingnpu_interleave_ropeto fail.The regression was confirmed across #144:
220c7626de5195bdcf24271659f0c1519a71f2a3: passes355a1cda4a1f170b0fd5db0029eba8c521151241: failsIssue
Scope
hf_config/hf_text_configidentity when converting a pure-textmodel config to its AFD architecture.
hf_text_configunchanged.Implementation Notes
get_afd_model_config()now creates one privatehf_configcopy, updates itsarchitecture, and assigns it to the copied
ModelConfig.When the source is a pure-text configuration:
the copied model config points both fields to the new private
hf_config.When the source has a genuinely distinct nested
hf_text_config, that objectremains distinct and unchanged.
This is a plugin-owned model configuration fix. It does not patch or modify
vLLM/vLLM-Ascend source code.
Test Plan
Focused unit coverage:
python -m pytest -vv \ tests/unit/package/test_package.py \ -k "afd_model_config"Ascend NPU Async CAM regression coverage:
Test Result
Both targeted tests passed:
afd_model_configunit tests: passedThe E2E result confirms that the existing main-branch Async CAM test can run
with Attention TP2 and FlashComm1 without the previous RoPE batch-dimension
failure.
Docs Impact
ModelConfiginvariantand does not introduce a new public configuration or runtime option. The
regression and reproduction are documented in [Bug]: PR #144 breaks pure-text model config identity and causes Async CAM TP2 FlashComm1 RoPE failure #172.
Essential PR Checklist