Skip to content

Conversation

@ivyilike
Copy link

@ivyilike ivyilike commented Nov 19, 2025

What this PR does / why we need it?

Some worker interfaces called by vLLM are missing in vLLM Ascend. It's good to add the support.
this PR add support to "update_config" interface. Update config in model_runner_v1.
For example, collective_rpc("update_config", overrides={"load_config": {"load_format": "auto"}})

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an update_config interface to the worker, allowing for dynamic configuration updates. The implementation is straightforward, adding the method to NPUWorker and NPUModelRunner, along with a corresponding unit test. My main feedback is to replace the use of assert for input validation with a ValueError to ensure robustness, as assertions can be disabled. This change also requires a small update to the new test case.

Comment on lines +4599 to +4601
assert config_name in allowed_config_names, \
f"Config `{config_name}` not supported. " \
f"Allowed configs: {allowed_config_names}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using assert for input validation is not recommended as assertions can be disabled with the -O Python flag, which would lead to this check being skipped. It's more robust to raise a ValueError for invalid configuration keys.

Additionally, sorting the allowed_config_names in the error message will ensure deterministic output, which is good practice.

Note that this change will require updating test_update_config to expect a ValueError.

Suggested change
assert config_name in allowed_config_names, \
f"Config `{config_name}` not supported. " \
f"Allowed configs: {allowed_config_names}"
if config_name not in allowed_config_names:
allowed = sorted(list(allowed_config_names))
raise ValueError(f"Config `{config_name}` not supported. Allowed: {allowed}")

model_runner.update_config({"load_config": {"load_format": "dummy"}})
assert model_runner.load_config.load_format == "dummy"
# Raise error on non-existing config
with pytest.raises(AssertionError):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

To align with the recommended change of raising a ValueError for invalid inputs in NPUModelRunner.update_config, this test should be updated to expect a ValueError.

Suggested change
with pytest.raises(AssertionError):
with pytest.raises(ValueError):

@xueliangyang-oeuler
Copy link

good job !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants