Skip to content

Conversation

@MrZ20
Copy link
Contributor

@MrZ20 MrZ20 commented Nov 18, 2025

What this PR does / why we need it?

Add accuracy test for multiple models:

  • ERNIE-4.5-21B-A3B-PT.yaml
  • Mistral-7B-Instruct-v0.1.yaml
  • Phi-4-mini-instruct.yaml

Does this PR introduce any user-facing change?

How was this patch tested?

Signed-off-by: MrZ20 <[email protected]>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds accuracy test configurations for four new models: ERNIE-4.5-21B-A3B-PT, MiniCPM3-4B, Mistral-7B-Instruct-v0.1, and Phi-4-mini-instruct. The changes include new YAML configuration files for each model and an update to the list of accuracy tests. The new configuration files are well-formed. I have one suggestion to improve the maintainability of the test list file by sorting it alphabetically, which will make it easier to manage in the future.

Comment on lines 12 to 16
InternVL3_5-8B.yaml
ERNIE-4.5-21B-A3B-PT.yaml
MiniCPM3-4B.yaml
Mistral-7B-Instruct-v0.1.yaml
Phi-4-mini-instruct.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

For improved maintainability and to prevent potential duplicates, it's a good practice to keep this list of configuration files sorted alphabetically. While the existing file isn't sorted, applying this convention now would be beneficial for future updates.

Could you please sort the entire file? For your convenience, here is the alphabetically sorted list:

DeepSeek-V2-Lite.yaml
ERNIE-4.5-21B-A3B-PT.yaml
InternVL3_5-8B.yaml
Meta-Llama-3.1-8B-Instruct.yaml
MiniCPM3-4B.yaml
Mistral-7B-Instruct-v0.1.yaml
Phi-4-mini-instruct.yaml
Qwen2.5-Omni-7B.yaml
Qwen2.5-VL-7B-Instruct.yaml
Qwen2-7B.yaml
Qwen2-Audio-7B-Instruct.yaml
Qwen2-VL-7B-Instruct.yaml
Qwen3-30B-A3B.yaml
Qwen3-8B.yaml
Qwen3-VL-30B-A3B-Instruct.yaml
Qwen3-VL-8B-Instruct.yaml

@vllm-ascend-ci vllm-ascend-ci added ready-for-test start test by label for PR accuracy-test enable all accuracy test for PR labels Nov 18, 2025
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Signed-off-by: MrZ20 <[email protected]>
@vllm-ascend-ci vllm-ascend-ci removed ready-for-test start test by label for PR accuracy-test enable all accuracy test for PR labels Nov 19, 2025
Signed-off-by: MrZ20 <[email protected]>
- name: "exact_match,strict-match"
value: 0.35
- name: "exact_match,flexible-extract"
value: 0.38
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an accuracy issue on this model?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants