Skip to content

Add update_g_idx flag for setting qweight&g_idx #2143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 21, 2025
Merged

Conversation

Kaihui-intel
Copy link
Contributor

@Kaihui-intel Kaihui-intel commented Mar 10, 2025

Type of Change

bug fix

Description

error:
 [ERROR][modeling_auto.py:128] index 133 is out of bounds for dimension 0 with size 128
 [ERROR][modeling_auto.py:129] Saved low bit model loading failed, please check your model.

add update_g_idx flag to avoid secondary value updates.
note:
waiting for the release of ipex-gpu v2.7 ...
related PR: https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-gpu/pull/5411

results:
meta-llama/Meta-Llama-3-8B-Instruct on PVC

Tasks Version Filter n-shot Metric Value Stderr
lambada_openai 1 none 0 acc 0.7093 ± 0.0063
none 0 perplexity 3.5535 ± 0.0997

Expected Behavior & Potential Risk

the expected behavior that triggered by this PR

How has this PR been tested?

how to reproduce the test (including hardware information)

Dependency Change?

any library dependency introduced or removed

Signed-off-by: Kaihui-intel <[email protected]>
@Kaihui-intel Kaihui-intel changed the title Add update g_idx flag for setting qweight&g_dix Add update_g_idx flag for setting qweight&g_idx Mar 10, 2025
@Kaihui-intel Kaihui-intel added the related to ipex-xpu v2.7 Waiting for ipex gpu v2.7 to be ready label Mar 10, 2025
@thuang6
Copy link
Contributor

thuang6 commented Mar 17, 2025

I saw IPEX PR has been merged. have we validated it? We do not need to wait for IPEX-XPU v2.7. If there is issue, we need to raise before their release.

@Kaihui-intel Kaihui-intel merged commit c4a8c77 into master Apr 21, 2025
23 checks passed
@Kaihui-intel Kaihui-intel deleted the kaihui/gidx_fix branch April 21, 2025 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
related to ipex-xpu v2.7 Waiting for ipex gpu v2.7 to be ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants