fix regression issue and command in mix-precision example

[xin3he]

User description

Type of Change

bug fix

---

PR Type

Enhancement, Bug fix

---

Description

• Added mem_per_param_scale and enable_torch_compile arguments

• Updated dtype handling for uNVFP4 and NVFP4+

• Fixed regression issues in dtype mapping and layer configuration

• Updated README to include enable_torch_compile in example command

---

Diagram Walkthrough

flowchart LR
  A["Add mem_per_param_scale"] -- "New argument" --> B["Update dtype handling"]
  B -- "Fix regression" --> C["Add enable_torch_compile"]
  C -- "Update README" --> D["Enhance quantization"]

Loading

File Walkthrough

	Relevant files
Enhancement	quantize.py Add mem_per_param_scale and enable_torch_compile                  examples/pytorch/nlp/huggingface_models/language-modeling/quantization/mix-precision/quantize.py Added mem_per_param_scale and enable_torch_compile arguments Updated dtype handling for uNVFP4 and NVFP4+ Fixed regression issues in dtype mapping and layer configuration +27/-27
Documentation	README.md Update README with enable_torch_compile                                    examples/pytorch/nlp/huggingface_models/language-modeling/quantization/mix-precision/README.md Updated example command to include enable_torch_compile +3/-3

---

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Command Consistency The command for running the script with deepspeed has different node specifications (localhost:0,1,2,3 vs localhost:4,5,6,7). Ensure these are consistent or clarify the reason for the difference. deepspeed --include="localhost:0,1,2,3" --master_port=29500 quantize.py \ --model_name_or_path meta-llama/Llama-3.3-70B-Instruct/ \

[PRAgent4INC]

PR Code Suggestions ✨

[thuang6]

Better change AR dependency to pip released v0.8 version after AR v0.8 released

[thuang6]

not see this arg is used in example, is it added for further tuning consideration? any guideline on how user set the value?

[xin3he]

Yes, It's for llama3.3 70b pipeline parallel. It's added in case that user wants to run 70b without TP.
It's not the suggested way, the suggested way is using main branch with my fix of compile, so I intend not to introduce it.

[thuang6]

merge can wait binary published