[model] support glm4_moe conversion

[Kuangdd01]

Description

• IMO, GLM4_MOE can be treated as a combination of deepseek_v3 and qwen/llama. It keep the first dense layer, one share expert per sparse layer and route_bias with GQA. So we just combine the template of qwen and deepseek_v3 and specify some arguments like rope_percent.

• This modification was verified with a tiny random model on LlamaFactory.

• training and merge process pass ✅

Help Need

• Real model should be tested
• MTP moudle should be address after GLM4.5

cc @hiyouga @chocoded

[chocoded]

It appears that the final layer (Layer 46) has a different structure compared to the intermediate layers. This requires some special handling which seems to have been overlooked in the current implementation. Could you please look into this?

[PanAndy]

@chocoded

[Kuangdd01]

Sure, the last layer is mtp-specific layer and we will look through this.

[chocoded]

For reference, you can check out the implementation here: converters and hf_invalid_keys. Could you please update the code to account for this?