Bug: Qwen3-VL-Embedding-8B has missing lm_head.weight despite tie_word_embeddings=false

[Footree]

Description

Qwen/Qwen3-VL-Embedding-8B config claims tie_word_embeddings: false, but none of the 4 safetensors shards contain lm_head.weight. Only model.language_model.embed_tokens.weight exists.

This causes loading failures both in HuggingFace from_pretrained() and in downstream conversion tools (e.g., mlx-vlm convert).

Error

ValueError: Missing 1 parameters: language_model.lm_head.weight.

Reproduction

from transformers import AutoModel
model = AutoModel.from_pretrained("Qwen/Qwen3-VL-Embedding-8B", trust_remote_code=True)
# ValueError: Missing lm_head.weight

Comparison

Qwen3-VL-Reranker-8B does NOT have this issue — its weight index correctly contains lm_head.weight alongside embed_tokens.weight.

Root Cause

The safetensors index (model.safetensors.index.json) has no entry for lm_head.weight, and:

$ grep lm_head model.safetensors.index.json
# (empty)
$ grep embed_tokens model.safetensors.index.json
"model.language_model.embed_tokens.weight": "model-00001-of-00004.safetensors"

Workaround

Either:

1. Set "tie_word_embeddings": true in config.json, or
2. Duplicate embed_tokens.weight as lm_head.weight in the safetensors and update the index file.

Environment

• Model: Qwen3-VL-Embedding-8B (bf16)
• Downloaded from: https://huggingface.co/Qwen/Qwen3-VL-Embedding-8B