TL;DR (English)
slime's default image slimerl/sglang:v0.5.15.post1-cu129 ships a SGLang version whose fused CUDA kernel fused_qk_gemma_rmsnorm_rope_gate reads Qwen3.5-VL's [3, T] M-RoPE positions as 1D [T] — it keeps only the temporal row and silently drops the height/width rows. Vision tokens get a corrupted 2D rotary encoding, so grounding / bbox rollouts are systematically wrong (~90px offset) while text/OCR look fine. In grounding RL this collapses the rollout rewards and the run looks like it is "degrading" when the model is fine — the rollout engine is encoding vision tokens wrong.
Root cause + fix are already upstream: sgl-project/sglang#35345 (kernel-level root cause) and fix PR sgl-project/sglang#35744 (extends the kernel to real [3,T] M-RoPE, keeps the fused fast path). This issue is a heads-up + workaround for slime users until that lands and the bundled image is rebuilt.
现象
用 slime 跑 Qwen3.5-VL 的 grounding / GUI 定位 RL 时,rollout 出的 bbox 系统性偏移约 90px,奖励迅速塌方,评测里定位命中率大幅下降(我这边 ~89% → ~44%),看着像"RL 把模型训退化了"。但 SFT 权重本身没问题——同一权重用 HF transformers / vLLM 推理 grounding 正常,是 rollout 推理这一侧就错,RL 只是把错误奖励放大。文本 / OCR / 数学任务不受影响。
根因
slime 的 docker/Dockerfile 默认 ARG SGLANG_IMAGE_TAG=v0.5.15.post1-cu129 / FROM slimerl/sglang:${SGLANG_IMAGE_TAG}。该 SGLang 版本里 Qwen3.5 的 full-attention 层走融合内核 fused_qk_gemma_rmsnorm_rope_gate(models/qwen3_5.py,_is_cuda and attn_output_gate 时),而该内核按 1D 取位置:
pos = tl.load(positions_ptr + token).to(tl.int64)
Qwen3.5-VL 多模态前向喂进来的是 [3, T] 的 M-RoPE(temporal / height / width 三行,mrope_section=[11,11,10]、mrope_interleaved=True、partial_rotary_factor=0.25)。内核只读了第一行 temporal,H/W 两维被静默丢弃 → vision token 的 2D 旋转位置编码坏掉。受影响模型类含 dense Qwen3_5ForConditionalGeneration 与 MoE Qwen3_5MoeForConditionalGeneration。
定位证据
组件级 bisection vs HF:pixel_values cos≈1.0、ViT+merger embeds cos≈0.9995、M-RoPE position ids 完全一致、GDN/线性层逐层无差异;第一个 full_attention 层 hidden states 发散约 10×。grounding center-in-box 命中率:融合(错)~26% → 修复后 ~81.7%(与 HF/vLLM 对齐)。
上游状态
临时 workaround(上游合并 + 镜像重建前)
在容器内 patch python/sglang/srt/models/qwen3_5.py 的 dispatch,让 CUDA + M-RoPE 走 forward_prepare_fused_gate(正确的 Python MRotaryEmbedding);或在 slime 的 Dockerfile 里临时叠一层该 patch。我有个幂等的 apply/rollback 脚本可以分享。
建议:#35744 合并后,把 slime 默认的 SGLANG_IMAGE_TAG bump 到含修复的版本 / 重建 slimerl/sglang 镜像。
详细排查记录
slime强化训练之采坑记续续续-sglang 定位能力退化 — 知乎 https://zhuanlan.zhihu.com/p/2076401110785471102
TL;DR (English)
slime's default image
slimerl/sglang:v0.5.15.post1-cu129ships a SGLang version whose fused CUDA kernelfused_qk_gemma_rmsnorm_rope_gatereads Qwen3.5-VL's[3, T]M-RoPE positions as 1D[T]— it keeps only the temporal row and silently drops the height/width rows. Vision tokens get a corrupted 2D rotary encoding, so grounding / bbox rollouts are systematically wrong (~90px offset) while text/OCR look fine. In grounding RL this collapses the rollout rewards and the run looks like it is "degrading" when the model is fine — the rollout engine is encoding vision tokens wrong.Root cause + fix are already upstream: sgl-project/sglang#35345 (kernel-level root cause) and fix PR sgl-project/sglang#35744 (extends the kernel to real
[3,T]M-RoPE, keeps the fused fast path). This issue is a heads-up + workaround for slime users until that lands and the bundled image is rebuilt.现象
用 slime 跑 Qwen3.5-VL 的 grounding / GUI 定位 RL 时,rollout 出的 bbox 系统性偏移约 90px,奖励迅速塌方,评测里定位命中率大幅下降(我这边 ~89% → ~44%),看着像"RL 把模型训退化了"。但 SFT 权重本身没问题——同一权重用 HF transformers / vLLM 推理 grounding 正常,是 rollout 推理这一侧就错,RL 只是把错误奖励放大。文本 / OCR / 数学任务不受影响。
根因
slime 的
docker/Dockerfile默认ARG SGLANG_IMAGE_TAG=v0.5.15.post1-cu129/FROM slimerl/sglang:${SGLANG_IMAGE_TAG}。该 SGLang 版本里 Qwen3.5 的 full-attention 层走融合内核fused_qk_gemma_rmsnorm_rope_gate(models/qwen3_5.py,_is_cuda and attn_output_gate时),而该内核按 1D 取位置:Qwen3.5-VL 多模态前向喂进来的是
[3, T]的 M-RoPE(temporal / height / width 三行,mrope_section=[11,11,10]、mrope_interleaved=True、partial_rotary_factor=0.25)。内核只读了第一行 temporal,H/W 两维被静默丢弃 → vision token 的 2D 旋转位置编码坏掉。受影响模型类含 denseQwen3_5ForConditionalGeneration与 MoEQwen3_5MoeForConditionalGeneration。定位证据
组件级 bisection vs HF:pixel_values cos≈1.0、ViT+merger embeds cos≈0.9995、M-RoPE position ids 完全一致、GDN/线性层逐层无差异;第一个
full_attention层 hidden states 发散约 10×。grounding center-in-box 命中率:融合(错)~26% → 修复后 ~81.7%(与 HF/vLLM 对齐)。上游状态
run-ci标签触发 CI)临时 workaround(上游合并 + 镜像重建前)
在容器内 patch
python/sglang/srt/models/qwen3_5.py的 dispatch,让 CUDA + M-RoPE 走forward_prepare_fused_gate(正确的 PythonMRotaryEmbedding);或在 slime 的 Dockerfile 里临时叠一层该 patch。我有个幂等的 apply/rollback 脚本可以分享。建议:#35744 合并后,把 slime 默认的
SGLANG_IMAGE_TAGbump 到含修复的版本 / 重建slimerl/sglang镜像。详细排查记录
slime强化训练之采坑记续续续-sglang 定位能力退化 — 知乎 https://zhuanlan.zhihu.com/p/2076401110785471102