Skip to content

Fix FlexAttention cross-attention masks for dynamic text lengths#107

Draft
Arison591 wants to merge 1 commit into
Robbyant:mainfrom
Arison591:fix/flex-attention-text-mask-length
Draft

Fix FlexAttention cross-attention masks for dynamic text lengths#107
Arison591 wants to merge 1 commit into
Robbyant:mainfrom
Arison591:fix/flex-attention-text-mask-length

Conversation

@Arison591

Copy link
Copy Markdown

Summary

  • derive the per-sample text sequence length from the projected text embeddings
  • pass that length into FlexAttnFunc.init_mask
  • build the cross-attention sequence IDs from the runtime length instead of a fixed 512 tokens

Root cause

During post-training, text embeddings are flattened from [B, L, C] to [1, B * L, C] before cross-attention. The FlexAttention cross mask was always built for B * 512 key/value tokens, so any embedding length other than 512 produced a mask/KV shape mismatch.

This keeps the existing per-sample masking semantics while making the mask length match the actual projected text sequence.

Fixes #82.

Validation

  • NVIDIA GeForce RTX 5090 (compute capability 12.0)
  • PyTorch 2.9.0+cu128 and Triton 3.5.0
  • reproduced the old behavior with B=2, L=20: mask shape (1, 1, 256, 1024) rejected runtime kv_len=40
  • validated the patched FlexAttnFunc with B=2, L=20: mask shape (1, 1, 256, 40), BF16 output shape (1, 256, 2, 64), all values finite
  • python -m py_compile wan_va/modules/model.py
  • git diff --check

Signed-off-by: Haoran Qian <haoran@hust.edu.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ValueError: block_mask

2 participants