About training details

[Leoooo333]

Hi LiveAvatar team, thanks again for the great work! I am trying to reimplement liveavatar training. I want to ask about the text prompt details. Will you use the same data process as wan2.2 S2V to use a MLLM to extract very detailed text prompt for the training data, and also tune the text embedding and text cross attention during the training and distillation?

[Yubo-Shankui]

Basically the pipeline is the same. We use Tarsier to generate more fine-grained action descriptions for captions. During training, the text condition is fed into both the student and teacher models. However, our captions are still imperfect, which may also limit the model’s prompt responsiveness.

[Leoooo333]

Thanks a lot! Would you mind to share the prompt for Tarsier2 that you are using? And also during the AR training part, does LiveAvatar using detailed action caption for every block, such as Flowact-R1 do to fine-grained control the motion?