[Looking for community contribution] support Wan 2.2 S2V: an audio-driven cinematic video generation model

[yiyixuxu]

We're super excited about the Wan 2.2 S2V (Speech-to-Video) model and want to get it integrated into Diffusers! This would be an amazing addition, and we're looking for experienced community contributors to help make this happen.

• Project Page: https://humanaigc.github.io/wan-s2v-webpage/
• Source Code: https://github.com/Wan-Video/Wan2.2#run-speech-to-video-generation
• Model Weights: https://huggingface.co/Wan-AI/Wan2.2-S2V-14B

This is a priority for us, so we will try review fast and actively collabrate with you throughout the process :)

[VittoriaLanzo]

Hi everyone, I'd love to take ownership of this integration!

My approach will be to leverage our existing Wan implementation, extending the cross-attention modules to natively handle the S2V audio-conditioning based on the original Wan 2.2 source repository.

To make sure everything is solid, I'm going to spend some time running local equivalence tests against the original repo's forward pass before I open a PR. I'll drop a [WIP] draft PR here in a week or two once I have a baseline forward pass passing local assertions.

Let me know if anyone is already actively poking at this locally, otherwise I will get to work!

[VittoriaLanzo]

Hi @tolgacangoz — Just a heads-up that I’m picking this up for the S2V integration.
I’ve been reviewing the technical mapping you did in #12258; thanks for the foundational work on the tensor logic, it’s a huge help for understanding the conditioning layers.
My goal is to standardize the implementation to fit the current Wan architecture as a strictly 'MVP' refactor. I’m focusing on a very clean, documented version that’s easy for the maintainers to review/merge.
I’ll be using your research as the primary technical reference and will make sure to credit your work in the final PR. If you have any 'lessons learned' on the audio conditioning you'd want me to watch out for, I'm all ears—otherwise, I'm getting to work on the local tests!

[tolgacangoz]

Actually, I didn't leave that PR :/

[VittoriaLanzo]

Ah, my apologies! I completely misread the thread and thought the PR was inactive.

I’ll step back so I don't disrupt your momentum. Good luck with the rest of the integration—I'll be looking forward to seeing it merged!

[LeonxLJX]

I'd like to take this one ([Looking for community contribution] support Wan 2.2 S2V: an). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)

[tolgacangoz]

Actually, I didn't leave that PR :/

[LeonxLJX]

I'd like to take this one ([Looking for community contribution] support Wan 2.2 S2V: an). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)