[Proposals Welcome] Fal Flashpack integration for faster model loading

[yiyixuxu]

Hey! 👋

We've had a request to explore integrating Fal's Flashpack for faster DiT and Text Encoder loading (#12550). Before we jump into implementation, we wanted to open this up to the community to gather ideas and hear from anyone who's experimented with this.

We'd love your input on:

1. Performance: Has anyone tried it? What kind of speedups did you see? Are there any performance trade-offs?
2. Integration Design: How would you approach it if you were to integrating this into Diffusers? Describe your design at a high level - how would we support this in our existing framework and what would the API look like?

We're looking for proposals and ideas rather than PRs at this stage. We're genuinely interested in hearing different approaches and perspectives from the community on this.

Feel free to share your thoughts!

[Tanmaypatil123]

I got this flashpack tested , it's really fast but problem comes when network storage comes in picture. torch.load outperforms both safetensors and flashpack with network storage asper my experience .

[Tanmaypatil123]

torch.load takes constant time on both without network storage and with network storage but flashpack and safetensors loading time increases with network storage. I got it tested on modal.com also and on runpod also .

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[rumutaydin]

Since flashpack support was merged in #12700, I've been testing the integration end to end and found the Hub download path is currently broken for FlashPack pipelines (filtering starves text encoders of weights, and use_flashpack is never forwarded to download) fixed in #14166. Building on that, #14167 proposes extending use_flashpack=True to transformers components so pipelines are FlashPack end to end; happy to adjust the design if you'd prefer a different approach for the text-encoder side.

[LeonxLJX]

I'd like to take this one ([Proposals Welcome] Fal Flashpack integration for faster mod). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)

[LeonxLJX]

I'd like to take this one ([Proposals Welcome] Fal Flashpack integration for faster mod). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)