This new version of diffusers causes problems when doing inference with Flux 1.D and Wan2.2 Text To Image (explanation below)

[ukaprch]

Describe the bug

A few days ago I decided to upgrade Torch (cuda:gpu) from v2.7.1 to 2.8.0. In addition I also decided to update diffusers from v0.35.0 to v0.36.0. I updated additional components as well due to the Torch upgrade as well as compatibility with the new version of diffusers.
llama-stack
transformers
huggingface-hub
tokenizers
xformers (for torch 2.8)
triton-windows-3.5 (for torch 2.8)
sageattention-2.2.0 (for torch 2.8)
flash_attn-2.8.2 (for torch 2.8)

There may have been additional components added as is the case when upgrading.

I didn't change any of my code for inference. I encountered 2 major problems with the new diffusion library which may or may not be specifically attributable to diffusers alone:

1. When running Flux 1.D text to image, the model/transformer seemed to ignore parts of the prompt. When running WAN 2.2 T2I it was even worse. The model/transformer ignored the entire prompt. So basically, something changed in the environment after the updates.

2. When running WAN 2.2 T2I using "Sage" attention, I get a black image. I traced this back to the transformer attn processing. After a few iterations of attn processing, Sage Attn starts experiencing problems with the Query Tensor producing nan's... which eventually cause my black image. The K and V components don't seem affected, only the Q part. Strange.

As I stated, I didn't change any of my code for inference, only the environment as pertains to what I stated above. Has anyone else experience this or a similar problem?

It doesn't help to produce the code / logs as they don't show anything regarding the problem I stated above. This is a problem that doesn't emit an error message, only a bad or wrong image.

Reproduction

N/A. Note that I use the diffuser library locally on my PC. Not using Focus, A111, or any other alternative library.
I should add that I'm using the Quanto Quantized (INT8) versions of the Transformers and T5 Encoders due to VRAM limitations for both Flux 1.D and WAN 2.2 and have experienced no previous problems in doing so with the diffusers library.

In the interim, I have reverted back to previous releases of diffusers-0.35.0.dist-info and transformers-4.55.4.dist-info (keeping my upgraded Torch 2,8 functionality) and am not experiencing the problems I note here.

Logs

System Info

System: Windows 10 OS; Python v12.5.0
GPU: RTX 4090
Diffusers: v0.36.0 (current)
Transformers: v5.0.0 (current)
Torch: v2.8.0 (cuda)
Cuda Toolkit: v12.8

Who can help?

No response

[asomoza]

Hi, there's two issues in one here, one is for the actual inference and the other one for sage attention. Lets try to approach the inference difference first.

You're not just upgrading diffusers only, you're upgrading multiple libraries at the same time but blaming it all on diffusers, is there a specific reason you're doing this?

If not, the first obvious approach is to isolate the problem, can you please upgrade only diffusers while keeping everything else in the same version, including transformers?

FYI, transformer 5.0 is a release candidate (not current), and my guess is that the problems are with it, since it has some major changes and personally I haven't found any issues with diffusers but I haven't tested it with transformers 5.0, for example the T5 tokenizer change mentioned in this PR (this is just an example not saying it's because of it)

[ukaprch]

Thanks for your reply. I did not specifically blame diffusers if you read my whole issue, however, it might make sense for me to add the new diffusers library w/o updating transformers to the new 5.0. That seems to make sense and we can take it from there. So you folks have not tested transformers 5.0 with your current release?
I will commence with re updating only diffusers and test again. BTW, updating Pytorch should have no impact, yes? Since this release 2.8 has been out for some time.

I also see this note posted as per your PR on transformers which others should heed before updating their transformer library to 5.0:
Note:

Diffusers will still work fine with Transformers < v5.0.0rc0 despite the aforementioned change, but this way it also enables Transformers v5.0.0rc0 onwards.

[asomoza]

Maybe I didn't express myself that clear, the release version of transformers is 4.57.3, you're using 5.0rc0 which is a release candidate (and just the fist one), is not a final version and you can read the information about it here

that version is not definitive and probably will change a lot, we will probably be adapting to it like with the PR I linked, but nothing is definitive with the 5.0 version yet, so the best recommendation is to not update to 5.0 if you're using diffusers and probably any other library that depends on it.

I'm using pytorch 2.9 with cuda 13 and didn't experience any problems so far, 2.8 works fine and I tested it a lot before upgrading.

[ukaprch]

Thanks for your reply. I think we can both agree that the current diffusers library isn't compatible with transformers 5.0. You may wish to take into account my original issues when testing on your side. I will continue by retesting w/o transformers 5.0 and instead use your recommendation of transformers-4.57.3.

Thanks again....now we shall see...

Update!!

Everything works as expected now. I will close this issue with the caveat that you keep my past issues in mind with future testing of transformers 5.0. especially the issue with sage attention.

Cheers and thanks again.