sd:sync to master-656-0e4ee04

[wbruna]

One-line fix for the TAE issue on Lens (and ERNIE), an LLM tensor name fix, and... another refactor 🙂

[wbruna]

@LostRuins , I'm having a strange issue with Lens + the taesd_f2.embd file on ROCm: it crashes the server when running. And with mainline sd.cpp, instead of a crash, I get an all-grey image. But both work fine with the original .safetensors TAE, and if I use -M convert to change taesd_f2.embd to either GGUF or safetensors format (which also doubles its size). Any idea about what could be happening?

[LostRuins]

@wbruna it could be due to the data format. The one in KoboldCpp is manually converted to fp8_ e4m3 format. This has normally worked fine with all prior cases.

Perhaps something here doesn't like that format? Either that or the loss of precision causes the vae to fail catastrophically. Maybe we can try to convert back to bf16 in memory?

[wbruna]

Found it: it's an issue with mmap. Looks like the f8 tensors are not being skipped as they should.

[LostRuins]

Ah, okay. Do I need to patch anything on my side?

[LostRuins]

btw i got ltx video gen working on my 16gb @wbruna

[wbruna]

Ah, okay. Do I need to patch anything on my side?

Only if you think it's worth including before it's accepted upstream: leejet/stable-diffusion.cpp#1575

[wbruna]

btw i got ltx video gen working on my 16gb @wbruna

Cool 🙂 I'll give it a try during the weekend. Did you use those same models and parameters?

[LostRuins]

Yeah, but you might also want to run the LLM on GPU for faster speeds. With CPU offload that works for me

[LostRuins]

Only if you think it's worth including before it's accepted upstream: leejet/stable-diffusion.cpp#1575

no reply from leejet so i just went ahead and merged

[wbruna]

no reply from leejet so i just went ahead and merged

He just merged it. He also pushed another fix; I'll sync everything as soon as the release gets tagged.