[Bug]: Aesthetic Gradients not working with SD 2.0/2.1 #5722

[MushroomFleet]

ClashSAN commented 16 minutes ago
leave an issue here: https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients
from here: AUTOMATIC1111/stable-diffusion-webui#5722

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

This is the expected output with no Aesthetic Gradient selected:

This is the Error produced when selecting any Aesthetic Gradient:

Error completing request
Arguments: ('KristoferHivju', '', 'None', 'None', 35, 0, False, False, 1, 1, 7, 2684044062.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 0, 0, 0, 0.9, 40, '0.0001', False, 'djzCyberSociety', '', 0.1, False, False, False, False, False, '', 1, '', 0, '', True, False, False) {}
Traceback (most recent call last):
File "D:\MACHINES\test\stable-diffusion-webui\modules\call_queue.py", line 45, in f
res = list(func(*args, **kwargs))
File "D:\MACHINES\test\stable-diffusion-webui\modules\call_queue.py", line 28, in f
res = func(*args, **kwargs)
File "D:\MACHINES\test\stable-diffusion-webui\modules\txt2img.py", line 49, in txt2img
processed = process_images(p)
File "D:\MACHINES\test\stable-diffusion-webui\modules\processing.py", line 464, in process_images
res = process_images_inner(p)
File "D:\MACHINES\test\stable-diffusion-webui\modules\processing.py", line 556, in process_images_inner
uc = prompt_parser.get_learned_conditioning(shared.sd_model, negative_prompts, p.steps)
File "D:\MACHINES\test\stable-diffusion-webui\modules\prompt_parser.py", line 138, in get_learned_conditioning
conds = model.get_learned_conditioning(texts)
File "D:\MACHINES\test\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 669, in get_learned_conditioning
c = self.cond_stage_model(c)
File "D:\MACHINES\test\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "D:\MACHINES\test\stable-diffusion-webui\modules\sd_hijack_clip.py", line 219, in forward
z1 = self.process_tokens(tokens, multipliers)
File "D:\MACHINES\test\stable-diffusion-webui\extensions\stable-diffusion-webui-aesthetic-gradients\aesthetic_clip.py", line 205, in call
tokenizer = shared.sd_model.cond_stage_model.tokenizer
File "D:\MACHINES\test\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1207, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'FrozenOpenCLIPEmbedderWithCustomWords' object has no attribute 'tokenizer'

I can confirm that the SD 1.5 model and Fine Tunes based on that is still working fine:

Steps to reproduce the problem

1. Clean Installation of Automatic1111 UI
2. Install the Aesthetic Gradient Embeddings Extension from the Extensions Tab
3. add any aesthetic Gradient to the folder:
   stable-diffusion-webui\extensions\stable-diffusion-webui-aesthetic-gradients\aesthetic_embeddings
4. Restart the UI on installed tab
5. Generate an image
6. lock/fix the seed
7. set the Aesthetic Weight to 0.9, Steps to 40
   (heavy bias to the gradient)

This works with the 1.5 model loaded, but gives the reported error with any 2.0/2.1 model

What should have happened?

you get a cool image :)

Commit where the problem happens

685f9631b56ff8bd43bce24ff5ce0f9a0e9af490

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--medvram --xformers --disable-safe-unpickle --opt-split-attention

Additional information, context and logs

I had also reported this to Vic Galle here: vicgalle/stable-diffusion-aesthetic-gradients#13

because it may be that the SD model has changed in 2.0/2.1 and the cause in unknown. At this time no other UI is able to use Aesthetic Gradient with SD 2.0/2.1
It is unclear whether a change is need to the extension or the UI.