I am trying to reproduce your result on SDXL(ddpm method not flow matching)

[shoutOutYangJie]

I am trying to reproduce your result on DDPM model (like SDXL ,but better), not flow matching method. I follow your configuration, don't use cfg when rollout step.

because SDXL need cfg in the inference step, so rollout step returns low quality image. with this trainging data, after 1200 step, training model outputs bad image. like this:

the output images almost lost any texture, the backgrounding color become to gray.

I use geneval reward model only, the reward score curve like this:

[shoutOutYangJie]

[Morefre]

This happens in their flow model as well. Shall we consider this as "good" samples if it aligns well with the prompts?

[shoutOutYangJie]

This happens in their flow model as well. Shall we consider this as "good" samples if it aligns well with the prompts?

but image is bad. very bad

[zhengkw18]

This is normal because the training only targets increasing the reward score. GenEval does not care about the background quality, so these images are considered good (high reward) under the GenEval environment. To improve the overall quality, you can add CFG at inference time or incorporate other reward models (e.g., PickScore, ClipScore, HPS) in training.