Inference speed seems very slow

[wssy1992]

I use the inference-image script to evaluate a single image.
scoring image:asset/test/test1.jpg: 83%|████████▎ | 25/30 [17:22<03:27, 41.56s/it]

20min ...

Did I use it wrongly somewhere? My device is an H800

[LexiPang]

Same problem, takes 30 minutes to score the default image on an A800

[weathon]

+1, 25min an image on L20. This is insanly slow. Could we distill a version? I am starting a project to distill it into a smaller model, if you are interested in working on this, please contact me.

[weathon]

@wssy1992 @LexiPang I fine tuned a Qwen-3B using their dataset, it is much faster. But I finetuned using questions with numbers as output and not binary questions.
https://huggingface.co/weathon/qwen_2_5_vision_reward_long/blob/main/README.md

[weathon]

https://huggingface.co/weathon/smolvlm2_anti_aesthetics_7b
this is a better version