Runitime

[sigeisler]

Hi,

for some reason evaluating a single image on ImageNet takes easily a whole minute. Is this normal? What should I expect?

Thanks!

[david2611]

The amount of time PDQ takes per image is dependent largely on the number of detections you are producing as the most time intensive part is calculating the pixel-wise probability scores.

It is a fairly long process and evaluating over the entire PrOD Challenge Test dataset (https://competitions.codalab.org/competitions/20597) of about 56,000 images could often take 20 mins for competitors.

That said, 1 minute per image seems excessive.

[david2611]

@sigeisler did you ever manage to figure out what was going on in your evaluation?

[sigeisler]

Hi, thanks for your response. I will check it again by the end of next week I think and then will follow up. How many detections per image would you say is a reasonable number?

[david2611]

to be getting 1 minute per image timings I would be (hopefully) expecting high hundreds of detections but I will be honest, I don't have a an accurate estimate for you. I haven't done timing tests in a while.

If you have anything below 100 detetions per image though I wouldn't be expecting it to take a minute per image.