What is the correct dataset to use in val.py?

[pretiand]

In my .yaml file I'm not using a test set. The outputs metrics of the model are automatically calculated on validation set, but I think that is not correct (I obtain very high mAP). Should I calculate the metrics on another test set, different from the validation?

[glenn-jocher]

@pretiand val.py computes mAP on val set by default. You can compute on any other part of your dataset using the --task argument:

python val.py --task train  # compute mAP on train set
python val.py --task val  # compute mAP on val set
python val.py --task test  # compute mAP on test set if it exists

yolov3/val.py

Line 312 in 0f80f2f

parser.add_argument('--task', default='val', help='train, val, test, speed or study')

[pretiand]

Thank you for the answer. But practically, what is the sense of calculating mAP on validation set? I tought that mAP has to be calculated on unseen data, such as test set. Thank you!

[glenn-jocher]

See https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets

[pretiand]

Thank you, sorry, I'm a newbie. For what concerns training, validation and test dataset I think there is a little bit of confusion in literature terminology.
Is here the val set provide in the .yaml file used in the training process (i.e. hyperparameters tuning...) ? Or it is only used for metrics calculation (as a "test" set of UNSEEN data)?