Check endpoint method in nemo-evaluator does not allow vlm endpoint type

[athitten]

Describe the bug

check_endpoint method in nemo-evaluator's utils does not allow endpoint_type="vlm". This causes the error:

    check_endpoint(
  File "/opt/Evaluator/packages/nemo-evaluator/src/nemo_evaluator/core/utils.py", line 348, in check_endpoint
    raise ValueError(f"Invalid endpoint type: {endpoint_type}")
ValueError: Invalid endpoint type: vlm

Steps/Code to reproduce bug

Can be reproduced with this evaluator snippet

from nemo_evaluator.api import check_endpoint, evaluate
from nemo_evaluator.api.api_dataclasses import (
    ApiEndpoint,
    ConfigParams,
    EvaluationConfig,
    EvaluationTarget,
)

# Configure the evaluation target
api_endpoint = ApiEndpoint(
    url="http://0.0.0.0:1235/v1/chat/completions/",
    type="vlm",
    model_id="megatron-model",
)
eval_target = EvaluationTarget(api_endpoint=api_endpoint)
eval_params = ConfigParams(top_p=0, temperature=0, limit_samples=2, parallelism=1)
eval_config = EvaluationConfig(type="ocrbench", params=eval_params, output_dir="results")

if __name__ == "__main__":
    check_endpoint(
        endpoint_url=eval_target.api_endpoint.url,
        endpoint_type=eval_target.api_endpoint.type,
        model_name=eval_target.api_endpoint.model_id,
    )
    evaluate(target_cfg=eval_target, eval_cfg=eval_config)                                                           

Expected behavior

Support for endpoint_type "vlm" needs to be added to check_endpoint utility to avoid error when evaluating on vlm benchmarks.

Additional context

Add any other context about the problem here.

[marta-sd]

Good catch. I now realized we also don't have embedding there.

Side note: we're refactoring endpoint specification and vlm type will be removed soon. We will have only the basic types (completions, chat, embedding) and additionally tags indicating what capabilities a given benchmark requires. What we currently call vlm is actually a chat endpoint that accepts images, so we will use chat type in near future.