Skip to content

question: does ng_run run wait for model endpoint #673

@cwing-nvidia

Description

@cwing-nvidia

Q: does ng_run wait for the policy_base_url to be ready (e.g. an sglang or vllm model)? or will that have to implemented by the client launching ng_run?

A: for external servers it will not wait to be ready.
but you can also try LocalVLLMModel which will wait https://github.com/NVIDIA-NeMo/Gym/blob/main/responses_api_models/local_vllm_model/configs/nano_v3_single_node.yaml

Metadata

Metadata

Assignees

No one assigned

    Labels

    model-serverModel server integrations

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions