Model Providers

Model Providers

LightAgent uses OpenAI-compatible chat completion APIs. Most providers work by setting:

• model
• api_key
• base_url

OpenAI

agent = LightAgent(
    model="gpt-4.1",
    api_key="your_openai_api_key",
    base_url="https://api.openai.com/v1",
)

OpenRouter

agent = LightAgent(
    model="openai/gpt-4.1",
    api_key="your_openrouter_api_key",
    base_url="https://openrouter.ai/api/v1",
)

OpenRouter model names are provider-routed names such as openai/gpt-4.1 or another model listed in your OpenRouter account.

DeepSeek, Qwen, and Other Compatible Providers

Use the provider's OpenAI-compatible endpoint:

agent = LightAgent(
    model="provider_model_name",
    api_key="provider_api_key",
    base_url="https://provider.example.com/v1",
)

Check whether the selected model supports tool calling if you use Tools, MCP, or Skills.

vLLM

Start vLLM with its OpenAI-compatible server:

vllm serve Qwen/Qwen2.5-7B-Instruct --host 0.0.0.0 --port 8000

agent = LightAgent(
    model="Qwen/Qwen2.5-7B-Instruct",
    api_key="local",
    base_url="http://localhost:8000/v1",
)

llama.cpp

Run llama.cpp in server mode:

llama-server -m ./models/model.gguf --host 0.0.0.0 --port 8080

agent = LightAgent(
    model="local-model",
    api_key="local",
    base_url="http://localhost:8080/v1",
)

Ollama

Recent Ollama versions expose an OpenAI-compatible API:

agent = LightAgent(
    model="llama3.1",
    api_key="ollama",
    base_url="http://localhost:11434/v1",
)

Troubleshooting

• [LA-401]: check API key, account status, or auth headers.
• [LA-404]: check base_url and exact model name.
• [LA-413]: reduce history, prompt size, or tool output.
• [LA-429]: reduce concurrency or use provider-side quota controls.
• Tool calls missing: confirm the provider and model support OpenAI-compatible function calling.

More detail: Model Provider Configuration.