Skip to content

feat: add LightLLM provider support #806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

kelvinguo0210
Copy link

@kelvinguo0210 kelvinguo0210 commented May 4, 2025

With this change, you can specify a LightLLM provider for Codex.

Command line:

export LIGHTLLM_API_KEY=
export LIGHTLLM_BASE_URL=http://xxx.xxx.xxx.xxx:port

node ./codex-cli/dist/cli.js --provider lightllm --model claude-3-7-sonnet --fullAuto
or:
codex --provider lightllm --model claude-3-7-sonnet --fullAuto

config.yaml
The example LightLLM configuration file:
model_list:

  • model_name: claude-3-7-sonnet
    litellm_params:
    model: bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0
    aws_access_key_id:
    aws_secret_access_key:
    aws_region_name: us-east-1
    api_key:

LightLLM Deploy:
docker run -d -v /tmp/litellm/config.yaml:/app/config/config.yaml -p 9080:4000 ghcr.io/berriai/litellm:main-latest --config /app/config/config.yaml --detailed_debug

Test:

  • codex can run against lightLLM with claude-3-7-sonnet as backend model normally.

Copy link

github-actions bot commented May 4, 2025

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kelvinguo0210
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant