Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请教一下,是否支持多模型源 #48

Open
luhengyu1 opened this issue Mar 13, 2025 · 1 comment
Open

请教一下,是否支持多模型源 #48

luhengyu1 opened this issue Mar 13, 2025 · 1 comment

Comments

@luhengyu1
Copy link

例如我在a,b,c三台机器部署了Ollama的DeepSeek r1 模型,是否可以在这之间动态分配

@LnYo-Cly
Copy link
Owner

LnYo-Cly commented Mar 13, 2025

例如我在a,b,c三台机器部署了Ollama的DeepSeek r1 模型,是否可以在这之间动态分配

可以调用下面的函数,手动传入url以及apikey。

chatCompletion(String baseUrl, String apiKey, ChatCompletion chatCompletion)

chatCompletionStream(String baseUrl, String apiKey, ChatCompletion chatCompletion, SseListener eventSourceListener)

如果ollama没有设置apikey,传入null或者""都可以。

目前没有内置轮询、负载均衡相关的实现。需要您自己实现何时调用哪一台机器上的模型。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants