Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: process stream events #1184

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: process stream events #1184

wants to merge 5 commits into from

Conversation

cr7258
Copy link
Collaborator

@cr7258 cr7258 commented Aug 7, 2024 •
edited
Loading

Ⅰ. Describe what this PR did

将处理 SSE 消息的逻辑抽离出来，开发 provider 时只需要处理消息本身的内容即可。

// 开发者只需要将处理消息内容的函数（本例中是 streamResponseClaude2OpenAI）传递给 processStreamEvent 函数即可。
// processStreamEvent 函数会负责 SSE 消息的处理与缓存
modifiedResponseChunk := processStreamEvent(ctx, chunk, isLastChunk, log, c.streamResponseClaude2OpenAI)

Ⅱ. Does this pull request fix one issue?

fixes #1164

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

qwen

image 
# 修改前
data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"！"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"我是"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"阿里云开发的一款超"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"大规模语言模型，我叫通义"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"千问。作为一个AI助手，我的"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"目标是帮助用户获得准确、有用"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"的信息，解决他们的问题和困惑。"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以回答各种问题、提供信息、"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"进行对话、创作文字，比如写"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"故事、文章、公文、邮件"}}],"created":1723042463,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"等，还能表达观点、参与讨论"}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"、提供建议等等。无论是"}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"学习、工作还是娱乐，只要是在"}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"语言范围内的需求，我都会尽力"}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{"role":"assistant","content":"协助。有什么我可以帮助你的吗？"}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data:{"id":"91935601-e164-9274-bc9f-d768c8272d25","choices":[{"index":0,"delta":{}}],"created":1723042464,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{"prompt_tokens":13,"completion_tokens":104,"total_tokens":117}}
------------------------------------------------------------------------------------------------------------------------
# 修改后
data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"}}],"created":1723042237,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"！"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"我是"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"阿里云开发的一款超"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"大规模语言模型，我叫通义"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"千问。作为一个AI助手，我的"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"目标是帮助用户获得准确、有用"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"的信息，解决他们的问题和困惑。"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以回答各种问题、提供信息、"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"进行对话、创作文字，比如写"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"故事、文章、公文、邮件"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"等，还能表达观点、参与讨论"}}],"created":1723042238,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"、提供建议，以及在"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"多个领域内协助工作。无论是学习"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"、工作还是娱乐，我都在这里"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"尽力提供帮助。请随时向我"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"提问或说明你的需求，我会尽力"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{"role":"assistant","content":"满足你！"}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{}}

data: {"id":"4263547c-57fa-9bb2-b5ce-a52c62279541","choices":[{"index":0,"delta":{}}],"created":1723042239,"model":"qwen-turbo","object":"chat.completion.chunk","usage":{"prompt_tokens":13,"completion_tokens":115,"total_tokens":128}}

claude

image 
# 修改前
data: {"id":"msg_011YdprPxdzVHjfzGBoxapZb","choices":[{"index":0,"delta":{"role":"assistant"}}],"created":1723042581,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"您"}}],"created":1723042581,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"好,我是"}}],"created":1723042581,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"一位由"}}],"created":1723042581,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"Open"}}],"created":1723042581,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"AI公"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"司训"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"练的"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"AI"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"助手。我"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"是"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"一个人工"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"智能系"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"统,没"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"有实"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"体形"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"态,"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"但是经"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"过大"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"量训练可"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"以像"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"人一"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"样进"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"行对话交"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"流"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":","}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"回"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"答问题,"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"协"}}],"created":1723042582,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"助工作等"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"。"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"很"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"高兴和"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"您打"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"招呼,"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"希望以"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"后可"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"以为"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"您提供"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"帮助。"}}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"created":1723042583,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}
------------------------------------------------------------------------------------------------------------------------
data: {"id":"msg_012dtJpwN2VdAX6WV3zbjBe6","choices":[{"index":0,"delta":{"role":"assistant"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"您"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"好,我是"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"一个由"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":" "}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"Anthropic "}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"公"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"司开发的"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"人工智能"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"助理"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"。我没有"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"具"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"体的名"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"字或"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"头"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"像"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":",但可"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"以用"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"\""}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"它"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"\"来称"}}],"created":1723163835,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"呼我。"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"我"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"的"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"主要功"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"能是为人"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"类提供有"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"用的信息"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"和帮助"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":","}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"回"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"答各"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"种问题,"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"协"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"助完"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"成一"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"些任务等"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"。我会"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"尽我"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"所能来满"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"足您的需"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"求,但作"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"为一个 "}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"AI 系"}}],"created":1723163836,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"统,"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"我也"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"有"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"自身"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"的限制,"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"有"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"些"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"事"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"情是无"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"法做到的"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"。不"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"过我会"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"一"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"直努力学"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"习和"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"提"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"高。"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"很"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"高兴为"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"您服务,"}}],"created":1723163837,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"有"}}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"什么需"}}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"要"}}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"尽管"}}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{"content":"问我。"}}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"created":1723163838,"model":"claude-3-sonnet-20240229","object":"chat.completion.chunk","usage":{}}

baidu

image 
# 修改前
data: {"id":"as-32qcw8gxhi","choices":[{"index":0,"message":{"role":"assistant","content":"你好！"}}],"created":1723161417,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"total_tokens":4}}

data: {"id":"as-32qcw8gxhi","choices":[{"index":0,"message":{"role":"assistant","content":"我是文心一言，英文名是ERNIE Bot。"}}],"created":1723161417,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"total_tokens":4}}

data: {"id":"as-32qcw8gxhi","choices":[{"index":0,"message":{"role":"assistant","content":"我可以完成的任务包括知识问答，文本创作，知识推理，数学计算，代码理解与编写，作画，翻译等。"}}],"created":1723161419,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"total_tokens":4}}

data: {"id":"as-32qcw8gxhi","choices":[{"index":0,"message":{"role":"assistant","content":"如果你有任何问题，请随时向我提问。"}}],"created":1723161419,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"total_tokens":4}}

data: {"id":"as-32qcw8gxhi","choices":[{"index":0,"message":{"role":"assistant"},"finish_reason":"stop"}],"created":1723161419,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"completion_tokens":51,"total_tokens":55}}
------------------------------------------------------------------------------------------------------------------------
# 修改后，顺便修复了 baidu 流式响应的错误，在 choices 中应该是 delta 而不是 message。
data: {"id":"as-ie0zjkqiqi","choices":[{"index":0,"delta":{"role":"assistant","content":"你好！"}}],"created":1723162386,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"total_tokens":4}}

data: {"id":"as-ie0zjkqiqi","choices":[{"index":0,"delta":{"role":"assistant","content":"我是文心一言，英文名是ERNIE Bot。"}}],"created":1723162387,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"completion_tokens":14,"total_tokens":18}}

data: {"id":"as-ie0zjkqiqi","choices":[{"index":0,"delta":{"role":"assistant","content":"我可以完成的任务包括知识问答，文本创作，知识推理，数学计算，代码理解与编写，作画，翻译等。"}}],"created":1723162388,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"completion_tokens":14,"total_tokens":18}}

data: {"id":"as-ie0zjkqiqi","choices":[{"index":0,"delta":{"role":"assistant","content":"如果你有任何问题，请随时向我提问。"}}],"created":1723162388,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"completion_tokens":14,"total_tokens":18}}

data: {"id":"as-ie0zjkqiqi","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":"stop"}],"created":1723162388,"model":"ERNIE-3.5-128K","object":"chat.completion","usage":{"prompt_tokens":4,"completion_tokens":51,"total_tokens":55}}

hunyuan

image 
# 修改前
data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"我"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":1,"total_tokens":8}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"是一个"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":2,"total_tokens":9}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"AI"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":3,"total_tokens":10}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"助手"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":4,"total_tokens":11}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"由"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":5,"total_tokens":12}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"人工智能"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":6,"total_tokens":13}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"技术"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":7,"total_tokens":14}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"驱动"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":8,"total_tokens":15}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"。"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":9,"total_tokens":16}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"您可以"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":10,"total_tokens":17}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"问我"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":11,"total_tokens":18}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"各种"}}],"created":1723167924,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":12,"total_tokens":19}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"问题"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":13,"total_tokens":20}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"，"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":14,"total_tokens":21}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"我会"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":15,"total_tokens":22}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"尽力"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":16,"total_tokens":23}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"回答"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":17,"total_tokens":24}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"您的"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":18,"total_tokens":25}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"疑问"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":19,"total_tokens":26}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"，"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":20,"total_tokens":27}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":21,"total_tokens":28}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"您"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":22,"total_tokens":29}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"解决问题"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":23,"total_tokens":30}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"delta":{"role":"assistant","content":"。"}}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":24,"total_tokens":31}}

data: {"id":"a4967a21-37ee-410b-b252-cf36ed48b063","choices":[{"index":0,"finish_reason":"stop"}],"created":1723167925,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":24,"total_tokens":31}}
------------------------------------------------------------------------------------------------------------------------
# 修改后
data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"我"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":1,"total_tokens":8}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"是一个"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":2,"total_tokens":9}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"AI"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":3,"total_tokens":10}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"助手"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":4,"total_tokens":11}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"由"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":5,"total_tokens":12}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"人工智能"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":6,"total_tokens":13}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"技术"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":7,"total_tokens":14}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"驱动"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":8,"total_tokens":15}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"。"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":9,"total_tokens":16}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"您可以"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":10,"total_tokens":17}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"问我"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":11,"total_tokens":18}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"各种"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":12,"total_tokens":19}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"问题"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":13,"total_tokens":20}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"，"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":14,"total_tokens":21}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"我会"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":15,"total_tokens":22}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"尽力"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":16,"total_tokens":23}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"回答"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":17,"total_tokens":24}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"您的"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":18,"total_tokens":25}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"疑问"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":19,"total_tokens":26}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"，"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":20,"total_tokens":27}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"帮助"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":21,"total_tokens":28}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"您"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":22,"total_tokens":29}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"解决问题"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":23,"total_tokens":30}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"delta":{"role":"assistant","content":"。"}}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":24,"total_tokens":31}}

data: {"id":"8ab41776-81b7-4ecd-b676-9a237ef563da","choices":[{"index":0,"finish_reason":"stop"}],"created":1723168003,"model":"hunyuan-lite","object":"chat.completion.chunk","usage":{"prompt_tokens":7,"completion_tokens":24,"total_tokens":31}}

minimax

image 
# 修改前
data: {"id":"0304abbf61242a0b13bc538a83b35a1a","choices":[{"index":0,"delta":{"content":"我","role":"assistant"}}],"created":1723168959,"model":"abab6.5g-chat","object":"chat.completion.chunk","output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0}

data: {"id":"0304abbf61242a0b13bc538a83b35a1a","choices":[{"index":0,"delta":{"content":"叫MiniMax，是MiniMax智能团队开发的一个大型语言模型，我的主要功能是通过自然语言处理技术和深度学习算法来模拟人类的思维和行为，为你提供智能对话、信息查询、智能推荐等服务。您可以使用中文、英文或其他语言与我交流，","role":"assistant"}}],"created":1723168960,"model":"abab6.5g-chat","object":"chat.completion.chunk","output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0}

data: {"id":"0304abbf61242a0b13bc538a83b35a1a","choices":[{"finish_reason":"stop","index":0,"delta":{"content":"我会尽我所能为您提供帮助。欢迎您与我交流，我会尽我所能为您提供服务。","role":"assistant"}}],"created":1723168961,"model":"abab6.5g-chat","object":"chat.completion.chunk","output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0}

data: {"id":"0304abbf61242a0b13bc538a83b35a1a","choices":[{"finish_reason":"stop","index":0,"message":{"content":"我叫MiniMax，是MiniMax智能团队开发的一个大型语言模型，我的主要功能是通过自然语言处理技术和深度学习算法来模拟人类的思维和行为，为你提供智能对话、信息查询、智能推荐等服务。您可以使用中文、英文或其他语言与我交流，我会尽我所能为您提供帮助。欢迎您与我交流，我会尽我所能为您提供服务。","role":"assistant"}}],"created":1723168961,"model":"abab6.5g-chat","object":"chat.completion","usage":{"total_tokens":170},"input_sensitive":false,"output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0,"base_resp":{"status_code":0,"status_msg":""}}

------------------------------------------------------------------------------------------------------------------------
# 修改后
data: {"id":"0304ad26ac4553e667d45f3f87fc80b6","choices":[{"index":0,"delta":{"content":"我是","role":"assistant"}}],"created":1723169318,"model":"abab6.5g-chat","object":"chat.completion.chunk","usage":{"total_tokens":0},"input_sensitive":false,"output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0,"output_sensitive_int":0}

data: {"id":"0304ad26ac4553e667d45f3f87fc80b6","choices":[{"finish_reason":"stop","index":0,"delta":{"content":"MiniMax,一个智能助理。很高兴认识你。","role":"assistant"}}],"created":1723169318,"model":"abab6.5g-chat","object":"chat.completion.chunk","usage":{"total_tokens":0},"input_sensitive":false,"output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0,"output_sensitive_int":0}

data: {"id":"0304ad26ac4553e667d45f3f87fc80b6","choices":[{"finish_reason":"stop","index":0,"message":{"content":"我是MiniMax,一个智能助理。很高兴认识你。","role":"assistant"}}],"created":1723169318,"model":"abab6.5g-chat","object":"chat.completion","usage":{"total_tokens":103},"input_sensitive":false,"output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0,"output_sensitive_int":0,"base_resp":{"status_code":0,"status_msg":""}}

Ⅴ. Special notes for reviews

@codecov-commenter
Copy link

codecov-commenter commented Aug 7, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 35.96%. Comparing base (ef31e09) to head (b03e18b).
Report is 36 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1184      +/-   ##
==========================================
+ Coverage   35.91%   35.96%   +0.05%     
==========================================
  Files          69       69              
  Lines       11576     9500    -2076     
==========================================
- Hits         4157     3417     -740     
+ Misses       7104     5767    -1337     
- Partials      315      316       +1

see 67 files with indirect coverage changes

@cr7258 cr7258 changed the title [WIP] feat: process stream events feat: process stream events Aug 9, 2024
@cr7258 cr7258 marked this pull request as ready for review August 9, 2024 02:49
johnlanni
johnlanni requested changes Aug 9, 2024
View reviewed changes
plugins/wasm-go/extensions/ai-proxy/provider/request_helper.go
type chatCompletionResponseConverter interface{}

// processStreamEvent 从上下文中取出缓冲区，将新 chunk 追加到缓冲区，然后处理缓冲区中的完整事件
func processStreamEvent(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我倾向于在 main.go 里封装好相关逻辑，组装成一个或多个完整的SSE message之后，再交给 provider 来处理，这样 provider 的代码可以不做改动。每次传入的 chunk 参数可以确保是完整的 SSE Message。

另外processStreamEvent的本质应该是多次迭代处理不完整的消息，进行合并处理，每次返回完整的 SSE 消息，这样的逻辑应该是通用的，不仅局限于AI Proxy插件使用，这个工具函数可以提出来放到上层。

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

plugins/wasm-go/extensions/ai-proxy/provider/request_helper.go
if isLastChunk || len(chunk) == 0 {
return nil
}
// 从上下文中取出缓冲区，将新 chunk 追加到缓冲区
Copy link
Collaborator

@johnlanni johnlanni Aug 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的SSE处理逻辑感觉比较复杂，可否参考下这里：

higress/plugins/wasm-go/extensions/ai-cache/main.go

Lines 237 to 270 in 04a9104

func processSSEMessage(ctx wrapper.HttpContext, config PluginConfig, sseMessage string, log wrapper.Log) string {
subMessages := strings.Split(sseMessage, "\n")
var message string
for _, msg := range subMessages {
if strings.HasPrefix(msg, "data:") {
message = msg
break
}
}
if len(message) < 6 {
log.Errorf("invalid message:%s", message)
return ""
}
// skip the prefix "data:"
bodyJson := message[5:]
if gjson.Get(bodyJson, config.CacheStreamValueFrom.ResponseBody).Exists() {
tempContentI := ctx.GetContext(CacheContentContextKey)
if tempContentI == nil {
content := TrimQuote(gjson.Get(bodyJson, config.CacheStreamValueFrom.ResponseBody).Raw)
ctx.SetContext(CacheContentContextKey, content)
return content
}
append := TrimQuote(gjson.Get(bodyJson, config.CacheStreamValueFrom.ResponseBody).Raw)
content := tempContentI.(string) + append
ctx.SetContext(CacheContentContextKey, content)
return content
} else if gjson.Get(bodyJson, "choices.0.delta.content.tool_calls").Exists() {
// TODO: compatible with other providers
ctx.SetContext(ToolCallsContextKey, struct{}{})
return ""
}
log.Debugf("unknown message:%s", bodyJson)
return ""
}

仅做参考，逻辑本身是识别openai协议用的，也不通用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@johnlanni johnlanni johnlanni requested changes

@WeixinX WeixinX Awaiting requested review from WeixinX WeixinX is a code owner

@CH3CHO CH3CHO Awaiting requested review from CH3CHO CH3CHO is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

ai-proxy 收口 SSE 消息分段机制
4 participants
@cr7258 @codecov-commenter @CH3CHO @johnlanni