File tree Expand file tree Collapse file tree 2 files changed +73
-0
lines changed Top Open diff view settings
doc/projects_llm_framework_doc Expand file tree Collapse file tree 2 files changed +73
-0
lines changed Top Open diff view settings Original file line number Diff line number Diff line change @@ -53,6 +53,43 @@ Response JSON:
5353- created: Message creation time, UNIX time.
5454- work_id: Successfully created work_id unit.
5555
56+ ## inference
57+
58+ ### streaming input
59+
60+ ``` json
61+ {
62+ "request_id" : " 2" ,
63+ "work_id" : " melotts.1003" ,
64+ "action" : " inference" ,
65+ "object" : " melotts.utf-8.stream" ,
66+ "data" : {
67+ "delta" : " What's ur name?" ,
68+ "index" : 0 ,
69+ "finish" : true
70+ }
71+ }
72+ ```
73+ - object: The data type transmitted is melotts.utf-8.stream, indicating a streaming input from the user's UTF-8.
74+ - delta: Segment data of the streaming input.
75+ - index: Index of the segment in the streaming input.
76+ - finish: A flag indicating whether the streaming input has completed.
77+
78+ ### non-streaming input
79+
80+ ``` json
81+ {
82+ "request_id" : " 2" ,
83+ "work_id" : " melotts.1003" ,
84+ "action" : " inference" ,
85+ "object" : " melotts.utf-8" ,
86+ "data" : " What's ur name?"
87+ }
88+ ```
89+
90+ - object: The data type transmitted is melotts.utf-8, indicating a non-streaming input from the user's UTF-8.
91+ - data: Data for non-streaming input.
92+
5693## link
5794
5895Link the output of the upper-level unit.
Original file line number Diff line number Diff line change 5151- created:消息创建时间,unix 时间。
5252- work_id:返回成功创建的 work_id 单元。
5353
54+ ## inference
55+
56+ ### 流式输入
57+
58+ ``` json
59+ {
60+ "request_id" : " 2" ,
61+ "work_id" : " melotts.1003" ,
62+ "action" : " inference" ,
63+ "object" : " melotts.utf-8.stream" ,
64+ "data" : {
65+ "delta" : " 今天天气真好!" ,
66+ "index" : 0 ,
67+ "finish" : true
68+ }
69+ }
70+ ```
71+ - object:传输的数据类型为 ` melotts.utf-8.stream ` 代表的是从用户 utf-8 的流式输入
72+ - delta:流式输入的分段数据
73+ - index:流式输入的分段索引
74+ - finish:流式输入是否完成的标志位
75+
76+ ### 非流式输入
77+
78+ ``` json
79+ {
80+ "request_id" : " 2" ,
81+ "work_id" : " melotts.1003" ,
82+ "action" : " inference" ,
83+ "object" : " melotts.utf-8" ,
84+ "data" : " 今天天气真好!"
85+ }
86+ ```
87+ - object:传输的数据类型为 ` melotts.utf-8 ` 代表的是从用户 utf-8 的非流式输入
88+ - data:非流式输入的数据
89+
5490## link
5591
5692链接上级单元的输出。
You can’t perform that action at this time.
0 commit comments