refactor(plugins/compat-oai): use ChatCompletionAccumulator for strea… #3823
+262
−172
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ming - Simplified generateStream by using openai-go's ChatCompletionAccumulator - Removed manual tool call accumulation logic (currentToolCall, toolCallCollects) - Created convertChatCompletionToModelResponse helper for unified response conversion - Added support for detailed token usage fields: - ThoughtsTokens (reasoning tokens) - CachedContentTokens (cached tokens) - Audio, prediction tokens in custom field - Added support for refusal messages and system fingerprint metadata - Refactored generateComplete to reuse convertChatCompletionToModelResponse
This commit adds support for configuring whether tools can be executed
concurrently or must be executed sequentially.
Changes:
- Add Concurrent() method to Tool interface to query concurrency support
- Add WithConcurrent() option for configuring tool concurrency settings
- Update all tool definition functions (DefineTool, NewTool, etc.) to
accept optional ToolOption parameters using the functional options pattern
- Store concurrency flag in tool metadata and tool struct
- Modify handleToolRequests() to execute sequential tools first, then
concurrent tools in parallel
- Extract toolExecution struct and executeToolRequest() helper function
to improve code organization and eliminate duplication
By default, all tools support concurrent execution to maintain backward
compatibility. Tools can opt-out by using WithConcurrent(false).
Example usage:
tool := ai.DefineTool(registry, "myTool", "description",
func(ctx *ai.ToolContext, input string) (string, error) {
return "result", nil
},
ai.WithConcurrent(false)) // Sequential execution
Contributor
Author
|
@hugoaguirre @apascal07 pls review it |
Collaborator
|
Hi Eric, Thank you for this PR. I have given it a quick look over and will do a more thorough review once I investigate how, if at all, the changes to the tools API will interact with our in-progress implementation for multi-part tool responses as you are eager to see supported. We'd like to roll multiple changes to the API into one release. Thanks for your patience! |
Contributor
Author
|
@apascal07 OK. Thank you very much for your patience in handling this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description here... Help the reviewer by:
Checklist (if applicable):