Tool call changes - #1060
Draft
Akshay Sonawane (apsonawane) wants to merge 1 commit into
Draft
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Copilot started reviewing on behalf of
Akshay Sonawane (apsonawane)
September 1, 2026 20:56
View session
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Tool-call inputs and reasoning output have correctness regressions, and the new documentation link is unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Improves C++ tool-calling, reasoning-stream handling, Responses API inputs, and BYOM metadata preservation.
Changes:
- Preserves local-model tool/reasoning metadata and adds function-call inputs.
- Enhances tool-call parsing, streaming, grammar guidance, and history replay.
- Adds focused tests and documentation navigation.
File summaries
| File | Description |
|---|---|
sdk_v2/cpp/test/internal_api/toolcalling/tool_call_utils_test.cc |
Tests malformed and aliased calls. |
sdk_v2/cpp/test/internal_api/toolcalling/tool_call_stream_accumulator_test.cc |
Tests flush recovery. |
sdk_v2/cpp/test/internal_api/toolcalling/grammar_test.cc |
Updates grammar expectations. |
sdk_v2/cpp/test/internal_api/response_converter_test.cc |
Tests function-call inputs. |
sdk_v2/cpp/test/internal_api/chat/chat_template_test.cc |
Tests tool-call history serialization. |
sdk_v2/cpp/src/service/responses_handler.cc |
Emits and normalizes streamed calls. |
sdk_v2/cpp/src/items/message_item.h |
Stores calls on messages. |
sdk_v2/cpp/src/items/message_item.cc |
Copies stored calls. |
sdk_v2/cpp/src/inferencing/generative/toolcalling/tool_call_utils.h |
Extends parser API. |
sdk_v2/cpp/src/inferencing/generative/toolcalling/tool_call_utils.cc |
Adds parsing recovery and normalization. |
sdk_v2/cpp/src/inferencing/generative/toolcalling/tool_call_stream_accumulator.h |
Recovers unterminated calls. |
sdk_v2/cpp/src/inferencing/generative/toolcalling/grammar.cc |
Constrains guided text. |
sdk_v2/cpp/src/inferencing/generative/openresponses/response_converter.cc |
Converts function-call inputs. |
sdk_v2/cpp/src/inferencing/generative/chat/reasoning_stream_splitter.h |
Supports prefilled reasoning. |
sdk_v2/cpp/src/inferencing/generative/chat/onnx_chat_generator.h |
Tracks reasoning-prefill state. |
sdk_v2/cpp/src/inferencing/generative/chat/onnx_chat_generator.cc |
Detects prefilled reasoning and expands guidance. |
sdk_v2/cpp/src/inferencing/generative/chat/chat_template.h |
Exposes message serialization. |
sdk_v2/cpp/src/inferencing/generative/chat/chat_template.cc |
Serializes assistant calls. |
sdk_v2/cpp/src/inferencing/generative/chat/chat_session.cc |
Updates parsing, streaming, and history. |
sdk_v2/cpp/src/contracts/responses.h |
Adds function-call input contract. |
sdk_v2/cpp/src/contracts/responses_json.cc |
Deserializes function-call inputs. |
sdk_v2/cpp/src/catalog/azure_model_catalog.cc |
Merges BYOM metadata. |
docs/README.md |
Links engineering notes. |
Review details
- Files reviewed: 23/23 changed files
- Comments generated: 8
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (ctx.tool_output) { | ||
| // Keep marker-delimited calls out of the free-text branch so they must | ||
| // pass through the schema-constrained functioncall rule. | ||
| grammar << (known_tool_tokens ? "TEXT: /[^{<][^<]*/\n" : "TEXT: /[^{<][^{]*/\n"); |
Comment on lines
+206
to
+212
| // Locally imported models may not have catalog metadata even when their | ||
| // chat template uses the standard Qwen tool-call markers. | ||
| if (tool_ctx.HasTools() && tool_ctx.tool_call_start.empty() && tool_ctx.tool_call_end.empty()) { | ||
| tool_ctx.supports_tool_calling = true; | ||
| tool_ctx.tool_call_start = "<tool_call>"; | ||
| tool_ctx.tool_call_end = "</tool_call>"; | ||
| } |
| : cached_tool_ctx_.reasoning_end) | ||
| : std::string()); | ||
| : std::string(), | ||
| cached_generator_ && cached_generator_->PromptEndsInReasoning()); |
Comment on lines
+893
to
+896
| auto tool_calls = assistant_reply ? std::move(assistant_reply->tool_calls) | ||
| : std::vector<ToolCallItem>{}; | ||
| assistant_reply = msg; | ||
| assistant_reply->tool_calls = std::move(tool_calls); |
Comment on lines
+329
to
+331
| } else if (auto* fc = std::get_if<FunctionCallInputItem>(&input_item)) { | ||
| auto i = std::make_unique<ToolCallItem>(fc->call_id, fc->name, fc->arguments); | ||
| request.AddOwnedItem(std::move(i)); |
Comment on lines
+327
to
+332
| // If a malformed call was left open before another call, parse the | ||
| // innermost complete block rather than combining both payloads. | ||
| size_t nested_start = text.rfind(tool_call_start, end_pos); | ||
| if (nested_start != std::string::npos && nested_start > start_pos) { | ||
| content_start = nested_start + tool_call_start.size(); | ||
| } |
|
|
||
| Documentation for Foundry Local can be found in the following resources: | ||
|
|
||
| - [MSBench Tool Calling Fixes](MSBench%20Tool%20Calling%20Fixes.md): Engineering notes for the Responses API and tool-calling changes that enabled a local Qwen model to produce SWE-bench patches through MSBench. |
Comment on lines
+132
to
+133
| std::string text = | ||
| R"(<tool_call><exec_command","arguments":{"cmd":"ls"}<tool_call>{"name":"exec_command","args":{"cmd":"pwd"}}</tool_call>)"; |
| struct MessageItem : Item { | ||
| flMessageRole role; | ||
| std::vector<MessagePart> content; | ||
| std::vector<ToolCallItem> tool_calls; |
Collaborator
There was a problem hiding this comment.
Does this need to be part of the MessageItem vs. multiple Items being provided in the Request (e.g. MessageItem and potentially multiple ToolResultItem instances) to keep them decoupled?
Collaborator
There was a problem hiding this comment.
Or multiple ToolCallItem instances in the Response items.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several improvements and fixes to the handling of tool-calling and reasoning support for locally imported models, as well as enhancements to the chat session and prompt generation logic. The changes ensure that important metadata from local models is preserved, improve the processing of tool calls and reasoning blocks, and add better support for assistant tool calls in chat history.
Key changes include:
Tool-calling and Reasoning Metadata for Local Models
inference_model.jsoninto theModelInfo, preserving tool-calling and reasoning tags provided by the model author. [1] [2] [3]Input Item and JSON Handling
FunctionCallInputItemtype to more accurately represent function call inputs, updated theInputItemvariant, and implemented corresponding JSON (de)serialization logic. The parser now distinguishes between function call inputs and outputs in the input array. [1] [2] [3] [4]Chat Session and Prompt Processing
Assistant Tool Call History
Prompt Generation Utilities
Reasoning Model Prompt Detection
Documentation: