[AI] Add automatic function calling support #15653
Conversation
This feature enables the Chat and GenerativeModel to automatically handle function calls from the model by executing registered handlers and feeding the results back. Includes: - AutomaticFunction struct to wrap function declarations and execution logic. - Support for FoundationModels.Tool via an adapter. - Chat loop logic for both unary and streaming requests to handle multi-turn function calls. - Integration tests for manual and Tool-based automatic function calling. - Helper methods for schema conversion.
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. |
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling capabilities to the Firebase AI SDK. Key changes include a new AutomaticFunction struct to encapsulate function declarations and their executable logic, along with an initializer to create these from FoundationModels.Tool instances. The Chat class's sendMessage and sendMessageStream methods have been significantly refactored to implement a multi-turn loop that detects function calls from the model, executes registered handlers concurrently, and feeds the results back to the model, with a maximum call limit to prevent infinite loops. The GenerativeModel now includes a functionHandlers property to store these executable functions and a new convenience initializer to configure models with AutomaticFunctions. Additionally, JSONSchema's makeInternal() and asSchema() methods were made accessible and a new asSchema() conversion was added to facilitate the FoundationModels.Tool integration. New integration tests were added to cover automatic function calling for both custom functions and FoundationModels.Tools, including streaming scenarios, and a unit test was added to verify basic automatic function calling setup. Review comments highlight the need to replace fatalError with throwing initializers/functions in AutomaticFunction and JSONSchema for graceful error handling, to use Dictionary(_:uniquingKeysWith:) to prevent crashes from duplicate function names in GenerativeModel, and to refactor the duplicated automatic function calling logic in Chat's sendMessage and sendMessageStream into a shared helper function.
|
/gemini review |
paulb777
changed the title
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling capabilities to the Firebase AI SDK. Key changes include a new AutomaticFunction struct to wrap function declarations and their execution logic, along with an extension to create AutomaticFunctions from FoundationModels.Tools. The Chat class's sendMessage and sendMessageStream methods were refactored to automatically detect and execute function calls returned by the model, incorporating a while loop with a maxFunctionCalls limit to prevent infinite loops. A new private executeFunctionCalls method was added to handle the concurrent execution of these functions. The GenerativeModel now includes a functionHandlers property and a new convenience initializer to accept AutomaticFunctions, converting their declarations into Tool objects and storing their execution logic. The JSONSchema conversion logic was updated to support this, making makeInternal() and ModelOutput extension public. Integration tests were added to verify automatic function calling for both custom and FoundationModels.Tool-based functions in unary and streaming contexts. Review comments highlighted the need to fix a state inconsistency in Chat.sendMessage by adopting a userContentCommitted pattern, suggested making the automaticFunctionTools parameter optional in the GenerativeModel convenience initializer for better API ergonomics, pointed out incomplete JSONSchema conversion for format and enumValues, and recommended extracting maxFunctionCalls and related error handling into shared private constants to reduce duplication in Chat methods, as well as extracting verbose AutomaticFunction definitions in tests into helper methods for improved readability.
FirebaseAI/Tests/TestApp/Tests/Integration/AutomaticFunctionCallingIntegrationTests.swift
Outdated
Show resolved
Hide resolved
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling, a significant feature that allows the model to interact with client-side functions. The implementation is well-structured, adding support for both unary and streaming chat modes.
Key changes include:
- The new
AutomaticFunctionstruct, which neatly wraps function declarations and their execution logic. - An adapter to support
FoundationModels.Tool, enhancing interoperability. - Updates to the
Chatclass to include a loop that handles multi-turn function call sequences, with safeguards against infinite loops. - Concurrent execution of function handlers using
withThrowingTaskGroupfor better performance. - Comprehensive integration and unit tests that cover the new functionality, including tests for the schema conversion logic.
The code is clean and the changes are well-thought-out. I have a few minor suggestions for simplification and to ensure future improvements for schema conversion are tracked. Overall, this is an excellent addition.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling capabilities, a significant and well-implemented feature. It adds the AutomaticFunction struct, updates the Chat class to handle multi-turn function calls for both unary and streaming responses, and includes support for FoundationModels.Tool. The code is robust, considering concurrency and edge cases, and is well-supported by new integration and unit tests. I have a couple of minor suggestions for refactoring to enhance code clarity and efficiency. Overall, this is an excellent contribution.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling, a significant and well-implemented feature. The changes include a new AutomaticFunction struct, updates to Chat to handle multi-turn function call execution, and modifications to GenerativeModel and schema types to support this. The logic for handling function calls, including concurrent execution and loop protection, is robust. The addition of comprehensive integration and unit tests is excellent.
My review includes a few suggestions for improvement:
- Propagating a potential JSON decoding error in
AutomaticFunctionto avoid silent failures. - A minor refactoring in
Chat.swiftto improve readability. - A suggestion to reduce code duplication in the new test files.
FirebaseAI/Tests/TestApp/Tests/Integration/GenerateObjectStreamIntegrationTests.swift
Show resolved
Hide resolved
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces robust support for automatic function calling in the Chat and GenerativeModel APIs. The changes are well-structured, with the new AutomaticFunction struct providing a clean way to define functions and their handlers. The core logic for handling the multi-turn function calling loop in both unary and streaming chat modes is implemented thoughtfully, including protection against infinite loops. The integration with FoundationModels.Tool via a schema-conversion adapter is a great addition for interoperability. The accompanying integration and unit tests are comprehensive and cover various scenarios, ensuring the reliability of this new feature. I have a couple of minor suggestions for code simplification, but overall, this is an excellent contribution.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling, a significant and well-implemented feature. The changes include a new AutomaticFunction struct, an adapter for FoundationModels.Tool, updates to the Chat class to handle the multi-turn function calling loop, comprehensive integration tests, and schema conversion helpers. The code is well-structured, and the logic for handling the chat loop and function execution is robust. I have a couple of minor suggestions for improving code conciseness and reducing duplication, but overall this is an excellent contribution.
FirebaseAI/Tests/TestApp/Tests/Integration/GenerateObjectStreamIntegrationTests.swift
Show resolved
Hide resolved
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces robust support for automatic function calling, a significant feature enhancement. The implementation is well-structured, with a new AutomaticFunction wrapper, and the chat loop logic in both sendMessage and sendMessageStream is thoughtfully updated to handle multi-turn function execution. The use of withThrowingTaskGroup for concurrent handler execution is a great choice for performance. The added integration and unit tests provide excellent coverage for the new functionality. I have a couple of minor suggestions for improvement regarding code duplication in tests and a small performance optimization.
FirebaseAI/Tests/TestApp/Tests/Integration/GenerateObjectStreamIntegrationTests.swift
Show resolved
Hide resolved
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling, a significant and well-implemented feature. The changes are comprehensive, including the new AutomaticFunction struct, robust chat loop logic for both unary and streaming requests, and compatibility with FoundationModels.Tool. The addition of extensive integration and unit tests is excellent and ensures the new functionality is reliable.
My main feedback is to consider refactoring the duplicated logic in Chat.swift between the sendMessage and sendMessageStream methods to improve long-term maintainability. I've also added a minor suggestion to improve code documentation. Overall, this is a high-quality contribution.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces automatic function calling, a significant and well-implemented feature. The changes include a new AutomaticFunction struct to encapsulate function declarations and execution logic, an adapter for FoundationModels.Tool, updated chat loop logic to handle multi-turn function calls, and comprehensive integration and unit tests. The code is well-structured and robust. I have one minor suggestion to improve maintainability.
|
/gemini summarize |
|
This pull request introduces automatic function calling support to the Key changes include:
|
1 participant
This pull request introduces automatic function calling support to the
ChatandGenerativeModelcomponents. This feature allows the model to automatically execute declared functions based on its output and feed the results back into the conversation.Key changes include:
AutomaticFunctionstruct: A new type (FirebaseAI/Sources/AutomaticFunction.swift) that wraps a function's declaration and its executable Swift closure. This allows developers to define functions that the model can call.GenerativeModelintegration: TheGenerativeModel(FirebaseAI/Sources/GenerativeModel.swift) now accepts a list ofAutomaticFunctions during initialization. These are stored asfunctionHandlerswithin the model.Chatlogic for multi-turn function calls: TheChatclass (FirebaseAI/Sources/Chat.swift) has been updated to handle multi-turn conversations involving function calls. When the model suggests a function call, theChatautomatically executes the corresponding registeredAutomaticFunctionand sends the result back to the model. AmaxFunctionCallssafeguard is in place to prevent infinite loops.FoundationModels.Tooladapter: Support has been added to convertFoundationModels.Toolinstances intoAutomaticFunctions, facilitating integration with existing tool definitions. This required modifications toJSONSchema(FirebaseAI/Sources/Types/Public/Generable/JSONSchema.swift) for schema conversion.FirebaseAI/Tests/TestApp/Tests/Integration/AutomaticFunctionCallingIntegrationTests.swift) and unit tests (FirebaseAI/Tests/Unit/ChatTests.swift,FirebaseAI/Tests/Unit/Types/Generable/JSONSchemaTests.swift) have been added to validate the automatic function calling mechanism, including both custom functions andFoundationModels.Toolbased functions, as well as structured output with complex parameters.