-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add output_json_schema method to Agent class #3454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
@DouweM is there a source of truth I can reference for what the schema output should be for tests? I think I will get stuck figuring this out so any guidance will be appreciated. Edit: I added snapshots of expected output to the tests. Please review them carefully as I'm guessing what they are supposed to be. |
|
@DouweM thanks for the reviews so far. Fixed:
allows_text to True. This doesn't seem right but I'll wait for confirmation.
|
|
I don't know how to properly fix the coverage issue 😬 |
`tool_defs` provide the name and description the LLM sees, and thus is more appropriate to use given the motivations for this feature request.
This reverts commit 74ec324.
|
@g-eoj Let me know when this is ready for review again! |
|
@DouweM it is ready! Apologies. |
| count = 1 | ||
| new_key = key | ||
| while new_key in unique_object_keys: | ||
| count += 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI The coverage CI step complains about this line and the next one being not being covered by tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought my tests triggered the loop, in fact I purposely added one with non-unique keys. However, the offending code is gone now.
|
@DouweM how should duplicate tool output types be handled? For example: Can it be: Or should duplicates be removed? |
|
@DouweM ready for review. The test failure doesn't appear to be from this PR. I have one question about whether we should remove duplicate output types: #3454 (comment) |
|
@g-eoj Yeah that's fine to be deduplicated. The code using I'll do another review later or early next week, I have some some more PRs to get through :) |
This PR adds a method to get the json schema for agent output.
It aims to support all output types, including special ones:
BinaryImageDeferredToolRequestsSimple example:
{'type': 'string'}Override agent output types:
{'properties': {'response': {'type': 'boolean'}}, 'required': ['response'], 'type': 'object', 'title': 'bool'}Discriminated JSON whenever there is more than one output type:
{'type': 'object', 'properties': {'result': {'anyOf': [{'type': 'object', 'properties': {'kind': {'type': 'string', 'const': 'final_result'}, 'data': {'properties': {'response': {'type': 'boolean'}}, 'required': ['response'], 'type': 'object'}}, 'required': ['kind', 'data'], 'additionalProperties': False, 'title': 'final_result'}, {'type': 'object', 'properties': {'kind': {'type': 'string', 'const': 'str'}, 'data': {'type': 'string'}}, 'required': ['kind', 'data'], 'additionalProperties': False}]}}, 'required': ['result'], 'additionalProperties': False}Closes #3225