Skip to content

Conversation

@g-eoj
Copy link
Contributor

@g-eoj g-eoj commented Nov 17, 2025

This PR adds a method to get the json schema for agent output.

It aims to support all output types, including special ones:

  • text only
  • BinaryImage
  • DeferredToolRequests

Simple example:

agent = Agent('test')
agent.output_json_schema()
{'type': 'string'}

Override agent output types:

agent = Agent('test')
agent.output_json_schema(output_type=bool)
{'properties': {'response': {'type': 'boolean'}},
 'required': ['response'],
 'type': 'object',
 'title': 'bool'}

Discriminated JSON whenever there is more than one output type:

agent = Agent('test', output_type=[str,bool])
agent.output_json_schema()
{'type': 'object',
 'properties': {'result': {'anyOf': [{'type': 'object',
     'properties': {'kind': {'type': 'string', 'const': 'final_result'},
      'data': {'properties': {'response': {'type': 'boolean'}},
       'required': ['response'],
       'type': 'object'}},
     'required': ['kind', 'data'],
     'additionalProperties': False,
     'title': 'final_result'},
    {'type': 'object',
     'properties': {'kind': {'type': 'string', 'const': 'str'},
      'data': {'type': 'string'}},
     'required': ['kind', 'data'],
     'additionalProperties': False}]}},
 'required': ['result'],
 'additionalProperties': False}

Closes #3225

@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 17, 2025

@DouweM is there a source of truth I can reference for what the schema output should be for tests? I think I will get stuck figuring this out so any guidance will be appreciated.

Edit: I added snapshots of expected output to the tests. Please review them carefully as I'm guessing what they are supposed to be.

@g-eoj g-eoj marked this pull request as ready for review November 17, 2025 21:23
@g-eoj g-eoj changed the title Add output_json_schema property to Agents Add output_json_schema method to Agent class Nov 19, 2025
@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 21, 2025

@DouweM thanks for the reviews so far.

Fixed: The PR has an issue where it always adds str to the output schema if the AutoOutputSchema is selected. However I think this is due to a bug not caused by the PR. I think it is because

text_processor=processor,
will always set allows_text to True. This doesn't seem right but I'll wait for confirmation.

@g-eoj g-eoj requested a review from DouweM November 21, 2025 23:38
@g-eoj g-eoj marked this pull request as draft November 21, 2025 23:45
@g-eoj g-eoj marked this pull request as ready for review November 22, 2025 00:43
@g-eoj g-eoj marked this pull request as draft November 22, 2025 01:50
@g-eoj g-eoj marked this pull request as ready for review November 24, 2025 04:16
@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 24, 2025

I don't know how to properly fix the coverage issue 😬

g-eoj and others added 4 commits November 24, 2025 09:44
`tool_defs` provide the name and description the LLM sees, and thus is
more appropriate to use given the motivations for this feature request.
@DouweM
Copy link
Collaborator

DouweM commented Nov 26, 2025

@g-eoj Let me know when this is ready for review again!

@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 26, 2025

@DouweM it is ready! Apologies.

count = 1
new_key = key
while new_key in unique_object_keys:
count += 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI The coverage CI step complains about this line and the next one being not being covered by tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought my tests triggered the loop, in fact I purposely added one with non-unique keys. However, the offending code is gone now.

@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 28, 2025

@DouweM how should duplicate tool output types be handled? For example:

output_type=[ToolOutput(bool, name='one'), ToolOutput(bool, name='two'), ToolOutput(bool, name='three')]

Can it be:

{'anyOf': [{'type': 'boolean'}, {'type': 'boolean'}, {'type': 'boolean'}]}  

Or should duplicates be removed?

@g-eoj g-eoj requested a review from DouweM November 28, 2025 18:17
@g-eoj
Copy link
Contributor Author

g-eoj commented Nov 28, 2025

@DouweM ready for review. The test failure doesn't appear to be from this PR. I have one question about whether we should remove duplicate output types: #3454 (comment)

@DouweM
Copy link
Collaborator

DouweM commented Nov 28, 2025

@g-eoj Yeah that's fine to be deduplicated. The code using agent.run(...).output wouldn't be able to tell the different bools apart either (except by inspecting the message history for tool call parts).

I'll do another review later or early next week, I have some some more PRs to get through :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add method to get an agent's output JSON schema

2 participants