You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As more AI models support structured output natively, you can take advantage of this feature by using the `AdvisorParams.WITH_NATIVE_STRUCTURED_OUTPUT` advisor parameter when calling the `ChatClient`.
289
+
You can use the `defaultAdvisors()` method on the `ChatClient.Builder` to set this parameter globally for all calls or set it per call as shown below:
Copy file name to clipboardExpand all lines: spring-ai-docs/src/main/antora/modules/ROOT/pages/api/structured-output-converter.adoc
+45-16Lines changed: 45 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,9 +2,6 @@
2
2
3
3
= Structured Output Converter
4
4
5
-
NOTE: As of 02.05.2024 the old `OutputParser`, `BeanOutputParser`, `ListOutputParser` and `MapOutputParser` classes are deprecated in favor of the new `StructuredOutputConverter`, `BeanOutputConverter`, `ListOutputConverter` and `MapOutputConverter` implementations.
6
-
the latter are drop-in replacements for the former ones and provide the same functionality. The reason for the change was primarily naming, as there isn't any parsing being done, but also have aligned with the Spring `org.springframework.core.convert.converter` package bringing in some improved functionality.
7
-
8
5
The ability of LLMs to produce structured outputs is important for downstream applications that rely on reliably parsing output values.
9
6
Developers want to quickly turn results from an AI model into data types, such as JSON, XML or Java classes, that can be passed to other application functions and methods.
10
7
@@ -17,6 +14,8 @@ Generating structured outputs from Large Language Models (LLMs) using generic co
17
14
18
15
Before the LLM call, the converter appends format instructions to the prompt, providing explicit guidance to the models on generating the desired output structure. These instructions act as a blueprint, shaping the model's response to conform to the specified format.
19
16
17
+
NOTE: As more AI models natively support structured outputs, you can leverage this capability using the xref:api/chatclient.adoc#_native_structured_output[Native Structured Output] feature with `AdvisorParams.WITH_NATIVE_STRUCTURED_OUTPUT`. This approach uses the generated JSON schema directly with the model's native structured output API, eliminating the need for pre-prompt formatting instructions and providing more reliable results.
18
+
20
19
After the LLM call, the converter takes the model's output text and transforms it into instances of the structured type. This conversion process involves parsing the raw text output and mapping it to the corresponding structured data representation, such as JSON, XML, or domain-specific data structures.
21
20
22
21
TIP: The `StructuredOutputConverter` is a best effort to convert the model output into a structured output.
List<String> list = this.listOutputConverter.convert(this.generation.getOutput().getText());
254
253
----
255
254
256
-
== Supported AI Models
255
+
== Native Structured Output
256
+
257
+
Many modern AI models now provide native support for structured output, which offers more reliable results compared to prompt-based formatting. Spring AI supports this through the xref:api/chatclient.adoc#_native_structured_output[Native Structured Output] feature.
258
+
259
+
When using native structured output, the JSON schema generated by `BeanOutputConverter` is sent directly to the model's structured output API, eliminating the need for format instructions in the prompt. This approach provides:
260
+
261
+
* **Higher reliability**: The model guarantees output conforming to the schema
262
+
* **Cleaner prompts**: No need to append format instructions
263
+
* **Better performance**: Models can optimize for structured output internally
264
+
265
+
=== Using Native Structured Output
266
+
267
+
To enable native structured output, use the `AdvisorParams.WITH_NATIVE_STRUCTURED_OUTPUT` parameter:
| xref:api/chat/vertexai-gemini-chat.adoc[Vertex AI Gemini] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/test/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModelIT.java[VertexAiGeminiChatModelIT.java]
269
-
|====
298
+
NOTE: Some AI models, such as OpenAI, don't support arrays of objects natively at the top level. In such cases, you can use the Spring AI default structured output conversion (without the native structured output advisor).
270
299
271
-
== Built-in JSON mode
300
+
=== Built-in JSON mode
272
301
273
302
Some AI Models provide dedicated configuration options to generate structured (usually JSON) output.
0 commit comments