feat(providers/google): Add reasoning token output support #6261

Und3rf10w · 2025-05-10T03:46:17Z

Background

Vertex now supports extraction of thinking tokens in certain Gemini models.

When the configuration is passed via providerOptions, the sdk:

Did not extract reasoning tokens
Did not pass include_thoughts to the provider

Summary

Added extraction logic to google-generative-ai package to parse reasoning tokens.

Added a includeThoughts switch to the thinkingConfig for vertex models.

Verification

I verified it manually. Testable via examples/ai-core/src/stream-text/google-vertex-reasoning.ts. Easily copiable to google provider.

Tests have been added.

Tasks

Tests have been added / updated (for bug fixes / features) - c1264af
Examples have been updated
Documentation has been added / updated (for bug fixes / features)
A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
Formatting issues have been fixed (run pnpm prettier-fix in the project root)

Related Issues

Fixes #6259

chore(docs/providers): Update google provider documentation to add logic around reasoning token support

Und3rf10w · 2025-05-10T04:34:29Z

Added example in 80eb2c3 to demonstrate:

lgrammel · 2025-05-10T07:27:45Z

content/providers/01-ai-sdk-providers/15-google-generative-ai.mdx

+### Reasoning (Thinking Tokens)
+
+Certain Google Gemini models support emitting "thinking" tokens, which represent the model's reasoning process before generating the final response. The AI SDK exposes these as reasoning information.
+
+To enable thinking tokens, set `includeThoughts: true` in the `thinkingConfig` provider option:
+
+```ts
+import { google } from '@ai-sdk/google';
+import { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google';
+import { generateText, streamText } from 'ai';
+
+// For generateText:
+const { text, reasoning, reasoningDetails } = await generateText({
+  model: google('gemini-2.5-flash-preview-04-17'), // Or other supported model
+  providerOptions: {
+    google: {
+      thinkingConfig: {
+        includeThoughts: true,
+        // thinkingBudget: 2048, // Optional
+      },
+    } satisfies GoogleGenerativeAIProviderOptions,
+  },
+  prompt: 'Explain quantum computing in simple terms.',
+});
+
+console.log('Reasoning:', reasoning);
+console.log('Reasoning Details:', reasoningDetails);
+console.log('Final Text:', text);
+
+// For streamText:
+const result = streamText({
+  model: google('gemini-2.5-flash-preview-04-17'), // Or other supported model
+  providerOptions: {
+    google: {
+      thinkingConfig: {
+        includeThoughts: true,
+        // thinkingBudget: 2048, // Optional
+      },
+    } satisfies GoogleGenerativeAIProviderOptions,
+  },
+  prompt: 'Explain quantum computing in simple terms.',
+});
+
+for await (const part of result.fullStream) {
+  if (part.type === 'reasoning') {
+    process.stdout.write(`THOUGHT: ${part.textDelta}\n`);
+  } else if (part.type === 'text-delta') {
+    process.stdout.write(part.textDelta);
+  }
+}
+```
+
+When `includeThoughts` is true, parts of the API response marked with `thought: true` will be processed as reasoning.
+
+- In `generateText`, these contribute to the `reasoning` (string) and `reasoningDetails` (array) fields.
+- In `streamText`, these are emitted as `reasoning` stream parts.
+
+<Note>
+  Refer to the [Google Generative AI
+  documentation](https://ai.google.dev/gemini-api/docs/thinking) for a list of
+  models that support thinking tokens and for more details on `thinkingBudget`.
+</Note>
+


@Und3rf10w have you tested this with the gemini api? couldn't find it in their docs

@lgrammel, I have not directly, but I'm making the assumption it works for the gemini API based off of this Google Provided Notebook: https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_thinking.ipynb

I HAVE successfully tested this with the Vertex API.

@lgrammel,

I tried a variation of https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_thinking.ipynb with a valid API key, and it turns out the GEMINI api doesn't yet OFFICIALLY support (read: documented) includeThoughts.

Looking at the proto definitions for the python client, include_thoughts isn't yet supported: (documented) https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform_v1.types.GenerationConfig.ThinkingConfig

Compare this to the vertex api documentation, where includeThoughts IS supported: https://cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/GenerationConfig#ThinkingConfig

However, trying this in the notebook, we can see when we set a thinkingBudget and includeThoughts, the request IS valid, but it doesn't return thought candidates, despite using thinking tokens:

So I suppose, in the Google Generative API, it WON'T provide the thoughts, but the request will still work likely it will eventually be supported. We should probably just remove the 15-google-generative-ai.mdx file edits for now?

Here's more photos playing around with the ThinkingConfig:

TL;DR: While the parameter include_thoughts works on the Google Generative AI platform, it doesn't currently return the thought tokens from the response. It does work as expected in Vertex AI. Likely way forward is to remove the edits to 15-google-generative-ai.mdx from this PR.

Maybe also:

Update changeset to be @ai-sdk/vertex instead of @ai-sdk/google

Throw a warning when includeThoughts is specified with an @ai-sdk/google provider for now? To be removed if/when that's specified?

Updated with 6586ef2.

Now add a warning when includeThoughts is used with the google provider

added a test for above

removed includeThoughts addition from Google provider.

Added a generateText example.

… thinking config chore(providers/google): Remove `includeThoughts` from google provider docs chore(providers/google): Add unit test for reasoning warning when google provider is used with `includeThoughts`

nileshtrivedi · 2025-05-12T11:06:25Z

Thanks @Und3rf10w for implementing this. I'm surprised that Gemini APIs do not return thought tokens, because https://aistudio.google.com/ does support this in the UI as well as generated code - for all displayed programming languages:

Und3rf10w · 2025-05-12T14:52:11Z

Thanks @Und3rf10w for implementing this. I'm surprised that Gemini APIs do not return thought tokens, because https://aistudio.google.com/ does support this in the UI as well as generated code - for all displayed programming languages:

@nileshtrivedi, I agree. I am sure it's supported but undocumented as of right now in the Gemini AI studio logic, and that the Vertex API is likely also how it's going to work.

thinkingBudget already worked in both Vertex and AI Studio before this PR, but only by passing includeThoughts in the request is what will actually trigger vertex to send thinking tokens, which I couldn't get to work in the AI Studio/Gemini API.

nileshtrivedi · 2025-05-18T04:21:26Z

I am seeing thinking tokens working only with 2.5-flash, not 2.5-pro (via Vertex). I get this error message:

[Error [AI_APICallError]: Unable to submit request because thinking is not configurable in this model; please remove the thinking_config setting and try it again. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini]

Und3rf10w · 2025-05-18T05:54:09Z

I am seeing thinking tokens working only with 2.5-flash, not 2.5-pro (via Vertex). I get this error message:

[Error [AI_APICallError]: Unable to submit request because thinking is not configurable in this model; please remove the thinking_config setting and try it again. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini]

I can confirm, I am experiencing this now, but it did work.

The vertex documentation still states support for the 2.5 pro models, so maybe it's a new model version they deployed or something?

Ports the changes from this PR: vercel#6261 to the v5 branch. Adds support for Gemini's thinking messages and the includeThoughts flag in the google and vertex configs.

vlrevolution · 2025-05-29T10:57:35Z

How to disable thinking on vertex provider altogether for 'gemini-2.5-pro-preview', I'm getting this error:

[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] [GenerateText] Non-fatal error reported by AI SDK streamText.onError:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] AI_APICallError: Unable to submit request because thinking is a default and constant feature of this model; To proceed, please remove the thinking_config.thinking_budget setting from your configuration and retry. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini

Oddly for 'gemini-2.5-flash-preview' it seems to work to disable it by setting the thinkingBudget to 0 (includeThoughts is not related to fully disabling it right, it's only about if thinking IS enabled to make the model send thinking tokens if my understanding is correct?).

Did they make thinking mandatory on 'gemini-2.5-pro-preview'? Is anybody else having difficulties? Such a wierd move if it is required, given they don't let us see the thinking. This is really keeping me back from using 2.5-pro for anything on the api as the thinking is a black box and is messing up a lot of prompt outputs because it gets confused by its own thinking lol...

vlrevolution · 2025-05-29T11:01:23Z

How to disable thinking on vertex provider altogether for 'gemini-2.5-pro-preview', I'm getting this error:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] [GenerateText] Non-fatal error reported by AI SDK streamText.onError:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] AI_APICallError: Unable to submit request because thinking is a default and constant feature of this model; To proceed, please remove the thinking_config.thinking_budget setting from your configuration and retry. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
Oddly for 'gemini-2.5-flash-preview' it seems to work to disable it by setting the thinkingBudget to 0 (includeThoughts is not related to fully disabling it right, it's only about if thinking IS enabled to make the model send thinking tokens if my understanding is correct?).

Did they make thinking mandatory on 'gemini-2.5-pro-preview'? Is anybody else having difficulties? Such a wierd move if it is required, given they don't let us see the thinking. This is really keeping me back from using 2.5-pro for anything on the api as the thinking is a black box and is messing up a lot of prompt outputs because it gets confused by its own thinking lol...

I've created an issue in google's issue tracker, please comment if you are having issues too:
https://issuetracker.google.com/issues/420952680

Und3rf10w · 2025-05-29T11:58:37Z

How to disable thinking on vertex provider altogether for 'gemini-2.5-pro-preview', I'm getting this error:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] [GenerateText] Non-fatal error reported by AI SDK streamText.onError:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] AI_APICallError: Unable to submit request because thinking is a default and constant feature of this model; To proceed, please remove the thinking_config.thinking_budget setting from your configuration and retry. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
Oddly for 'gemini-2.5-flash-preview' it seems to work to disable it by setting the thinkingBudget to 0 (includeThoughts is not related to fully disabling it right, it's only about if thinking IS enabled to make the model send thinking tokens if my understanding is correct?).

Did they make thinking mandatory on 'gemini-2.5-pro-preview'? Is anybody else having difficulties? Such a wierd move if it is required, given they don't let us see the thinking. This is really keeping me back from using 2.5-pro for anything on the api as the thinking is a black box and is messing up a lot of prompt outputs because it gets confused by its own thinking lol...

Did you try includeThoughts: false though? Or not including a thinkingConfig all for that model?

At one point of the 2.5 pro exp model, you were able to configure the thinking, but now that's only limited to 2.5 flash

vlrevolution · 2025-05-29T12:32:35Z

How to disable thinking on vertex provider altogether for 'gemini-2.5-pro-preview', I'm getting this error:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] [GenerateText] Non-fatal error reported by AI SDK streamText.onError:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] AI_APICallError: Unable to submit request because thinking is a default and constant feature of this model; To proceed, please remove the thinking_config.thinking_budget setting from your configuration and retry. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
Oddly for 'gemini-2.5-flash-preview' it seems to work to disable it by setting the thinkingBudget to 0 (includeThoughts is not related to fully disabling it right, it's only about if thinking IS enabled to make the model send thinking tokens if my understanding is correct?).
Did they make thinking mandatory on 'gemini-2.5-pro-preview'? Is anybody else having difficulties? Such a wierd move if it is required, given they don't let us see the thinking. This is really keeping me back from using 2.5-pro for anything on the api as the thinking is a black box and is messing up a lot of prompt outputs because it gets confused by its own thinking lol...
Did you try includeThoughts: false though? Or noti ncluding a thinkingConfig all to get for that model?

At one point of the 2.5 pro exp model, you were able to configure the thinking, but now that's only limited to 2.5 flash

Both or either result in the same error that thinking config is not allowed :(

Adebesin-Cell · 2025-06-02T14:29:52Z

How to disable thinking on vertex provider altogether for 'gemini-2.5-pro-preview', I'm getting this error:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] [GenerateText] Non-fatal error reported by AI SDK streamText.onError:
[Nest] 298  - 05/29/2025, 10:53:54 AM   ERROR [GenerateTextService] AI_APICallError: Unable to submit request because thinking is a default and constant feature of this model; To proceed, please remove the thinking_config.thinking_budget setting from your configuration and retry. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
Oddly for 'gemini-2.5-flash-preview' it seems to work to disable it by setting the thinkingBudget to 0 (includeThoughts is not related to fully disabling it right, it's only about if thinking IS enabled to make the model send thinking tokens if my understanding is correct?).
Did they make thinking mandatory on 'gemini-2.5-pro-preview'? Is anybody else having difficulties? Such a wierd move if it is required, given they don't let us see the thinking. This is really keeping me back from using 2.5-pro for anything on the api as the thinking is a black box and is messing up a lot of prompt outputs because it gets confused by its own thinking lol...
I've created an issue in google's issue tracker, please comment if you are having issues too: issuetracker.google.com/issues/420952680

I'm using gemini-2.5-flash-preview-05-20, but every time the response includes reasoning or thoughts, even though I've clearly asked in the prompt not to include them. I also set thinkingBudget: 0 and includeThoughts: false, but it didn't help.

feat(providers/google): Add reasoning token output support

d44aa57

chore(docs/providers): Update google provider documentation to add logic around reasoning token support

Und3rf10w mentioned this pull request May 10, 2025

Google-Vertex: Support include_thinking in reasoning configuration and extraction of model thoughts. #6259

Closed

Und3rf10w added 2 commits May 10, 2025 00:30

feat(examples/google): Add reasoning example for vertex

7e595e7

fix(examples/google): Prettier-fix on vertex reasoning example

80eb2c3

Und3rf10w marked this pull request as ready for review May 10, 2025 04:34

chore(providers/vertex): Add reasoning tests

c1264af

lgrammel reviewed May 10, 2025

View reviewed changes

fix(providers/google): Add warning when includeThoughts is added to…

6586ef2

… thinking config chore(providers/google): Remove `includeThoughts` from google provider docs chore(providers/google): Add unit test for reasoning warning when google provider is used with `includeThoughts`

Und3rf10w requested a review from lgrammel May 10, 2025 20:04

lgrammel approved these changes May 11, 2025

View reviewed changes

lgrammel merged commit fe24216 into vercel:main May 11, 2025
7 of 8 checks passed

lgrammel mentioned this pull request May 13, 2025

@ai-sdk/google-vertex support thinking in gemini-2.5 #6009

Closed

Und3rf10w mentioned this pull request May 16, 2025

Add codeExecution support to google providers and more safely handle provider and user tools #6354

Closed

4 tasks

ap-inflection mentioned this pull request May 22, 2025

[v5]: Adds Google/VertexAI reasoning messages #6428

Open

4 tasks

Und3rf10w mentioned this pull request May 22, 2025

feat(providers/google): Add codeExecution and reasoning support in v5 #6365

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(providers/google): Add reasoning token output support #6261

feat(providers/google): Add reasoning token output support #6261

Uh oh!

Und3rf10w commented May 10, 2025 •

edited

Loading

Uh oh!

Und3rf10w commented May 10, 2025

Uh oh!

lgrammel May 10, 2025

Uh oh!

Und3rf10w May 10, 2025

Uh oh!

Und3rf10w May 10, 2025 •

edited

Loading

Uh oh!

Und3rf10w May 10, 2025

Uh oh!

Uh oh!

nileshtrivedi commented May 12, 2025

Uh oh!

Und3rf10w commented May 12, 2025

Uh oh!

nileshtrivedi commented May 18, 2025

Uh oh!

Und3rf10w commented May 18, 2025 •

edited

Loading

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

Und3rf10w commented May 29, 2025 •

edited

Loading

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

Adebesin-Cell commented Jun 2, 2025

Uh oh!

Uh oh!

feat(providers/google): Add reasoning token output support #6261

feat(providers/google): Add reasoning token output support #6261

Uh oh!

Conversation

Und3rf10w commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Summary

Verification

Tasks

Related Issues

Uh oh!

Und3rf10w commented May 10, 2025

Uh oh!

lgrammel May 10, 2025

Choose a reason for hiding this comment

Uh oh!

Und3rf10w May 10, 2025

Choose a reason for hiding this comment

Uh oh!

Und3rf10w May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Und3rf10w May 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nileshtrivedi commented May 12, 2025

Uh oh!

Und3rf10w commented May 12, 2025

Uh oh!

nileshtrivedi commented May 18, 2025

Uh oh!

Und3rf10w commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

Und3rf10w commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vlrevolution commented May 29, 2025

Uh oh!

Adebesin-Cell commented Jun 2, 2025

Uh oh!

Uh oh!

Und3rf10w commented May 10, 2025 •

edited

Loading

Und3rf10w May 10, 2025 •

edited

Loading

Und3rf10w commented May 18, 2025 •

edited

Loading

Und3rf10w commented May 29, 2025 •

edited

Loading