Skip to content

[v5]: Adds Google/VertexAI reasoning messages #6428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: v5
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/khaki-poets-remember.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@ai-sdk/google-vertex': patch
'@ai-sdk/google': patch
---

Adds support for Gemini/Vertex thinking messages
66 changes: 66 additions & 0 deletions content/providers/01-ai-sdk-providers/16-google-vertex.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,72 @@ const { text } = await generateText({
Google Vertex language models can also be used in the `streamText` function
(see [AI SDK Core](/docs/ai-sdk-core)).

#### Reasoning (Thinking Tokens)

Google Vertex AI, through its support for Gemini models, can also emit "thinking" tokens, representing the model's reasoning process. The AI SDK exposes these as reasoning information.

To enable thinking tokens for compatible Gemini models via Vertex, set `includeThoughts: true` in the `thinkingConfig` provider option. Since the Vertex provider uses the Google provider's underlying language model, these options are passed through `providerOptions.google`:

```ts
import { vertex } from '@ai-sdk/google-vertex';
import { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google'; // Note: importing from @ai-sdk/google
import { generateText, streamText } from 'ai';

// For generateText:
const { content } = await generateText({
model: vertex('gemini-2.5-flash-preview-04-17'), // Or other supported model via Vertex
providerOptions: {
google: {
// Options are nested under 'google' for Vertex provider
thinkingConfig: {
includeThoughts: true,
// thinkingBudget: 2048, // Optional
},
} satisfies GoogleGenerativeAIProviderOptions,
},
prompt: 'Explain quantum computing in simple terms.',
});

for (const part of content) {
if (part.type === 'text') {
process.stdout.write(part.text);
} else if (part.type === 'reasoning') {
process.stdout.write(`THOUGHT: ${part.text}\n`);
}
}

// For streamText:
const result = streamText({
model: vertex('gemini-2.5-flash-preview-04-17'), // Or other supported model via Vertex
providerOptions: {
google: {
// Options are nested under 'google' for Vertex provider
thinkingConfig: {
includeThoughts: true,
// thinkingBudget: 2048, // Optional
},
} satisfies GoogleGenerativeAIProviderOptions,
},
prompt: 'Explain quantum computing in simple terms.',
});

for await (const part of result.fullStream) {
if (part.type === 'reasoning') {
process.stdout.write(`THOUGHT: ${part.text}\n`);
} else if (part.type === 'text') {
process.stdout.write(part.text);
}
}
```

When `includeThoughts` is true, parts of the API response marked with `thought: true` will be processed as reasoning.

<Note>
Refer to the [Google Vertex AI documentation on
"thinking"](https://cloud.google.com/vertex-ai/generative-ai/docs/thinking)
for model compatibility and further details.
</Note>

#### File Inputs

The Google Vertex provider supports file inputs, e.g. PDF files.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

async function main() {
const { content } = await generateText({
model: vertex('gemini-2.5-flash-preview-04-17'),
prompt:
"Describe the most unusual or striking architectural feature you've ever seen in a building or structure.",
providerOptions: {
google: {
thinkingConfig: {
thinkingBudget: 2048,
includeThoughts: true,
},
},
},
});

for (const part of content) {
if (part.type === 'text') {
process.stdout.write(part.text);
} else if (part.type === 'reasoning') {
process.stdout.write('\x1b[34m' + part.text + '\x1b[0m');
}
}
}

main().catch(console.log);
35 changes: 35 additions & 0 deletions examples/ai-core/src/generate-text/google-vertex-reasoning.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import { vertex } from '@ai-sdk/google-vertex';
import { streamText } from 'ai';

async function main() {
const result = streamText({
model: vertex('gemini-2.5-flash-preview-04-17'),
prompt:
"Describe the most unusual or striking architectural feature you've ever seen in a building or structure.",
providerOptions: {
google: {
thinkingConfig: {
thinkingBudget: 2048,
includeThoughts: true,
},
},
},
});

for await (const part of result.fullStream) {
if (part.type === 'reasoning') {
process.stdout.write('\x1b[34m' + part.text + '\x1b[0m');
} else if (part.type === 'text') {
process.stdout.write(part.text);
}
}

console.log();
console.log('Warnings:', await result.warnings);

console.log();
console.log('Token usage:', await result.usage);
console.log('Finish reason:', await result.finishReason);
}

main().catch(console.log);
Loading