Skip to content

[JS] Simulated structured output instructions are injected into the system message when a system message is present, but user message when one is not #1923

@chrisraygill

Description

@chrisraygill

Genkit version: 1.0.4

Describe the bug
Simulated structured output instructions are injected into the system message when a system message is present, but user message when one is not.

For example...

In the below code, the output instructions will be appended to the system message.

const response = await ai.generate({
  model: gemini20Flash,
  tools: [exampleTool],
  output: { schema: outputSchema },
  system: `A system message`,
  prompt: `Use exampleTool to respond to the user`,
});

In the below code, the output instructions will be appended to the user message.

const response = await ai.generate({
  model: gemini20Flash,
  tools: [exampleTool],
  output: { schema: outputSchema },
  prompt: `Use exampleTool to respond to the user`,
});

This is particularly jarring when using ai.chat and ai.send because if you do ai.send({prompt, tools, output}), then the output instructions will be added to the system message, not the prompt being sent.

Expected behavior
By default, the output instructions should be added to the user message unless specifically configured to inject into the system message.

Repro
Full code sample to try below. Run the reproFlow in the DevUI and view the traces. See that in promptWithSystem, the output instructions are in the system message - while they are in the user message for promptWithoutSystem.

import { z, genkit } from "genkit";
import { gemini, googleAI } from "@genkit-ai/googleai"

const ai = genkit({
    plugins: [googleAI()],
    model: gemini("gemini-2.0-flash")
});

const inputSchema = z.object({
    username: z.string().default("testUsername"),
});

const outputSchema = z.object({
    username: z.string(),
    email: z.string(),
});

const getEmailTool = ai.defineTool({
    name: "getEmailTool",
    description: "Tool that returns the corresponding email address for a given username as a string.",
    inputSchema: z.object({ username: z.string().describe("The username for which to return the corresponding email address.") }),
}, async ({username}) => {
    return "[email protected]";
});

const promptWithSystem = ai.definePrompt({
    name: "promptWithSystem",
    input: { schema: inputSchema },
    output: { schema: outputSchema },
    tools: [getEmailTool],
    system: "Your job is to return the email address for a given username.",
    prompt: "My username is {{username}}. Get my email address using the getEmailTool."
});

const promptWithoutSystem = ai.definePrompt({
    name: "promptWithoutSystem",
    input: { schema: inputSchema },
    output: { schema: outputSchema },
    tools: [getEmailTool],
    prompt: "My username is {{username}}. Get my email address using the getEmailTool."
});

const reproFlow = ai.defineFlow("reproFlow",
async (input) => {
    promptWithSystem({ username: "testUsername" });
    promptWithoutSystem({ username: "testUsername" });
});

reproFlow();

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingjs

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions