Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/harness-p13-agent-middleware.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@tanstack/ai': minor
'@tanstack/ai-harness': minor
---

`@tanstack/ai`: the chat middleware context has `subagentName`, the agent name of a child that runs through `ctx.chat`, and `parentSubagentRunId`, the id of the child that started a nested child. `chat()` takes both as options. A child that calls `ctx.chat({ subagents })` now gives its own children the host middleware (`subagents.binding.chatMiddleware` and `generationMiddleware`) before their own, also when the binding has no budget.

`@tanstack/ai-harness`: plugins can contribute `agentMiddleware`, chat middleware for every agent run: subagents the lead model calls (coding agents too), background agents, and their children. It does not run in the lead turn. Put the same middleware in `middleware` and `agentMiddleware` to see every model call. Run plugins' `generationMiddleware` now also reaches the agents of their turn. The first-party `usage()` plugin now counts the model calls of every agent, not only the lead turn.
2 changes: 2 additions & 0 deletions docs/chat/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,8 @@ A child is its own `chat()` call. Put the child's middleware in that call. The p

Inside a child, the middleware context has `subagentRunId`. Use it to tell a child run from a top-level run, and to link a child trace to its card.

When the child calls `ctx.chat`, the context also has `subagentName`, the name of the agent. For a nested child, `parentSubagentRunId` is the id of the child that started it.

On a routed turn where a child runs and main does not, the parent's middleware does not run. Only `withPersistence` records that turn. A turn where main runs, including a handoff, runs the parent's middleware as usual.

## Persistence
Expand Down
5 changes: 3 additions & 2 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@
"label": "Subagents",
"to": "chat/subagents",
"addedAt": "2026-09-21",
"updatedAt": "2026-09-26"
"updatedAt": "2026-09-28"
},
{
"label": "Agentic Cycle",
Expand Down Expand Up @@ -933,7 +933,8 @@
{
"label": "Run agents from a harness",
"to": "harness/subagents",
"addedAt": "2026-09-26"
"addedAt": "2026-09-26",
"updatedAt": "2026-09-28"
},
{
"label": "Deploy a harness",
Expand Down
2 changes: 1 addition & 1 deletion docs/harness/coding-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Run the file with `npx tsx coder.ts`. Ask for a change. The agent reads files fr
| `projectInstructions({ root })` | Adds `AGENTS.md` and `CLAUDE.md` to the system prompt. |
| `fileCommands({ dir })` | Each `.md` file becomes a slash command. `$ARGUMENTS` is replaced by what you type after it. |
| `compact({ adapter })` | `/compact` replaces a long conversation with a summary. |
| `usage()` | `/usage` shows the tokens of the session. |
| `usage()` | `/usage` shows the tokens of the session: the lead turn and every agent. |
| `goal({ judge })` | `/goal <text>` keeps the agent working until a judge model says that the goal is met. See [Work until a goal is met](./goal). |

## Add your own rules
Expand Down
39 changes: 38 additions & 1 deletion docs/harness/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Add it with `plugins: () => [today]` in `defineHarness`. `setup` runs once per s

- `tools`: tools for the model, made with `toolDefinition`.
- `prompts`: text for the system prompt. A function runs for each turn, so it can show current state.
- `middleware`: chat middleware, the same type as `chat({ middleware })`.
- `middleware`: chat middleware for the lead turn, the same type as `chat({ middleware })`. `agentMiddleware` is for agent runs, see [Middleware in every agent](#middleware-in-every-agent).
- `generationMiddleware`: middleware for the activities agents call.
- `agents`: agents added to `session.agents`.
- `subagents`: agents the model can call as tools. They are also added to `session.agents`. [Delegate to coding agents](./coding-agents) uses them.
Expand Down Expand Up @@ -99,6 +99,42 @@ export const counter = definePlugin({

When two writers race, `update` runs your function again with fresh state.

## Middleware in every agent

You want to track token cost, or apply a policy, for every model call. But `middleware` runs only in the lead turn. The agents that the lead model calls, the agents you start in the background, and their own children each run a separate chat. Put the same middleware in `agentMiddleware` too. Then it runs in each of those chats.

```ts group=harness-plugins
import type { ChatMiddleware } from '@tanstack/ai'

export const usageByAgent = definePlugin({
name: 'acme/usage-by-agent',
setup: () => {
const tokens = new Map<string, number>()
const tracker: ChatMiddleware = {
name: 'acme/usage-by-agent',
onUsage: (ctx, usage) => {
const agent = ctx.subagentName ?? 'lead'
tokens.set(agent, (tokens.get(agent) ?? 0) + usage.totalTokens)
},
}
return {
middleware: [tracker],
agentMiddleware: [tracker],
commands: {
tokens: defineCommand({
description: 'Show the tokens of each agent',
run: () => Object.fromEntries(tokens),
}),
},
}
},
})
```

- `ctx.subagentName` is the name of the agent. In the lead turn, it is `undefined`.
- `ctx.subagentRunId` is different for each run of an agent. For a nested child, `ctx.parentSubagentRunId` is the id of the child that started it.
- Each model call goes to `onUsage` one time, in the run that made the call. The lead turn does not count the usage of a child again.

## Let plugins work together

Three ways, from simple to loose:
Expand Down Expand Up @@ -152,6 +188,7 @@ Set `lifetime: 'run'` to set a plugin up again for each turn.
## What you have now

- A plugin that adds tools, prompts, commands, settings, and state to any harness.
- Middleware that sees every model call, in the lead turn and in every agent.
- Plugins that share services, lists, and events without knowing each other.

Next: see the [first-party plugins](./coding-agent) that turn a harness into a coding agent.
2 changes: 2 additions & 0 deletions docs/harness/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ export const review = definePlugin({
- `ctx.agents.start(agent, input, { wake: true })` runs it in the background and starts a turn when it is done.
- `ctx.agents.group(options, body)` runs several. With `onFailure: 'cancel-siblings'`, one failure cancels the others. With `'collect'`, use `group.runSettled` to get every result or error. Every child settles before `group` returns.

To track usage or apply a policy in each of these runs, see [Middleware in every agent](./plugins#middleware-in-every-agent).

## Call a harness as a child

`harnessAgent` turns a harness into an agent. Put it in `subagents.agents`, and the main model calls it as a tool. The child harness keeps its own tools, plugins, and history.
Expand Down
40 changes: 24 additions & 16 deletions packages/ai-harness/src/first-party/session-tools.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import { chat } from '@tanstack/ai'
import { defineCommand } from '../commands'
import { definePlugin } from '../plugins'
import type { AnyTextAdapter, ModelMessage } from '@tanstack/ai'
import type {
AnyChatMiddleware,
AnyTextAdapter,
ModelMessage,
} from '@tanstack/ai'

const SUMMARY_PROMPT =
'Summarize the conversation so far for yourself. Keep decisions, open tasks, file names, and facts you still need. Leave out small talk.'
Expand Down Expand Up @@ -78,7 +82,10 @@ interface UsageTotals {
totalTokens: number
}

/** Count tokens across the session. `/usage` shows the totals. */
/**
* Count tokens across the session: the lead turn and every agent run
* (subagents, background agents, and their children). `/usage` shows the totals.
*/
export function usage() {
return definePlugin({
name: 'tanstack/usage',
Expand All @@ -91,21 +98,22 @@ export function usage() {
})
const show = (totals: UsageTotals) =>
`${totals.turns} model calls, ${totals.promptTokens} input tokens, ${totals.completionTokens} output tokens, ${totals.totalTokens} total.`
const count = {
name: 'tanstack/usage',
onUsage: async (_run, info) => {
await state.update((totals) => ({
turns: totals.turns + 1,
promptTokens: totals.promptTokens + (info.promptTokens ?? 0),
completionTokens:
totals.completionTokens + (info.completionTokens ?? 0),
totalTokens: totals.totalTokens + (info.totalTokens ?? 0),
}))
},
} satisfies AnyChatMiddleware
return {
middleware: [
{
name: 'tanstack/usage',
onUsage: async (_run, info) => {
await state.update((totals) => ({
turns: totals.turns + 1,
promptTokens: totals.promptTokens + (info.promptTokens ?? 0),
completionTokens:
totals.completionTokens + (info.completionTokens ?? 0),
totalTokens: totals.totalTokens + (info.totalTokens ?? 0),
}))
},
},
],
// The same counter in the lead turn and in every agent run.
middleware: [count],
agentMiddleware: [count],
commands: {
usage: defineCommand({
description: 'Show token usage for this session',
Expand Down
11 changes: 11 additions & 0 deletions packages/ai-harness/src/plugins.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ export interface PluginContributions {
prompts?: ReadonlyArray<string | (() => string) | PluginPrompt>
/** Chat middleware, the same type as `chat({ middleware })`. */
middleware?: ReadonlyArray<AnyChatMiddleware>
/**
* Chat middleware for every agent run: subagents, background agents, and
* their children. Not the lead turn: add the same middleware to
* `middleware` for that.
*/
agentMiddleware?: ReadonlyArray<AnyChatMiddleware>
/** Middleware for the activities agents call (`ctx.generateImage`, ...). */
generationMiddleware?: ReadonlyArray<AnyGenerationMiddleware>
/** Agents added to `session.agents`. */
Expand Down Expand Up @@ -281,6 +287,8 @@ export interface MountedPlugins {
prompts: Array<{ id: string; owner: string }>
}
middleware: Array<AnyChatMiddleware>
/** Chat middleware for every agent run. See `PluginContributions`. */
agentMiddleware: Array<AnyChatMiddleware>
generationMiddleware: Array<AnyGenerationMiddleware>
/**
* Chat middleware that provides every plugin capability to each chat run,
Expand Down Expand Up @@ -443,6 +451,7 @@ export async function mountPlugins(
}))
const prompts: Array<Owned<PluginPrompt>> = []
const middleware: Array<AnyChatMiddleware> = []
const agentMiddleware: Array<AnyChatMiddleware> = []
const generationMiddleware: Array<AnyGenerationMiddleware> = []
const commands = new Map<string, { command: AnyCommand; owner: string }>()
const config = new Map<string, { option: ConfigOption; owner: string }>()
Expand Down Expand Up @@ -554,6 +563,7 @@ export async function mountPlugins(
prompts.push({ value: section, owner: plugin.name })
})
middleware.push(...(contributions.middleware ?? []))
agentMiddleware.push(...(contributions.agentMiddleware ?? []))
generationMiddleware.push(...(contributions.generationMiddleware ?? []))
for (const [name, command] of Object.entries(
contributions.commands ?? {},
Expand Down Expand Up @@ -647,6 +657,7 @@ export async function mountPlugins(
.map((entry) => entry.value),
prompts: prompts.map((entry) => entry.value.text),
middleware,
agentMiddleware,
generationMiddleware,
commands,
config,
Expand Down
16 changes: 12 additions & 4 deletions packages/ai-harness/src/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1066,10 +1066,18 @@ export class HarnessSession<THarness extends AnyHarness = AnyHarness> {
return root.child()
}

private binding(): SubagentBinding {
/** What plugins add to every agent run: session plugins, then run plugins. */
private binding(runPlugins?: MountedPlugins) {
return {
generationMiddleware: this.sessionPlugins?.generationMiddleware ?? [],
}
generationMiddleware: [
...(this.sessionPlugins?.generationMiddleware ?? []),
...(runPlugins?.generationMiddleware ?? []),
],
chatMiddleware: [
...(this.sessionPlugins?.agentMiddleware ?? []),
...(runPlugins?.agentMiddleware ?? []),
],
} satisfies SubagentBinding
}

private async runTurn(turn: QueuedTurn): Promise<void> {
Expand Down Expand Up @@ -1171,7 +1179,7 @@ export class HarnessSession<THarness extends AnyHarness = AnyHarness> {
...this.harness.subagents,
agents: subagentList,
limits: this.limits(),
binding: this.binding(),
binding: this.binding(runPlugins),
},
}
: {}),
Expand Down
Loading
Loading