Skip to content

test: add Anthropic Messages recording mode + gateway-tool cassettes#123

Open
ashwing wants to merge 1 commit into
vllm-project:mainfrom
ashwing:feat/messages-cassette-recorder
Open

test: add Anthropic Messages recording mode + gateway-tool cassettes#123
ashwing wants to merge 1 commit into
vllm-project:mainfrom
ashwing:feat/messages-cassette-recorder

Conversation

@ashwing

@ashwing ashwing commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Part of #115 (Claude Code — Stage 2). Adds a messages mode to the cassette recorder for the Anthropic Messages API (/v1/messages), and records the upstream gateway-tool traffic that the Claude Code server-side tool loop will replay in tests.

The Stage 2 server-side loop itself is gated on making the executor loop API-agnostic (it's currently typed to Responses RequestPayload/ResponsePayload) — that's a core refactor for a separate change. Recording the acceptance cassette has no such dependency and is the unblocked prerequisite #115 calls out, so this lands the recorder + fixtures ahead of the loop.

What's here:

  • messages recorder mode (record_cassette.py): sends Anthropic /v1/messages with tools, stateless (resends full messages history each turn). With --tool-outputs, a turn following a tool_use feeds back matching tool_result blocks keyed by tool name — mirroring how a gateway tool loop feeds results back. Streaming reconstructs the final message from the Anthropic SSE (content_block_delta / input_json_delta / message_delta).
  • Recorded cassettes (messages/): a two-turn web_search gateway-tool session — model emits the gateway tool_use, tool_result is fed back, model produces the final answer. Both streaming and non-streaming.
  • Reproducer record_messages_cassettes.sh + a README modes-table row.

Test Plan

  • Recorded against vLLM 0.25.1 (the current release, which serves /v1/messages natively), Qwen/Qwen3-30B-A3B-FP8, hermes tool parser + qwen3 reasoning parser.
  • python -m py_compile record_cassette.py clean; --help shows the new mode; cargo build --workspace unaffected (no Rust changed).
  • Recorded cassettes verified: 2 turns each, POST /v1/messages, turn 1 emits tool_use(web_search) with stop_reason: tool_use, turn 2 request carries the full Anthropic history (userassistant with tool_useuser with tool_result), turn 2 produces the final end_turn answer.
  • No Rust consumer yet: the Messages cassette loader lands with the Stage 2 loop these fixtures are for. Recording them now (per [Claude Code - Stage 2] Server-side gateway tool loop for Messages #115's carved-out sub-task) de-risks that work and keeps the fixtures reproducible.
  • Note: streaming reasoning arrives as <think>-tagged text deltas while non-streaming arrives as thinking blocks — this is an accurate capture of what vLLM emits on each path, recorded intentionally so the future parser handles both.

Adds a `messages` mode to the cassette recorder for the Anthropic Messages
API (/v1/messages), and records the upstream gateway-tool traffic that the
Claude Code server-side tool loop (vllm-project#115) will replay in tests.

The mode is stateless (resends full `messages` history each turn); with
`--tool-outputs`, a turn following a `tool_use` feeds back matching
`tool_result` blocks keyed by tool name, mirroring how a gateway tool loop
feeds results back. Streaming reconstructs the final message from the
Anthropic SSE (content_block_delta / input_json_delta / message_delta).

Recorded against vLLM 0.25.1 (Qwen3-30B-A3B-FP8, hermes parser) serving
/v1/messages natively — a two-turn web_search turn: model emits the gateway
tool_use, tool_result is fed back, model answers. Both streaming and
non-streaming. Reproducer: record_messages_cassettes.sh.

These are fixtures for the (maintainer-gated) vllm-project#115 loop; there is no Rust
consumer yet, since the Messages cassette loader lands with that loop.

Part of vllm-project#115.

Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant