test: add Anthropic Messages recording mode + gateway-tool cassettes#123
Open
ashwing wants to merge 1 commit into
Open
test: add Anthropic Messages recording mode + gateway-tool cassettes#123ashwing wants to merge 1 commit into
ashwing wants to merge 1 commit into
Conversation
Adds a `messages` mode to the cassette recorder for the Anthropic Messages API (/v1/messages), and records the upstream gateway-tool traffic that the Claude Code server-side tool loop (vllm-project#115) will replay in tests. The mode is stateless (resends full `messages` history each turn); with `--tool-outputs`, a turn following a `tool_use` feeds back matching `tool_result` blocks keyed by tool name, mirroring how a gateway tool loop feeds results back. Streaming reconstructs the final message from the Anthropic SSE (content_block_delta / input_json_delta / message_delta). Recorded against vLLM 0.25.1 (Qwen3-30B-A3B-FP8, hermes parser) serving /v1/messages natively — a two-turn web_search turn: model emits the gateway tool_use, tool_result is fed back, model answers. Both streaming and non-streaming. Reproducer: record_messages_cassettes.sh. These are fixtures for the (maintainer-gated) vllm-project#115 loop; there is no Rust consumer yet, since the Messages cassette loader lands with that loop. Part of vllm-project#115. Signed-off-by: Ashwin Giridharan <girida@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #115 (Claude Code — Stage 2). Adds a
messagesmode to the cassette recorder for the Anthropic Messages API (/v1/messages), and records the upstream gateway-tool traffic that the Claude Code server-side tool loop will replay in tests.The Stage 2 server-side loop itself is gated on making the executor loop API-agnostic (it's currently typed to Responses
RequestPayload/ResponsePayload) — that's a core refactor for a separate change. Recording the acceptance cassette has no such dependency and is the unblocked prerequisite #115 calls out, so this lands the recorder + fixtures ahead of the loop.What's here:
messagesrecorder mode (record_cassette.py): sends Anthropic/v1/messageswith tools, stateless (resends fullmessageshistory each turn). With--tool-outputs, a turn following atool_usefeeds back matchingtool_resultblocks keyed by tool name — mirroring how a gateway tool loop feeds results back. Streaming reconstructs the final message from the Anthropic SSE (content_block_delta/input_json_delta/message_delta).messages/): a two-turnweb_searchgateway-tool session — model emits the gatewaytool_use,tool_resultis fed back, model produces the final answer. Both streaming and non-streaming.record_messages_cassettes.sh+ a README modes-table row.Test Plan
/v1/messagesnatively),Qwen/Qwen3-30B-A3B-FP8, hermes tool parser + qwen3 reasoning parser.python -m py_compile record_cassette.pyclean;--helpshows the new mode;cargo build --workspaceunaffected (no Rust changed).POST /v1/messages, turn 1 emitstool_use(web_search)withstop_reason: tool_use, turn 2 request carries the full Anthropic history (user→assistantwithtool_use→userwithtool_result), turn 2 produces the finalend_turnanswer.<think>-taggedtextdeltas while non-streaming arrives asthinkingblocks — this is an accurate capture of what vLLM emits on each path, recorded intentionally so the future parser handles both.