feat(endpoints): Add OpenAI Responses API endpoint with fixes and integration tests#43
Open
acere wants to merge 2 commits intoawslabs:mainfrom
Open
feat(endpoints): Add OpenAI Responses API endpoint with fixes and integration tests#43acere wants to merge 2 commits intoawslabs:mainfrom
acere wants to merge 2 commits intoawslabs:mainfrom
Conversation
… test suite - Add ResponseEndpoint and ResponseStreamEndpoint classes for OpenAI Responses API support - Implement non-streaming and streaming response handling with proper error management - Add structured output support with response format validation and serialization - Create comprehensive unit test suite covering response parsing, error handling, format validation, model parameters, payload parsing, properties, and serialization - Add integration tests for Bedrock response endpoint functionality - Export new response endpoint classes from endpoints module - Update integration test configuration with response endpoint fixtures
- Rename max_tokens to max_output_tokens in create_payload (Response API parameter name) - Fix _parse_response to handle usage=None (Bedrock Mantle) and use input_tokens/output_tokens with fallback to prompt_tokens/completion_tokens - Rewrite _parse_stream_response to process typed events (response.output_text.delta, response.completed) instead of the old chunk-with-output-array format - Fix test_response_bedrock.py to use ResponseUsage attribute names (input_tokens/output_tokens) - Add integration tests for ResponseEndpoint and ResponseStreamEndpoint - Add example notebook for Response API on Bedrock - Update all unit test mocks to match new behavior
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the OpenAI Responses API endpoint support for LLMeter, with fixes to align with the actual API behavior.
Changes
Endpoint fixes (
llmeter/endpoints/openai_response.py)max_tokenstomax_output_tokensincreate_payload(Response API parameter name)_parse_responseto handleusage=None(Bedrock Mantle doesn't always return it) and useinput_tokens/output_tokenswith fallback toprompt_tokens/completion_tokens_parse_stream_responseto process typed events (response.output_text.delta,response.completed) instead of the old chunk-with-output-array formatIntegration tests
tests/integ/test_response_endpoint.py— integration tests forResponseEndpointandResponseStreamEndpointwrappers against Bedrock Mantletests/integ/test_response_bedrock.pyto useResponseUsageattribute names (input_tokens/output_tokens)Unit test updates
spec-based usage mocks (input_tokens/output_tokens) and event-based streaming mocksExample notebook
examples/LLMeter with OpenAI Response API on Bedrock.ipynbdemonstrating non-streaming and streaming usage with Runner and plottingTesting