Skip to content

Add OpenAI Responses API support to parallel processor #1972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

milistu
Copy link

@milistu milistu commented Jul 24, 2025

Summary

This PR adds support for the new OpenAI Responses API to the api_request_parallel_processor.py script. The enhancement enables efficient batch processing of Responses API requests while maintaining full backwards compatibility with existing OpenAI API endpoints (chat completions and embeddings).

The implementation includes error handling for the Responses API's unique response format, token counting for flexible input types, and appropriate rate limiting management.

Motivation

The OpenAI Responses API is a next-generation API that improves upon the Chat Completions API with several key advantages:

  • Built-in state management: Server-side conversation history management eliminates the need to resend the entire conversation context
  • Built-in tools: Native support for web search, file search, and computer use capabilities
  • Improved performance: Faster response times and more efficient processing - most important for this script
  • Simplified workflows: Streamlines tool use, code execution, and multi-turn conversations
  • Future-focused: OpenAI's strategic direction as they phase out the Assistants API in favour of the Responses API

As developers adopt the Responses API for production workloads, they need the same parallel processing capabilities that this script provides for other endpoints.

The changes improve the cookbook by:

  • Extending utility: Supports OpenAI's newest and most advanced API endpoint
  • Maintaining consistency: Uses the same established patterns for error handling and rate limiting
  • Preserving compatibility: All existing functionality remains unchanged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant