Skip to content

P5: Implement streaming chat completions gateway #19

@wauputr4

Description

@wauputr4

Goal

Add streaming support for /v1/chat/completions while preserving request context, usage capture, and cleanup behavior.

Acceptance Criteria

  • Streaming request path proxies provider chunks to the client.
  • Final usage metadata is captured when upstream provides it.
  • Gateway releases any runtime resources after stream completion or error.
  • Streaming works with OpenAI-compatible clients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: gatewayGateway routing and OpenAI-compatible API surfaceenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions