This repository was archived by the owner on Jun 7, 2026. It is now read-only.
feat(stream-object): add /stream-object endpoint with SDK support - #73
Merged
Conversation
Add a new /stream-object endpoint that streams partial JSON objects as they're generated using Server-Sent Events (SSE). This follows Anthropic's input_json_delta pattern for real-time structured output. Features: - Partial JSON parsing using partial-json library - Emits partial-object events with parsed objects as tokens arrive - Emits object event with final validated object - Full SSE event schema: session, partial-object, object, result, done - Uses --json-schema CLI flag for constrained decoding Closes #67
- Add logging for unexpected errors in partial JSON parsing - Distinguish interrupts from real errors in buffer processing - Send error event when structured_output missing and JSON parse fails - Send error event for internal stream processing errors instead of re-throwing - Add JSON Schema validation requiring valid schema keywords - Apply JSON Schema validation to generateObjectRequest as well Tests: - Add test for buffer processing on CLI close - Add test for non-JSON line handling - Add test for unparseable JSON fallback case - Add test for stdin.end() verification - Add tests for JSON Schema validation
Add streamObject() function to the TypeScript SDK that consumes the gateway's /stream-object endpoint: - Stream partial objects via partialObjectStream (async iterable) - Get final validated object via object promise - Session ID resolves early, usage resolves at end - Zod schema validation for partial objects (best-effort) and final object (strict) - Comprehensive test coverage (17 tests) - Example usage in examples/stream-object.ts Part of #67
…artial streaming The --json-schema flag doesn't stream JSON tokens incrementally - it streams natural language text and only provides the JSON at the end in structured_output. This change uses prompt injection instead: - Inject schema into prompt with JSON output instructions - Add JSON generator system prompt - Parse accumulated text with partial-json library - Add parseJsonResponse() fallback for markdown/text wrapping Workaround for: anthropics/claude-code#15511
- Number updates based on meaningful changes (day count) shown to user - Track total stream updates from gateway separately - Display debug info at the bottom with both counts and token usage
Addresses review feedback from PR review agents: Gateway improvements: - Fix silent failure in close handler catch block (now logs and emits error) - Add diagnostic info to parseJsonResponse errors (tracks all parse attempts) - Add logging for content_block_delta events missing text field - Improve cleanup function with try/catch for kill(), SIGKILL escalation logging - Simplify buildStreamObjectArgs to return string[] (remove unused values) - Update endpoint docstring with precise streaming limitation explanation SDK improvements: - Use StreamObjectOptions<T> interface in function signature (was inline type) - Optimize isCriticalEvent check with const Set instead of array New tests: - Empty response handling (CLI returns no JSON) - Markdown code block stripping fallback - JSON embedded in surrounding text extraction - Array schema parsing
Adds test infrastructure and integration test for aborting streams mid-flight: - createDelayedSSEStream: mock SSE stream with configurable delays - createDelayedMockSSEResponse: wrapper for delayed SSE responses - Test verifies: partial objects received, abort triggered, AbortError thrown
Add stream_object() function to the Python SDK for streaming structured JSON responses via SSE. Follows patterns from TypeScript SDK and stream_text() implementation. Features: - StreamObjectResult[T] dataclass with partial_object_stream async iterator - Pydantic validation: best-effort for partials, strict for final object - Futures for session_id(), usage(), and object() resolution - HTTPObjectStreamContext async context manager for resource cleanup Includes: - 10 comprehensive tests covering success, errors, and edge cases - Example script demonstrating travel itinerary streaming - NO_OBJECT error code for incomplete streams
Align TypeScript SDK example with Python SDK example for parity. Both now generate a 3-day Tokyo itinerary with 2-3 activities per day.
Improvements from comprehensive PR review:
Python SDK:
- Add T = TypeVar("T", bound=BaseModel) for better type safety
- Make SSE event models frozen with ConfigDict
- Refine SSE event type annotations (dict[str, Any] | None)
- Add explicit httpx exception handling (ReadTimeout, RemoteProtocolError)
- Improve docstrings documenting validation rejection
- Add cancellation mid-stream test (parity with TypeScript abort test)
TypeScript SDK:
- Add @throws JSDoc annotations for validation errors
- Clarify partial object comments
- Document object promise rejection on validation failure
Gateway:
- Fix silent failure: parseJsonResponse now returns extraction strategy
- Emit warning event when fallback extraction is used (markdown-block, etc.)
- Fix silent failure: emit warning event on buffer data loss (clean CLI exit)
- Add streamObject integration tests
All tests passing (204 total)
- Add timeout behavior test for TypeScript SDK streamObject - Add mid-stream connection error test for TypeScript SDK - Add connection/timeout/protocol error tests for Python SDK - Fix Python SDK to wrap httpx exceptions during request phase
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
/stream-objectendpoint for streaming structured JSON objects, with full SDK support in both TypeScript and Python.Closes #67
Changes
Gateway
/stream-objectendpoint that streams partial JSON objects via SSEsession,partial-object,object,result, anddoneeventsTypeScript SDK
streamObject()function with Zod schema validationpartialObjectStream(async iterable),object,sessionId,usagepromisesPython SDK
stream_object()async context manager with Pydantic schema validationStreamObjectResult[T]with partial object stream and futuresTest Coverage
Test Plan