fix: Handle generate=false WebSocket requests locally#124
Merged
franciscojavierarceo merged 2 commits intoJul 15, 2026
Merged
Conversation
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
franciscojavierarceo
approved these changes
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Handle Codex's non-generating WebSocket establishment request locally instead of forwarding it to vLLM.
When Codex sends
generate: false, the gateway now persists a response checkpoint and returnsresponse.createdandresponse.completedwithout running inference. This prevents Qwen 3.5/6 from failing withNo user query found in messageswhile preservingprevious_response_idsupport for the first real user request.generateis a WebSocket-only request control used on theresponse.createevent. It is not a standard HTTPPOST /v1/responsesrequest parameter or a field in the returned Response object. The gateway therefore reads it from the raw WebSocket message, handlesgenerate: falselocally, and does not add it to or forward it through the generic HTTP/vLLM request payload.OpenAI models and Qwen 3 tolerate an empty initial message, but Qwen 3.5/6 does not and requires a user query. Forwarding Codex's empty WebSocket establishment request directly to these stricter models therefore causes the chat template to fail.
Test Plan
Added tests for the WebSocket
generate: falsebehavior.Added an integration test confirming:
generate: falserequest does not reach vLLM.generate: falsestill follows the normal inference path.Ran
cargo clippy --workspace --all-targets -- -D warnings.Ran the full workspace test suite with eight test threads using
cargo test --workspace -- --test-threads=8.Started Qwen 3.5 locally with vLLM:
Started the gateway against the local vLLM server:
Ran Qwen 3.5 through the Codex CLI interactively over WebSocket and confirmed that the non-generating establishment request is handled locally and subsequent prompts work correctly.
This behavior is difficult to reproduce with the existing cassettes because cassette tests pass request inputs directly and do not reproduce Codex CLI's initial empty WebSocket establishment request. The issue is most reliably reproduced by running the Codex CLI locally and interacting with it over WebSocket.