Skip to content

fix: Handle generate=false WebSocket requests locally#124

Merged
franciscojavierarceo merged 2 commits into
vllm-project:mainfrom
EmbeddedLLM:fix-websocket-codex-qwen3.5
Jul 15, 2026
Merged

fix: Handle generate=false WebSocket requests locally#124
franciscojavierarceo merged 2 commits into
vllm-project:mainfrom
EmbeddedLLM:fix-websocket-codex-qwen3.5

Conversation

@maralbahari

Copy link
Copy Markdown
Collaborator

Summary

Handle Codex's non-generating WebSocket establishment request locally instead of forwarding it to vLLM.

When Codex sends generate: false, the gateway now persists a response checkpoint and returns response.created and response.completed without running inference. This prevents Qwen 3.5/6 from failing with No user query found in messages while preserving previous_response_id support for the first real user request.

generate is a WebSocket-only request control used on the response.create event. It is not a standard HTTP POST /v1/responses request parameter or a field in the returned Response object. The gateway therefore reads it from the raw WebSocket message, handles generate: false locally, and does not add it to or forward it through the generic HTTP/vLLM request payload.

OpenAI models and Qwen 3 tolerate an empty initial message, but Qwen 3.5/6 does not and requires a user query. Forwarding Codex's empty WebSocket establishment request directly to these stricter models therefore causes the chat template to fail.

Test Plan

  • Added tests for the WebSocket generate: false behavior.

  • Added an integration test confirming:

    • A generate: false request does not reach vLLM.
    • The generated response ID is reusable.
    • The following user request reaches vLLM unchanged.
    • An empty input without generate: false still follows the normal inference path.
  • Ran cargo clippy --workspace --all-targets -- -D warnings.

  • Ran the full workspace test suite with eight test threads using cargo test --workspace -- --test-threads=8.

  • Started Qwen 3.5 locally with vLLM:

    vllm serve Qwen/Qwen3.5-35B-A3B-FP8 \
      --reasoning-parser qwen3 \
      --enable-auto-tool-choice \
      --tool-call-parser qwen3_coder \
      --port 5050
  • Started the gateway against the local vLLM server:

    RUST_LOG=agentic_server=debug,agentic_core=debug \
      cargo run -p agentic-server -- --llm-api-base http://127.0.0.1:5050
  • Ran Qwen 3.5 through the Codex CLI interactively over WebSocket and confirmed that the non-generating establishment request is handled locally and subsequent prompts work correctly.

  • This behavior is difficult to reproduce with the existing cassettes because cassette tests pass request inputs directly and do not reproduce Codex CLI's initial empty WebSocket establishment request. The issue is most reliably reproduced by running the Codex CLI locally and interacting with it over WebSocket.

Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
@franciscojavierarceo franciscojavierarceo merged commit acd6846 into vllm-project:main Jul 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants