Your current environment
main / nightly
vLLM Router version: main / nightly
Component
Other
Router Configuration
Requests are rountripped with Serde without preserve_order. This means that JSON schema properties inside response_format and tools are sorted in alphabetical order before being sent to the upstream.
This is not desirable, because the schema is part of the prompt and some structured output backends may enforce the generation order to match the property order as well.
In our case, this lead to major regression in response quality. Consider a simple case of something like { reasoning: string; label: 'foo' | 'bar'} -- you typically want the model to generate the reasoning first.
The fix is simple (see here: main...DreamGenOrg:vllm-router:preserve_order).
🐛 Describe the bug
No related error messages
Additional context
No response
Your current environment
main / nightly
Component
Other
Router Configuration
Requests are rountripped with Serde without preserve_order. This means that JSON schema properties inside response_format and tools are sorted in alphabetical order before being sent to the upstream.
This is not desirable, because the schema is part of the prompt and some structured output backends may enforce the generation order to match the property order as well.
In our case, this lead to major regression in response quality. Consider a simple case of something like
{ reasoning: string; label: 'foo' | 'bar'}-- you typically want the model to generate thereasoningfirst.The fix is simple (see here: main...DreamGenOrg:vllm-router:preserve_order).
🐛 Describe the bug
No related error messages
Additional context
No response