Skip to content

fix: preserve Responses cache salt#96

Merged
maralbahari merged 2 commits into
mainfrom
codex/responses-cache-salt
Jul 16, 2026
Merged

fix: preserve Responses cache salt#96
maralbahari merged 2 commits into
mainfrom
codex/responses-cache-salt

Conversation

@franciscojavierarceo

Copy link
Copy Markdown
Collaborator

Summary

  • Preserve the optional Responses cache_salt when parsing RequestPayload and serializing the normalized request sent upstream.
  • Add a regression test proving that a client/internal salt survives RequestPayload::to_upstream_request.
  • Initialize the new optional field in existing test, benchmark, and internal request literals.

Why

The gateway previously accepted JSON containing cache_salt but silently discarded the unknown field during deserialization. In a KV-aware llm-d deployment, llm-d folds the salt into its first canonical request key and vLLM includes it in APC block identity. Dropping the field weakens tenant isolation and prevents llm-d's salted request keys from matching vLLM's salted KV events.

This implementation is intentionally separate from the ADR and benchmark evidence in #65.

Test Plan

  • cargo test -p agentic-server-core request_payload_forwards_cache_salt_upstream -- --nocapture
  • cargo test
  • cargo clippy --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • uvx pre-commit run --all-files

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
@maralbahari
maralbahari merged commit 3a8a964 into main Jul 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants