Does kthena fully support LMCache as a PD disaggregation connector?

[0xjasoncao]

Please provide an in-depth description of the question you have:

I'm trying to run disaggregated prefill (PD) with vLLM + LMCache as the KV connector, routed by kthena-router. After tracing both codebases
against the LMCache official PD example (examples/disagg_prefill/disagg_proxy_server.py @ v0.4.4), I'm not sure kthena currently provides
what the LMCache LMCacheConnectorV1 path actually requires. Opening this to ask the community: does kthena fully support LMCache as a PD
connector, and if not, what's the intended path?

What LMCache requires (per its official example)

In LMCache's official PD example, an external proxy (disagg_proxy_server.py) does two things kthena doesn't appear to do:

1. Injects transfer_spec (a.k.a. disagg_spec) into the prefill request body — this is how the prefill side learns which decode node to send
   KV to. It's mandatory: without it, the producer won't transfer KV (and on older LMCache it outright crashes). The proxy picks a decode
   node and adds to the prefill request:

 disagg_spec = {
      "req_id": req_id,
      "receiver_host": decode_client.host,        
      "receiver_init_port": decode_client.init_port, 
      "receiver_alloc_port": decode_client.alloc_port,
  }

     req_data["kv_transfer_params"] = {
      "ret_first_tok": True,
      "disagg_spec": disagg_spec,
  }

2. Receives a completion notification from prefill and gates the decode request until KV transfer finishes. This part is optional as of
   LMCache v0.4.4 (pd_skip_proxy_notification), but the disagg_spec injection in (1) is not optional.

What kthena currently does

Looking at pkg/kthena-router/connectors/, the LMCache-relevant path seems to be the simple HTTPConnector (http.go), whose Proxy just
forwards req_data to prefill then to decode sequentially — it does not inject disagg_spec. The NIXLConnector (nixl.go) injects a different,
vLLM-native shape (do_remote_decode / remote_block_ids / remote_host / remote_port, see types.go:20-26), which LMCache's
LMCacheConnectorV1 does not consume.

What do you think about this question?:

So when a PD workload uses --kv-transfer-config '{"kv_connector":"LMCacheConnectorV1",...}', the fields kthena injects don't match what
LMCache reads. The prefill node never receives disagg_spec, so KV is never transferred to the decode node.

Questions for the community

1. Is full LMCache (LMCacheConnectorV1) PD support a goal of kthena-router? If yes, is there a connector (existing or planned) that injects
   the LMCache disagg_spec shape above and does decode-node selection?

Environment:

• Kthena version: v1.0.0
• Kubernetes version: v1.35
• Others:

[YaoZengzeng]

We may not have caught up with the updates from the LMCache community.

I will implement this feature in this release cycle if no one in the community is interested.

[nXtCyberNet]

hi @YaoZengzeng , i will take this up ,

/assign

[YaoZengzeng]

I think this issue is related to #1069

[hzxuzhonghu]

This is because kthena does not elvove that fast as lmcache connector, previously https://github.com/vllm-project/vllm/pull/16625/changes#diff-256116cc09c571191268bad7b8ad35c71f609ed0d497f18aa24b6a870475f016 we can see it does not need to pass kv_transfer_params

[nXtCyberNet]

I think this issue is related to #1069

i agree its already solved by the pr #1432

[hzxuzhonghu]

can you explain where kv_transfer_params is augmented https://github.com/vllm-project/vllm/blob/9c22668436a4d94aab87ea74a220e060415cf1d8/examples/disaggregated/lmcache/disagg_prefill_lmcache_v1/disagg_proxy_server.py#L190-L202

[0xjasoncao]

The VLLM official website examples only show a regular HTTP proxy, but I saw related content in the LMCache official examples.
https://github.com/LMCache/LMCache/blob/v0.4.4/examples/disagg_prefill/disagg_proxy_server.py#L504-L515

[YaoZengzeng]

Thanks for the detailed write-up. After tracing the kthena connector code (pkg/kthena-router/connectors/) against both the vLLM official example and the LMCache v0.4.4 example, I think the confusion comes from LMCacheConnectorV1 having two distinct PD modes, and the two examples referenced here belong to different modes.

The key distinction: who consumes kv_transfer_params

There are two different shapes of kv_transfer_params, consumed by different components:

Shape	Consumer	Status
do_remote_decode / do_remote_prefill / remote_block_ids / remote_host / remote_port	vLLM core NixlConnector	Officially supported; this is what kthena's NIXLConnector injects
disagg_spec (receiver_host / init_port / alloc_port)	LMCache's private pd_backend (not vLLM core)	LMCache-specific P2P feature

So to answer the questions directly:

1. Is disagg_spec a vLLM-supported path?

Not in vLLM core. vLLM core just passes kv_transfer_params through to the connector; the disagg_spec shape is parsed by LMCache's own pd_backend (lmcache.v1.storage_backend.pd_backend), which also relies on LMCache-private PDMsg/ProxyNotif msgspec structures and a ZMQ notification channel. The vLLM official LMCacheConnectorV1 example (examples/disaggregated/lmcache/disagg_prefill_lmcache_v1/disagg_proxy_server.py) deliberately does not inject kv_transfer_params — it just forwards the prompt to prefill (max_tokens=1, non-stream) and then streams from decode, relying on a shared KV backend (e.g. Redis) for KV reuse. As @hzxuzhonghu noted (vLLM PR #16625), the official path does not need kv_transfer_params.

2. What does kthena currently support?

• LMCacheConnectorV1 shared-backend mode — supported today via HTTPConnector (factory.go maps ConnectorTypeLMCache -> NewHTTPConnector). This matches the vLLM official V1 example: max_tokens=1 + strip stream on prefill, then forward the same body to decode. KV reuse via the shared store works.
• NIXL point-to-point mode — supported via NIXLConnector, which does the prefill→decode handshake (do_remote_decode, then relays the returned kv_transfer_params into the decode request). This already covers the "no shared store, direct instance-to-instance transfer" use case.

3. The actual gap

kthena does not implement the LMCache-private disagg_spec P2P path (the v0.4.4 disagg_proxy_server.py linked above). That path requires the proxy to: select the decode node, inject disagg_spec, run a ZMQ PULL server to receive ProxyNotif, gate decode with wait_decode_kv_ready, and manage a PD-buffer semaphore. This tightly couples the router to LMCache-internal, version-sensitive interfaces.

Recommendation

I'd suggest not injecting disagg_spec into the existing HTTPConnector, because:

• The vLLM-official LMCacheConnectorV1 path (shared backend) already works in kthena.
• The "P2P transfer without a shared store" need is already covered by the NIXL connector.
• disagg_spec is an LMCache-private, fast-moving interface — coupling to it is exactly the "kthena can't keep pace with LMCache" concern raised above.

If there's a concrete need to support LMCache's native P2P disaggregation, it should be a separate LMCacheConnector (not a modification of HTTPConnector) with a pinned LMCache version, given the ZMQ notification + buffer-gating requirements.

In short: for the vLLM-official LMCacheConnectorV1 flow, kthena already works (shared backend), and NIXL covers P2P. The only missing piece is LMCache's own disagg_spec P2P mode, which is a targeted "unsupported mode" gap rather than "LMCache PD is broken."

[YaoZengzeng]

A clearer user guide for PD disaggregation based on lmcache #1559

[LiZhenCheng9527]

@0xjasoncao Userguide has updated
#1559