[KV Offload] Define clean backend configuration boundary

[Change72]

Purpose

Part of #47929.

This PR defines plain offloading configuration structs in vllm/v1/kv_offload and moves the existing KVCacheSpec and KVCacheConfig parsing for block geometry, CPU sizing inputs, and file identity into the offloading connector, which translates the raw configs into those structs. The struct shape (model-, cache-, and parallel-level sub-configs with tokens_per_block, blocks_per_chunk, and a precomputed parallelism-agnostic flag) follows the review suggestion.

• Migrate the experimental OffloadingSpec plugin constructor to a single translated OffloadingConfig argument (a breaking change for external specs loaded via spec_module_path; the migration is mechanical, and a transition period accepting both signatures can be added if preferred).
• Adopt the requested chunk terminology across the offload domain (the offloaded unit is a chunk): blocks_per_chunk, tokens_per_chunk, tokens_per_hash, tokens_per_block, applied to config, spec, scheduler, FileMapper, and tests. Values are preserved; only names change.
• Reuse one packed-layout predicate for CPU sizing and worker canonicalization.
• Add characterization coverage for packed and unpacked layouts, hybrid block sizes, parallel dimensions, file identity, and external plugin construction.

This refactor has no intended runtime behavior change, with one review-requested exception: the FileMapper namespace now records the resolved hash granularity, the context-parallel-scaled per-group block sizes, and the renamed identity keys. All FS/OBJ namespace digests therefore change: existing on-disk caches are safely cold-missed, and mixed-version P2P peers fail closed on the config-fingerprint mismatch. The breaking change to the experimental plugin surface covers the renamed config/spec/FileMapper fields as well, not just the constructor. CPU sizing is unchanged, and this PR does not implement MLA replica deduplication.

This ownership split was requested in #47929. It establishes connector-owned parsing helpers that the next PR can extend on the scheduler side to infer replicated MLA layouts from KVCacheSpec.

Duplicate-work check

I checked the issue comments and searched open PRs by issue number and area keywords. No other open PR references #47929 or implements this parsing move.

The closest related work—#42374, #46954, #44865, and #38261—covers core KV representation, canonical layout metadata, transfer descriptors, or hybrid planning rather than this ownership refactor.

Test Plan

• Run focused factory and FileMapper characterization tests.
• Run the applicable tests/v1/kv_offload suite.
• Run pre-commit and the manual Python 3.12 mypy hook on every changed Python file.
• No model evaluation is needed because this refactor does not affect model execution or output.

Test Result

On NVIDIA L4 with PyTorch 2.11.0+cu128 and CUDA 12.8:

pytest \
  tests/v1/kv_offload/test_factory.py \
  tests/v1/kv_offload/test_file_mapper.py -q

Result: 34 passed.

pytest \
  tests/v1/kv_offload/tiering/test_fs_tier.py \
  tests/v1/kv_offload/tiering/test_obj_tier.py -q

Result: 46 passed, 2 skipped.

The P2P manager tests (including the data-parallel port-offset cases from #47636, adapted to the translated config) pass: 60 passed.

The full tests/v1/kv_offload suite was additionally compared against the latest-main merge-base on the same host: the failing cases are identical pre-existing environment-dependent tests on that setup — the change introduces no new failures (347 passed, 2 skipped otherwise).

Changed-file pre-commit, manual mypy-3.12, and git diff --check also passed.

AI Assistance

AI assistance was used for implementation and review.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[orozery]

Thanks @Change72 !

As a principle, the kv_offload directory should not import from the offloading connector or vllm internals.
I think we should define clean structs on kv_offload, and have the offloading connector build them (translating from vllm_config and kv_cache_spec).
There should be one struct for model-level config and one for group-level config.

[Change72]

Thanks @orozery — I reworked the PR around that boundary:

• Added model-level and per-group config structs under vllm/v1/kv_offload/config.py.
• The connector now translates VllmConfig and KVCacheConfig into the normalized config.
• OffloadingSpec now takes a single OffloadingConfig; scheduler and worker components receive any raw configs directly.
• Added tests for the import boundary and existing sizing, packed-layout, and file-identity behavior.

One scope question: cpu/manager.py and tiering/manager.py already import connector metrics at runtime. Should I clean those up in this PR or leave them for a follow-up?

[orozery]

Thanks @Change72 ! Great work!

One scope question: cpu/manager.py and tiering/manager.py already import connector metrics at runtime. Should I clean those up in this PR or leave them for a follow-up?

I actually think it's fine to leave it as is for now.
We have some external dependencies which I think are fine as they are minimal and should be stable.
e.g. KVConnectorStats, kv event mediums.

[orozery]

Suggesting some changes.
Here are the adaptations needed:

1. namespace_block_size dropped → FileMapper uses cache.hash_block_size instead.
2. Per-group block_size dropped → FileMapper uses tokens_per_block in the namespace hash.
3. parallel_agnostic pre-computed → FileMapper no longer decides this itself.
4. num_gpu_blocks dropped → Consumers check worker_kv_bytes_per_block == 0 as the "no blocks" sentinel.
5. engine_id made non-optional → Builder asserts it exists (offloading is always active when this config is constructed).

Suggested change

	@dataclass(frozen=True)
	class OffloadingGroupConfig:
	"""Normalized configuration for one KV cache group."""
	block_size: int
	gpu_block_size: int
	layer_names: tuple[str, ...]
	is_non_mla_full_attention: bool
	@dataclass(frozen=True)
	class OffloadingConfig:
	"""Model-level configuration for a native offloading backend."""
	groups: tuple[OffloadingGroupConfig, ...]
	hash_block_size: int
	block_size_factor: int
	num_gpu_blocks: int
	worker_kv_bytes_per_gpu_block: int
	world_size: int
	enable_kv_cache_events: bool
	extra_config: Mapping[str, Any]
	model_name: str
	kv_cache_dtype: str
	namespace_block_size: int
	tp_size: int
	pp_size: int
	pcp_size: int
	dcp_size: int
	rank: int
	use_v2_model_runner: bool
	engine_id: str | None
	@dataclass(frozen=True)
	class OffloadingGroupConfig:
	# Total token span covered by one block across all workers
	# (accounts for context parallelism).
	tokens_per_block: int
	# Layer names belonging to this group.
	layer_names: tuple[str, ...]
	@dataclass(frozen=True)
	class OffloadingModelConfig:
	# Model identifier (e.g. HuggingFace model path).
	name: str
	# KV cache data type (e.g. "float16").
	dtype: str
	@dataclass(frozen=True)
	class OffloadingCacheConfig:
	# Tokens per block hash.
	hash_block_size: int
	# Blocks per offload key.
	blocks_per_key: int
	@dataclass(frozen=True)
	class OffloadingParallelConfig:
	# Worker index in [0, world_size). 0 on the scheduler side.
	rank: int
	# Total number of workers.
	world_size: int
	# Tensor parallel size.
	tp_size: int
	# Pipeline parallel size.
	pp_size: int
	# Prefill context parallel size.
	pcp_size: int
	# Decode context parallel size.
	dcp_size: int
	# True when concatenating a block's data across all workers yields
	# the same result regardless of the parallelism configuration.
	is_parallelism_agnostic: bool
	@dataclass(frozen=True)
	class OffloadingConfig:
	groups: tuple[OffloadingGroupConfig, ...]
	# KV bytes stored by one worker per block.
	worker_kv_bytes_per_block: int
	# Whether the scheduler emits KV cache events. When true,
	# the offloading backend should emit events as well.
	enable_kv_cache_events: bool
	# Offloading-specific configuration from kv_connector_extra_config.
	extra_config: Mapping[str, Any]
	# Unique identifier for this engine, distinct per DP rank.
	engine_id: str
	model: OffloadingModelConfig
	cache: OffloadingCacheConfig
	parallel: OffloadingParallelConfig

[Change72]

Applied

[orozery]

This test is nice, but it's kind of cumbersome, and also hard to maintain with all the paths hard-coding consts.
I suggest we remove it.

[Change72]

Removed

[Change72]

Thanks @orozery — applied all of it:

Adopted your struct suggested shape and removed the import-guard test.

(from claude:)
One pre-existing inconsistency surfaced while adapting this (not introduced here, and left as-is): tokens_per_block applies the DCP×PCP factor to every group, faithful to the old gpu_block_size computation — but resolve_kv_cache_block_sizes deliberately leaves Mamba groups unscaled ("Mamba groups keep their full per-rank state and are not scaled"). So for Mamba-hybrid + context-parallel setups the offload path disagrees with the core resolver, as it did before. Happy to fix that here or in a follow-up — your call.

[orozery]

(from claude:)
One pre-existing inconsistency surfaced while adapting this (not introduced here, and left as-is): tokens_per_block applies the DCP×PCP factor to every group, faithful to the old gpu_block_size computation — but resolve_kv_cache_block_sizes deliberately leaves Mamba groups unscaled ("Mamba groups keep their full per-rank state and are not scaled"). So for Mamba-hybrid + context-parallel setups the offload path disagrees with the core resolver, as it did before. Happy to fix that here or in a follow-up — your call.

Let's leave it as it is.

[orozery]

Can we revert back to generating a mocked spec instead of building a real one?

[Change72]

Done.

[orozery]

Re-thinking this, we use "block" as the unit being stored in the accelerated (GPU) hardware.
For the unit being offloaded, we use "key", which is confusing.
Claude suggested we use "chunk" instead.
It would be good to rename block_size_factor to blocks_per_chunk throughout the code.
Can also rename block_size -> tokens_per_blocks
offloaded_block_size -> tokens_per_chunk
hash_block_size -> tokens_per_hash
hash_block_size_factor -> hashes_per_chunk
src_block_size_factor -> src_blocks_per_chunk
dst_block_size_factor -> dst_blocks_per_chunk
sub_block_size -> block_page_size (gpu_worker.py)
sub_block_size -> tokens_per_hash (events.py)
full_attn_offloaded_block_sizes -> full_attn_tokens_per_chunk
offload_block_idx -> chunk_idx
gpu_blocks_per_file -> blocks_per_file
kv_bytes_per_offloaded_block -> kv_bytes_per_chunk

Suggested change

	# Blocks per offload key.
	blocks_per_key: int
	# Blocks coalesced into one offload chunk.
	blocks_per_chunk: int

[Change72]

we use "key", which is confusing.

Agreed. tokens_per_blocks and hash_block_size were also confused for me in the previous commit.

block_size -> tokens_per_blocks
offloaded_block_size -> tokens_per_chunk
hash_block_size -> tokens_per_hash

are more clear. Will fix

[orozery]

I made a typo it should be tokens_per_block not tokens_per_blocks.

[orozery]

Thanks @Change72 !
I still see some gpu_block_size. Can we rename those remaining to tokens_per_block?

[Change72]

All requested changes are done:

• Completed the chunk terminology and config-boundary cleanup.
• Restored the mocked FileMapper tests and integrated [KVOffload][P2P] Well-known default host/port env vars and per-DP-rank control port #47636 through the translated config.
• Verified no new failures against the merge base.

[orozery]

Keep up the good work! :)