With prefix caching enabled, an instance that has gone idle retains far more physical GPU memory than the KVCACHED_MAX_CACHED_TOKENS bound implies, measured ~8.9 GiB pinned vs. the nominal ~2.2 GiB (16000 tokens) on Qwen3-4B. Cached blocks are retained in place, scattered across the 2MB pages they were originally allocated from, and free_pages() can only unmap a page when it is completely empty. The pinned set is therefore bounded by pages touched, not tokens cached.
With prefix caching enabled, an instance that has gone idle retains far more physical GPU memory than the KVCACHED_MAX_CACHED_TOKENS bound implies, measured ~8.9 GiB pinned vs. the nominal ~2.2 GiB (16000 tokens) on Qwen3-4B. Cached blocks are retained in place, scattered across the 2MB pages they were originally allocated from, and free_pages() can only unmap a page when it is completely empty. The pinned set is therefore bounded by pages touched, not tokens cached.