Skip to content

[SM120][Bugfix] Fix packed KV block zeroing stride - #34

Open
wangxian001 wants to merge 1 commit into
jasl:ds4-sm120-preview-devfrom
wangxian001:fix-sm120-packed-kv-zeroing-stride
Open

[SM120][Bugfix] Fix packed KV block zeroing stride#34
wangxian001 wants to merge 1 commit into
jasl:ds4-sm120-preview-devfrom
wangxian001:fix-sm120-packed-kv-zeroing-stride

Conversation

@wangxian001

@wangxian001 wangxian001 commented Jul 29, 2026

Copy link
Copy Markdown

Purpose

Apply the packed KV cache zeroing fix to ds4-sm120-preview-dev. The upstream counterpart is vllm-project#50276.

The zeroing path currently uses a segment's page size for both its logical block stride and its zero span. Packed DeepSeek KV views can have a larger physical stride than an individual segment's page, so clearing a high block ID may touch adjacent packed data or cross the end of the allocation.

This change:

  • tracks logical block stride independently from page size;
  • represents virtual kernel blocks as independent zeroing segments;
  • derives the zero span from the tensor shape and strides; and
  • adds a final-block regression test for a packed backing allocation.

The SM120 prefill/decode paths, DeepSeek-V4 kernels, and DSpark speculative decoding behavior are unchanged.

Duplicate-work check

The matching upstream fix is vllm-project#50276. Searches of open JASL issues and PRs for KVBlockZeroer and packed KV zeroing found no other implementation on this branch.

Test Plan

  • .venv/Scripts/python.exe -m py_compile vllm/v1/worker/utils.py tests/v1/worker/test_kv_block_zeroer.py
  • ruff 0.14.0 check vllm/v1/worker/utils.py tests/v1/worker/test_kv_block_zeroer.py
  • ruff 0.14.0 format --check vllm/v1/worker/utils.py tests/v1/worker/test_kv_block_zeroer.py
  • pytest -q tests/v1/worker/test_kv_block_zeroer.py on CUDA
  • End-to-end DeepSeek-V4 serving with FP8 KV cache, block size 256, prefix caching, DSpark speculative decoding, and mixed long-context refill and short structured requests

Test Result

  • Python 3.12 syntax checks passed.
  • Ruff lint and format checks passed.
  • Targeted CUDA regression suite: 3 passed.
  • End-to-end validation passed on two RTX PRO 6000 Blackwell GPUs under repeated cache pressure; the API remained healthy and no CUDA Xid was observed after the fix.

AI assistance disclosure: OpenAI Codex helped prepare the patch and PR text. The submitter participated in the diagnosis, reviewed the change, and validated it end-to-end.

@github-actions

Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

@wangxian001
wangxian001 force-pushed the fix-sm120-packed-kv-zeroing-stride branch 2 times, most recently from fab94df to 5bfa63a Compare July 29, 2026 14:49
Track logical block strides separately from segment page sizes so packed KV views do not clear adjacent memory. Expand virtual blocks into independent segments and add a last-block regression test.

Assisted-by: OpenAI Codex
Signed-off-by: wangxian001 <120719093+wangxian001@users.noreply.github.com>
@wangxian001
wangxian001 force-pushed the fix-sm120-packed-kv-zeroing-stride branch from 5bfa63a to 409defc Compare July 30, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant