[yuxianq]

@coderabbitai summary

Description

Extract general PrimTS compilation, caching, and paged-context fixes from the persistent compatibility patch into the locked FlashInfer source. On current main, this reduces the patch from 883 to 377 lines (506 fewer lines).

Upstream implementation: flashinfer-ai/flashinfer#8005

The source branch yuxianq/flashinfer:prims-ts-upstream-refactor--backport is rebased onto the new canonical branch trtllm-prims-ts-dev-20261107 at 911d6a9d62308b6c8a5249e8dc4aabc7518020f1. The lock pins the rebased source commit 52be15bbc078ad86710fa2a92367c80bcead3a76. The consumer PR is rebased onto TensorRT-LLM main at 00b59a491841aa72b5d704d2b5f709f13a846277.

The source owns precompile/probe modes, placeholder-safe compiler memos, validated/device-specific sparse plan caching, prepared sparse MLA launches, and import-order-safe paged helpers. Conflict resolution preserves the new dev branch's public APIs, planned host/tensor scale ABI, and SM107 dispatch. Geometry cache keys include the effective compile target; tensor-scale launches keep the full path. The positional output-quantization ABI fix is already in the new base.

The regenerated compatibility patch preserves main's metadata-extension, reduction, and Rubin compatibility adaptations. The metadata and parallel-reduction compatibility files remain byte-identical to main. The lock, patch, and digests were generated with scripts/vendor/manage.py.

Keep the PR as a draft while upstream #8005 is reviewed. Promotion should advance the new canonical branch after upstream merge.

schema_version: 1
vendors:
  flashinfer-prims-ts:
    upstream_prs:
      - https://github.com/flashinfer-ai/flashinfer/pull/8005
    resolutions:
      52be15bbc078ad86710fa2a92367c80bcead3a76:
        upstream_pr: https://github.com/flashinfer-ai/flashinfer/pull/8005
        reason: >-
          Rebases the attention precompile, plan-cache, prepared-launch, and
          paged-helper extraction onto trtllm-prims-ts-dev-20261107 at 911d6a9d.
          It preserves the new base's planned scalar/tensor scale ABI and
          SM107 compile-target dispatch, adding the effective target to the
          geometry key and keeping tensor-scale launches on the full path.
          The compile helper seam is retained for consumer regression tests.
          The positional output_quant_args fix is already in the new base.
          Tests, benchmarks, and CLAUDE.md are outside vendor selection.

Test Coverage

Validated after the 2026-11-10 rebase:

• Source ancestry: the pinned commit contains the fetched trtllm-prims-ts-dev-20261107 tip.
• Before/after git range-diff audits completed for both branches. The consumer changed-file set remains the same nine files; intentional differences adapt to the new scale APIs and SM107 target selection. No follow-up repair commit was needed.
• scripts/vendor/manage.py check flashinfer-prims-ts --repo <source-worktree>: PASS-OFFLINE and PASS-LOCAL. All vendor entries pass offline checks.
• tests/unittest/others/test_vendor_sources.py: 72 passed.
• Source CPU cache, locking, import-order, device/compile-target regressions: 8 passed.
• B200/SM100 source validation: 9 passed — five cache-lifecycle cases, tensor-scale graph replay, two forced D576 split cases, and paged-context numerical accuracy.
• Actual TRT-LLM vendored files loaded through the isolated source harness: 12 passed (7 CPU + 5 GPU lifecycle/graph cases).
• Targeted syntax validation, git diff --check, source mypy/Ruff, and applicable pre-commit checks in both repositories pass.

Test consolidation: the source branch's added suite collects 13 cases instead of 14 (CPU 9 → 8; GPU unchanged at 5). Removed the redundant normal-order import subprocess, covered by existing paged-context testing; retained the reverse-order cycle regression. The four dtype × pool combinations select distinct JIT paths, and the fifth GPU case checks cold persistence/reload. No runtime speedup is inferred from case count. This consumer PR adds no CI test entries.

Full model/executor integration tests were not run locally. CI is retriggered with the previous command, /bot run --disable-fail-fast.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[yuxianq]

Vendor validation: flashinfer-prims-ts

@yuxianq — Vendor promotion needs author action:

Some listed upstream PRs cover no new lock-selected changes: upstream PR #6774.

Fix: Remove these unrelated PRs from vendors.flashinfer-prims-ts.upstream_prs in the PR description. PRs covering only excluded tests/docs do not need to be listed.

The bot will recheck automatically on a later poll.

Checked PR revision: 3a707c9a8284. This is a snapshot; newer results are posted as new comments.

Checked source: https://github.com/yuxianq/flashinfer.git / prims-ts-upstream-refactor--backport, pin f1dc866792fbcc02bef7b9c7c2f0ce7798a5b7f5.

[yuxianq]

/bot run --disable-fail-fast

[yuxianq]

Vendor validation: flashinfer-prims-ts

@yuxianq — Vendor promotion metadata and source attribution are valid. This is not a code-review approval. Promotion still requires the source PR to merge.

Checked PR revision: 3a707c9a8284. This is a snapshot; newer results are posted as new comments.

Checked source: https://github.com/yuxianq/flashinfer.git / prims-ts-upstream-refactor--backport, pin f1dc866792fbcc02bef7b9c7c2f0ce7798a5b7f5.

Supersedes previous vendor validation.

[tensorrt-cicd]

PR_Github #81677 [ run ] triggered by Bot. Commit: 3a707c9 Link to invocation

[tensorrt-cicd]

PR_Github #81677 [ run ] completed with state SUCCESS. Commit: 3a707c9
/LLM/main/L0_MergeRequest_PR pipeline #67340 completed with status: 'SUCCESS'

CI Report

Link to invocation

[yuxianq]

Vendor validation: flashinfer-prims-ts

@yuxianq — Vendor promotion needs maintainer action:

Waiting for the preceding vendor promotion: the base lock still points to repository https://github.com/harrisonzhy/flashinfer.git, branch harrisonzhy/trtllm-prims-ts-dev_vc-attn-20261029.

Fix: A maintainer must finish the preceding lock-only promotion back to yuxianq/flashinfer / trtllm-prims-ts-dev-20261012. After it merges, the author should rebase this consumer PR onto the latest base branch and rebase its temporary source branch onto the new canonical pin, then update the lock/vendor snapshot. No metadata edit can unblock the preceding promotion.

The bot will recheck automatically on a later poll.

Checked PR revision: 3a707c9a8284. This is a snapshot; newer results are posted as new comments. Observed dev/upstream tip SHAs may be stale: tip movement alone does not post a new comment when the result is unchanged. Fetch the live branch before rebasing or choosing a new pin.

Supersedes previous vendor validation.

[yuxianq]

Vendor validation: flashinfer-prims-ts

@yuxianq — Vendor promotion needs author action:

Source update is not a fast-forward.

Fix: Fetch canonical repository https://github.com/yuxianq/flashinfer.git, branch trtllm-prims-ts-dev-20261012, and rebase the temporary source branch prims-ts-upstream-refactor--backport onto its latest tip without merge commits, not just the consumer lock's pin 7087b06c22ceb116b2236a938038a00b39f752d1. Then use scripts/vendor/manage.py to update 3rdparty/vendor_sources.lock.yaml and the vendored snapshot to the rebased SHA, and push the source branch and consumer PR. Editing metadata alone will not fix this.

The bot will recheck automatically on a later poll.

Checked PR revision: 3a707c9a8284. This is a snapshot; newer results are posted as new comments. Observed dev/upstream tip SHAs may be stale: tip movement alone does not post a new comment when the result is unchanged. Fetch the live branch before rebasing or choosing a new pin.

Checked source: https://github.com/yuxianq/flashinfer.git / prims-ts-upstream-refactor--backport, pin f1dc866792fbcc02bef7b9c7c2f0ce7798a5b7f5.

Supersedes previous vendor validation.

[yuxianq]

Vendor validation: flashinfer-prims-ts

@yuxianq — Vendor promotion needs maintainer action:

The base lock now uses refresh branch trtllm-prims-ts-dev-20261107, but this monitor still publishes to trtllm-prims-ts-dev-20261012.

Fix: A maintainer must stop this monitor and migrate its workdir with --canonical-branch trtllm-prims-ts-dev-20261107 --migrate-canonical-branch-from trtllm-prims-ts-dev-20261012. Verify a dry run before resuming publication. Do not promote back to the old branch or change contributor metadata to bypass this cutover.

The bot will recheck automatically on a later poll.

Checked PR revision: 3a707c9a8284. This is a snapshot; newer results are posted as new comments. Observed dev/upstream tip SHAs may be stale: tip movement alone does not post a new comment when the result is unchanged. Fetch the live branch before rebasing or choosing a new pin.

Supersedes previous vendor validation.

[yuxianq]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #83770 [ run ] triggered by Bot. Commit: 4cf4e67 Link to invocation

[tensorrt-cicd]

PR_Github #83770 [ run ] completed with state SUCCESS. Commit: 4cf4e67
/LLM/main/L0_MergeRequest_PR pipeline #69267 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation