[Roadmap] 2026 Q2 vLLM × RL Roadmap

[aoshen02]

[Roadmap] 2026 Q2 vLLM × RL Roadmap

This tracks the Q2 2026 vLLM-side work needed to make RL workloads (training & rollout) first-class. Each item links its own RFC / issue / PR — please discuss there, and use this thread for cross-cutting prioritization.

Training-Inference Consistency

• Support R3 routing replay: [RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support #39568

• Support PD + R3 Relax routed_experts capture KV-connector check to a warning for P/D #45419

• Support KV offload + R3 @xhx1022 [WIP] Enable return_routed_experts support with KV offload #45635

• Support flashinfer + R3 [RL Infra][FlashInfer] Enable router replay output from FlashInfer monolithic MoE kernel #44214

• render/derender endpoint [Render] Add reasoning/tool parsing to /derender + fix byte-fallback FFFD #45919

• Fix logprobs / logits surface consistency: [RFC]: Logprobs/Logits Semantics and Determinism Across the vLLM Ecosystem #42259

Runtime State Switching

• Standardize weight sync lifecycle: [RFC]: Native Weight Syncing APIs #31848

• Sparse weight updates support [Feature]: Support sparse in-place weight updates in weight transfer API #39451

• Make pause / resume coordinator-safe [RFC]: Extending vLLM Pause/Resume API to support in-flight requests and DPEP scenarios #32103

• CUDA Graph / NCCL context offload / resume to save the reconstruction time. https://github.com/NVIDIA/nccl/releases/tag/v2.29.7-1 [Core] Release NCCL communicator memory in sleep mode #46234

Rollout Performance / Efficiency

• Improve KV cache / prefix reuse: https://vllm.ai/blog/mooncake-store https://github.com/vllm-project/vllm/pull/42694/changes#diff-1f7101183bf6c2328171823032cc29e8ba98087d03fef92993abc88716dbcf7c @aoshen02

• Stabilize prefill decode disaggregation rollout: verl-project/verl#6243

• Add phase-aware performance modes to dynamically switch between throughput-optimized and latency-optimized configs [RFC]: Adaptive throughput/latency profile for RL rollout long-tail #41821 https://docs.vllm.ai/en/latest/features/speculative_decoding/dynamic_speculative_decoding

• More mature FP8 WA KV-cache rollout support

• RDMA-based cross-cluster transport for vLLM intermediate results — vLLM internally produces large per-request artifacts (expert routing indices are one example, but it generalizes — any large per-step or per-layer signal a downstream RL system might need). We need a generic plugin mechanism to export these artifacts. Doing it over RDMA in a distributed fashion (peer-to-peer between vLLM workers and downstream consumers) instead of going through a host-side aggregation buffer scales much better when multiple nodes pull at once. [RFC]: Rollout Artifact Transfer Connector for Post-Training Workloads #45221

Framework & Workload Enablement

• Support multimodal RL: https://github.com/verl-project/verl-omni/ vllm-omni & verl

Misc

• Add rollout liveness and debug signals: [RFC]: Endpoints to Retrieve User Configuration and Runtime Data #38147
• RL CI [CI][RL] Add E2E test suite for sleep/wake/pause/weight-update lifecycle #45586

Sync / Discussion

• Slack channel: #sig-reinforcement-learning on the vLLM Slack workspace

• Weekly call: every Friday 06:30 Beijing / 15:30 Pacific (Thursday) PDT / 18:30 Eastern (Thursday) EDT — https://meet.google.com/hpi-znch-gcx?hs=224

• New contributors / observers welcome — leave a comment on this thread or ping in Slack to be added to the agenda

[xiefan46]

Friday 06:30 might be too early for someone based on China. Is it possible to make it later?

[aoshen02]

Well, I don't think so. Do you have anything want to talk? You can also dm me in feishu/lark group chat. I've noticed that you are there?

[AYou0207]

Excited to see the roadmap! Just a quick heads-up: issue #40692 (I have met the same issue) shows that routing replay still has some stability gaps. It might be worth considering it as a prerequisite or a "stability polish" item within the current timeline to ensure a smooth user experience. Thanks for the hard work! @aoshen02

[huangyibo]

Please add me to the agenda! Thanks! email: ybhuang.cs@gmail.com

[aoshen02]

Excited to see the roadmap! Just a quick heads-up: issue #40692 (I have met the same issue) shows that routing replay still has some stability gaps. It might be worth considering it as a prerequisite or a "stability polish" item within the current timeline to ensure a smooth user experience. Thanks for the hard work! @aoshen02

Thanks for the info, I see.

[tpx818]

Well, I don't think so. Do you have any thing want to talk? You can also dm me in feishu/lark group chat. I guess we are all there?

I can’t find the link to the Feishu group. Could you please share it?

[aoshen02]

https://applink.feishu.cn/client/chat/chatter/add_by_link?link_token=57ch40b1-3e81-443b-81d4-733ee04593bc

[SamitHuang]

For mult-modal RL supports, PTAL at https://github.com/verl-project/verl-omni/ released recently, which uses vLLM and vLLM-Omni as rollout/reward model engine

[aoshen02]

Nice, is there any urgent todos you recommend for this domain?

For mult-modal RL supports, PTAL at https://github.com/verl-project/verl-omni/ released recently, which uses vLLM and vLLM-Omni as rollout/reward model engine

[joyang-nv]

Hi @aoshen02:
Some questions

1. Make pause / resume coordinator-safe, can you provide more details about this?
2. Any work on global kv cache cross multiple LLM instances?

[aoshen02]

Hi @aoshen02: Some questions

1. Make pause / resume coordinator-safe, can you provide more details about this?
2. Any work on global kv cache cross multiple LLM instances?

Hi, thanks for the advise, I've update it.

[joyang-nv]

And we do notice about global kv cache work between mooncake and vllm. That answers all my questions.

[aoshen02]

And we do notice about global kv cache work between mooncake and vllm. That answers all my questions.

Just need to add a kvcache reset support and then I think that's enough for RL scenario.

[SohamRajpure]

Interested! Please add me to the agenda. Thank you
email: srajpure@outlook.com