Feature/python native v2#1108

Open

baohengyi wants to merge 131 commits into

alibaba:feature/python_native_v2from

baohengyi:feature/python_native_v2

Commits on May 14, 2026

feat: add MoriEP expert parallelism support for ROCm

jacobwin-ai
authored and
LLLLKKKK
committed
feat: add fake balance expert and MoriEP tests

jacobwin-ai
authored and
LLLLKKKK
committed
fix: address P1 issues in MoriEP PR

jacobwin-ai
authored and
LLLLKKKK
committed
fix: route PD smoke prompt_batch through / to preserve pd_sep

jacobwin-ai
authored and
LLLLKKKK
committed
fix: update golden apostrophe to match model output (U+2019)

jacobwin-ai
authored and
LLLLKKKK
committed
fix: use ep_size instead of world_size in MoriEP wrapper initialization

jacobwin-ai
authored and
LLLLKKKK
committed

Commits on May 15, 2026

Commits on May 18, 2026

perf(rocm): add fused QK RMSNorm V2 warp-per-head wave64 single-pass kernel

authored and
LLLLKKKK
committed
perf(rocm): add fused QKV+bias+RoPE+paged-cache V3 all-heads-per-block kernel

authored and
LLLLKKKK
committed
perf(rocm): skip q_output zero-fill when kernel writes every cell

authored and
LLLLKKKK
committed
perf(rocm): hoist FMHA seqlen_k computation out of per-layer hot path

authored and
LLLLKKKK
committed
fix(smoke): add response alternative for rocm_eagle_qwen2_14b smoke test

chengshu-lcc
authored and
LLLLKKKK
committed
fix(rocm): reject Mrope at host dispatch to avoid silent wrong RoPE positions

authored and
LLLLKKKK
committed
feat: add PureCP/PureDP MoE routers with allgather+reduce_scatter

intermezzi
authored and
LLLLKKKK
committed
feat: register PureCP/PureDP MoE strategies and gate use_all_gather

intermezzi
authored and
LLLLKKKK
committed
test(smoke): add pure DP and pure CP MoE smoke cases

authored and
LLLLKKKK
committed
fix(moe): address review feedback (P2/P3, default to DeepEP, build deps)

intermezzi
authored and
LLLLKKKK
committed

Commits on May 19, 2026

feat: N-aware dispatch for FP8 PTPC GEMM — route large-M and large-N shapes to cktile

liaocz
authored and
LLLLKKKK
committed
test(rocm): add dispatch boundary tests for N-aware cktile routing

liaocz
authored and
LLLLKKKK
committed

Commits on May 20, 2026

fix(loader): inline FP8 quantization for online PTPC to fix 397B MoE OOM

authored and
LLLLKKKK
committed
test(loader): add unit tests for inline FP8 MoE quantization

authored and
LLLLKKKK
committed
perf(rocm): scatter q/k/v in GDN prefill via Triton kernel to eliminate view copies

authored and
LLLLKKKK
committed

Commits on May 21, 2026

perf(rocm): fuse GDN in_proj_qkvz + in_proj_ba into single GEMM (BF16 path)

authored and
LLLLKKKK
committed

Commits on May 22, 2026

fix(rocm): sanitize and pad paged prefill block_table for CK speculative prefetch

liaocz
authored and
LLLLKKKK
committed
fix: collect tensor names from QuantWeight sub_weights in checkpoint filter

authored and
LLLLKKKK
committed
fix(rocm): apply RoPE for embedding models without KV cache

siluzhou
authored and
LLLLKKKK
committed
perf(rocm): vectorize K/V unpad on no-cache RoPE hot path + tests

siluzhou
authored and
LLLLKKKK
committed

Commits on May 26, 2026

update aiter & add flydsl

hxy0118
authored and
LLLLKKKK
committed
fix: update qwen3 fp8 smoke test tokens due to aiter package changes

hxy0118
authored and
LLLLKKKK
committed

Commits on May 27, 2026

fixed trtallreduce bug

hxy0118
authored and
LLLLKKKK
committed
chunk-gdn flydsl

hxy0118
authored and
LLLLKKKK
committed
(v,k) layout 改动以及 beta fp32

hxy0118
authored and
LLLLKKKK
committed
fix: update smoke test expected values due to V-first layout transpose

hxy0118
authored and
LLLLKKKK
committed

Commits on May 28, 2026

feat(kimi-linear): enable tool-call protocol and add e2e smoke

theNiemand
authored and
LLLLKKKK
committed

Commits on Jun 2, 2026

Fix MTP streaming tool-call arg truncation at UTF-8 token boundaries

authored and
LLLLKKKK
committed

Commits on Jun 3, 2026

Commits on Jun 4, 2026

feat: add support of beam search on ROCm devices

zhangjianning-zjn
authored and
LLLLKKKK
committed

Commits on Jun 8, 2026

perf: ensure torch inference mode throughout engine

zhangjianning-zjn
authored and
LLLLKKKK
committed
test: add server_args for server_test

zhangjianning-zjn
authored and
LLLLKKKK
committed
fix(rocm): preserve column-major layout in GDN qkvz+ba fusion to fix swizzle core dump

authored and
LLLLKKKK
committed

Commits on Jun 9, 2026

Commits on Jun 10, 2026

feat(logits_processor): robust prefix-dict SAX parsing and DFA error handling

authored and
LLLLKKKK
committed

Commits on Jun 11, 2026

Commits on Jun 12, 2026

Commits on Jun 15, 2026

feat: fix XQA CUDA graph support and enable adaptive multi-block decode

authored and
LLLLKKKK
committed

Commits on Jun 17, 2026

feat: migrate flashinfer renorm kernel (#1027)
Vinkle-hzt
authored

Commits on Jun 21, 2026

Commits on Jun 26, 2026

feat: update to transformers and fix compatibility issues

HongminTan
authored and
LLLLKKKK
committed
feat: support glm 4.7 flash and add smoke

HongminTan
authored and
LLLLKKKK
committed
Merge remote-tracking branch 'upstream/main' into feature/python_native_v2
baohengyi
committed
fix: fix metric reporting on waiting time of generate stream

zhangjianning-zjn
authored and
LLLLKKKK
committed
fix(perf): propagate per-host errors in multi_runner.sh subshells

baohengyi
and
claude
committed
fix(moe): avoid cross-warp stale read in ep_scatter_1 prefix sum

HoniiTro19
authored and
LLLLKKKK
committed
remove test_old_kernel_reads_poison assertion

HoniiTro19
authored and
LLLLKKKK
committed
fix code formatting to comply with line width limits

HoniiTro19
authored and
LLLLKKKK
committed
refactor(attn): unify PyAttentionInputs host/device tensors with _host/_device suffixes

Jonathan-hwx
authored and
LLLLKKKK
committed

Commits on Jun 27, 2026

Commits on Jun 28, 2026

fix(review): address review batch (correctness, perf, safety)

baohengyi
and
claude
committed

Commits on Jun 29, 2026

fix(review): pickle back-compat, XQA fixes, test correctness & cleanup

baohengyi
and
claude
committed

Commits on Jul 1, 2026

Commits on Jul 2, 2026

Commits on Jul 3, 2026