perf(v4): batch decode sampling into a single sync (+3.6-11.7% tps)

[nosyndicate]

Replaces v4's per-row decode sampling loop (up to batch host↔device .item() syncs per step) with a single batched sample_tokens call plus one .tolist() sync. Pure host-side optimization confined to ScheduleInferenceEngine (v4-only); randomness stays deterministic and batch-invariant via uniforms_from_seeds, keyed on each request's (seed, output-token count).

• Adds noise_salt: a stable per-request random seed assigned at admission for unseeded requests, so the counter-based batched noise source always has a concrete integer seed
• Refactors token-finalization (EOS/detok/max-len) into _decode_result_for_token, shared by the prefill path (_sample_one) and the new batched decode path (_post_decode)
• _post_decode now gathers per-row temperature/top_k/top_p into tensors, draws deterministic uniforms for the whole batch, and calls sample_tokens once instead of looping sample_token per sequence
• Adds test_post_decode_mixed_greedy_sampled_topk_batch, covering a batch mixing greedy, top-k=1, and top-k=2 rows

Benchmarked on RTX A6000 / Qwen3-1.7B (v4 paged attention), batch 8→64, concurrency = batch (no starvation confound):

batch	Δ throughput	Δ TPOT
8	+3.6%	−3.7%
16	+7.0%	−6.8%
32	+6.8%	−6.2%
64	+11.7%	−10.8%

Peak GPU memory identical at every batch; pytest tests/executor 135 passed (main: 134). Benefit scales with batch size (more eliminated per-row syncs).

[Copilot]

Pull request overview

This PR optimizes v4 ScheduleInferenceEngine decode-time sampling by batching per-sequence sampling into a single sample_tokens call, reducing host↔device synchronization overhead while keeping sampling deterministic and batch-invariant via uniforms_from_seeds.

Changes:

• Batch decode sampling in _post_decode using per-row parameter tensors + uniforms_from_seeds, replacing the prior per-row sampling loop.
• Introduce GenerationRequestState.noise_salt and engine-side _noise_seed() to ensure unseeded requests still have a stable per-request seed for counter-based randomness.
• Refactor token finalization into _decode_result_for_token and add a targeted unit test for mixed greedy/top-k rows in a single batched sampling step.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
server/executor/engine.py	Implements batched decode sampling (sample_tokens), adds per-request noise seeding, and centralizes token finalization logic.
server/executor/types.py	Adds noise_salt to request state to support deterministic counter-based sampling for unseeded requests.
tests/executor/test_schedule_engine.py	Adds coverage for batched _post_decode with mixed greedy and top-k sampling rows.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.