Skip to content

Our c1 arm has a reproducible ~4 s TTFT outlier on request 3 of every leg; the pinned oracle does not #1381

Description

@localai-bot

Our c1 arm has a reproducible ~4 s TTFT outlier on request 3 of every leg that the pinned oracle does not have.

Found 2026-08-19 in the raw vllm bench serve --save-detailed files of the Qwen3.8-27B bf16 c1/c8 re-measure (#915, #979), out/bench-20260819T035148Z/.

The measurement

Concurrency is 1, so the six requests are strictly serialized. Read straight out of the ttfts array of each leg (seconds):

leg r1 r2 r3 r4 r5 r6
warmup 77.005 0.727 3.981 0.850 0.845 0.839
c1-r1 0.738 0.826 3.924 0.844 0.905 0.848
c1-r2 0.729 0.808 4.006 0.915 0.925 0.852
c1-r3 0.729 0.836 3.955 0.890 0.892 0.890

Four legs of four, always the same index. Every other request in the same legs is 0.727-0.925 s.

The pinned oracle 0.1.dev1+g555967922, on the byte-identical client invocation, same box and same lease (out/vllm-20260819T095758Z/), has no such point:

leg r1 r2 r3 r4 r5 r6
warmup 43.767 0.838 0.842 0.852 0.846 0.878
c1-r1 0.838 0.876 0.851 0.902 0.897 0.877
c1-r2 0.834 0.941 0.868 0.874 0.865 0.917
c1-r3 0.843 1.015 0.905 0.897 0.858 0.883

18 requests across three legs, every TTFT between 0.834 and 1.015 s.

What does not explain it

  • Not prompt length. Request 3 carries a 1024-token prompt exactly as requests 4, 5 and 6 do.
  • Not first inference. The 77.005 s first value in our warmup leg is the separate, already-known first-inference cost behind a liveness-only /health; the oracle pays its own (43.767 s) at the same index. The outlier is at index 2, three requests later, and recurs in every subsequent leg after the server is fully warm.
  • Not noise. A fixed request index reproducing in four legs of four is a behaviour.

What is and is not affected

Nothing published is wrong. The repository quotes the MEDIAN and labels it, and the median of six averages ranks three and four, which the outlier never occupies.

It moves the MEAN — ours 1347.6 / 1372.6 / 1365.4 ms against vLLM's 873.3 / 883.4 / 900.2 ms, while the medians read 883.78 against 876.4 — and it costs wall time: request 4 starts 31.63 s after request 3 where every other gap is ~28.4 s, so roughly 3.1 s of the 174.39 s c1 wall.

Why this is filed rather than left in the record

A reproducible outlier at a fixed request index is a behaviour rather than noise, and the record that mentions it is a benchmark row that writes no product code and holds no GPU. The cause is deliberately not chased here.

Record defect this also corrects

The finding above was appended to .agents/issue-index.md under the link #1365, in 9e1a5e573 (#1369). #1365 is a different issue — the O(n^2) BPE merge loop — so that landed row anchors this finding to unrelated work, and the index carries no row for the BPE issue's own number. The row landed on origin/main, so the append-only rule forbids editing it in place (the #979 precedent permits in-place correction only before the row lands). The correction is therefore an appended row naming this issue as the finding's real anchor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions