benchmark: empirically calibrate all 8 router-benchmark scenarios and detect request-level saturation by vanshika2720 · Pull Request #1472 · volcano-sh/kthena

vanshika2720 · 2026-08-01T12:13:38Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

The router benchmark scenarios were using guessed load parameters that could saturate the mocker backend before meaningfully exercising the router/plugin under test, making A/B comparison unreliable.

This PR:

Empirically calibrates all 8 router-benchmark smoke-test scenarios against the actual 4c/16g CI runner.
Replaces the previously unsafe/guessed rate, concurrency, duration, and backend configurations with values validated through repeated CI runs.
Adds router-level success-rate regressions to the CI exit-status decision.
Adds request-level saturation detection for:
- genuine AIPerf-reported errors;
- 503s not explained by end-of-window cancellations;
- an evidence-backed 90% success-rate floor.
Documents validated parameters, rejected configurations, CI run IDs, and known limitations in the scenario YAMLs and benchmark README.

The calibrated scenarios now provide a stable basis for comparing router behavior without the mocker backend becoming the primary bottleneck.

Which issue(s) this PR fixes:

Fixes #1452

Bug evidence (required for bug-related PRs):

The bug was reproduced through the actual router benchmark CI path.

The original benchmark configurations could saturate the mocker backend before the router could be meaningfully compared. In the original evidence, success rates dropped to approximately 13–41% while mocker pods remained running, with router logs showing repeated 503 proxy failures and requests being retried across backend pods.

The issue was then reproduced and characterized through repeated CI experiments on the 4c/16g runner. For example, the original s2 configuration at rate=20 on the 8-pod backend produced only 31.35% / 33.37% success for the two plugin configurations. Lowering the load to empirically calibrated values removed the saturation behavior.

The final validated configurations were established through repeated CI runs:

s1: rate=2, 4 pods
s2: rate=3, 8 pods, 250m/1CPU
s3: concurrency=5, 8 pods, 250m/1CPU
s4: rate=2, 6 pods, 250m/1CPU
s5: rate=0.5, 2048-token prompts, 8 pods, 250m/1CPU
s6: rate=0.5, 1024-token completions, 120s, 8 pods, 250m/1CPU
s7: rate=0.5, 3 heterogeneous pods
s8: rate=1, 4 heterogeneous pods

Each scenario has repeated CI evidence documented in its YAML and in the benchmark README.

The benchmark also previously had a framework-level gap where router-level degradation did not affect the CI exit status, and pod health alone was used to detect saturation. This PR closes those gaps by incorporating router-level regressions and request-level saturation signals into the benchmark verdict.

Special notes for your reviewer:

All 8 scenarios were calibrated using actual CI runs rather than theoretical/local estimates.
The validated parameters and CI run IDs are documented in the scenario YAML headers and README.
The 90% success-rate floor is based on the observed separation between the lowest validated-safe sample (93.94%) and highest confirmed-bad sample (87.88%).
s1's intended role as either a safe comparison scenario or deliberate throughput/saturation probe remains a design question and is explicitly documented.
The p95/p50 warning is known to be unreliable for the intentionally heterogeneous s7/s8 scenarios and is documented as a limitation rather than treated as a definitive saturation signal.
No router-overhead/decision-quality framework abstraction was added because the existing homogeneous and heterogeneous scenarios already provide these measurements.
Tests: 67/67 unit tests passed, 67/67 pytest tests passed, and ruff is clean.

Does this PR introduce a user-facing change?:

Yes. The benchmark behavior and CI validation are changed: unsafe/saturated benchmark runs can now be marked invalid or cause CI failure instead of producing misleading successful results.

Benchmark router scenarios are now empirically calibrated for CI resource constraints and detect request-level backend saturation.

…scenarios - Use ai-dynamo/aiperf for Load Gen and dynamo mocker for Mock Backend - Deploy mocker with kthena resource definitions - Provide s2 scenario: routing latency under different QPS; provide s7 scenario: latency variance on heterogeneous backends; provide s8 scenario: schedule strategy comparison Signed-off-by: Leo Xie <stleox@qq.com>

Signed-off-by: Leo Xie <stleox@qq.com>

…others use `rate` Signed-off-by: Leo Xie <stleox@qq.com>

Signed-off-by: Leo Xie <stleox@qq.com>

…xperiment 1 Issue volcano-sh#1452 showed the router-benchmark mockers saturating before the router/plugin comparison becomes meaningful: CI run 30509964471 measured 13.2%/40.7% success rates dominated by 503s. Later commits on this branch raised backend replicas to 8 and lowered s2/s3 load, but CI runs 30544274253 (s2, rate=20) and 30532320062 (s3, concurrency=100) still show 16-33% success with 20-50s p95 latency, so the backend is still far past capacity on the 4c/16g runner. Start an empirical calibration sweep rather than guessing further: narrow the workflow matrix to smoke-test-s3 only (concurrency mode, one job per run) to keep each CI iteration cheap, and drop concurrency from 100 to 10 as the first, most conservative point on the sweep. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Experiment 1 (concurrency=10, CI run 30619293479) showed concurrency=10 is not a safe, plugin-agnostic operating point: router-config-random hit 94.78% success (230 total, 12x503) but router-config-least-request hit only 72.55% (306 total, 84x503) at the same nominal concurrency. AIPerf's closed-loop concurrency mode issues more total requests when a config fails fast (least-request's 503s averaged 365ms vs random's 7.87s), so the two arms did not receive equivalent load even at identical concurrency. Halve concurrency to 5, changing nothing else (same s3 scenario, same 8-pod backend, same resources, same token sizes), to see whether both arms clear a safe success-rate threshold. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

s2's current rate=20 was already known to saturate this 8-pod backend (CI run 30544274253: 31.35%/33.37% success for random/least-request). s2 uses open-loop rate mode, unlike s3's closed-loop concurrency mode, so s3's validated concurrency=5 does not carry over directly - an open-loop rate keeps injecting requests on schedule regardless of whether the backend keeps up, so it saturates harder than the same nominal concurrency number. The validated s3 runs (30620977224, 30686693047) show this exact 8-pod/250m-1CPU backend sustains roughly 3-4 successfully completed requests/sec in steady state. Start the s2 sweep at rate=5, just above that observed ceiling, instead of re-probing near the already-failed 20. Narrow the workflow matrix to smoke-test-s2 only (one scenario per run, same pattern used for s3's experiments) to keep the iteration cheap. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Three identical rate=5/45s runs (CI 30687307473, 30688773166, 30689203624) showed the random arm swinging 91.87% / 96.65% / 93.30% success and least-request 98.09% / 98.56% / 95.69% - two of three random runs and the latest least-request run fell below the 95% target, despite errors=0 and 0 mocker restarts on every run. In all six samples the 503 count was bounded by AIPerf's own end-of-window cancellation count, suggesting the 45s duration's fixed cutoff is a large fraction of the measurement and may be amplifying run-to-run noise in the raw success-rate metric. Test that hypothesis directly: raise duration from 45s to 60s (matching s3's already-validated duration) and change nothing else, to see whether the raw success rate stabilizes once trailing in-flight requests are a smaller share of the total. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

… 45s Experiment 4 (duration=60s at rate=5, CI 30689646184) falsified the "45s cutoff is the whole problem" hypothesis: extending the window made both arms worse (random 77.39%, least-request 74.2%) and introduced genuine AIPerf-reported errors (19 and 53) that hadn't appeared in any of the three 45s/rate=5 runs, where errors were 0 every time. Sustained rate=5 for longer therefore looks like a real capacity problem, not a window-cutoff artifact. Revert duration to 45s (undoing experiment 4) and instead lower the offered rate from 5 to 3, to test whether reducing load rather than extending the window closes the gap. Three rate=5/45s runs oscillated around the 95% success line (random: 91.87/96.65/93.30, least-request: 98.09/98.56/95.69) without ever showing a comfortable margin on both arms simultaneously; rate=3 tests whether a lower offered rate produces a clearer margin instead of borderline noise. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

…quest-level saturation Issue volcano-sh#1452's investigation found the router-benchmark framework had three concrete gaps beyond the scenario parameters themselves: 1. reporter.py already computed a router-level success-rate regression (router_comparison) but ab_test.py's exit code only checked the AIPerf-level comparison, so a run with a severely degraded router success rate could still report CI green. 2. compute_run_verdict only detects mocker pod restarts/OOMKill/CrashLoop. Every saturated run produced during this issue's calibration sweep (s2 rate=5/60s, rate=20 baseline) showed 0 mocker restarts, so a backend can be fully saturated at the request level while the existing verdict reports "valid". 3. Nothing distinguished a benign end-of-window cancellation (the load generator cutting off in-flight requests when the fixed benchmark duration elapses) from a genuine mid-run failure, so a naive verdict would need to treat every 503 as equally suspicious. Changes: - ab_test.py: main() now also checks router_comparison for regressions, not just comparison. - load_generator.py: AIPerfRunner now sums AIPerf's own error_summary into aiperf_genuine_errors, and parses the one place AIPerf exposes its end-of-window cancellation count (the "Phase profiling complete" log line) into aiperf_cancelled - neither was read anywhere before. - models.py: new apply_request_level_verdict() layers two evidence-backed checks onto the existing pod-based verdict: genuine AIPerf errors > 0, and router 503s in excess of AIPerf's own cancellation count. A high p95/p50 tail-latency ratio is recorded as a non-fatal warning only (the evidence for a hard cutoff there is a single ambiguous sample, not a clean separator). Thresholds and their evidence are documented inline and in the accompanying investigation. - reporter.py: build_report() now applies the new verdict checks before compare(), so a request-level-saturated run correctly skips the A/B comparison via the existing invalid-run gate (issue volcano-sh#1271). Warnings are surfaced in the printed report alongside reasons. - Documents the two operating points empirically validated during this issue's CI experiments (s2 rate=3/45s, s3 concurrency=5) directly in their scenario YAML comments and in a new README section, including the rejected alternatives and why, so future scenario tuning starts from evidence instead of guessing again. s1, s4-s8 are intentionally not recalibrated in this change - they still need the same empirical treatment before their current values can be trusted. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

s2/s3 are validated (issue volcano-sh#1452); s1 and s4-s8 have not been through the same empirical process and their existing rate values should not be assumed safe. Run all six as parallel matrix jobs in one CI trigger instead of six sequential dispatches: - s1: rate left at its current 75 (unchanged) specifically to get a real CI data point rather than assume it's unsafe without evidence. - s4: rate 50 -> 2. Shares s2/s3's validated resource shape (250m/1CPU), just with 6 pods instead of 8. Proportional starting point from s2's validated ceiling (8 pods safe at rate=3): 6 pods -> ~2.25, rounded down to 2. - s5: rate 30 -> 1. Default (unshrunk) resource shape, 4 pods, and 2048-token prompts (4x the validated baseline) - no transferable safe- rate data exists for this workload, so this starts at the same conservative floor s2's search began from. - s6: rate 20 -> 1. Default resources, 4 pods, 1024-token completions (8x baseline) - output tokens are typically the more compute-expensive side of generation, so this is a conservative starting point for what is likely the single most expensive scenario per request. - s7: rate 50 -> 1. Only 3 total pods (heterogeneous fast/normal/slow), default resources - fewest pods of any scenario. - s8: rate 50 -> 1. 4 total pods (heterogeneous), default resources. None of these are claimed safe yet. This is the first data point in an iterative search, same methodology used for s2/rate=3 and s3/concurrency=5. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Round 1 calibration of s1/s4-s8 (issue volcano-sh#1452, CI run 30693603161) surfaced a failure mode the genuine-error and unexplained-503 checks added in 8a85a89 both miss: smoke-test-s6's least-request arm had 0 genuine AIPerf errors and its 10 503s were fully covered by 12 end-of-window cancellations, so neither check fired - yet the run was clearly unhealthy (81.48% success, p50=19.8s). The whole latency distribution had shifted up together rather than producing a fast median with a stretched tail, so the p95/p50 ratio check (1.47x) stayed well under its warning threshold too. Add an absolute success-rate floor (90.0%) as a third, coarser check. This is now evidence-backed by more than the original s2 data: every validated-safe sample collected for this issue stayed >=95%, and every confirmed-bad sample (s2 rate=20's 31-33%, s2 rate=5/60s's 74-77%, s1's 5-73%, s6's 81.48%) sat well under 90% - a gap wide enough to place a floor in without it being an arbitrary guess. Like the other checks, missing success_rate_pct is treated as unknown, not passing. This required updating three tests that relied on the shared _PROM_FIXTURE (80% success, chosen for exercising the metrics-parsing paths, not meant to represent a healthy run) to use a new _PROM_FIXTURE_SAFE (96% success) where the test's actual intent is to demonstrate valid-run behavior. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Round 1 (CI run 30693603161) results: - s1 (rate=75): catastrophic, confirmed unsafe (random 73.33%/9 total higher, least-request 5.24% with 1175/1240 503s). - s4 (rate=2): 100%/100% success both plugins, 0 errors, 0 restarts. - s5 (rate=1): still unsafe even at this floor - 19/26 genuine AIPerf errors, p95 26-29s. 2048-token prompts are expensive enough that the most conservative starting point still saturates. - s6 (rate=1): random arm invalid (7 genuine errors); least-request arm formally valid but with p50=19.8s - this is what motivated the new absolute success-rate floor added in 36afef4. - s7 (rate=1): random arm valid with a tail-latency warning (5.6x); least-request arm invalid (2 genuine errors). - s8 (rate=1): both arms formally valid, but both show prominent tail-latency warnings (13.4x, 5.9x) well beyond the one ambiguous historical sample that justified that threshold - not confident calling this safe on a single run with that signal present. Round 2: - s1: rate 75 -> 2, a fresh probe on this topology (4 pods, default resources) rather than continuing from the catastrophic rate=75. - s4: rerun at the same rate=2 for the same stability-run standard used to validate s2/s3. - s5: rate 1 -> 0.5, since rate=1 itself produced genuine errors. - s6: rate 1 -> 0.5, same reasoning. - s7: rate 1 -> 0.5, since least-request already failed at rate=1. - s8: rerun at the same rate=1 to see whether the tail-latency warnings persist or were a one-off. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Round 2 (CI run 30694269799): - s1 (rate=2): clean pass, 94.9%/97.96% success, 0 errors/restarts, latency identical in shape to every other validated point. One sample. - s4 (rate=2, rerun): confirmed, 96.72%/100% (vs round 1's 100%/100%). Two consistent runs -> validated, same standard as s2/s3. - s5 (rate=0.5): still failing on the original 4-pod default-resource shape - config_a has 3 genuine errors, config_b's 87.88% is caught by the new success-rate floor. Even 0.5 req/s saturates 2048-token prompts on this topology. - s6 (rate=0.5): both arms caught by the floor (81.25%/78.12%) on the same topology. - s7 (rate=0.5): both arms 100% success, 0 errors, but with large tail-latency warnings (5.3x, 40.0x). This is consistent with s7's heterogeneous pods by design (fast=10x speedup, slow=0.2x speedup) - config_b's p50=222ms (fast pod) vs p95=8.9s (slow pod) is the scenario working as intended, not a saturation signal the ratio check should be trusted on for heterogeneous scenarios. One sample. - s8 (rate=1, rerun): confirmed, 98.36%/98.36% (vs round 1's 96.72%/96.72%), same recurring tail-latency warnings both times (13.4x/5.9x -> 11.4x/6.2x) - consistent with the same heterogeneity explanation as s7. Two consistent runs -> validated, with that caveat documented. Round 3: rerun s1 and s7 at their unchanged rates for the second stability sample. For s5/s6, apply the resource shape that already fixed s2/s3/s4 (8 pods at 250m/1CPU instead of 4 default-resource pods) rather than continuing to lower an already-very-low rate - the original 4-pod shape failed even at 0.5 req/s, so the bottleneck looks structural (guaranteed CPU too concentrated across too few pods) rather than purely a rate problem. Narrow the matrix to these four; s4/s8 are done. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Round 3 (CI run 30695729677): - s1 (rate=2, rerun): 95.92%/96.94% success, 0 genuine errors - matches round 2's 94.9%/97.96% closely. Two consistent runs -> validated. (Artifact upload hit a transient ECONNRESET; numbers recovered from the job's own stdout log - "Run A/B Test" step itself succeeded.) - s5 (rate=0.5, new 8-pod/250m-1CPU shape): fixed - 93.94%/96.97% success, 0 genuine errors, no warnings. Confirms the topology fix (more, smaller pods, same shape validated for s2-s4) was the right lever, not further rate cuts. One clean sample. - s6 (same new shape): still failing, 81.25%/81.25% both invalidated by the floor. But cancelled >= 503 both times (not "unexplained"), and p50/p95 (7-28s) are a large fraction of the 60s duration - a structural symptom of individual-request latency being long relative to the window, not necessarily load-level saturation. - s7 (rate=0.5, rerun): 96.77%/100%, 0 genuine errors, consistent heterogeneity-driven tail-latency warnings (7.6x/34.3x vs round 2's 5.3x/40.0x). Two consistent runs -> validated, with that caveat. Round 4: s5 stability rerun at the same params. s6 duration 60s -> 120s (same rate/shape) to test the "window too short for this scenario's inherent per-request latency" hypothesis before cutting rate further. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

Rounds 1-5 of issue volcano-sh#1452 calibration validated all six remaining smoke-test scenarios via repeated real CI runs: - s1: rate=75 -> 2 (4 pods, default resources). CI 30694269799, 30695729677. - s4: rate=50 -> 2 (6 pods, 250m/1CPU). CI 30693603161, 30694269799. - s5: rate=30 -> 0.5, plus the 250m/1CPU/8-pod shape already validated for s2-s4 (was 4 default-resource pods) - the topology, not just the rate, was the fix. CI 30695729677, 30696222084, 30696682560 (one run had a single isolated genuine error, not a recurring pattern). - s6: rate=20 -> 0.5, same 8-pod/250m-1CPU shape, plus duration 60s -> 120s. Unlike s5, the topology fix alone did not help here - success stayed at 81.25%/81.25% until duration was extended, because 1024- token completions' 7-28s latency was a large enough fraction of the original 60s window that the harness's own end-of-window cutoff was inflating the failure count independent of real saturation. CI 30696222084, 30696682560. - s7: rate=50 -> 0.5 (3 heterogeneous pods, default resources). CI 30694269799, 30695729677. - s8: rate=50 -> 1 (4 heterogeneous pods, default resources). CI 30693603161, 30694269799. s7/s8 show large p95/p50 tail-latency warnings on every validated run (5.3x-40.0x) - documented as expected given their deliberately mismatched pod speeds (10x/1x/0.2x), not a saturation signal. The tail-latency check in apply_request_level_verdict was calibrated on homogeneous scenarios and is not a reliable signal for heterogeneous ones; success rate and the genuine-error/cancelled checks are. All eight smoke-test scenarios now have empirically validated parameters. The workflow matrix defaults to running s2/s3 on every triggering PR to keep routine CI cost bounded; s1/s4-s8 are left commented but ready to enable or dispatch manually without re-guessing their values. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

…eats in README Pre-PR review found smoke-test-s1.yaml's own comment contradicted itself: it cited two CI runs (30694269799, 30695729677) validating rate=2, then two lines later said "rate=2 has only been validated once here". Fix the wording to reflect the actual 2-run evidence and keep the caveat scoped to what's actually unresolved (s1's design intent, not the number of runs). Also surface the s1 design-intent caveat and s5's one-anomalous-run note directly in the README (previously only in the per-scenario YAML comments), so they aren't only visible to someone reading the raw scenario files. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

gemini-code-assist · 2026-08-01T12:13:41Z

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

volcano-sh-bot · 2026-08-01T12:13:44Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lizhencheng9527 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

StLeoX · 2026-08-03T02:28:19Z

Those scenario arguments seem reasonable, and the runs/30699272001 worked. lgtm

StLeoX · 2026-08-03T02:36:59Z

I don't think you should inlcude yourself among these commit authors, like fe6e083

vanshika2720 · 2026-08-03T05:40:09Z

@StLeoX Those commits had my Git identity as the committer because they were created on my branch. I rewrote the metadata to correctly show you as author/committer and force-pushed the corrected history.

LiZhenCheng9527 · 2026-08-03T07:45:31Z

Why does your PR need to include someone else's commit?

LiZhenCheng9527 · 2026-08-03T07:48:46Z

Have you tried triggering the newly added GitHub Action? Can you display the results?

StLeoX · 2026-08-03T08:12:20Z

It's on the top of #1285, frist 1285 then 1472, 1464

… non-fatal in verdict logic This is a benchmark-verdict classification fix, not a scenario calibration change - no scenario YAML, workflow, or README was touched, and s3's parameters (concurrency=5, 8 pods, 250m/1CPU) are unaffected. Evidence-gathering for PR volcano-sh#1472's smoke-test-s3 failure (issue volcano-sh#1452) showed AIPerf's InvalidInferenceResultError, specifically its "No responses with actual content were received from the server" note, occurs at a bounded 0-1.61% rate across every one of 12 concurrency-mode (s3) config-arm samples collected across 6 independent CI runs, and in zero of 8 rate-mode (s2) samples across 4 runs. Every occurrence found in the router's own access log was a severe latency outlier (6-8x that run's p50, several beyond its p99 bucket) - HTTP 200, zero completion tokens. The existing hard-invalidating genuine_errors check does not distinguish this from a real failure, so validated s3 runs fail CI intermittently on this alone. Add classify_aiperf_errors() (models.py), which splits AIPerf's error_summary into an invalidating count and an empty-content count by matching on error type and note text (currently only exposed embedded in error_summary's message field). Only InvalidInferenceResultError carrying the empty-content note, and not also a timestamp-invalid note, is classified as empty-content; every other error, including InvalidInferenceResultError with a different note, remains invalidating exactly as before. apply_request_level_verdict gets one new, additive check: a nonzero empty_content_errors count is recorded as a warning, never added to offenders/reasons. The existing genuine_errors/cancelled/success-rate- floor/tail-latency-ratio logic is untouched. load_generator.py now calls classify_aiperf_errors instead of blindly summing error_summary, producing aiperf_genuine_errors (now the classified, non-exempt count) and a new aiperf_empty_content_errors. reporter.py passes the new field through to request_stats. This does not establish the condition is universally harmless - the comment next to the matching constants documents the actual evidence and its scope explicitly. Refs volcano-sh#1452 Signed-off-by: Vanshika <pahalvanshikaa@gmail.com>

vanshika2720 · 2026-08-04T14:46:13Z

@LiZhenCheng9527 The PR includes those commits because #1472 is built on top of #1285, as @StLeoX mentioned. The commits are part of the existing branch history, not duplicated work introduced by this PR.

I also corrected the commit metadata so StLeoX is properly credited as author/committer for his commits.

I triggered the newly added GitHub Actions as requested. The benchmark checks passed: s2 , s3 , along with build, tests, lint, licenses, and DCO. No conflicts remain.

StLeoX and others added 30 commits August 1, 2026 17:31

refactor: spilt python modules in router_ab_test

278005a

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Add metrics_collector module

6cd4282

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Support router endpoint: 1.pf, 2.lb

e699bc3

Signed-off-by: Leo Xie <stleox@qq.com>

chore: Add mocker-deployment-local-hf.yaml

8f5b2a8

Signed-off-by: Leo Xie <stleox@qq.com>

fix: Fix copilot reviews

7601d71

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Remove routing in scenario

c2d230d

Signed-off-by: Leo Xie <stleox@qq.com>

feat: implement ramp-up strategy

063f329

Signed-off-by: Leo Xie <stleox@qq.com>

feat: redesign scenario data model, implement DeploymentBuilder

d02378d

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Cold-start between a/b, uses requests for ping

c04f902

Signed-off-by: Leo Xie <stleox@qq.com>

fix: Remove hard-code model name, bind with scenario data

2d85deb

Signed-off-by: Leo Xie <stleox@qq.com>

feat: ab_test support dry-run mode

a677e63

Signed-off-by: Leo Xie <stleox@qq.com>

chore: Add requirement.txt

585a475

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Support resolving k8s resources requests

a0284c6

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Support full scenarios

f5bb5e7

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Add verdict for backend restarts after traffic arrival

eada067

Signed-off-by: Leo Xie <stleox@qq.com>

refactor: Separate two load models of aiperf, s3 uses concurrency, …

84fc32b

…others use `rate` Signed-off-by: Leo Xie <stleox@qq.com>

chore: Workflow enable s3

5af731f

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Deep insight plugin-specific metrics

971f49c

Signed-off-by: Leo Xie <stleox@qq.com>

test: Relax load-backend constrain in s2 and s3

14813a7

Signed-off-by: Leo Xie <stleox@qq.com>

test: Testing least-request plugin

0f7b2ea

Signed-off-by: Leo Xie <stleox@qq.com>

feat: Make pprof collection async

c85539f

Signed-off-by: Leo Xie <stleox@qq.com>

vanshika2720 added 5 commits August 1, 2026 17:31

volcano-sh-bot added the kind/bug label Aug 1, 2026

volcano-sh-bot requested review from LiZhenCheng9527 and hzxuzhonghu August 1, 2026 12:13

volcano-sh-bot added the size/XXL label Aug 1, 2026

vanshika2720 force-pushed the feat/router-benchmark branch from e541dd9 to b9b7d10 Compare August 3, 2026 05:34

Navigation Menu

Uh oh!

benchmark: empirically calibrate all 8 router-benchmark scenarios and detect request-level saturation - #1472

benchmark: empirically calibrate all 8 router-benchmark scenarios and detect request-level saturation#1472
vanshika2720 wants to merge 36 commits into
volcano-sh:mainfrom
vanshika2720:feat/router-benchmark

vanshika2720 commented Aug 1, 2026

Uh oh!

gemini-code-assist Bot commented Aug 1, 2026

Uh oh!

volcano-sh-bot commented Aug 1, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

vanshika2720 commented Aug 3, 2026

Uh oh!

LiZhenCheng9527 commented Aug 3, 2026

Uh oh!

LiZhenCheng9527 commented Aug 3, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

vanshika2720 commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

vanshika2720 commented Aug 1, 2026

Uh oh!

gemini-code-assist Bot commented Aug 1, 2026

Uh oh!

volcano-sh-bot commented Aug 1, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

vanshika2720 commented Aug 3, 2026

Uh oh!

LiZhenCheng9527 commented Aug 3, 2026

Uh oh!

LiZhenCheng9527 commented Aug 3, 2026

Uh oh!

StLeoX commented Aug 3, 2026

Uh oh!

vanshika2720 commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants