Skip to content

benchmark: integrate pprof analysis into reports - #1464

Open
avinxshKD wants to merge 23 commits into
volcano-sh:mainfrom
avinxshKD:fix/benchmark-pprof-analysis
Open

benchmark: integrate pprof analysis into reports#1464
avinxshKD wants to merge 23 commits into
volcano-sh:mainfrom
avinxshKD:fix/benchmark-pprof-analysis

Conversation

@avinxshKD

@avinxshKD avinxshKD commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Depends on #1285. This change is based on StLeoX:feat/router-benchmark

What type of PR is this?

/kind feature

What this PR does / why we need it:

Integrates pprof analysis into ResultReporter.build_report().

The reporter parses collected CPU and heap profiles, resolves samples to functions, and adds the hottest scheduler/plugin functions to each configuration's pprof_analysis report section. Individual profile errors are recorded without preventing the benchmark report from being generated.

Which issue(s) this PR fixes:

Fixes #1462

Bug evidence (required for bug-related PRs):

N/A

Special notes for your reviewer:

Uses the checked-in protobuf binding generated from profile.proto. The tests use captured CPU and heap profiles from the router pprof endpoint.

Validation performed:

  • All 50 router benchmark tests pass
  • ruff check passes
  • git diff --check passes

Does this PR introduce a user-facing change?:

Integrate CPU and heap pprof analysis into router benchmark reports.

StLeoX added 22 commits July 2, 2026 16:35
…scenarios

- Use ai-dynamo/aiperf for Load Gen and dynamo mocker for Mock Backend
- Deploy mocker with kthena resource definitions
- Provide s2 scenario: routing latency under different QPS; provide s7 scenario: latency variance on heterogeneous backends; provide s8 scenario: schedule strategy comparison

Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
…others use `rate`

Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Signed-off-by: Leo Xie <stleox@qq.com>
Copilot AI review requested due to automatic review settings July 31, 2026 16:31
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@volcano-sh-bot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yaozengzeng for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@avinxshKD

Copy link
Copy Markdown
Contributor Author

@StLeoX PTAL when you get a chance. added the pprof parser with scheduler/plugin filtering, CPU and heap support, and fixture-based tests

cc @YaoZengzeng @LiZhenCheng9527

Copilot AI review requested due to automatic review settings July 31, 2026 16:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@StLeoX

StLeoX commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

I expect the Analyzer can be integrated into build_report, instead of a separate module. You can checkout this branch first.

Signed-off-by: Avinash Kumar Deepak <avinash8655279@gmail.com>
Copilot AI review requested due to automatic review settings August 2, 2026 06:30
@avinxshKD
avinxshKD force-pushed the fix/benchmark-pprof-analysis branch from a93ad6a to b8bae40 Compare August 2, 2026 06:30
@avinxshKD

Copy link
Copy Markdown
Contributor Author

@StLeoX Reworked this on top of #1285. The pprof analyzer is now integrated into ResultReporter.build_report().

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

benchmark/router-benchmark/scripts/router_ab_test/kubernetes.py:506

  • _start_port_forward() always targets "{target_type}/{self.ROUTER_DEPLOYMENT}". This happens to work today because ROUTER_DEPLOYMENT and ROUTER_SVC_NAME are both "kthena-router", but it couples service and deployment naming and will break silently if they ever diverge. Consider selecting the target resource based on target_type.
        local_endpoint = f"localhost:{local_port}"
        print(f"  Starting port-forward ({local_endpoint} → {description})")

        target = f"{target_type}/{self.ROUTER_DEPLOYMENT}"
        proc = subprocess.Popen(

Comment on lines +131 to +137
mode = str(schedule.get("mode", "")).lower()
if mode in {"rate", "constant_rate"}:
cmd.extend(["--request-rate", str(schedule.get("rate", 10))])
self._append_traffic_args(cmd, traffic, benchmark_secs)
else:
cmd.extend(["--concurrency", str(concurrency.get("connections", 10))])
self._append_concurrency_args(cmd, concurrency)
@avinxshKD avinxshKD changed the title benchmark: add pprof analysis module benchmark: integrate pprof analysis into reports Aug 2, 2026
@StLeoX

StLeoX commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

lgtm, it's stacked now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FE] [Benchmark] Add pprof analysis module in benchmark suite

4 participants