Skip to content

[test][charts] Port the benchmarks to the benchmark CLI - #24028

Open
Janpot wants to merge 18 commits into
mui:masterfrom
Janpot:janpot/benchmark-cli
Open

Janpot wants to merge 18 commits into
mui:masterfrom
Janpot:janpot/benchmark-cli

Conversation

@Janpot

@Janpot Janpot commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Moves the benchmarks from Vitest to the benchmark CLI of @mui/internal-benchmark (mui/mui-public#1898), as in mui/material-ui#49329, mui/base-ui#5974 and mui/base-ui-charts#1186.

Today a pull request is compared against a master run from another machine, so unrelated changes alarm (#23939, an ARIA role: SparkLineChart +186%). The CLI builds both sides in the same job and samples them in alternating rounds in one browser.

  • test/performance-charts becomes test/benchmark (pnpm test:benchmark). Cases live in src/<product>/scenarios.tsx; the charts' in src/charts/<Chart>/scenarios.tsx, unchanged.
  • src/smoke/x.bench.tsx runs all of MUI X in one page: every chart case, plus a few for Data Grid, Pickers, Tree View, Scheduler and Chat. 30 rounds each, alarming past 10%.
  • CI (test_benchmark, was test_charts_benchmark) runs on every master commit against its parent, and on pull requests changing packages/x-* (but x-codemod) against the fork point. About 7 min on medium+; the baseline's build is cached.
  • WebGL benchmarks no longer wait a fixed 300 ms: the paint of the frame that submits the first draw is recorded as bench:paint#webgl.

Pro and Premium components render with the license watermark on both sides: packed builds reject the test keys.

Run the charts benchmarks with @mui/internal-benchmark's benchmark CLI, which builds both sides in the same job and compares them on paired rounds. Pull requests run a smoke file with one case per kind of chart.
@code-infra-dashboard

code-infra-dashboard Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Deploy preview

https://deploy-preview-24028--material-ui-x.netlify.app/
QR code for https://deploy-preview-24028--material-ui-x.netlify.app/

Bundle size

Bundle Parsed size Gzip size
@mui/x-data-grid 0B(0.00%) 0B(0.00%)
@mui/x-data-grid-pro 0B(0.00%) 0B(0.00%)
@mui/x-data-grid-premium 0B(0.00%) 0B(0.00%)
@mui/x-charts 0B(0.00%) 0B(0.00%)
@mui/x-charts-pro 0B(0.00%) 0B(0.00%)
@mui/x-charts-premium 0B(0.00%) 0B(0.00%)
@mui/x-date-pickers 0B(0.00%) 0B(0.00%)
@mui/x-date-pickers-pro 0B(0.00%) 0B(0.00%)
@mui/x-tree-view 0B(0.00%) 0B(0.00%)
@mui/x-tree-view-pro 0B(0.00%) 0B(0.00%)
@mui/x-scheduler 0B(0.00%) 0B(0.00%)
@mui/x-scheduler-premium 0B(0.00%) 0B(0.00%)
@mui/x-chat 0B(0.00%) 0B(0.00%)
@mui/x-license 0B(0.00%) 0B(0.00%)

Details of bundle changes

Performance

No regressions in 35 benchmarks · ran 7m 55s · details


Check out the code infra dashboard for more information about this PR.

@github-actions github-actions Bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Oct 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

The WebGL benchmarks waited at least 300ms per sample, and their render and paint metrics ended before the chart drew. Observe the draw calls instead: after a canvas's first draw pass, a marker's paint is recorded as bench:paint#webgl, and a fence times the GPU's work, without blocking, as webgl:gpu.
In CI the GPU time is most likely software rendering's, which doesn't track a real GPU, and it varies too much from sample to sample to alarm on, while waiting for it took most of each WebGL sample.
# Conflicts:
#	.circleci/config.yml
#	pnpm-lock.yaml
@github-actions github-actions Bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged. label Oct 28, 2026
Master runs every chart's benchmark file; a pull request runs the smoke file and the files of the charts whose source folder it changes. The smoke cases keep their names, so they read as the same benchmarks as in their chart's file. Also removes the Vitest setup files the port left behind.
@github-actions github-actions Bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Oct 29, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

# Conflicts:
#	pnpm-lock.yaml
#	test/performance-charts/package.json
@github-actions github-actions Bot added PR: out-of-date The pull request has merge conflicts and can't be merged. and removed PR: out-of-date The pull request has merge conflicts and can't be merged. labels Oct 29, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@Janpot Janpot added the scope: code-infra Involves the code-infra product (https://www.notion.so/mui-org/5562c14178aa42af97bc1fa5114000cd). label Nov 12, 2026
# Conflicts:
#	.circleci/config.yml
#	pnpm-lock.yaml
#	test/benchmark/package.json
#	test/benchmark/src/charts/Heatmap/scenarios.tsx
#	test/benchmark/src/charts/HeatmapPremium/scenarios.tsx
@github-actions github-actions Bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged. label Nov 12, 2026
@Janpot Janpot added the type: enhancement It’s an improvement, but we can’t make up our mind whether it's a bug fix or a new feature. label Nov 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: code-infra Involves the code-infra product (https://www.notion.so/mui-org/5562c14178aa42af97bc1fa5114000cd). type: enhancement It’s an improvement, but we can’t make up our mind whether it's a bug fix or a new feature.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant