[v1.0.0] feat(perf): add wp-tooling perf CLI, setup/perf scaffold, and xhprof normalization by Adi-ty · Pull Request #37 · rtCamp/wp-tooling

Adi-ty · 2026-07-15T04:49:57Z

What this PR does

Implements wp-tooling perf — a zero-runtime-dep CLI runner that collects Core Web Vitals (web-vitals + Lighthouse) and optional server-side xhprof profiling per URL, with sensible degradation for every failure mode
Adds setup/perf scaffold so consumers can bootstrap config, npm scripts, and the PHP xhprof shim in one command
Normalises xhprof output (ct/wt/cpu/mu/pmu) into the report's server section alongside frontend metrics, with a fidelity note explaining CLI-context limitations

Closes

Closes https://github.com/rtcamp/wp-devtools/issues/22
Closes https://github.com/rtcamp/wp-devtools/issues/36
Closes https://github.com/rtcamp/wp-devtools/issues/37

Changes

Perf runner: per-URL collector that runs web-vitals under headless Chromium, Lighthouse, and optional xhprof server profiling — each layer degrades independently on failure. Config resolution with built-in DEFAULTS and section-level merge, supporting --url override and optional config files. Binary/module resolution walks consumer node_modules with npx --no-install fallback, never fetching from registry. Normalisation pipeline rates metrics against CWV thresholds, extracts Lighthouse scores + failing audits, builds human-readable assessment lines from a single THRESHOLDS registry. CLI layer with dry-run plan printer, text/json emitter, exit codes 0/1/2/3 matching the a11y convention, and a RunnerError hierarchy with machine-readable codes.
Setup/perf scaffold: scaffold manifest with six configurable inputs (base URL, page paths, server enable toggle, WP-CLI env-cwd path). Minimal .perfrc.json template emitting only URLs and server settings — all other sections fall through to built-in defaults, preventing silent drift. Server-enabled boolean separated from env-cwd path so profiling the WordPress root (--env-cwd=.) is expressible. Hardened PHP shim that detects xhprof/tideways backends, degrades gracefully to [] when the profiler is absent, unhooks redirect_canonical to prevent early exit(), installs a shutdown-fallback emitter with buffer drain, populates $_GET and REQUEST_URI from the positional path argument, and writes a route diagnostic to stderr.
xhprof normalisation: WP-CLI invocation layer that spawns the shim with correct argument order, parses JSON output with preamble tolerance, and captures the stderr diagnostic — every failure degrades to { data: null, error } instead of throwing. URL splitter separating origin from path+query for the shim's two-argument contract. Normaliser maps raw xhprof function data into the report's server section with ct/wt/cpu/mu/pmu fields and a fidelity note explaining CLI-context limitations. Server layer wired into the per-URL collector independently of the browser layer — still profiles even when the frontend scan failed.

How I verified

npm run check

Fixture testing for all exit codes and degradation paths. Scaffold render tests confirm default and custom config output. Also tested end-to-end by running wp-tooling perf against the features skeleton plugin.

Acceptance criteria

Runtime behavior

wp-tooling perf --dry-run resolves and prints the plan without executing anything.
A JSON report for two URLs validates the shape above; exit codes match a11y (0/1/2/3).
The engine core has no TTY-UI dependency (runs headless/CI/AI).
wp-tooling add setup/perf yields a runnable test:perf on a consumer, with the dev deps surfaced as developer actions (not auto-installed).
wp eval-file server-profile.php --url=... --top=15 --format=json returns top-N JSON when the extension is present, and [] when it is not.
No file overwrite, no package-manager execution, no secret values.
A perf report from a consumer run carries both the frontend and the server layer per URL.
The fidelity note is present in the report.
Fixture tests for the normalizer are green (frontend-only, server-only, and combined).

Code quality

Lint — zero errors
Static analysis — zero errors
Tests pass

Housekeeping

CHANGELOG.md entry under ## Unreleased
.claude/issues/<N>-<slug>.md updated with final state

_{Full decision log and verification history for this work lives in .claude/issues/wp-devtools-22-36-37-perf-runner.md on this branch.}

…ver xhprof) Mirrors the a11y runner: two-layer normalized report (lab Core Web Vitals + Lighthouse, plus optional server-side xhprof hotspots over WP-CLI). Ships src/perf/*, the perf CLI command, package.json exports, tests, the setup/perf scaffold with a hardened server-profile.php shim, and CHANGELOG/issue-tracking entries. Includes two pre-existing lint-gate fixes (no-shadow + prettier in src/init/index.js and tests/ui/selects.test.js) needed for npm run check to pass, matching the fix already on the a11y branch.

… retention - server-profile: catch malformed URL instead of throwing - run.js: skip Lighthouse when scanError set; extractLighthouse immediately after run; show server.error in text output; guard dry-run against string command - normalize.js: derive METRIC_NAMES from THRESHOLDS - collect-vitals.js: import METRIC_NAMES from normalize - scaffold: split server_enabled from server_env_cwd - test: de-flake dry-run assertion

No safe default for server_env_cwd — wrong guess silently misroutes the server layer. Malformed config now exits 2 (usage error, not run failure).

Copilot

Pull request overview

Adds a two-layer performance CLI combining browser metrics, Lighthouse, and optional server-side XHProf profiling.

Changes:

Implements performance collection, normalization, reporting, and CLI handling.
Adds the setup/perf consumer scaffold and PHP profiling shim.
Adds comprehensive fixtures and Jest coverage.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`node-packages/wp-tooling/tests/scaffolds/bundled-manifests.test.js`	Tests scaffold rendering.
`node-packages/wp-tooling/tests/perf/server-profile.test.js`	Tests WP-CLI profiling.
`node-packages/wp-tooling/tests/perf/resolve-module.test.js`	Tests consumer module resolution.
`node-packages/wp-tooling/tests/perf/normalize.test.js`	Tests report normalization.
`node-packages/wp-tooling/tests/perf/lighthouse.test.js`	Tests Lighthouse invocation.
`node-packages/wp-tooling/tests/perf/fixtures/xhprof.json`	Provides XHProf fixture data.
`node-packages/wp-tooling/tests/perf/fixtures/web-vitals.attribution.iife.js`	Provides web-vitals fixture script.
`node-packages/wp-tooling/tests/perf/fixtures/partial.perfrc.json`	Provides partial configuration fixture.
`node-packages/wp-tooling/tests/perf/fixtures/malformed.perfrc.json`	Provides malformed configuration fixture.
`node-packages/wp-tooling/tests/perf/fixtures/lighthouse-lhr.json`	Provides Lighthouse fixture data.
`node-packages/wp-tooling/tests/perf/fixtures/.perfrc.no-urls.json`	Covers empty URL configuration.
`node-packages/wp-tooling/tests/perf/fixtures/.perfrc.json`	Provides complete configuration fixture.
`node-packages/wp-tooling/tests/perf/config.test.js`	Tests configuration resolution.
`node-packages/wp-tooling/tests/perf/collect-vitals.test.js`	Tests browser metric collection.
`node-packages/wp-tooling/tests/perf/cli.test.js`	Tests CLI behavior and exit codes.
`node-packages/wp-tooling/src/perf/server-profile.js`	Invokes server profiling through WP-CLI.
`node-packages/wp-tooling/src/perf/run.js`	Orchestrates layers and CLI output.
`node-packages/wp-tooling/src/perf/resolve-module.js`	Resolves consumer-installed modules.
`node-packages/wp-tooling/src/perf/resolve-bin.js`	Resolves consumer binaries.
`node-packages/wp-tooling/src/perf/normalize.js`	Builds normalized performance reports.
`node-packages/wp-tooling/src/perf/lighthouse.js`	Runs Lighthouse scans.
`node-packages/wp-tooling/src/perf/index.js`	Exposes the performance API.
`node-packages/wp-tooling/src/perf/errors.js`	Defines structured runner errors.
`node-packages/wp-tooling/src/perf/config.js`	Loads and merges performance configuration.
`node-packages/wp-tooling/src/perf/collect-vitals.js`	Collects lab web-vitals metrics.
`node-packages/wp-tooling/src/cli/commands/perf.js`	Registers the `perf` command.
`node-packages/wp-tooling/scaffolds/setup/perf/templates/server-profile.php`	Implements the PHP profiling shim.
`node-packages/wp-tooling/scaffolds/setup/perf/templates/.perfrc.json.mustache`	Generates consumer configuration.
`node-packages/wp-tooling/scaffolds/setup/perf/scaffold.json`	Defines scaffold inputs and actions.
`node-packages/wp-tooling/package.json`	Exports the performance API.
`node-packages/wp-tooling/CHANGELOG.md`	Documents the new functionality.
`.claude/issues/wp-devtools-22-36-37-perf-runner.md`	Records implementation decisions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…t rendering, report gaps

Copilot

Pull request overview

Copilot reviewed 35 out of 35 changed files in this pull request and generated 8 comments.

Copilot

Pull request overview

Copilot reviewed 35 out of 35 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

node-packages/wp-tooling/src/perf/config.js:39

Using npx wp-env without --no-install allows npm to download @wordpress/env when it is absent locally, violating the runner's no-fetch behavior and potentially prompting or modifying the npm cache in CI. Make the fallback explicitly local-only.

		command: ['npx', 'wp-env', 'run', 'cli', '--env-cwd=.', '--', 'wp'],

node-packages/wp-tooling/scaffolds/setup/perf/templates/.perfrc.json.mustache:10

The generated config likewise invokes npx wp-env without --no-install, so running the optional server layer can fetch @wordpress/env instead of degrading when it is not installed. Include the no-install flag here and update the scaffold expectations accordingly.

		"command": ["npx", "wp-env", "run", "cli", "--env-cwd={{server_env_cwd}}", "--", "wp"]

node-packages/wp-tooling/src/perf/run.js:311

The default text report drops both server.note and server.diagnostic. As a result, users do not see the required CLI-context fidelity warning or the captured route diagnostic unless they choose JSON output; emit both fields when present.

			lines.push(`  server top: ${top || 'none'}`);

…, server resilience - read attribution.target (not element) for web-vitals v5; use ?? for settleMs/timeoutMs - skip malformed xhprof entries instead of aborting the entire perf run - unwind output buffers to pre-render level in server-profile.php - tryParse iterates [/{ candidates past preambles; non-zero exit checked before parsing - dry-run uses resolveBin so lighthouse --version is never spawned - pre-render script templates before file writes; validate checks scripts alongside files - emit server fidelity note in default text output

Copilot

Pull request overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 3 comments.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

node-packages/wp-tooling/src/perf/normalize.js:101

A configured topAudits: 0 is replaced with 5 because 0 is falsy, so consumers cannot request zero audit details while retaining Lighthouse scores. Use nullish defaulting so the documented maximum of zero is preserved.

	const topAudits = options.topAudits || 5;

node-packages/wp-tooling/src/perf/run.js:286

failedUrls also includes a vitalsError for pages that loaded successfully but returned no metrics, so this summary can incorrectly say those URLs “failed to load.” Use wording that covers both navigation and metric-collection failures.

This issue also appears on line 495 of the same file.

	const failed =
		summary.failedUrls > 0 ? `, ${summary.failedUrls} failed to load` : '';

node-packages/wp-tooling/src/perf/run.js:498

This diagnostic is also emitted for vitalsError, where the page did load (and Lighthouse may have run). Saying every failed URL “failed to load” obscures the actual metric-harvest failure reported in the per-URL notes.

	if (report.summary.failedUrls > 0) {
		process.stderr.write(
			`perf: ${report.summary.failedUrls} URL(s) failed to load — treating as a run failure.\n`
		);

+		const candidate = path.join(dir, 'node_modules', '.bin', binName);
+		if (fs.existsSync(candidate)) {


+	const command = server.command[0];
+	let result;
+	try {
+		// spawnSync itself can throw synchronously (e.g. command undefined),
+		// separately from the return-based result.error handled below.
+		result = spawnSync(command, args, {


Copilot

🟡 Changes recommended

Server execution can fetch packages, and browser-layer error handling can incorrectly suppress Lighthouse.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (8)

Previously missed (6) — in code that hasn't changed since the last review.

node-packages/wp-tooling/src/perf/config.js:39

The default server command invokes bare npx, so enabling server without a locally installed wp-env can download and execute it from the registry. That contradicts this PR's never-fetch/no-package-manager-execution contract. Use the same no-install resolution policy as Lighthouse, and keep the scaffold template in sync.
node-packages/wp-tooling/src/perf/normalize.js:101
An explicit topAudits: 0 is replaced by the default because 0 is falsy, so consumers cannot request zero audit details. Use nullish defaulting so the documented maximum is respected at this boundary.
node-packages/wp-tooling/src/perf/run.js:183
scanError covers every exception from collectVitals, including injection, evaluation, and page-close failures after navigation succeeded. This guard then skips Lighthouse as if the URL were unreachable, so a collector-specific failure disables the supposedly independent Lighthouse layer. Distinguish navigation failures from collection failures, or run Lighthouse after non-navigation errors.
node-packages/wp-tooling/package.json:30
Adding this public export leaves the package guide stale: node-packages/wp-tooling/AGENTS.md:61-72 still says the exports map has eight entries and omits ./perf; its CLI directory listing also omits the new command. Update that authoritative package documentation so future changes do not treat this export as unsupported.
node-packages/wp-tooling/src/perf/resolve-module.js:96
This variable-path require violates the package rule forbidding dynamic require paths (node-packages/wp-tooling/AGENTS.md:104). Replace the generic loader with a constrained consumer-module mechanism, such as a dedicated Puppeteer loader using module.createRequire with a statically named package.
node-packages/wp-tooling/src/perf/run.js:286
failedUrls also includes vitalsError results where navigation succeeded but no metrics were harvested, so reporting all of them as “failed to load” is inaccurate. Use wording such as “failed to collect metrics” that covers both load and harvest failures.

This issue also appears on line 495 of the same file.

node-packages/wp-tooling/src/perf/resolve-bin.js:32

On Windows, npm's extensionless .bin/<name> file is a POSIX shim; execFileSync cannot execute it. Because this path is preferred whenever it exists, locally installed Lighthouse is detected but its version probe fails. Resolve the package's JavaScript bin entry and invoke it with process.execPath, rather than selecting an extensionless or shell-based shim.

		const candidate = path.join(dir, 'node_modules', '.bin', binName);
		if (fs.existsSync(candidate)) {

node-packages/wp-tooling/src/perf/run.js:498

This message says every failed URL failed to load, but failedUrls is also incremented for an empty web-vitals harvest after a successful load. Report a generic collection failure, or track load and harvest failures separately.

	if (report.summary.failedUrls > 0) {
		process.stderr.write(
			`perf: ${report.summary.failedUrls} URL(s) failed to load — treating as a run failure.\n`
		);

Files reviewed: 38/38 changed files
Comments generated: 1
Review effort level: Balanced

00kaku · 2026-09-15T04:58:17Z

+	],
+	"server": {
+		"enabled": {{#server_enabled}}true{{/server_enabled}}{{^server_enabled}}false{{/server_enabled}},
+		"command": ["npx", "wp-env", "run", "cli", "--env-cwd={{server_env_cwd}}", "--", "wp"]


Please review this.

Adi-ty marked this pull request as ready for review July 15, 2026 04:51

Adi-ty requested a review from AnuragVasanwala July 15, 2026 04:51

Adi-ty self-assigned this Jul 21, 2026

Adi-ty added 4 commits August 31, 2026 11:42

docs(issues): rename tracking file to reflect #36 and #37 scope

5153373

fix(perf): require server_env_cwd, fix EBADJSON exit code

9255880

No safe default for server_env_cwd — wrong guess silently misroutes the server layer. Malformed config now exits 2 (usage error, not run failure).

Adi-ty force-pushed the v1.0.0/task/perf-runner branch from 8252bf2 to 9255880 Compare August 31, 2026 06:16

Adi-ty changed the base branch from release/v1.0.0 to main August 31, 2026 06:16

Swanand01 requested a balanced review from Copilot August 31, 2026 14:04

Copilot started reviewing on behalf of Swanand01 August 31, 2026 14:05 View session

Copilot AI reviewed Aug 31, 2026

View reviewed changes

Swanand01 reviewed Aug 31, 2026

View reviewed changes

fix(perf): Copilot review fixes — spawnSync guard, vitalsError, scrip…

64a3237

…t rendering, report gaps

Copilot AI review requested due to automatic review settings September 1, 2026 04:20

Copilot started reviewing on behalf of Adi-ty September 1, 2026 04:20 View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

perf: readability refactors in normalize and resolve-module

fd4257a

Copilot AI review requested due to automatic review settings September 1, 2026 04:37

Copilot started reviewing on behalf of Adi-ty September 1, 2026 04:37 View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Comment thread node-packages/wp-tooling/src/perf/server-profile.js

Copilot AI review requested due to automatic review settings September 1, 2026 05:57

Copilot started reviewing on behalf of Adi-ty September 1, 2026 05:58 View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

fix(scaffolds): scan script maps in inferPlaceholders fallback

c7221b6

Adi-ty requested review from Swanand01 and Copilot September 2, 2026 06:31

Copilot started reviewing on behalf of Adi-ty September 2, 2026 06:32 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

00kaku approved these changes Sep 15, 2026

View reviewed changes

Adi-ty mentioned this pull request Sep 17, 2026

feat(scaffolds): add lint/i18n scaffold #51

Open

		const candidate = path.join(dir, 'node_modules', '.bin', binName);
		if (fs.existsSync(candidate)) {

Conversation

Adi-ty commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Closes

Changes

How I verified

Acceptance criteria

Runtime behavior

Code quality

Housekeeping

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (8)

Uh oh!

00kaku Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Adi-ty commented Jul 15, 2026 •

edited

Loading