Convert local setup to a runner #63

wesleytodd · 2025-10-01T13:45:46Z

Took a quick look at converting @RafaelGSS's local server/client into a runner. I didn't have time this morning to look at the wrk2/autocannon stuff, but can assuming that #62 was not trying to say "ditch that abstraction entirely", which maybe it is and then I would like to discuss why we should do that.

wesleytodd · 2025-10-03T13:53:50Z

There are still quite a few gaps:

Sorting out local installs for the CLI's (global installs are a no go imo)
Finding the right format for passing requests to wrk2 like we do to autocannon
Applying overrides
Decide how we would want to support the --node option (maybe this one doesn't since it would use what node you have locally)

But @RafaelGSS are you alright with this direction so that we align with the other runner styles and features?

RafaelGSS

Sorry delay, @wesleytodd, a lot of things are going on recently, as you might know.

To be honest, I was trying to say "ditch that abstraction entirely" - It's a personal preference, so I'm fine leaving it as internal packages. My feeling that this increases the contribution barrier and makes things overly complex.

Also, I do believe Wrk2 (at least) should be a global binary - making it available through this CLI will make things very complex as the binary needs different dependencies according to each supported environment. Leaving that open for the "sysadmin" seems a better choice to not make things more complex than it is nowadays.

(That's what we do for Node.js too)

RafaelGSS · 2025-11-30T13:49:33Z

Ping me when this is ready to review @wesleytodd

wesleytodd · 2025-12-01T13:29:03Z

Will do, just cleaning things up and will move it from draft when it is ready.

cli: add local-bench to exbf Output example > @expressjs/perf-wg@1.0.0 local-load > expf local-load Running autocannon load... Result Autocannon: 121610.67 Running wrk2 load... Result Wrk2: [ { percentile: 50, ms: 0.829 }, { percentile: 75, ms: 1.1 }, { percentile: 90, ms: 1.41 }, { percentile: 99, ms: 1.75 }, { percentile: 99.9, ms: 1.88 }, { percentile: 99.99, ms: 2 }, { percentile: 99.999, ms: 2.2 }, { percentile: 100, ms: 2.2 } ]

wesleytodd · 2025-12-02T15:54:18Z

packages/requests/index.mjs

+
+  // Start here, await in .results()
+  const toAwait = requesters.flatMap((requester) => {
+    return requests.map((request) => {


By iterating the requests array we start one cli process per request since the cli's don't support making multiple request shapes well, but also so that we can compare to see if one specific request was the cause of problems. We may want to have a setup to separate each out and compose them back together in the future, but for now this means we can support patterns like #77 for more "real world" tests.

Honestly, I'd make it opt-in.

if (parallelRequests) return Promise.all(requests.map(req => requester.start(req)) else { const results = [] for (const req of requests) results.push(await requester.start(req)) }

We definitely don't want this to be default for the simple process.

wesleytodd · 2025-12-02T15:55:07Z

packages/requests/package.json

  "name": "@expressjs/perf-requests",
  "version": "1.0.0",
  "exports": {
+    ".": "./index.mjs",


I think after this we may be able to get rid of the exports with a small refactor to make adding request sets easier in the future. I am not going to do that now since this needs to land so we can get the CI integration over the finish line, but just wanted to call out why I added this file here.

wesleytodd · 2025-12-02T15:56:31Z

packages/wrk2/index.mjs

+
+    // TODO: I don't see docs on how to do this with wrk2
+    if (opts.method || this.method) {
+      throw new Error('not yet supported');


method and body did not show up in the readme and since I didn't get wrk2 building on my mac I just left these here for now. We should fix this before merging.

I can test it.

wesleytodd · 2025-12-02T15:57:31Z

packages/wrk2/index.mjs

+    });
+  }
+
+  processResults (output) {


Since I didn't get wkr2 built locally I also didn't test this after the refactor. I think I faithfully copied it but this needs more testing before we land this.

wesleytodd · 2025-12-02T15:58:02Z

servers/express-body-parser/index.js

 app.post(expressVersion.startsWith('4.') ? '*' : '*path', (req, res) => {
  res.status(200).json({
    hello: 'body!',
+    method: req.method,


This was just to test that the new request were passing method through correctly. Not necessary for this PR.

wesleytodd · 2025-12-02T16:29:02Z

packages/autocannon/spawn.mjs

+    }
+
+    // -H/--headers K=V
+    for (const [header, value] of Object.entries(opts.headers || this.headers)) {


opts.headers and this.headers should probably merge?

RafaelGSS

I think it's also missing a doc update.

RafaelGSS · 2025-12-02T21:19:08Z

packages/autocannon/spawn.mjs

+      // -c/--connections NUM
+      '-c', this.connections,
+      // -w/--workers NUM
+      '-w', Math.min(this.connections, availableParallelism() || 8),


Suggested change

'-w', Math.min(this.connections, availableParallelism() || 8),

'-w', Math.min(this.connections, availableParallelism() || 4),

8 sounds too much for the dedicated machine (availableParallelism is not available on v20<)

RafaelGSS · 2025-12-02T21:25:50Z

packages/requests/index.mjs

+
+  // Start here, await in .results()
+  const toAwait = requesters.flatMap((requester) => {
+    return requests.map((request) => {


Honestly, I'd make it opt-in.

if (parallelRequests) return Promise.all(requests.map(req => requester.start(req)) else { const results = [] for (const req of requests) results.push(await requester.start(req)) }

We definitely don't want this to be default for the simple process.

RafaelGSS · 2025-12-02T21:26:50Z

packages/wrk2/index.mjs

+
+    // TODO: I don't see docs on how to do this with wrk2
+    if (opts.method || this.method) {
+      throw new Error('not yet supported');


I can test it.

wesleytodd added the perf-wg-agenda label Oct 1, 2025

github-project-automation bot added this to Perf WG Tooling Oct 1, 2025

wesleytodd added the review requested label Oct 1, 2025

github-actions bot mentioned this pull request Oct 2, 2025

2025-10-15 Express Performance Working Group Meeting #64

Closed

wesleytodd force-pushed the add-local-server-runner branch from 3c7d75f to 3c0dfb0 Compare October 3, 2025 13:48

wesleytodd force-pushed the add-local-server branch from 596a16a to 8de4e78 Compare October 3, 2025 13:49

wesleytodd force-pushed the add-local-server-runner branch from 3c0dfb0 to bf79959 Compare October 3, 2025 13:50

wesleytodd mentioned this pull request Oct 3, 2025

Add support for wrk2 #60

Closed

github-actions bot mentioned this pull request Oct 4, 2025

2025-10-15 Express Performance Working Group Meeting #65

Closed

github-actions bot mentioned this pull request Oct 16, 2025

2025-10-29 Express Performance Working Group Meeting #66

Closed

RafaelGSS reviewed Oct 21, 2025

View reviewed changes

github-actions bot mentioned this pull request Oct 30, 2025

2025-11-12 Express Performance Working Group Meeting #67

Closed

RafaelGSS mentioned this pull request Nov 12, 2025

cli: add local-server and local-bench to expf #62

Closed

wesleytodd mentioned this pull request Nov 12, 2025

Isolated and Consistent Performance Testing in CI #38

Open

github-actions bot mentioned this pull request Nov 13, 2025

2025-11-26 Express Performance Working Group Meeting #71

Closed

wesleytodd force-pushed the add-local-server-runner branch 2 times, most recently from b5706bc to 7e872b6 Compare November 21, 2025 14:57

wesleytodd changed the base branch from add-local-server to main November 21, 2025 14:59

github-actions bot mentioned this pull request Nov 27, 2025

2025-12-10 Express Performance Working Group Meeting #75

Open

wesleytodd force-pushed the add-local-server-runner branch from 7e872b6 to 5a5c185 Compare November 28, 2025 20:55

wesleytodd force-pushed the add-local-server-runner branch from 5a5c185 to 3499db7 Compare December 1, 2025 13:35

wesleytodd force-pushed the add-local-server-runner branch from 3499db7 to e764ef6 Compare December 1, 2025 14:07

wesleytodd removed perf-wg-agenda review requested labels Dec 1, 2025

wesleytodd force-pushed the add-local-server-runner branch 2 times, most recently from 2870e11 to ce4622a Compare December 2, 2025 15:33

WIP: convert to local runner

a61605e

wesleytodd force-pushed the add-local-server-runner branch from ce4622a to a61605e Compare December 2, 2025 15:51

wesleytodd commented Dec 2, 2025

View reviewed changes

wesleytodd marked this pull request as ready for review December 2, 2025 15:58

wesleytodd requested a review from a team December 2, 2025 15:58

wesleytodd commented Dec 2, 2025

View reviewed changes

RafaelGSS reviewed Dec 2, 2025

View reviewed changes

	'-w', Math.min(this.connections, availableParallelism() \|\| 8),
	'-w', Math.min(this.connections, availableParallelism() \|\| 4),

Uh oh!

Convert local setup to a runner #63

Are you sure you want to change the base?

Convert local setup to a runner #63

Uh oh!

Conversation

wesleytodd commented Oct 1, 2025

Uh oh!

wesleytodd commented Oct 3, 2025

Uh oh!

RafaelGSS left a comment

Choose a reason for hiding this comment

Uh oh!

RafaelGSS commented Nov 30, 2025

Uh oh!

wesleytodd commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wesleytodd Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RafaelGSS left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wesleytodd commented Dec 1, 2025 •

edited

Loading

wesleytodd Dec 2, 2025 •

edited

Loading