Pass the blob URL for preloads in WasmEMCCBenchmark. #74

kmiller68 · 2025-05-27T16:26:18Z

Right now we pass the original path when loading preloads for WasmEMCCBenchmark. This means we might skip the cache and fetch the content from the network again. We don't want to do this because it means the OS might spin down the CPU, which can punish running faster.

I also moved this logic to the prerunCode rather than adding it to the runnerCode since logically that makes more sense.

Lastly, have the preloads tuples contain the path for cli runs. This means we no longer have to duplicate the preload paths into the benchmark.js file.

netlify · 2025-05-27T16:26:23Z

✅ Deploy Preview for webkit-jetstream-preview ready!

Name	Link
🔨 Latest commit	`510e017`
🔍 Latest deploy log	https://app.netlify.com/projects/webkit-jetstream-preview/deploys/686e885078338e0008d26e52
😎 Deploy Preview	https://deploy-preview-74--webkit-jetstream-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

kmiller68 · 2025-07-01T16:02:15Z

Ugh, unfortunately now the benchmark jetsams on iOS... I might need to get creative with a solution here...

danleh · 2025-07-02T10:40:46Z

8bitbench/benchmark.js

@@ -34,14 +34,24 @@ function dumpFrame(vec) {

 class Benchmark {
  isInstantiated = false;
+  romBinary;
+
+  async init() {


IIUC, this is essentially shared code with the getBinary function in WasmEMCCBenchmark below, just that there it's only used for initializing the Module["wasmBinary"] = getBinary(...) and here it's for an arbitrary (non-Wasm) file, that happens to be the emulator ROM, right?

Can we unify this, ideally also with the mechansim for preloading blogs of JavaScript line items (ARES-6/Babylon below)?

danleh · 2025-07-02T10:40:50Z

Dart/benchmark.js

-      } catch {
-        this.dart2wasmJsModule = await import("./Dart/build/flute.dart2wasm.mjs");
-      }
+      if (!isInBrowser)


So what happens in !isInBrowser environments, where the previous dynamic import threw an exception? In that case, wouldn't we simply not set this.dart2wasmJsModule and fail later? Or do we assume that we only reach here if isInBrowser == false? In that case, can we instead change this to an explicit assertion, e.g.

console.assert(!isInBrowser, "relative imports should always succeed in browsers, this code is only for shells");

or something.

danleh · 2025-07-02T10:44:53Z

JetStreamDriver.js


-        const promise = Promise.all(filePromises).then((texts) => {
+        if (isInBrowser) {
+            this._resourcesPromise = Promise.resolve();


I haven't fully understood the (somewhat intertwined, complex) preload, resource loading code. At a high-level, why do we just early return here having added anything to this.scripts?

Yeah, I don't fully understand it either. My guess is that various parts were edited by different people (or the same person that forgot how it worked) over time so it has lots of now dead code and/or duplicated logic. It seems like this could be greatly simplified but I'd rather do that in a follow up since this PR blocks "correct" results from the benchmark as a whole.

Sure, happy to have more cleanups later and land this first.

danleh · 2025-07-02T10:50:09Z

JetStreamDriver.js

+            return this._resourcesPromise;
+        }
+
+        const filePromises = this.plan.files.map((file) => fileLoader.load(file));


I don't get this in conjunction with fileLoader._loadInternal: Here, we never reach if isInBrowser, because of the early return above. But in fileLoader._loadInternal (from line 199,

JetStream/JetStreamDriver.js

Lines 199 to 225 in 6947a46

async _loadInternal(url) {

if (!isInBrowser)

return Promise.resolve(readFile(url));

let response;

const tries = 3;

while (tries--) {

let hasError = false;

try {

response = await fetch(url);

} catch (e) {

hasError = true;

}

if (!hasError && response.ok)

break;

if (tries)

continue;

globalThis.allIsGood = false;

throw new Error("Fetch failed");

}

if (url.indexOf(".js") !== -1)

return response.text();

else if (url.indexOf(".wasm") !== -1)

return response.arrayBuffer();

throw new Error("should not be reached!");

}

) we early return just early return if (!isInBrowser) Promise.resolve(readFile(url)). In other words, isn't this whole remaining code of fileLoader superfluous?

It does look like it.

Alright: let's try to remember to remove _loadInternal for the follow-up clenaup.

danleh · 2025-07-02T10:54:29Z

JetStreamDriver.js

@@ -994,9 +993,16 @@ class Benchmark {
        if (this._resourcesPromise)
            return this._resourcesPromise;

-        const filePromises = !isInBrowser ? this.plan.files.map((file) => fileLoader.load(file)) : [];
+        this.preloads = [];
+        this.blobs = [];


AFAICT, this.blobs is never read, can't this be removed?

danleh · 2025-07-02T11:01:07Z

JetStreamDriver.js

+        }
+
+        const filePromises = this.plan.files.map((file) => fileLoader.load(file));
+        this._resourcesPromise = Promise.all(filePromises).then((texts) => {
            if (isInBrowser)


The if (isInBrowser) return; in lines 1006/1007 is dead code now, because of line 999, right?

danleh · 2025-07-02T11:04:38Z

JetStreamDriver.js

    }

-    // FIXME: Why is this part of the runnerCode and not prerunCode?
+    // This is in runnerCode rather than prerunCode because prerunCode isn't currently structured to be async by default.


What is the difference between prerunCode() and Benchmark.init()? The latter is async / called with await for AsyncBenchmark, see

JetStream/JetStreamDriver.js

Line 1138 in 6947a46

await __benchmark.init();

Could one unify the two, or move this code into async init()?

I think the Benchmark.init is specific to a given benchmark.js file but prerunCode is part of the driver's Benchmark variant, thus shared between all the line items using that variant.

Right, thanks.

danleh · 2025-07-02T11:07:23Z

JetStreamDriver.js

-                };
-                xhr.send(null);
+                async function getBinary(key, blobURL) {
+                    const response = await fetch(blobURL);


As I said above, this getBinary function is only used for the Wasm binary of Emscripten benchmarks, but could we use some more general mechanism from the JavaScript line items?

Done, although, since it uses fetch so it only works for async benchmarks. There doesn't appear to be an easy way to switch the response type synchronously in Chrome/Firefox.

Alright, so reminder for another follow-up cleanup (unless I miss some reason why it's not possible): Let's make everything an AsyncBenchmark and remove Benchmark altogether.

danleh · 2025-07-02T11:08:07Z

JetStreamDriver.js

-        for (let i = 0; i < keys.length; ++i) {
-            str += `loadBlob("${keys[i]}", "${this.plan.preload[keys[i]]}", async () => {\n`;
+        for (let [ preloadKey, blobURLOrPath ] of this.preloads) {
+            if (preloadKey == "wasmBinary") {


Can we get rid of this special casing of just the wasmBinary preloadKey? There are not that many Emscripten workloads, so I'd be happy to just add a line Module["wasmBinary"] = genericPreloadFunc(wasmBinary) to every one of those (for a little less "magic").

danleh

Thanks a lot for starting this, the preload/blob loading code is certainly ripe for a simplification/cleanup! (Unfortunately, I haven't fully understood all parts / why it's so complex, so some clarification questions in the review.)

danleh · 2025-07-02T11:10:59Z

Ugh, unfortunately now the benchmark jetsams on iOS... I might need to get creative with a solution here...

Could you explain what the issue is? (Also TIL a new word "jetsams".)

Right now we pass the original path when loading preloads for WasmEMCCBenchmark. This means we might skip the cache and fetch the content from the network again. We don't want to do this because it means the OS might spin down the CPU, which can punish running faster. I also moved this logic to the `prerunCode` rather than adding it to the `runnerCode` since logically that makes more sense. As a drive by change, this patch has the preloads tuples contain the path for cli runs. This means we no longer have to duplicate the preload paths into the benchmark.js file. Additionally, remove a console.log of the test that just finished which broke the dumpJSONResults option for the CLIs.

…ion to get the blob as an array buffer in modern browsers.

… that process I also had to change how 8bitbench loads the rom it's going to use.

There's now three helpers `getBinary`/`getString`/`dynamicImport` that help fetch resources from blobs/paths in the various `benchmark.js`'s `init` functions. All of the preloads in JetStream have been switched to using one of those, thus an AsyncBenchmark with the exception of WasmLegacyBenchmark (aka tfjs-wasm). Lastly, there's also no more special casing for the wasmBinary preload, instead it's set in each relevant `benchmark.js`'s `init` function.

kmiller68 · 2025-07-09T15:20:29Z

Ugh, unfortunately now the benchmark jetsams on iOS... I might need to get creative with a solution here...

Nvm, I take this back it must have been my setup. It seems to be working now.

Could you explain what the issue is? (Also TIL a new word "jetsams".)

iOS doesn't have paging to disk so when a process uses too much memory (or the system is under memory pressure) the OS will just "jetsam" your process (aka send it a SIGTERM).

danleh

Thanks for the changes, LGTM.

…terationCode This makes it easier for Benchmark subclasses to also call. This fixes a bug in WebKit#74 which caused inspector code load benchmarks to not set up properly ending the benchmark in 0ms. Also, use optional chaining for the call (and few other similar optional helpers).

kmiller68 force-pushed the preload-blobs-wasm branch from a9e68ec to 21aa2dc Compare May 29, 2025 14:02

danleh reviewed Jul 2, 2025

View reviewed changes

kmiller68 added 4 commits July 8, 2025 14:29

Move blob fetching to runnerCode since it needs to do an async operat…

8869c79

…ion to get the blob as an array buffer in modern browsers.

Fix between benchmark network requests for Dart-flute's JS Module. In…

bcfccc3

… that process I also had to change how 8bitbench loads the rom it's going to use.

kmiller68 force-pushed the preload-blobs-wasm branch from 07c8db1 to 510e017 Compare July 9, 2025 15:18

danleh approved these changes Jul 10, 2025

View reviewed changes

kmiller68 mentioned this pull request Jul 10, 2025

Refactor blob preloading code. #86

Open

kmiller68 merged commit df99e62 into WebKit:main Jul 10, 2025
10 checks passed

kmiller68 mentioned this pull request Jul 10, 2025

Add .NET on WebAssembly workload #73

Open

danleh mentioned this pull request Jul 11, 2025

Wasm benchmark runner does duplicate requests? #48

Closed

kmiller68 mentioned this pull request Jul 11, 2025

Move prepareForNextIteration call from Benchmark's runnerCode to preIterationCode #90

Merged

	async _loadInternal(url) {
	if (!isInBrowser)
	return Promise.resolve(readFile(url));

	let response;
	const tries = 3;
	while (tries--) {
	let hasError = false;
	try {
	response = await fetch(url);
	} catch (e) {
	hasError = true;
	}
	if (!hasError && response.ok)
	break;
	if (tries)
	continue;
	globalThis.allIsGood = false;
	throw new Error("Fetch failed");
	}
	if (url.indexOf(".js") !== -1)
	return response.text();
	else if (url.indexOf(".wasm") !== -1)
	return response.arrayBuffer();

	throw new Error("should not be reached!");
	}

Pass the blob URL for preloads in WasmEMCCBenchmark. #74

Pass the blob URL for preloads in WasmEMCCBenchmark. #74

Uh oh!

Conversation

kmiller68 commented May 27, 2025

Uh oh!

netlify bot commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for webkit-jetstream-preview ready!

Uh oh!

kmiller68 commented Jul 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danleh left a comment

Choose a reason for hiding this comment

Uh oh!

danleh commented Jul 2, 2025

Uh oh!

kmiller68 commented Jul 9, 2025

Uh oh!

danleh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

netlify bot commented May 27, 2025 •

edited

Loading