Skip to content

Pass the blob URL for preloads in WasmEMCCBenchmark. #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion 8bitbench/benchmark.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright 2025 the V8 project authors. All rights reserved.
// Copyright 2025 Apple Inc. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

Expand Down Expand Up @@ -34,14 +35,20 @@ function dumpFrame(vec) {

class Benchmark {
isInstantiated = false;
romBinary;

async init() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this is essentially shared code with the getBinary function in WasmEMCCBenchmark below, just that there it's only used for initializing the Module["wasmBinary"] = getBinary(...) and here it's for an arbitrary (non-Wasm) file, that happens to be the emulator ROM, right?

Can we unify this, ideally also with the mechansim for preloading blogs of JavaScript line items (ARES-6/Babylon below)?

Module.wasmBinary = await getBinary(wasmBinary);
this.romBinary = await getBinary(romBinary);
}

async runIteration() {
if (!this.isInstantiated) {
await wasm_bindgen(Module.wasmBinary);
this.isInstantiated = true;
}

wasm_bindgen.loadRom(Module.romBinary);
wasm_bindgen.loadRom(this.romBinary);

const frameCount = 2 * 60;
for (let i = 0; i < frameCount; ++i) {
Expand Down
32 changes: 8 additions & 24 deletions ARES-6/Babylon/benchmark.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2017 Apple Inc. All rights reserved.
* Copyright (C) 2017-2025 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
Expand All @@ -26,34 +26,18 @@
"use strict";

class Benchmark {
constructor(verbose = 0)
{
async init(verbose = 0) {
let sources = [];

const files = [
[isInBrowser ? airBlob : "./ARES-6/Babylon/air-blob.js", {}]
, [isInBrowser ? basicBlob : "./ARES-6/Babylon/basic-blob.js", {}]
, [isInBrowser ? inspectorBlob : "./ARES-6/Babylon/inspector-blob.js", {}]
, [isInBrowser ? babylonBlob : "./ARES-6/Babylon/babylon-blob.js", {sourceType: "module"}]
[airBlob, {}]
, [basicBlob, {}]
, [inspectorBlob, {}]
, [babylonBlob, {sourceType: "module"}]
];

for (let [file, options] of files) {
function appendSource(s) {
sources.push([file, s, options]);
}

let s;
if (isInBrowser) {
let request = new XMLHttpRequest();
request.open('GET', file, false);
request.send(null);
if (!request.responseText.length)
throw new Error("Expect non-empty sources");
appendSource(request.responseText);
} else {
appendSource(read(file));
}
}
for (let [file, options] of files)
sources.push([file, await getString(file), options]);

this.sources = sources;
}
Expand Down
22 changes: 3 additions & 19 deletions Dart/benchmark.js
Original file line number Diff line number Diff line change
Expand Up @@ -261,25 +261,9 @@ class Benchmark {
async init() {
// The generated JavaScript code from dart2wasm is an ES module, which we
// can only load with a dynamic import (since this file is not a module.)
// TODO: Support ES6 modules in the driver instead of this one-off solution.
// This probably requires a new `Benchmark` field called `modules` that
// is a map from module variable name (which will hold the resulting module
// namespace object) to relative module URL, which is resolved in the
// `preRunnerCode`, similar to this code here.
if (isInBrowser) {
// In browsers, relative imports don't work since we are not in a module.
// (`import.meta.url` is not defined.)
const pathname = location.pathname.match(/^(.*\/)(?:[^.]+(?:\.(?:[^\/]+))+)?$/)[1];
this.dart2wasmJsModule = await import(location.origin + pathname + "./Dart/build/flute.dart2wasm.mjs");
} else {
// In shells, relative imports require different paths, so try with and
// without the "./" prefix (e.g., JSC requires it).
try {
this.dart2wasmJsModule = await import("Dart/build/flute.dart2wasm.mjs");
} catch {
this.dart2wasmJsModule = await import("./Dart/build/flute.dart2wasm.mjs");
}
}

Module.wasmBinary = await getBinary(wasmBinary);
this.dart2wasmJsModule = await dynamicImport(jsModule);
}

async runIteration() {
Expand Down
129 changes: 71 additions & 58 deletions JetStreamDriver.js
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,6 @@ class Driver {
}

benchmark.updateUIAfterRun();
console.log(benchmark.name)

if (isInBrowser) {
const cache = JetStream.blobDataCache;
Expand Down Expand Up @@ -776,8 +775,8 @@ class Benchmark {

if (this.plan.preload) {
let str = "";
for (let [variableName, blobUrl] of this.preloads)
str += `const ${variableName} = "${blobUrl}";\n`;
for (let [ variableName, blobURLOrPath ] of this.preloads)
str += `const ${variableName} = "${blobURLOrPath}";\n`;
addScript(str);
}

Expand Down Expand Up @@ -994,9 +993,15 @@ class Benchmark {
if (this._resourcesPromise)
return this._resourcesPromise;

const filePromises = !isInBrowser ? this.plan.files.map((file) => fileLoader.load(file)) : [];
this.preloads = [];

if (isInBrowser) {
this._resourcesPromise = Promise.resolve();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't fully understood the (somewhat intertwined, complex) preload, resource loading code. At a high-level, why do we just early return here having added anything to this.scripts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't fully understand it either. My guess is that various parts were edited by different people (or the same person that forgot how it worked) over time so it has lots of now dead code and/or duplicated logic. It seems like this could be greatly simplified but I'd rather do that in a follow up since this PR blocks "correct" results from the benchmark as a whole.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, happy to have more cleanups later and land this first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed: #86

return this._resourcesPromise;
}

const promise = Promise.all(filePromises).then((texts) => {
const filePromises = this.plan.files.map((file) => fileLoader.load(file));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get this in conjunction with fileLoader._loadInternal: Here, we never reach if isInBrowser, because of the early return above. But in fileLoader._loadInternal (from line 199,

async _loadInternal(url) {
if (!isInBrowser)
return Promise.resolve(readFile(url));
let response;
const tries = 3;
while (tries--) {
let hasError = false;
try {
response = await fetch(url);
} catch (e) {
hasError = true;
}
if (!hasError && response.ok)
break;
if (tries)
continue;
globalThis.allIsGood = false;
throw new Error("Fetch failed");
}
if (url.indexOf(".js") !== -1)
return response.text();
else if (url.indexOf(".wasm") !== -1)
return response.arrayBuffer();
throw new Error("should not be reached!");
}
) we early return just early return if (!isInBrowser) Promise.resolve(readFile(url)). In other words, isn't this whole remaining code of fileLoader superfluous?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does look like it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright: let's try to remember to remove _loadInternal for the follow-up clenaup.

this._resourcesPromise = Promise.all(filePromises).then((texts) => {
if (isInBrowser)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The if (isInBrowser) return; in lines 1006/1007 is dead code now, because of line 999, right?

return;
this.scripts = [];
Expand All @@ -1005,10 +1010,11 @@ class Benchmark {
this.scripts.push(text);
});

this.preloads = [];
this.blobs = [];
if (this.plan.preload) {
for (const prop of Object.getOwnPropertyNames(this.plan.preload))
this.preloads.push([ prop, this.plan.preload[prop] ]);
}

this._resourcesPromise = promise;
return this._resourcesPromise;
}

Expand Down Expand Up @@ -1130,6 +1136,51 @@ class DefaultBenchmark extends Benchmark {
}

class AsyncBenchmark extends DefaultBenchmark {
get prerunCode() {
let str = "";
// FIXME: It would be nice if these were available to any benchmark not just async ones but since these functions
// are async they would only work in a context where the benchmark is async anyway. Long term, we should do away
// with this class and make all benchmarks async.
if (isInBrowser) {
str += `
async function getBinary(blobURL) {
const response = await fetch(blobURL);
return new Int8Array(await response.arrayBuffer());
}

async function getString(blobURL) {
const response = await fetch(blobURL);
return response.text();
}

async function dynamicImport(blobURL) {
return await import(blobURL);
}
`;
} else {
str += `
async function getBinary(path) {
return new Int8Array(read(path, "binary"));
}

async function getString(path) {
return read(path);
}

async function dynamicImport(path) {
try {
return await import(path);
} catch (e) {
// In shells, relative imports require different paths, so try with and
// without the "./" prefix (e.g., JSC requires it).
return await import(path.slice("./".length))
}
}
`;
}
return str;
}

get runnerCode() {
return `
async function doRun() {
Expand Down Expand Up @@ -1197,58 +1248,19 @@ class WasmEMCCBenchmark extends AsyncBenchmark {
Module.setStatus(left ? 'Preparing... (' + (this.totalDependencies-left) + '/' + this.totalDependencies + ')' : 'All downloads complete.');
},
};
globalObject.Module = Module;
`;
return str;
}

// FIXME: Why is this part of the runnerCode and not prerunCode?
get runnerCode() {
let str = `function loadBlob(key, path, andThen) {`;
globalObject.Module = Module;
${super.prerunCode};
`;

if (isInBrowser) {
str += `
var xhr = new XMLHttpRequest();
xhr.open('GET', path, true);
xhr.responseType = 'arraybuffer';
xhr.onload = function() {
Module[key] = new Int8Array(xhr.response);
andThen();
};
xhr.send(null);
`;
} else {
if (isSpiderMonkey) {
str += `
Module[key] = new Int8Array(read(path, "binary"));

Module.setStatus = null;
Module.monitorRunDependencies = null;

Promise.resolve(42).then(() => {
try {
andThen();
} catch(e) {
console.log("error running wasm:", e);
console.log(e.stack);
throw e;
}
})
// Needed because SpiderMonkey shell doesn't have a setTimeout.
Module.setStatus = null;
Module.monitorRunDependencies = null;
`;
}

str += "}";

let keys = Object.keys(this.plan.preload);
for (let i = 0; i < keys.length; ++i) {
str += `loadBlob("${keys[i]}", "${this.plan.preload[keys[i]]}", async () => {\n`;
}

str += super.runnerCode;
for (let i = 0; i < keys.length; ++i) {
str += `})`;
}
str += `;`;

return str;
}
};
Expand Down Expand Up @@ -1585,7 +1597,7 @@ let BENCHMARKS = [
iterations: 60,
tags: ["ARES"],
}),
new DefaultBenchmark({
new AsyncBenchmark({
name: "Babylon",
files: [
"./ARES-6/Babylon/index.js"
Expand Down Expand Up @@ -1621,7 +1633,7 @@ let BENCHMARKS = [
tags: ["CDJS"],
}),
// CodeLoad
new DefaultBenchmark({
new AsyncBenchmark({
name: "first-inspector-code-load",
files: [
"./code-load/code-first-load.js"
Expand All @@ -1631,7 +1643,7 @@ let BENCHMARKS = [
},
tags: ["CodeLoad"],
}),
new DefaultBenchmark({
new AsyncBenchmark({
name: "multi-inspector-code-load",
files: [
"./code-load/code-multi-load.js"
Expand Down Expand Up @@ -2091,7 +2103,8 @@ let BENCHMARKS = [
"./Dart/benchmark.js",
],
preload: {
wasmBinary: "./Dart/build/flute.dart2wasm.wasm"
jsModule: "./Dart/build/flute.dart2wasm.mjs",
wasmBinary: "./Dart/build/flute.dart2wasm.wasm",
},
iterations: 15,
worstCaseCount: 2,
Expand Down
17 changes: 2 additions & 15 deletions code-load/code-first-load.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,21 +25,8 @@

let indirectEval = eval;
class Benchmark {
constructor(iterations) {

let inspectorText;
if (isInBrowser) {
let request = new XMLHttpRequest();
request.open('GET', inspectorPayloadBlob, false);
request.send(null);
if (!request.responseText.length)
throw new Error("Expect non-empty sources");

inspectorText = request.responseText;
} else
inspectorText = readFile("./code-load/inspector-payload-minified.js");

this.inspectorText = `let _____top_level_____ = ${Math.random()}; ${inspectorText}`;
async init() {
this.inspectorText = `let _____top_level_____ = ${Math.random()}; ${await getString(inspectorPayloadBlob)}`;

this.index = 0;
}
Expand Down
16 changes: 2 additions & 14 deletions code-load/code-multi-load.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,8 @@

let indirectEval = eval;
class Benchmark {
constructor(iterations) {

let inspectorText;
if (isInBrowser) {
let request = new XMLHttpRequest();
request.open('GET', inspectorPayloadBlob, false);
request.send(null);
if (!request.responseText.length)
throw new Error("Expect non-empty sources");
inspectorText = request.responseText;
} else
inspectorText = readFile("./code-load/inspector-payload-minified.js");

this.inspectorText = `let _____top_level_____ = ${Math.random()}; ${inspectorText}`;
async init() {
this.inspectorText = `let _____top_level_____ = ${Math.random()}; ${await getString(inspectorPayloadBlob)}`;
this.index = 0;
}

Expand Down
5 changes: 5 additions & 0 deletions sqlite3/benchmark.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright 2024 the V8 project authors. All rights reserved.
// Copyright 2025 Apple Inc. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

Expand Down Expand Up @@ -51,6 +52,10 @@ delete globalThis.FileSystemHandle;
class Benchmark {
sqlite3Module;

async init() {
Module.wasmBinary = await getBinary(wasmBinary);
}

async runIteration() {
if (!this.sqlite3Module) {
// Defined in the generated SQLite JavaScript code.
Expand Down
5 changes: 5 additions & 0 deletions wasm/HashSet/benchmark.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
// Copyright 2025 the V8 project authors. All rights reserved.
// Copyright 2025 Apple Inc. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

class Benchmark {
async init() {
Module.wasmBinary = await getBinary(wasmBinary);
}

async runIteration() {
if (!Module._main)
await setupModule(Module);
Expand Down
Loading
Loading