Call stack size exceeded error when sending large UInt8Array

[hansottowirtz]

The library seems to have trouble sending large files. The following code throws:

Error sending file of size 112640 bytes: RangeError: Maximum call stack size exceeded
    at _Devaluator.devaluateImpl (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/serialize.ts:162:40)
    at _Devaluator.devaluateImpl (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/serialize.ts:144:28)
    at Function.devaluate (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/serialize.ts:83:25)
    at RpcSessionImpl.sendCall (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/rpc.ts:535:32)
    at RpcMainHook.call (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/rpc.ts:195:28)
    at doCall (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/core.ts:261:15)
    at Object.apply (<project>/node_modules/.pnpm/capnweb@0.4.0/node_modules/capnweb/src/core.ts:285:27)
    at <anonymous> (<project>/client.ts:10:34)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

Repro: https://github.com/hansottowirtz/capnweb-large-file-repro.git

server.ts:

import http from "node:http";
import { WebSocketServer } from 'ws';
import { RpcTarget, newWebSocketRpcSession } from "capnweb";

export class Api extends RpcTarget {
  async getFileSize(file: Uint8Array): Promise<number> {
    return file.length;
  }
}

const httpServer = http.createServer();

const wsServer = new WebSocketServer({ server: httpServer })
wsServer.on('connection', (ws) => {
  newWebSocketRpcSession(ws as any, new Api());
});

httpServer.listen(8000);

client.ts:

import type { Api } from "./server";
import { newWebSocketRpcSession } from "capnweb";

const session = newWebSocketRpcSession<Api>("ws://localhost:8000");

for (let i = 0; i < 100; i++) {
  const size = 1024 * (10 * i);
  try {
    const file = new Uint8Array(size);
    const result = await session.getFileSize(file);
    console.log(`Sent file of size ${size} bytes, received result ${result}`);
  } catch (error) {
    console.error(`Error sending file of size ${size} bytes:`, error);
    break;
  }
}

[hansottowirtz]

The culprit is this piece of code:

        if (bytes.toBase64) {
          return ["bytes", bytes.toBase64({omitPadding: true})];
        } else {
          return ["bytes",
              btoa(String.fromCharCode.apply(null, bytes as number[]).replace(/=*$/, ""))];
        }

More specifically String.fromCharCode.apply(null, bytes as number[]).

When adding import "core-js/proposals/array-buffer-base64", which adds bytes.toBase64, it seems to work.

[kentonv]

Ah, I suppose this is because each byte is being converted into an argument of fromCharCode, and V8 is not so happy about a 112,640-argument function call.

toBase64() is now available in all modern browsers and Cloudflare Workers but I guess supporting Node means supporting V8 that is a few years old, ugh. Will fix.

Also... Looking closely at that code I think the .replace() is in the wrong spot and may be corrupting the result. :(