Support (a subset of) the Web Audio API

[nasso]

Today, Cloudflare Workers are one of the easiest ways to work with image data on the edge: optimization, transformation, generation and rendering with WebGPU and Cloudflare Images. Sadly, working with audio isn't as easy.

Proposed solution

The Web Audio API is a standard API for processing and rendering audio by manipulating an audio graph using a relatively high-level API.

In a web browser, this allows real-time procedural audio to be generated. While this could already be useful in the context of a server-side Cloudflare Worker (e.g. streaming procedurally generated audio using Web RTC), the Web Audio API also offers a way to generate audio "offline" (not real-time) to record the output into an audio buffer. This has several use cases:

• Applying effects to audio files, e.g. to add a watermark
• Generating audio procedurally for playback on devices where doing it on the client isn't feasible (e.g. low power embedded devices)
• Rendering audio on the server-side using algorithms we don't want to ship to the client (e.g. proprietary virtual instrument or effect)
• Caching procedurally generated audio for quicker playback the client (maybe generating it is expensive or requires a lot of external samples/resources to be downloaded and mixed together)

Alternatives

Right now, it is of course possible to simply write custom audio processing logic and simply run it in the worker (maybe even using WebAssembly). But it would be nice to see Cloudflare Workers natively support at least the "offline" subset of the Web Audio API, enabling the aforementioned use cases plus more without the need to ship custom audio rendering logic. This would allow Workers to remain interoperable with the Web platform with minimal modifications and reduce the amount of code required.

The Web Audio API is not available in Web Workers, but most of it doesn't depend on the DOM. See: WebAudio/web-audio-api#2423