WebAssembly/WASI support

[talentlessguy]

Describe the feature

Hey, I'm trying to find a Solidity compiler that compiles to WASM (other than solc-js but with a simpler API) so it is easy to write bindings for it.

I was trying to compile Solar to WASM/WASI and hit these errors with different targets:

• wasm32-wasip2

thread 'main' panicked at /Users/user/Coding/solar/crates/interface/src/session.rs:326:10:
called `Result::unwrap()` on an `Err` value: ThreadPoolBuildError { kind: IOError(Error { kind: Unsupported, message: "operation not supported on this platform" }) }
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: the compiler unexpectedly panicked; this is a bug.

note: we would appreciate a bug report: https://github.com/paradigmxyz/solar/issues/new/?labels=C-bug%2C+I-ICE&template=ice.yml

Error: failed to run main module `target/wasm32-wasip2/release/solar.wasm`

Caused by:
    0: failed to invoke `run` function
    1: error while executing at wasm backtrace:
           0: 0x28ec00 - solar-110efb72ae4bd129.wasm!rust_panic
           1: 0x28eb14 - solar-110efb72ae4bd129.wasm!std::panicking::rust_panic_with_hook::h10e13bf722482b78
           2: 0x28db77 - solar-110efb72ae4bd129.wasm!std::panicking::begin_panic_handler::{{closure}}::h37d0d2c2e3408baf
           3: 0x28dad8 - solar-110efb72ae4bd129.wasm!std::sys::backtrace::__rust_end_short_backtrace::h6ec5d07b1f7674cb
           4: 0x28e42d - solar-110efb72ae4bd129.wasm!rust_begin_unwind
           5: 0x68eb7 - solar-110efb72ae4bd129.wasm!core::panicking::panic_fmt::h1f868571e48d1093
           6: 0x69e99 - solar-110efb72ae4bd129.wasm!core::result::unwrap_failed::hda2d75f66b656df9
           7: 0x156c3b - solar-110efb72ae4bd129.wasm!solar_interface::session::run_in_thread_pool_with_globals::h233d0b559770b743
           8: 0x155090 - solar-110efb72ae4bd129.wasm!solar_interface::globals::SessionGlobals::with_source_map::hb0551a48acbbd5b4
           9: 0x1567fe - solar-110efb72ae4bd129.wasm!scoped_tls::ScopedKey<T>::set::h716aba159ec5b008
          10: 0x155af3 - solar-110efb72ae4bd129.wasm!solar_cli::run_compiler_args::hde755acc0b6fffb1
          11: 0x13827b - solar-110efb72ae4bd129.wasm!solar::main::h8d11087b6b5eaeba
          12: 0x138ba8 - solar-110efb72ae4bd129.wasm!std::sys::backtrace::__rust_begin_short_backtrace::h00df648946c1df3b
          13: 0x1389e8 - solar-110efb72ae4bd129.wasm!std::rt::lang_start::{{closure}}::h65325146fb8fad3b
          14: 0x284acc - solar-110efb72ae4bd129.wasm!std::rt::lang_start_internal::h2d7f0f783d9cce6b
          15: 0x1382ea - solar-110efb72ae4bd129.wasm!__main_void
          16: 0x34da - solar-110efb72ae4bd129.wasm!_start
          17: 0x3c0ed5 - wit-component:adapter:wasi_snapshot_preview1!wasi:cli/[email protected]#run
    2: wasm trap: wasm `unreachable` instruction executed

• wasm32-wasip1-threads

Error: failed to run main module `target/wasm32-wasip1-threads/release/solar.wasm`

Caused by:
    0: failed to instantiate "target/wasm32-wasip1-threads/release/solar.wasm"
    1: unknown import: `env::memory` has not been defined

• wasm32-unknown-unknown compiles successfully, but throws "RuntimeError: unreachable" when trying to import it

import { main } from './solar.wasm'

main()

run via deno test.js

• wasm32-unknown-emscripten fails because jemalloc doesn't support emscripten

Additional context

No response

[DaniPopes]

Hey, we haven't explicitly supported wasm yet, can you try the following:

• you can disable jemalloc by compiling with --no-default-features --features cli
• by default solar is multi threaded, it should be possible to run with -j1

For bare wasm I think we would have to define a separate package which exposes JS bindings with wasm-bindgen

[talentlessguy]

hey thank you for a quick response!

I'm not sure if I'm doing this right, but the compilation fails with these flags:

 cargo build --target wasm32-unknown-emscripten --release --no-default-features --features cli -j1

[DaniPopes]

you want to pass -j1 to solar

[talentlessguy]

@DaniPopes alright, but, the build still fails because of jemalloc:

cargo build --target wasm32-unknown-emscripten --release --no-default-features --features cli
   Compiling tikv-jemalloc-sys v0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7
error: failed to run custom build command for `tikv-jemalloc-sys v0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7`

Caused by:
  process didn't exit successfully: `Coding/solar/target/release/build/tikv-jemalloc-sys-d145f1fbaed16ce0/build-script-build` (exit status: 101)
  --- stdout
  TARGET=wasm32-unknown-emscripten
  HOST=aarch64-apple-darwin
  NUM_JOBS=8
  OUT_DIR="Coding/solar/target/wasm32-unknown-emscripten/release/build/tikv-jemalloc-sys-1cdef298ec2882db/out"
  BUILD_DIR="Coding/solar/target/wasm32-unknown-emscripten/release/build/tikv-jemalloc-sys-1cdef298ec2882db/out/build"
  SRC_DIR=".cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tikv-jemalloc-sys-0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7"

  --- stderr

  thread 'main' panicked at .cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tikv-jemalloc-sys-0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7/build.rs:99:9:
  jemalloc does not support target: wasm32-unknown-emscripten
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

also I wouldn't recommend wasm-pack, it's kind of abandoned. Just having a regular .wasm file that's loadable is more than enough imo

[DaniPopes]

Can you try again, pushed a fix 20ea26c

[talentlessguy]

it compiled but is now throwing a similar error as wasm32-wasip1-threads:

Error: failed to run main module `target/wasm32-unknown-emscripten/release/solar.wasm`

Caused by:
    0: failed to instantiate "target/wasm32-unknown-emscripten/release/solar.wasm"
    1: unknown import: `a::a` has not been defined

I'm testing it with wasmtime

[DaniPopes]

huh ok, I'll have a deeper look tomorrow to see if I can make it work

[DaniPopes]

Here are my findings.

wasm32-wasip1

Works fine without any changes:

cargo b -r --target wasm32-wasip1
wasmtime --dir . ./target/wasm32-wasip1/release/solar.wasm -j1 file.sol

I'll make a PR to force -j1 so it doesn't have to be specified.

(you can avoid passing --dir . by using stdin through - as the file name)

wasm32-wasip1-threads

This requires a few more flags.

Execution will panic on -j greater than 1 if building on stable, since it looks like parking_lot requires nightly to work with wasm:

cargo b -r --target wasm32-wasip1-threads # --features nightly # to actually allow threads
wasmtime -Sthreads --dir . ./target/wasm32-wasip1-threads/release/solar.wasm -j1 file.sol

wasm32-unknown-emscripten

Requires disabling jemalloc:

cargo b -r --target wasm32-unknown-emscripten --no-default-features --features cli
bun ./target/wasm32-unknown-emscripten/release/solar.js -j1 - < file.sol

Not sure how to allow directories to pass files directly, don't know much about emscripten.

wasm32-unknown-unknown

Compiles, but this will require more work to be usable, including C FFI / JS bindings

[talentlessguy]

happy to confirm that both wasm32-wasip1 and wasm32-wasip2 work! I'll get to writing JS wrappers with WASI to have a first ever alternative to solc-js

I think it's safe to close the issue, since there's not really a need to support all possible wasm targets (I don't need support for threads in my usecase)

also if that's helpful, wasm32-unknown-unknown compiles but the CLI interface isn't working at all, not sure why, as you said probably requires additional bindings (for me it throws RuntimeError: Unreachable)

Huge thanks for putting your time and effort into investigating this

[talentlessguy]

It's probably worth documenting usage with WebAssembly in the readme I think.

This is how you use it with wasip1 (in Node.js):

import { WASI } from 'node:wasi'
import { readFile } from 'node:fs/promises'

const wasi = new WASI({
  version: 'preview1',
  args: ['-j1','-V']
})

const wasm = await WebAssembly.compile(
  await readFile(new URL('./solar.wasm', import.meta.url)),
)
const instance = await WebAssembly.instantiate(wasm, wasi.getImportObject())

wasi.start(instance)

with wasm32-unknown-unknown it throws "Unreachable" when trying to call main().

Reproduction with Deno/Node.js:

import fs from 'node:fs/promises';

const wasmBuffer = await fs.readFile('./solar.wasm');

const wasmModule = await WebAssembly.instantiate(wasmBuffer);

console.log(wasmModule.instance.exports.main(2, 3))

emscripten target's not really importable as a module, it runs only in CLI mode (for example if you try to import solar.js, it will print a help message)

[talentlessguy]

Including WASM targets in the releases would also be great

[talentlessguy]

Just published this: https://github.com/deno-web3/solar

a JS SDK that uses WASI (wasip1) to call the compiler binary