Allow compiling Wasm components with worlds besides `wasi:cli/command`

[ouillie]

AFAICT, TinyGo already supports the Wasm component model, but only if you're compiling a component for the wasi:cli/command world. That's hardwired into the wasip2 target.

What about compiling components with a different WIT interface? Is that something other people already know how to do, but it's just not well documented? Or are there changes that have to be made? I'm happy to help develop this, but would appreciate some pointers from anybody with context.

[daedric]

Hey, I think I'm hitting the same limit you seem to describe.
I've a wit file exposing a world, I can generate the binding without problem, but when I try to compile it I get the following error:

$> GOOS=wasip2 GOARCH=wasm tinygo build -o strats.wasm --wit-world myworld
error: no world named `myworld` in package
`wasm-tools component embed` failed: exit status 1

I did not encounter such issues with the rust bindings, so I'm pretty sure my wit is good.

I tried to pass --wit-package $mywit --wit-world myworld, but then it can't find standard wasi declaration:

$> GOOS=wasip2 GOARCH=wasm tinygo build -o strats.wasm --wit-package ../wit --wit-world myworld
error: failed to encode a component from module

Caused by:
    0: failed to decode world from module
    1: module was not valid
    2: failed to resolve import `wasi:cli/environment@0.2.0::get-environment`
    3: module requires an import interface named `wasi:cli/environment@0.2.0`
`wasm-tools component new` failed: exit status 1

[daedric]

ok, I wanted to dig a bit, actually it could be quite easy, I have a working binary with the following patch:

diff --git i/builder/build.go w/builder/build.go
index 9d73a036..3659b5b1 100644
--- i/builder/build.go
+++ w/builder/build.go
@@ -917,6 +917,11 @@ func Build(pkgName, outpath, tmpdir string, config *compileopts.Config) (BuildRe
 			}
 
 			// Run wasm-tools for component-model binaries
+			wasiPackage := strings.ReplaceAll(config.Target.WASIPackage, "{root}", goenv.Get("TINYGOROOT"))
+			if config.Options.WASIPackage != "" {
+				wasiPackage = config.Options.WASIPackage
+			}
+
 			witPackage := strings.ReplaceAll(config.Target.WITPackage, "{root}", goenv.Get("TINYGOROOT"))
 			if config.Options.WITPackage != "" {
 				witPackage = config.Options.WITPackage
@@ -925,8 +930,38 @@ func Build(pkgName, outpath, tmpdir string, config *compileopts.Config) (BuildRe
 			if config.Options.WITWorld != "" {
 				witWorld = config.Options.WITWorld
 			}
+
 			if witPackage != "" && witWorld != "" {
 
+				if wasiPackage != "" {
+
+					// wasm-tools component embed -w wasi:cli/command
+					// 		$$(tinygo env TINYGOROOT)/lib/wasi-cli/wit/ main.wasm -o embedded.wasm
+					componentEmbedInputFile := result.Binary
+					result.Binary = result.Executable + ".wasm-component-embed-1"
+					args := []string{
+						"component",
+						"embed",
+						"-w", "wasi:cli/command",
+						wasiPackage,
+						componentEmbedInputFile,
+						"-o", result.Binary,
+					}
+
+					wasmtools := goenv.Get("WASMTOOLS")
+					if config.Options.PrintCommands != nil {
+						config.Options.PrintCommands(wasmtools, args...)
+					}
+					cmd := exec.Command(wasmtools, args...)
+					cmd.Stdout = os.Stdout
+					cmd.Stderr = os.Stderr
+
+					err := cmd.Run()
+					if err != nil {
+						return fmt.Errorf("`wasm-tools component embed` failed: %w", err)
+					}
+				}
+
 				// wasm-tools component embed -w wasi:cli/command
 				// 		$$(tinygo env TINYGOROOT)/lib/wasi-cli/wit/ main.wasm -o embedded.wasm
 				componentEmbedInputFile := result.Binary
diff --git i/compileopts/options.go w/compileopts/options.go
index ed248c80..7a4d380c 100644
--- i/compileopts/options.go
+++ w/compileopts/options.go
@@ -56,6 +56,7 @@ type Options struct {
 	Monitor         bool
 	BaudRate        int
 	Timeout         time.Duration
+	WASIPackage     string // pass trough to wasm-tools component first embed invocation
 	WITPackage      string // pass through to wasm-tools component embed invocation
 	WITWorld        string // pass through to wasm-tools component embed -w option
 	ExtLDFlags      []string
diff --git i/compileopts/target.go w/compileopts/target.go
index 6917a944..5d83f10b 100644
--- i/compileopts/target.go
+++ w/compileopts/target.go
@@ -65,6 +65,7 @@ type TargetSpec struct {
 	JLinkDevice      string   `json:"jlink-device,omitempty"`
 	CodeModel        string   `json:"code-model,omitempty"`
 	RelocationModel  string   `json:"relocation-model,omitempty"`
+	WASIPackage      string   `json:"wasi-package,omitempty"`
 	WITPackage       string   `json:"wit-package,omitempty"`
 	WITWorld         string   `json:"wit-world,omitempty"`
 }
diff --git i/main.go w/main.go
index d4562fc4..1faef615 100644
--- i/main.go
+++ w/main.go
@@ -1511,7 +1511,7 @@ func main() {
 	printSize := flag.String("size", "", "print sizes (none, short, full, html)")
 	printStacks := flag.Bool("print-stacks", false, "print stack sizes of goroutines")
 	printAllocsString := flag.String("print-allocs", "", "regular expression of functions for which heap allocations should be printed")
-	printCommands := flag.Bool("x", false, "Print commands")
+	printCommands := flag.Bool("x", true, "Print commands")
 	parallelism := flag.Int("p", runtime.GOMAXPROCS(0), "the number of build jobs that can run in parallel")
 	nodebug := flag.Bool("no-debug", false, "strip debug information")
 	ocdCommandsString := flag.String("ocd-commands", "", "OpenOCD commands, overriding target spec (can specify multiple separated by commas)")
@@ -1549,8 +1549,9 @@ func main() {
 		flag.StringVar(&outpath, "o", "", "output filename")
 	}
 
-	var witPackage, witWorld string
+	var witPackage, witWorld, wasiPackage string
 	if command == "help" || command == "build" || command == "test" || command == "run" {
+		flag.StringVar(&wasiPackage, "wasi-package", "", "wasi package for wasm component embedding")
 		flag.StringVar(&witPackage, "wit-package", "", "wit package for wasm component embedding")
 		flag.StringVar(&witWorld, "wit-world", "", "wit world for wasm component embedding")
 	}
@@ -1638,6 +1639,7 @@ func main() {
 		Monitor:         *monitor,
 		BaudRate:        *baudrate,
 		Timeout:         *timeout,
+		WASIPackage:     wasiPackage,
 		WITPackage:      witPackage,
 		WITWorld:        witWorld,
 	}
diff --git i/targets/wasip2.json w/targets/wasip2.json
index 66e79eda..88cf4d67 100644
--- i/targets/wasip2.json
+++ w/targets/wasip2.json
@@ -28,6 +28,7 @@
 		"src/runtime/asm_tinygowasm.S"
 	],
 	"emulator": "wasmtime run --wasm component-model -Sinherit-network -Sallow-ip-name-lookup --dir={tmpDir}::/tmp {}",
+	"wasi-package": "{root}/lib/wasi-cli/wit/",
 	"wit-package": "{root}/lib/wasi-cli/wit/",
 	"wit-world": "wasi:cli/command"
 }

If that's something that could be upstreamed, I don't mind proposing a PR :)

[ouillie]

Thanks for the insights @daedric. Seems your suggestion obviates the need to explicitly include wasi:cli/imports in WIT as described here, but would still produce a compiled component that requires those imports to be supplied by the runtime.

Seems like that would be useful in situations where you want to share a single WIT interface for use in e.g. a Rust toolchain and a Go toolchain, and you don't want to impose the Go-specific baggage on the Rust component. However, I'm also wondering about the feasibility of producing a true "reactor" component that does not even have those imports.

[daedric]

Hey,
With pleasure. I'm still a total beginner on WASM, and I'm still assessing my ideas' feasibility. In other words, I'm really out of my depth.

I've to admit that I find the docs about components sparse and at the same time difficult to really understand. The differences between guest implementations (e.g. rust vs go) as well are not making the stuff any simpler.

You described perfectly what I'm trying to achieve on my end: a guest that implements an interface defined by a WIT file. I want my guest to either be in Go or Rust. The Rust part has mostly been pain-free. The Go part slightly harder.

Thanks for linking again this doc, I could not find it back, I was sure to have read it on Tinygo, this is where I assumed it should have been.

From what I gather(ed):
— The doc linked should probably be at least linked in the Additional Resources of the WASM section;
— If Tinygo does assume wasi:cli/command is targeted, my change does make sense: we use the bundled WIT, and the user need not worry about this part. I think that what wit-bindgen/wastime do in the end.
— If Tinygo wants to be as agnostic as possible, the component-model doc should actually be the one on tinygo.org, there should not be any default package nor world.

Again, I'm not in my element, so take everything with a grain of salt. The only thing I'm sure of is that component model documentation is a bit subpar ATM and spread too much between the different projects :)

[daedric]

I tried following the documentation and yeah, it worked well as well (the one with wkg).
What's unclear now is whether the compiled version of a wkg could be used as wit-bindgen input but that's not tinygo concern :D

[tomhoule]

We're also interested in this at Grafbase. I have a working branch based on the changes @daedric mentioned above, and I'd also be happy to upstream it. I'm looking into how to test this properly.