|
| 1 | +# Adding compat.wamr (WebAssembly Micro Runtime 2.4.5) |
| 2 | + |
| 3 | +Shape decision, mirror status, feature evaluation, verification results, and the |
| 4 | +things a later reader should not have to rediscover. |
| 5 | + |
| 6 | +## Source and shape |
| 7 | + |
| 8 | +Upstream is [bytecodealliance/wasm-micro-runtime](https://github.com/bytecodealliance/wasm-micro-runtime), |
| 9 | +Apache-2.0 WITH LLVM-exception, pure C. Latest tag at time of writing is |
| 10 | +`WAMR-2.4.5` (`git ls-remote --tags | sort -V | tail`). It offers no mcpp |
| 11 | +support, so this is case (a) — a third-party upstream adapted as `compat`. |
| 12 | + |
| 13 | +Shape is **C-source compat**, the same template as `compat.mbedtls`: a source |
| 14 | +glob compiled into one static archive with the public headers exposed. The |
| 15 | +tarball wraps everything in `wasm-micro-runtime-WAMR-2.4.5/`, absorbed by the |
| 16 | +leading `*/` in every glob. |
| 17 | + |
| 18 | +`sha256 = 1ab09d51099f276ca4a1d6629f6b589aab2bd0caa01445e05031a4bed22c199b`, |
| 19 | +computed twice on separate downloads to rule out a repacking archive source. |
| 20 | + |
| 21 | +## The one thing this shape could not express: per-architecture selection |
| 22 | + |
| 23 | +WAMR needs architecture-specific input in two places, and the descriptor schema |
| 24 | +has a per-OS hook but no per-architecture one (`archs` is package metadata that |
| 25 | +declares support, not a selector). |
| 26 | + |
| 27 | +1. **A `BUILD_TARGET_*` define.** The whole invoke-native section of |
| 28 | + `core/iwasm/common/wasm_runtime_common.c` sits inside |
| 29 | + `#if defined(BUILD_TARGET_X86_64) || defined(BUILD_TARGET_AMD_64) || |
| 30 | + defined(BUILD_TARGET_AARCH64) || …`. With none defined the file still |
| 31 | + compiles — it just contains no `invokeNative` caller — and the failure |
| 32 | + arrives at link time. Upstream's CMake sets the right one from |
| 33 | + `WAMR_BUILD_TARGET`. |
| 34 | +2. **The `invokeNative` implementation**, which is hand-written assembly, one |
| 35 | + file per architecture under `core/iwasm/common/arch/`. |
| 36 | + |
| 37 | +Both are resolved by moving the decision into the preprocessor, which does know |
| 38 | +the target: |
| 39 | + |
| 40 | +* `generated_files` emits `mcpp_generated/include/mcpp_wamr_config.h`, mapping |
| 41 | + `__x86_64__`/`__aarch64__` to the matching `BUILD_TARGET_*` and erroring out |
| 42 | + on anything else. It reaches every TU through `cflags = { "-include", |
| 43 | + "mcpp_wamr_config.h" }`, the mechanism `compat.zlib` already uses for |
| 44 | + `Z_HAVE_UNISTD_H`. |
| 45 | +* `generated_files` also emits `mcpp_generated/mcpp_wamr_invoke_native.S`, |
| 46 | + which `#include`s the chosen `arch/invokeNative_*.s` as text. |
| 47 | + |
| 48 | +**Why the dispatcher must be `.S` and the upstream files could not simply be |
| 49 | +listed.** `compat.libffi` lists `src/x86/unix64.S` and friends directly and lets |
| 50 | +each guard itself, because those are uppercase `.S` — clang runs the |
| 51 | +preprocessor on them. WAMR's are lowercase `.s`, which clang assembles with no |
| 52 | +preprocessing at all, so an `#ifdef` inside them is not evaluated. (They do |
| 53 | +contain `#ifndef BH_PLATFORM_DARWIN`, which upstream's build does honour; that |
| 54 | +guard starts working again once the file is pulled into a `.S`.) Listing all of |
| 55 | +them is also not an option: each defines `invokeNative`, and the aarch64 file |
| 56 | +does not assemble on x86_64. |
| 57 | + |
| 58 | +Verified directly, before any mcpp involvement: |
| 59 | + |
| 60 | +``` |
| 61 | +$ clang -c mcpp_wamr_invoke_native.S -I…/common/arch -o x64.o && nm x64.o | grep -i invokenative |
| 62 | +0000000000000000 T invokeNative |
| 63 | +$ clang --target=aarch64-unknown-linux-gnu -c mcpp_wamr_invoke_native.S -I…/common/arch -o a64.o && nm a64.o | grep -i invokenative |
| 64 | +0000000000000000 T invokeNative |
| 65 | +``` |
| 66 | + |
| 67 | +## `-std=gnu11` through cflags, not `c_standard` |
| 68 | + |
| 69 | +`core/shared/platform/linux/platform_internal.h` writes the GS base with a bare |
| 70 | +`asm volatile`. `c_standard = "c11"` emits `-std=c11`, which defines |
| 71 | +`__STRICT_ANSI__`, under which `asm` is not a keyword — only `__asm__` is — and |
| 72 | +`wasm_memory.c` and `wasm_interp_fast.c` both fail with *use of undeclared |
| 73 | +identifier 'asm'*. Appending `-std=gnu11` via `cflags` wins over the earlier |
| 74 | +flag and matches how upstream builds. |
| 75 | + |
| 76 | +The alternative is `-DWASM_DISABLE_WRITE_GS_BASE=1`, a real upstream knob that |
| 77 | +also compiles cleanly (both call sites are the only bare-`asm` uses in the |
| 78 | +build). It was not chosen: it turns off an x86_64 fast path to work around a |
| 79 | +language-mode choice, which is a behaviour change made for a formatting reason. |
| 80 | + |
| 81 | +This is adjacent to the `compat.libaio` lesson — there `-std=c11` hid |
| 82 | +`syscall()` and `sigset_t` behind `__STRICT_ANSI__` disabling `_DEFAULT_SOURCE`, |
| 83 | +fixed with `-D_GNU_SOURCE`. Same macro, different consequence; `_GNU_SOURCE` |
| 84 | +does not help with the `asm` keyword. |
| 85 | + |
| 86 | +## Linux only, on purpose |
| 87 | + |
| 88 | +`xpm` carries a `linux` section and nothing else, and consumers gate with |
| 89 | +`[target.'cfg(linux)'.dependencies]` — the `compat.libaio` shape. WAMR itself is |
| 90 | +portable: there are `core/shared/platform/darwin` and `…/windows` trees, and the |
| 91 | +assembly files carry Darwin guards that this descriptor's `.S` dispatcher would |
| 92 | +honour. macOS is likely to be a small delta. But neither macOS nor Windows was |
| 93 | +built or run here, and Windows additionally has an unresolved choice between |
| 94 | +upstream's MASM `.asm` and the MinGW `.s` variant. Declaring a platform because |
| 95 | +it looks symmetric is how a red CI job gets iterated on blind, so the sections |
| 96 | +are left for someone who can verify them. |
| 97 | + |
| 98 | +The single-platform `xpm` keeps `check_platform_version_parity.lua` quiet by |
| 99 | +design: it only compares platforms that both carry versions. |
| 100 | + |
| 101 | +## What the base is, and why |
| 102 | + |
| 103 | +Interpreter runtime only — classic and fast interpreter, bulk memory and |
| 104 | +reference types (both on in upstream's own default configuration), no AOT, no |
| 105 | +JIT, no guest-facing libc. That is what an embedder wants when wasm modules are |
| 106 | +plugins reached only through host-provided imports, and it keeps AOT's LLVM |
| 107 | +dependency out of the index entirely. |
| 108 | + |
| 109 | +Sources follow upstream's cmake fragments: `iwasm/common/*.c` |
| 110 | +(`iwasm_common.cmake`), the loader + runtime + one interpreter from |
| 111 | +`iwasm_interp.cmake`, `shared/platform/linux/*.c` plus `platform/common/posix` |
| 112 | +(`shared_platform.cmake` → `platform_api_posix.cmake`), `mem_alloc.c` + `ems/` |
| 113 | +(`mem_alloc.cmake` — its `tlsf/` glob matches nothing in 2.4.5), and |
| 114 | +`shared/utils/*.c` (`shared_utils.cmake`). |
| 115 | + |
| 116 | +`wasm_mini_loader.c` and `wasm_interp_classic.c` are deliberately absent: they |
| 117 | +are upstream's *alternatives* to `wasm_loader.c` and `wasm_interp_fast.c`, not |
| 118 | +additions, and listing both would define the same symbols twice. |
| 119 | + |
| 120 | +## Features |
| 121 | + |
| 122 | +Both were evaluated against the ABI rule that keeps `compat.recastnavigation`'s |
| 123 | +`DT_POLYREF64` out of a feature: a feature's `defines` reach only the package's |
| 124 | +own TUs, so a macro that changes the layout of a type crossing the library |
| 125 | +boundary cannot be a feature. Neither macro here does — `WASM_ENABLE_LIBC_WASI` |
| 126 | +and `WASM_ENABLE_LIBC_BUILTIN` appear **zero** times in `core/iwasm/include/` |
| 127 | +outside comments, so a consumer compiling without them sees the same |
| 128 | +`wasm_export.h` the library was built against. |
| 129 | + |
| 130 | +* **`libc-builtin`** — `libraries/libc-builtin/*.c` plus |
| 131 | + `WASM_ENABLE_LIBC_BUILTIN=1`. One TU. |
| 132 | +* **`libc-wasi`** — `libraries/libc-wasi/**/*.c` plus `WASM_ENABLE_LIBC_WASI=1` |
| 133 | + **and `WASM_ENABLE_MODULE_INST_CONTEXT=1`**. The second define is not |
| 134 | + optional and not obvious: `build-scripts/runtime_lib.cmake:97-99` sets |
| 135 | + `WAMR_BUILD_MODULE_INST_CONTEXT` whenever `WAMR_BUILD_LIBC_WASI` is on, and |
| 136 | + without it `wasm_native.c` calls `wasm_native_get_context`, |
| 137 | + `wasm_native_set_context` and the context-key helpers that `wasm_native.h` |
| 138 | + only declares under that macro. |
| 139 | + |
| 140 | +### Two things the feature table cannot do, and what was done instead |
| 141 | + |
| 142 | +**A feature cannot carry `include_dirs`.** With `WASM_ENABLE_LIBC_WASI=1`, |
| 143 | +`common/wasm_runtime_common.h` itself opens `#include "posix.h"`, so it is not |
| 144 | +only the feature's own sources that need the WASI header roots — |
| 145 | +`wasm_loader.c` and `wasm_runtime.c` stop compiling without them. The five |
| 146 | +header roots therefore live in the base `include_dirs`, where they cost nothing |
| 147 | +when the features are off (they are `-I` paths into directories no base source |
| 148 | +includes from). |
| 149 | + |
| 150 | +**An exclusion glob is global and beats a feature's own entry for the same |
| 151 | +file.** Upstream's `platform_api_posix.cmake` drops `posix_file.c`, |
| 152 | +`posix_clock.c` and `posix_socket.c` (and pulls in `libc-util`) only when WASI |
| 153 | +is on, and the natural translation is `!`-exclusions in the base with the |
| 154 | +feature adding them back. That produced a build that compiled and then failed |
| 155 | +to link: |
| 156 | + |
| 157 | +``` |
| 158 | +ld.lld: error: undefined symbol: os_file_get_access_mode |
| 159 | +ld.lld: error: undefined symbol: os_is_dir_stream_valid |
| 160 | +ld.lld: error: undefined symbol: os_closedir |
| 161 | +``` |
| 162 | + |
| 163 | +Each was referenced from `libc-wasi/sandboxed-system-primitives/src/posix.c`, |
| 164 | +i.e. the feature's own sources, while the file defining them stayed excluded. |
| 165 | +All four files were then compiled individually **with WASI off** and all four |
| 166 | +succeeded, so the base simply carries them unconditionally. The cost is a few KB |
| 167 | +of unreferenced objects the linker drops. |
| 168 | + |
| 169 | +## Verification |
| 170 | + |
| 171 | +CI's pinned mcpp (`MCPP_VERSION: 2026.8.27.2`, matching `index.toml`'s |
| 172 | +`min_mcpp`) was used throughout, with `MCPP_INDEX_MIRROR=GLOBAL`. |
| 173 | + |
| 174 | +Descriptor: |
| 175 | + |
| 176 | +``` |
| 177 | +$ mcpp xpkg parse pkgs/c/compat.wamr.lua |
| 178 | +package compat.wamr (namespace 'compat') |
| 179 | +versions linux 2.4.5 |
| 180 | +sources 11 includes 16 |
| 181 | +generated mcpp_generated/include/mcpp_wamr_config.h (679 bytes) |
| 182 | +generated mcpp_generated/mcpp_wamr_invoke_native.S (724 bytes) |
| 183 | +target wamr features 2 |
| 184 | +parse OK |
| 185 | +``` |
| 186 | + |
| 187 | +Lint, reproduced locally: lua syntax, required fields, no leading `v`, |
| 188 | +`check_mirror_urls.lua`, `check_package_name.lua`, no `c++fly` — plus |
| 189 | +`check_cross_package_refs.lua`, `check_duplicate_versions.lua` and |
| 190 | +`check_platform_version_parity.lua`. All pass. |
| 191 | + |
| 192 | +Members, from a cleaned `target/` and `.mcpp/` **and a cleared global build |
| 193 | +cache**, so both runs show `Compiling compat.wamr` rather than `Cached`: |
| 194 | + |
| 195 | +``` |
| 196 | +$ mcpp test -p wamr |
| 197 | +compute(6,7) = 42 (expected 42) |
| 198 | +expecting env.putchar to be unlinked (libc-builtin is off): |
| 199 | + exception: Exception: failed to call unlinked import function (env, putchar) |
| 200 | +libc-builtin gated out: yes |
| 201 | +run_module ... ok |
| 202 | + test result ok. 1 passed; 0 failed |
| 203 | +
|
| 204 | +$ mcpp test -p wamr-features |
| 205 | +libc-builtin load ok |
| 206 | +libc-builtin instantiate ok |
| 207 | +libc-builtin putchar call ok |
| 208 | +libc-wasi load ok |
| 209 | +libc-wasi instantiate ok |
| 210 | + call=0 exception=Exception: wasi proc exit exit_code=3 |
| 211 | +libc-wasi proc_exit linked ok |
| 212 | +libc-wasi exit code ok |
| 213 | +libc_features ... ok |
| 214 | + test result ok. 1 passed; 0 failed |
| 215 | +``` |
| 216 | + |
| 217 | +### What the tests actually assert, and one dead end |
| 218 | + |
| 219 | +Both modules are hand-assembled wasm bytes in the test source, so nothing here |
| 220 | +needs a wasm toolchain at build time. |
| 221 | + |
| 222 | +`tests/examples/wamr` runs a module that calls **back into the host**: |
| 223 | +`compute(6,7)` returns 42 only because a native `host_mul` was invoked through |
| 224 | +`invokeNative`. That is the assertion that notices a missing `BUILD_TARGET_*` or |
| 225 | +a mis-dispatched assembly file; a "does it link" test would not. |
| 226 | + |
| 227 | +The feature gate is asserted in both directions — positively in |
| 228 | +`wamr-features`, negatively in `wamr`, where the same `env.putchar` module must |
| 229 | +be refused. |
| 230 | + |
| 231 | +**The dead end worth recording:** the negative check was first written as *does |
| 232 | +the module instantiate*. It does. WAMR does not reject an unresolved import at |
| 233 | +load or instantiation — it logs `warning: failed to link import function (env, |
| 234 | +putchar)` and carries on, and the refusal only materialises when the guest calls |
| 235 | +the import, as `Exception: failed to call unlinked import function`. Any feature |
| 236 | +gate test for this package has to call, not instantiate. |
| 237 | + |
| 238 | +Two smaller ones: `putchar_wrapper` returns **1**, not the character written, so |
| 239 | +an assertion copied from C's `putchar` contract fails; and WAMR refuses to load |
| 240 | +a module importing WASI apis unless it exports a memory (*"a module with WASI |
| 241 | +apis must export memory by default"*), so the WASI test module carries a |
| 242 | +one-page memory and exports it. |
| 243 | + |
| 244 | +## CN mirror |
| 245 | + |
| 246 | +Not created — no `mcpp-res` write access. `url` uses the plain-string upstream |
| 247 | +form, which lint accepts and which makes CN users fall back to GitHub. A |
| 248 | +maintainer can promote it to `{ GLOBAL=…, CN=… }` after mirroring the identical |
| 249 | +tarball. |
| 250 | + |
| 251 | +## Note on the skill's step 8 |
| 252 | + |
| 253 | +`.agents/skills/add-mcpp-index-package` says to add a row to the category table |
| 254 | +in `README.md` and `README.zh-CN.md`. Those tables no longer exist — the README |
| 255 | +now points at the online index site and keeps only a short "Reference examples" |
| 256 | +list. The equivalent record was added to `docs/descriptor-examples.md` and |
| 257 | +`docs/zh/descriptor-examples.md` instead, as a new shape row: *C-source compat |
| 258 | +whose sources are chosen by architecture*. |
0 commit comments