feat(host-component): couchbase plugins over the Data API, the KV SDK, and wasi:sockets by ricochet · Pull Request #4 · wasmCloud/awesome-wasmcloud

ricochet · 2026-07-31T21:04:54Z

Three component host plugins that provide wasmcloud:couchbase@0.2.0, document CRUD and SQL++, to every workload on a host that imports it, plus the interface they share and a harness that runs all three against a real cluster.

All three are WebAssembly components, not native plugins, so they run inside the sandbox and reach Couchbase through granted capabilities. They differ only in how they get to the cluster, and a workload cannot tell them apart.

	`couchbase`	`couchbase-kv-sdk`	`couchbase-wasi-sockets-p3`
Transport	Data API over `wasi:http/client@0.3.0`	KV protocol, via the Couchbase Rust SDK	KV protocol, spoken directly on `wasi:sockets@0.3.0`
Endpoint	`https://…`	`couchbases://…`	`couchbase://…`
Needs a gateway	yes (Cloud Native Gateway, or Capella)	no	no
Concurrent calls	interleave	serialize	interleave
TLS	yes	yes	no
Dependencies	`serde_json`	the SDK, tokio, rustls	`serde_json`

Pick by what the deployment allows. The Data API needs a gateway in front of the cluster. The two KV plugins do not, and between them the SDK one has TLS while the sockets one has concurrency — so a stock Couchbase Server under concurrent load is the case the third plugin exists for.

The interface

wasmcloud:couchbase@0.2.0 mirrors the published @0.1.0-draft from Couchbase-Ecosystem/wasmcloud-provider-couchbase so code ports across, with four changes forced by the plugin contract or by Couchbase itself:

Every function is async. The cross-store shim registers and type-matches only asynchronous imports, so a synchronous interface cannot be served by a component host plugin at all.
A document is list<u8> plus document-flags, not a JSON string. Couchbase stores binary documents, and a string cannot hold one.
sqlpp-value gains a json case. The draft declares null as its only case, which cannot carry a parameter in or a row out.
get-any-replicas — the draft spells it get-any-repliacs.

Concurrency is the real difference

A component host plugin is one store shared by every workload on the host, so an implementation that blocks blocks everyone. The SDK's futures need a Tokio context, which the component executor is not, so they run under block_on on one pinned instance. The other two await their I/O on the same executor that drives their exports.

Eight concurrent ~600 ms queries:

	wall
no plugin at all (control)	4,904 ms
`couchbase-wasi-sockets-p3`	4,933 ms
`couchbase` (Data API)	4,942 ms
`couchbase-kv-sdk`	10,221 ms

Being within 29 ms of the control is the point: the plugin is transparent under concurrency, where a blocking one costs 2×.

Verification

host-plugins/component/couchbase/verification/demo.sh brings up Couchbase fronted by the Cloud Native Gateway, builds all three plugins, and runs the same workload against each in turn. 34/34 on all three. The workload component is byte-identical across the runs; only the host's plugin declaration changes. The Data API plugin has also been run against live Capella.

CNG serves the Data API self-hosted, so this tests the real implementation rather than a reading of its documentation, with no cloud account.

Requirements

Component host plugins are opt-in and absent from released wash builds:

cargo build --bin wash --features host-component-plugins

wasmcloud:couchbase@0.2.0 is not published to any registry, so each project resolves it from this checkout via wit.sources in .wash/config.yaml.

Known limitations

Replica reads return unsupported on all three, each for its own reason: the Data API is a single endpoint in front of the cluster, the Rust SDK exposes no replica read, and the sockets plugin reads the cluster map for its vbucket count rather than its topology. The interface keeps them, because a transport that addresses nodes can serve them.
The SDK plugin's calls serialize, as measured above. Where that matters and TLS does not, couchbase-wasi-sockets-p3 is the same wire protocol without the blocking.
The sockets plugin has no TLS, so no couchbases:// and no Capella. couchbases:// is refused rather than quietly downgraded. Adding it means wasi:tls@0.3.0-draft, whose WIT is not in a registry.
The sockets plugin assumes one node. A not-my-vbucket is reported rather than retried elsewhere. Fine for a single node or behind a balancer; not yet a multi-node client.
Locking over the Data API is /v1.alpha, which a gateway serves only with --alpha-endpoints. Capella need not set it.
The Data API is itself opt-in. CNG's --data-port defaults to 18098, but --dapi-port defaults to -1, meaning disabled — a gateway started without it serves Protostellar gRPC and nothing the couchbase plugin can use. The KV plugins are unaffected.
Reaching a cluster on the developer's own machine: 127.0.0.1 in a guest is the virtual network. The KV plugins can use host.wasmcloud.internal with allowedHostLoopbackPorts (wasmCloud#5577). wasi:http does not resolve that name, so the Data API plugin needs the machine's LAN address.

A wasmCloud component host plugin serving `wasmcloud:couchbase@0.2.0` over the Capella Data API's HTTPS surface, async throughout — a component host plugin can only serve async functions, since the cross-store shim type-matches on them. The interface is a port of wasmCloud v1's `wasmcloud:couchbase`, made async and trimmed to what an implementation can actually serve. Replica reads are gone: neither the Data API nor the Couchbase Rust SDK exposes them, so a call that could only ever fail was dead surface. `verification/` stands up a real Couchbase Server plus a Data API server implementing the documented endpoints over the SDK's native KV operations. No Couchbase release ships the Data API, so that server is what makes end-to-end testing possible at all. A run against live Capella settled several behaviours that the spec alone did not: CAS arrives as bare 16-digit hex rather than decimal, the RFC-style quoted `If-Match` is rejected, and `project` must be one comma-separated parameter rather than repeated ones. Each of those would have silently corrupted every CAS-conditional write. `api.rs`, `config.rs` and `mutation.rs` are bindings-free and unit-tested on the host; `plugin.rs` is the bindings glue and compiles only for wasm. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

A second implementation of the same `wasmcloud:couchbase@0.2.0` interface, reaching Couchbase over the binary KV protocol with `couchbases://` by embedding the official Couchbase Rust SDK. The interface is resolved from `../couchbase/interface`, so the two implementations cannot drift, and a workload cannot tell which one is serving it. It serves what the Data API has no endpoint for — get-and-lock/unlock, preserve-expiry, and `consistent-with` as at_plus scan consistency built from real mutation tokens — and works against self-hosted clusters, which have no Data API at all. Getting the SDK into a component took four fixes. The first two are build-time; the last two present as a hang or a panic far from their cause: - tokio's wasm socket support is gated behind `--cfg tokio_unstable` - `couchbase-connstr` reads /etc/resolv.conf, which does not compile for wasm - `Cluster::bucket` reads like a plain accessor but spawns the task that resolves the bucket's agent, so it panics outside a Tokio context. The guard has to be dropped again before `block_on`, which panics from *inside* one. - tokio resolves names on its blocking pool, which wasm cannot spawn, so it aborts with "Not supported (os error 58)" before any lookup happens — not something a DNS grant can fix. `std`'s resolver goes straight to wasi:sockets/ip-name-lookup, so the name is resolved in the plugin and the SDK only ever sees an address literal. SQL++ runs at scope level. A cluster-level query resolves an unqualified keyspace against the cluster, so `FROM _default` fails with "No bucket named _default" — the binding's bucket is never consulted. Both plugins score 32/32 on the shared verification scenario, which is transport-aware where the two genuinely differ: an operation must either report `unsupported` or be shown to have actually worked. That caught a weak test — asserting a write fails against a locked document was really testing the SDK's retry policy, which waits the lock out and then succeeds. The lock is proven by CAS instead. Reaching a cluster on the developer's own machine uses `allowedHostLoopbackPorts` (wasmCloud#5577) with `couchbase://host.wasmcloud.internal`, which needs neither `allowedHosts` nor `allowedIpNameLookups` — the `*.wasmcloud.internal` zone resolves inside the host, ahead of the name allowlist. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

`verification/demo.sh` runs the same workload against both implementations in turn and prints what was identical and where the transports genuinely differ. Nothing about the workload changes between the runs — only the host's plugin declaration — which is the point being demonstrated. Writing it surfaced two real defects: - The Couchbase healthcheck was `curl -sf .../pools`, which only works on an *unconfigured* node. Once the init step provisions the cluster that endpoint answers 401, `-f` turns that into a failure, and the container flips to unhealthy. A second `up` against a kept volume then died with "dependency failed to start" — pointing at the wrong thing entirely. It now accepts 200 or 401, which is what "the node is answering" actually means. - The scenario had no way to resolve `wasmcloud:couchbase`: `wit/deps/` is generated rather than committed, and the package is not published, so a clean checkout could not build it. The demo writes a `wit.sources` config pointing at ../../interface and fetches once up front, since `wash dev` deliberately skips fetching. Verified: 32 OK / 0 unexpected on both transports, from a clean checkout. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

The harness assumed the Data API was Capella-only and stood in a hand-written Python server implementing the published reference. That was wrong: the Data API is served by Couchbase's Cloud Native Gateway (couchbase/stellar-gateway), which runs self-hosted in front of any cluster, and the public couchbase/cloud-native-gateway image runs standalone -- no Kubernetes, no operator. docker-compose.yml now runs CNG 1.2.1 in place of the Python server, with a CA and leaf certificate generated per machine into tls/ (gitignored). The Data API is HTTPS only; `dev.http_client_ca_paths` makes wash trust the CA, and it does reach a host plugin's wasi:http, so the plugin verifies CNG rather than skipping verification. CNG_SAN names the address the plugin dials in the leaf. The plugin passed against CNG with no change. The formats first discovered against live Capella -- bare 16-digit hex ETags, `bucket:vbid:vbuuid:seqno` mutation tokens -- come back identically, so those findings are reproducible without a cloud account. It exposed one gap the stand-in had hidden: a read reports a document's expiry in an `Expires` header, which the plugin ignored, so `with-expiry` never returned `expires-at`. That is now filled. CNG writes the zone as `UTC` where HTTP-date requires `GMT`, so a strict parser would reject every value; ours takes both. Scenario step 31 checks it on both transports. Also: readiness has to probe the SQL++ passthrough, which lags document reads by ~30s while CNG loads the cluster map; a step-29 failure was mislabelled 31; and the KV README credited it with `consistent-with`, which the Data API plugin serves too. The Python server's /__log wire log goes with it; CNG has no equivalent. Verified: demo.sh from a torn-down stack, 33/33 on both transports. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

`get-and-lock` and `unlock` are `POST …/documents/{id}/lock` and `/unlock` under `/v1.alpha`, which a gateway serves with `--alpha-endpoints`. The compose stack passes it. The lock's CAS is what unlocks the document and what writes through the lock; a wrong CAS is `cas-mismatch` and an unlocked document is `not-locked`. Both transports now pass scenario steps 15 and 16 identically. `get-any-replicas`, `get-all-replicas`, `replica-read-level`, `document-get-replica-result` and `document-error.unretrievable` are back in the interface. A replica read needs a transport that addresses individual nodes, so both plugins report `unsupported`; step 34 holds them to that. Docs and comments state current behaviour only, and are terser for it. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

Calls a gRPC service from a component over `wasi:http/client@0.3.0`, exposed as an HTTP endpoint: POST /<package>.<Service>/<Method> with a protobuf message becomes one gRPC call. gRPC is HTTP/2 with three conventions on top, so no transport beyond the one every component already has is needed. src/grpc.rs is the reusable part. It depends on no protobuf crate, and it resolves `grpc-status` from headers or trailers: a call returning a message puts the status in trailers after the body, while one returning none — most errors — is sent trailers-only, putting it in the HEADERS frame. Reading only one place either misses every success or misses every error. Verified against Couchbase's Cloud Native Gateway, which the sibling couchbase plugin's harness already runs: a present document returns its content with grpc-status 0 and one frame; a missing one returns grpc-status 5 with a message and no body. The approach is Laurent Doguin's, from github.com/ldoguin/wasmcloud-couchbase-cng-conduit. This is an independent implementation, not a port: that repository carries no license, so it cannot be copied from. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

The root `vendor/` ignore rule is for Go, but it also swallowed `couchbase-kv/vendor/couchbase-connstr`, which `[patch.crates-io]` resolves by path. Nothing under it was tracked, so `cargo build` failed from a clean clone with "path not found", and the README's two links to WASM-PATCH.md were dead. It is 9 files and 96K, Apache-2.0, with its provenance and patch documented. Also corrects this plugin's index entry, which described it as blocked. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

- KV `get-and-lock` treated `lock-time: 0` as one second when options were present. `lock-time` is not optional in the WIT, so `0` is how a caller asks for the default; a caller setting only `timeout-ns` got a lock that lapsed almost at once. It now defaults to 15s, matching the sibling. - The KV plugin accepted `timeout-ms` and every per-call `timeout-ns` and used neither. `couchbase` 1.0.1 takes no timeout on a document operation, so the binding's value now bounds SQL++ (`server_timeout`), and a per-call timeout on a document call is refused. Accepting it would leave the caller believing in a bound that is not there, and because these calls serialize, one unbounded call stalls every workload on the host. - Neither plugin read `use-replica`, so a caller asking for a replica read got an ordinary active read with no sign the request was dropped. Both now refuse it, consistently with `get-any-replicas`. - `split_frames` computed `pos + len` from a peer-controlled `u32`. `usize` is 32 bits on wasm32, so a `0xFFFFFFFF` frame header overflowed: a panic in debug, an accidental `None` in release. Now `checked_add`. - Query service failures classified as `server`, because its numeric codes matched no arm and its HTTP 200 matched no status. 3000, 12003/12021 and 13014 now map, so a malformed statement is `invalid-argument` on both transports. - The Data API per-call timeout cast instead of clamping, so `u64::MAX` as "effectively no timeout" wrapped to a very short deadline. - The scenario's `get-and-lock` error arm reported one step where the others report two, which would desync demo.sh's line-by-line compare for every later step. - KV `to_wit_time` reported the sub-second part twice, as both `milliseconds` and `nanoseconds`. Scenario step 27 now asserts a per-call timeout is honoured or refused, never ignored. 34/34 on both transports. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

`couchbase` 1.0.1 takes no timeout on a document operation, so the previous commit refused a per-call `timeout-ns` rather than accept one it could not honour. That was the wrong conclusion: the SDK's lack of a knob does not prevent the plugin from bounding the future itself. Document operations now run under `tokio::time::timeout`, with the call's own deadline or the binding's. It matters more here than for the sibling plugin: these calls hold the plugin's only store, so one call waiting forever stops every workload on the host. Build the timer inside `block_on`. A `Sleep` registers with the timer driver when constructed, so constructing one outside a runtime context panics with `CONTEXT_MISSING_ERROR` before anything is awaited -- the same trap as `Cluster::bucket`. Verified against a real cluster: `timeout-ms: 1` reports `timeout` on every document call, `timeout-ms: 30000` serves them. 34/34 on both transports, with step 27 now reporting identically on each. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

… build The READMEs and demo.sh said `wasmcloud:wash` is unpublished and that `--skip-fetch` is therefore required. Neither holds: nothing here imports `wasmcloud:wash`, and `wash build` resolves everything on its own. What is actually true: `wasmcloud:couchbase@0.2.0` is not in a registry (only `0.1.0-draft` is), and `wasmcloud:host@0.1.1` is vendored from wasmCloud main. Both are mapped through `wit.sources`, so a plain `wash build` takes them from the checkout and fetches only the `wasi:*` packages. `--skip-fetch` was also wrong for demo.sh specifically. `wit/deps/` is generated and gitignored, so a clean checkout has none and skipping the fetch fails to resolve the world. Verified by clearing every generated `wit/deps/` and running demo.sh: 34/34 on both transports. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

The Data API needs the Cloud Native Gateway in front of the cluster, and the SDK-based KV plugin serializes concurrent calls behind `block_on`. Neither is an answer for a stock Couchbase Server under concurrent load. This third implementation of `wasmcloud:couchbase@0.2.0` speaks the KV binary protocol directly over `wasi:sockets@0.3.0`. Every socket operation is awaited on the executor that drives the plugin's exports, so callers interleave. Measured with eight concurrent ~600ms queries: 4,933ms, against a no-plugin control of 4,904ms. The Data API plugin is 4,942ms and the SDK one 10,221ms. 34/34 on the shared verification scenario, the same score as the other two. Two protocol rules are easy to get wrong and are unit-tested here: the vbucket hashes the bare document id while the wire key carries the leb128 collection prefix, and on a single node a misrouted read returns `not-found` rather than `not-my-vbucket`, so a routing bug is indistinguishable from a missing key. No TLS, one node, and no replica reads. Each is refused with a message naming the reason rather than answered wrongly. Two plugins now reach the cluster over the KV protocol, so name both for the transport that distinguishes them: `couchbase-kv` becomes `couchbase-kv-sdk`, and the new one is `couchbase-wasi-sockets-p3`. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

The harness built and ran two of the three, so the new plugin's 34/34 rested on a hand-written config rather than on anything a reader could reproduce. demo.sh now builds and runs couchbase-wasi-sockets-p3 as run 3 of 3, and the comparison is three-way. It counts files with `FNR == 1` rather than `ARGIND`, which is GNU-only, because macOS ships BSD awk. All three score 34/34 against a real cluster. Thirty-one of the thirty-four steps are identical across every transport; the three that differ are the ones the scenario asserts on coherence rather than a fixed answer. Signed-off-by: Bailey Hayes <bailey@cosmonic.com>

ricochet force-pushed the worktree-couchbase-host-plugin branch from cbc250c to a19e42b Compare September 16, 2026 14:09

ricochet added 2 commits September 16, 2026 10:12

ricochet force-pushed the worktree-couchbase-host-plugin branch from a19e42b to f4b855d Compare September 16, 2026 14:14

ricochet added 3 commits September 16, 2026 12:38

ricochet changed the title ~~feat(host-component): async couchbase plugin~~ feat(host-component): couchbase plugins over the Data API and the KV protocol Sep 22, 2026

ricochet added 7 commits September 22, 2026 12:19

ricochet changed the title ~~feat(host-component): couchbase plugins over the Data API and the KV protocol~~ feat(host-component): couchbase plugins over the Data API, the KV SDK, and wasi:sockets Sep 22, 2026

Navigation Menu

feat(host-component): couchbase plugins over the Data API, the KV SDK, and wasi:sockets - #4

feat(host-component): couchbase plugins over the Data API, the KV SDK, and wasi:sockets#4
ricochet wants to merge 12 commits into
mainfrom
worktree-couchbase-host-plugin

ricochet commented Jul 31, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ricochet commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The interface

Concurrency is the real difference

Verification

Requirements

Known limitations

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ricochet commented Jul 31, 2026 •

edited

Loading