Compatibility with unwasm

[pi0]

unjs/unwasm will be become soon the default wasm loaded for Nitro/Nuxt and already supported with shikiji!

Usage when unwasm enabled:

import { getHighlighterCore, loadWasm } from 'shikiji/core'

await loadWasm(() => import("shikiji/onig.wasm") as any)

const shiki = await getHighlighterCore()

I am also wondering if we could remove the requirement of loadWasm entirely. Especially when import maps support lands (unjs/unwasm#10), we could directly provide those dependencies. For this we might support shikiji/esm that does not conflict with current subpath exports or breaking current users but a path to the future. Wdyt @antfu?

TODO list:

• Document unwasm/nuxt/nitro usage
• Try zero-config support with subpath imports and shikiji/esm

[antfu]

Wasm will be loaded automatically from the binary on the main entry shikiji. While the core aims to be side-effect-free and composable. While I am not against loading the .wasm in the environments that supports it, my concern is that the major tooling support hasn't catch up yet. Wonder if there would be an elegant way to prefer .wasm and fallback to inline-binary with a single entry.

So I guess:

• For core, I think it would be better to keep as-is, loadWasm would be still needed for explicit compsoable
• If we don't have a way to do the efficient fallback yet, I am ok with exporting sub-exports for bundled usage. Note that we have multiple bundles: https://shikiji.netlify.app/guide/bundles that we might need .wasm export for each of them.
• I feel that the name shikiji/esm can be confusing, as Shikiji itself is ESM. I guess you are referring esm-wasm here. I don't have a better name in mind yet, maybe shikiji/unwasm and shikiji/bundle/full/unwasm?

Happy to update the docs for sure!

[pi0]

As an alternative to shikiji/esm, we could try to adopt a new export condition like unwasm in shikiji pkg so that users don't worry and can normally import shikiji or shiliji/bundle/* and when unwasm plugin is enabled, we can leverage import maps and they worry one less step of calling loadWasm. I guess the (one) /onig.wasm subpath export can also leverage export conditions so we don't need it for each build.

My hope with unwasm initiative is to push forward both tooling/libs and also end-users to adopt (ESM native) wasm without major changes. Otherwise it will be and Chicken-and-egg situation forever and we always have to deal with😆, so really hope to find a solution migrating major libs in ecosystem that require wasm dependencies toward a unified way, with less changes.

[pi0]

@antfu Do you think we might be able to support imports map as well removing need of manual loadWasm?

[antfu]

Sorry, what are you proposing specifically?

[pi0]

https://github.com/unjs/unwasm?tab=readme-ov-file#auto-imports

For using shikiji with unwasm, ideally I love to see loadWasm requirement being completely removed. This function can be introduced as a dependency of wasm import.

[antfu]

Oh I think I did that on purpose.

• When importing the main entry shikiji, users don't need to care about wasm at all (a.k.a. out-of-box, implicit)
• When importing the core shikiji/core, I want everything to be maximum explicitness, composable, and side-effect free so that users can pass their own wasm instance or load it remotely elsewhere. (a.k.a. low-level composable parts, explicit)

The reason we used shikiji/core in Nuxt MDC was because that CF does not support inline-buffer wasm. But now with #72, I assume ppl could directly use the main shikiji entry with unwasm and no need to worry about loadWasm.

I still do not fully understand what what imports is exactly for yet, but I wonder how it could help the situation here.