feat: Add ESM Import Maps support

[stipsan]

This follows up on @developit's signal that ESM Import Maps is an interesting feature for microbundle.

TL;DR

This PR is about supporting the ahead-of-time rewriting strategy that the draft spec for ESM Import Maps talks about.

Background info, super detailed

At FINN.no we run a diverse and distributed infra, with a constant stream of deployments that's only been interrupted when we moved data centers in the middle of the night.

To make it possible for each team to own their part of the page (for example the Recommendations team owns the feed of "Anbefalinger" on the front page) and deploy updates and changes anytime they want we developed Podium. Podium answers the question "how can we update a snippet of shared html everywhere at once without restricting the server stack" by composing pages over HTTP. The benefits of a distributed frontend system are many, but it came with a challenging cost: if two page fragments use the same dependency, how do we ensure it's only ever downloaded once?

We built Asset Pipe to solve bundling for distributed systems. When a layout is starting up it figures out what assets each fragment needs, concatinates them, and sends them to our asset pipe server. When it's done bundling it'll create an optimized file for the js and another for the css, serving them on hashed URLs that are guaranteed to be unique:

<link href="https://static.finncdn.no/_podium-assets/caeb0dc97f9e53242615f218cdb35b6ddb426e69b2803bc2e91e22e8a84a29d5.css" media="all" type="text/css" rel="stylesheet">
<script src="https://static.finncdn.no/_podium-assets/2a920eeee4c178549a1e1ca23869c55571750ddce280a16e4f8f07e0629539db.js" crossorigin defer></script>

The downside to this approach by "bundling as a service" is that bundling is a slow process and it's become a huge bottleneck. If a page fragment is deploying a new update it triggers bundling on each layout using it, causing a cascade of bundling processes starting up. We were also unhappy with the performance tradeoff by creating one hashed asset per layout that bundles everything. When a user moves from layout A to B ideally react shouldn't be redownloaded on layout B for example.

We decided to set out to replace Asset Pipe and explored different concepts to find the optimal trade-offs. It turns out that building a CDN asset service that is designed for ESM cache heuristics, HTTP2 and ESM Import Maps is a really good idea. We call this solution Eik.
It lets us publish assets to the CDN in a similar way as you would to npm, allowing assets to always be ready ahead-of-time for the page fragments that'll use them, with stable and immutable URLs. Making deployments as well as rollbacks fast as no bundling/rebundling is necessary.
The server part wasn't all we needed, we also had to create plugins for the bundlers people are using (rollup, esbuild, we're still waiting for webpack to support libraryTarget: "module" before we can add support there).

Many of our teams are used to the "bundling as a service" flow and don't have any bundler setup locally for their JSX or babel syntax. That's why we started looking at microbundle and saw it fit most of our needs, except that it didn't let us add custom rollup plugins and we needed to make a fork.

How we're using Import Maps in production

At FINN.no we've been testing this in production for a couple of months now, in combination with our ESM optimized CDN service. By using ESM Import Maps as our foundation we've been able to dedupe shared dependencies like react in our distributed system.

Here's an example from our frontpage on what this looks like in production:

       <script src="https://assets.finn.no/pkg/login-box/1.0.6/esm.js" type="module" crossorigin defer></script>
       <script src="https://assets.finn.no/pkg/login-box/1.0.6/ie11.js" crossorigin nomodule defer></script>

Browsers that support native ESM will load esm.js which Import Maps react like this:

import e from"https://assets.finn.no/npm/@pika/react/v16/index.js";
import t from"https://assets.finn.no/npm/@pika/react-dom/v16/index.js";

The assets we import map to are aliased to latest major, similar to how unpkg.com works except we cache the 302 redirect for 20 minutes instead of 1 minute. This speeds things up considerably as the user browse around on our marketplace. We also use -f modern to generate the esm.js files to further reduce the amount of JS our users have to download. More info on the setup in the docs.

The performance results we've seen in the last couple of months in production are promising and encouraging. We believe that betting on the import map spec (over competing solutions like Webpack's Module Federation) is the way to go.

[changeset-bot]

🦋 Changeset detected

Latest commit: 4e028e6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
microbundle	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[developit]

This looks pretty good! I'm wondering if it would make sense to have the --import-map option be a path to the JSON import map - would that match what you're using in other tools?

[stipsan]

Hey @developit I've discussed your suggestion with @trygve-lie and @digitalsadhu.
It would indeed be a good match for our tooling.
Downloading our Eik import maps to a local file is something we want our @eik/cli client to be able to do anyway.
It's much easier to test native import mapping in the browser when it's as simple as:

res.send(`
  <script type="importmap">
    ${JSON.stringify(require('./import-map.json'))}
  </script>
`)

While production is a simple matter of eik --download && microbundle --import-map ./eik/import-map.json for us 😄

[stipsan]

@developit alright PR updated with your suggestion 😄

[stipsan]

@developit oki ready for review, just had to fight node v12's "exports" behavior a little 😄

[developit]

Approved! There's a discussion around whether Microbundle should be support anything other than the original "bundling for npm" use-case, so I'm going to hold off on merging this just yet.

[stipsan]

@developit cool! I see the value in having that discussion. When a tool solves a particular problem well, it's always tempting to scale it up to solve a similar but not quite the same problem.
Maybe the use case "bundling for CDN" warrants a new library instead of scope creeping microbundle.
I don't know enough to form an opinion on it yet. But I must say that we haven't found an elegant setup when using microbundle outside npm. We often need to resort to --no-pkg-main, losing benefits like watch mode for multiple targets without resorting to spawn or lerna.

Is the discussion happening somewhere public?

[developit]

Heh - glad we are on the same page there. Yes, the discussion is happening in the #microbundle channel on the Preact slack. You can come commiserate with us on how much work it is to maintain things! :D

[stipsan]

@developit hahaha you got me at "commiserate" 😂

[stipsan]

Completely forgot about this 🙌 did we reach a consensus/conclusion on this @developit?