Native: WebGPU support

[birkskyum]

With the Modularization and Metal work finalizing, there is an opportunity to leverage it to add another target, focused on running the MapLibre Native through the WebGPU API.

It's still early days for WebGPU, but it's a good time to start exploring the space.

Browser usage

After years in the making the WebGPU API is slowly stabilizing, and already now active in Chrome Desktop by default. WebGPU has a modern architecture, more aligned with i.e. Metal than OpenGL 2, and having a target that utilizes the webgpu.h, and WGSL shaders, would allow for a WebAssembly build using Emscripten. This build would use the Dawn/wgpu backend that is shipped with the browser.

It's been demonstrated that MapLibre Native can run in the browser as WebGL, by utilizing the OpenGL target and Qt for WebAssembly. A modern WebGPU target would allow us to see much more clearly what the boundaries of this approach are. Here is MapLibre RS in WebGPU, so get a feel of this approach.

Alternative Portable Desktop Target.

By including Dawn or wgpu, a WebGPU target can, similar to the OpenGL target, act as a portable desktop target, and it would be able to tap into the more modern DX12, Vulkan, Metal backends. A WebGPU target can thus be considered a more future-proof general-purpose target, that doesn't get affected by i.e. Apple sunsetting OpenGL v3.

[sjg-wdw]

Lot of different issues here, so I'm just getting some time to respond. I think there are a number of fascinating questions you raise, only partly in the domain of a Native version.

WebGPU for MapLibreJS

Adding WebGPU support for the Javascript version of MapLibre is probably a great idea. I suspect MapLibreJS is not particularly well optimized for WebGL2, let alone WebGPU. Doing a WebGPU version would give you an opportunity to optimize things.

These are some of the same issues we see for OpenGL ES, like the lack of instancing, single uniforms and overly simple shaders. There are opportunities for Native even in OpenGL ES 3.0 to greatly improve performance. We're just starting to take advantage of those for Metal. WebGPU for the JS version would allow you to explore those on the JS side.

WebGPU for Native: Technical Side

Purely from an engineering perspective, it should be easier now. We've got two renderer SDKs going so adding a third should not be a huge burden. Getting it performant.... well that's impossible to say at this point. You're going through a translation step here from WebGPU to some local SDK, so it could be fine or it could be absolutely horrible. You'd just have to see.

If there's enough interest and budget, sure! But I'm not sure it'll meet your expectations.

Who is This For?

On mobile we've seen a lot of these cross-platform attempts come and go. The one approach that seems to have stuck are toolkits like React Native which are careful to point out they use native components and organize them with custom logic. But we have a path forward for Metal and Vulkan and a better OpenGL ES 3 version if we want it, so this isn't about mobile.

A lot of those cross-platform toolkits were just Javascript in a wig. WebAssembly allows us to turn the tables on the web, but with C++. I'm not sure JS developers are going to react better than Native developers did. I'd tread carefully.

One more thing to ask is how many organizations really develop for multiple platforms? In my experience it's usually one or the other and even when it's both, one is often vestigial. Actually a good argument for "a" WebAssembly version.

Maybe Do It Anyway

That's not to say it isn't worth trying, I just wouldn't bet the future of the toolkit on it. How do you try this out with scaring off any of your users?

• Improve the WebAssembly support. Not actually too hard and you might find some use cases.
• WebGPU support via translator on Native. Sure! You could argue it'll help with a later JS version.
• Develop some pieces on Native and share them via WebAssembly (coordinate systems?). New features are good!

To expand on that last one, I don't see why a developer would want to use the version you're talking about. You need to give them new functionality that they care about. That's how you make the case.

[HarelM]

Actually, I think developers like to use modern tools, even if they don't add new functionality.
And from a security perspective (business perspective), the push towards latest and maintainable versions is fierce.
Just my two cents.

[sjg-wdw]

Developers do love the new thing, that's true. So then the question is for how many new developers is WebGPU a draw?

As for maintainability, it might seem like moving to a single unified rendering approach is more maintainable, but you're talking about technology that's brand new and you need to see how it matures.

On mobile, many companies found it was actually more expensive to maintain a single cross-platform app because they needed experts in the toolkit they were using and then experts on how that toolkit interacted with the platform. I suspect this was on the more complex side of things. A small company would probably just work around bugs, not having the people to track them down.

All of this is to say, that I wouldn't bet on this approach. Stick a toe in and see what you learn.

[birkskyum]

Thanks @sjg-wdw , that was very insightful, and it's a great advice to consider this as an experimental solution for the foreseeable future, rather than bet heavily on it before it's significantly more proven. I have a few comments regarding some big trends in the mobile/web ecosystem, that I believe this would play nicely into.

Portability and consistency appear be desired - the approach React Native took was good, because using the native UI make the apps feel like they belong on the respective platforms - but it does come with the cost of demanding some amount of platform-specific work done, and if you really want a fully consistent experience cross-platform that can be very hard, bordering impossible, to achieve. This problem gave room for Flutter, where the entire canvas is controlled similar to a game engine like Unity, and with Flutter for Web/iOS/Android the deployment target can be considered almost like an afterthought. This flexibility seems to be very desired by a lot of app devs who suddenly can take their native apps and make a browser version, and we currently mix GL JS / Native to reach something similar which bring some inconsistency:

Scaring off users - I'm generally not so worried about i.e. presenting a native library to the web community if there is decent api, because there are examples of popular libraries that does exactly that. An example would be the most popular image manipulation library, sharp, which is largely C++, and as long as it's fast nobody seem to take any issue with that. As a sidenote - they're soon shipping a wasm version for increased portability:

[sjg-wdw]

Those are great examples with some interesting lessons.

React Native, of course, uses Native components, but gives the dev a consistent architecture between platforms.

Flutter is more like the traditional cross-platform approach. But what's interesting even there is that for certain key pieces of functionality they integrate with the local toolkit and then provide a consistent interface.

Which begs the question: How does MapLibre see itself. Is it a high level interface or is it native core functionality?

If it's a high level interface, then trying to consolidate all of the rendering bits on a single toolkit at the cost of performance does make sense. But if it's core functionality then local performance matters quite a lot. Consistency in that high level interface is vital in that second case too, but that's not what we're talking about.

The last one, sharp, is an interesting case on the web side and suggests how to best integrate WASM into MapLibreJS: Give the devs something new. If you make it worthwhile to move to the new thing because it does something they've always wanted, then you have a compelling case.

[sjg-wdw]

In my experience it's harder to mess with things in wide distribution. You have to be careful you don't change too much because you risk the new version not being adopted. But when you're doing something new, you have a lot more freedom.

That's part of the thinking behind the Optimized Vector Map Pipeline proposal. If we take a step back and look at what most users are trying to do (display a vector map) and ask how we can simplify that process, a lot more opens up.

In that proposal I talk about "Teeny Tiny Toolkits" that only read optimized style and vector data and display it. One of those would be a good candidate for cross compiling to WASM and having a unified core.

Then it's up to the devs of that toolkit to make it more appealing than the current versions. You replace yourself with yourself, essentially.

[birkskyum]

Right, so there are interesting areas things to do with WebGPU, but for web there are other areas where an improvement today would bring more value to more people. Arguably, if say MapLibre Native had a lot more features than GL JS, then there would be a better value proposition - but it's the other way around today, which is likely a hard sell as you mention.

[sjg-wdw]

That exactly it!

Not to say you shouldn't try WebGPU. It just needs value on its own, IMO.

[birkskyum]

@sjg-wdw , ignoring browsers, rust etc. for a second, what is your view of this recurrent argument that if we had a single WebGPU target in C++ MapLibre Native, that could replace the OpenGL, Metal and potential Vulkan targets? What are the key benefits of having say Metal compared to WebGPU->Metal? Does it i.e. deliver better DX on apple platforms, a smaller build sizes, performance or access to some Metal-specific features?

[sjg-wdw]

I could be very definitive about vector tile rendering performance as concerns Metal and OpenGL because we have our own toolkit (WhirlyGlobe) that's similar enough to compare.

There are too many unknowns for me to say much about the WebGPU centralization plan. I can list some of the unknowns if that helps.

WebGPU is very new. We don't know how the performance is going to play out in browsers yet. I'm rooting for it on the browser side, but I wouldn't call it mature.

We don't know much about translation performance. OpenGL is well understood so writing a translation layer from it to something else is much easier. It's also quite a bit older, and it tends to be easier to go from an older to a newer SDK (e.g. OpenGL -> Metal). Even so, you have to be clever. The translation layer here is new so we don't know how it's going to perform.

Is WebGPU going to be popular? If it is, then you can find people to more easily write to it. If not, then we've exchanged a complex, but widely known set of SDKs, for one that's not well known.

How buggy is the translation layer and how quickly will it improve? I'm on an Angle mailing list and they're still finding some weird things there even though it's pretty stable. Here you run the risk of having to debug both your use of WebGPU, how the translation layer is interpreting it and what might be buggy in the translation layer.

Can this approach handle threading? There's a lot of performance improvement to be had in threading. Would be a shame to give it up.

How does all of this work in the offline case? Now you're translating from WebGPU to a software OpenGL renderer. That's going to be a bit odd, but maybe not all that different.

How big is the translation layer? Right now we've got MapLibre -> Metal and MapLibre -> OpenGL. This would be MapLibre -> WebGPU -> OpenGL. Seems a bit like the OpenGL -> Metal approach which added a lot of size.

It would definitely be larger, it would very likely be slower, it might even be harder to maintain.

There's a whole lot of uncertainty baked into a solution that's meant to be easier. That gives me pause.

[birkskyum]

Thank you for your input, that was very helpful.

[andreamancuso]

Hi,

I've been slowly developing a library that provides React bindings for Dear Imgui based on React Native's Fabric engine, (https://reactnative.dev/architecture/fabric-renderer), emscripten, GLFW3, WGPU, ReactivePlusPlus, rxjs, Yoga, IconFontCppHeaders.

Yoga support in particular is very much WIP however I've made good progress overall and I'm hoping to merge my own PR into the main branch soon.

What will still be missing is the possibility to embed maps. I see that what I am after is already achievable: https://github.com/maplibre/maplibre-rs

I was wondering whether you or anyone else has managed to compile maplibre native using emscripten. If you do, please get in touch!

Thank you.

[louwers]

I am not busy with Qt stuff. I just wanted to point out that using Qt is probably the easiest way to get a WASM build going.

I would need to know the commands you are using to build the library to help. But I don't know anything about vcpkg.

[andreamancuso]

Understood. Fair enough.

I would use Qt but I've gone "all in" on Dear Imgui for now.

I'll give it another try in the evening.

[andreamancuso]

I successfully built maplibre-native (with MLN_WITH_CORE_ONLY ON) using emscripten. But I had to disable a setting whereby warnings are treated as errors (/WX as documented here https://learn.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/compiler-error-c2220)

cmake . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DMLN_WITH_CORE_ONLY=1 -DMLN_WITH_WERROR=OFF

Unfortunately I then get:

wasm-ld: warning: C:/dev/react-imgui/packages/dear-imgui/cpp/deps/maplibre-native/build/mbgl-core.lib: archive member 'CMakeFiles\mbgl-core.dir\src\mbgl\style\layers\location_indicator_layer_properties.cpp.obj' is neither Wasm object file nor LLVM bitcode
wasm-ld: warning: C:/dev/react-imgui/packages/dear-imgui/cpp/deps/maplibre-native/build/mbgl-core.lib: archive member 'CMakeFiles\mbgl-core.dir\src\mbgl\style\layers\location_indicator_layer_impl.cpp.obj' is neither Wasm object file nor LLVM bitcode
wasm-ld: warning: C:/dev/react-imgui/packages/dear-imgui/cpp/deps/maplibre-native/build/mbgl-core.lib: archive member 'CMakeFiles\mbgl-core.dir\src\mbgl\style\layers\location_indicator_layer.cpp.obj' is neither Wasm object file nor LLVM bitcode
[...]

But it still feels like moving forward, albeit rather slowly 😅

[wipfli]

@andreamancuso Thanks for sharing these insights. The repo here is more about tracking organizational things rather than technical details of an implementation. Can you open an issue in the maplibre-native repo and copy your progress over to there? Thanks!

[birkskyum]

moved this discussion to maplibre-native, so that we can keep using it.

[wcandillon]

I am not sure if this is helpful but we are working on a WebGPU binding for React Native and we can provide you with the prebuilt binaries for Dawn WebGPU (C++). We build them via github actions. From our experience, setting up a Vulkan and/or a Metal backend from it is pretty straightforward.

[andreamancuso]

I'm definitely interested.

[wcandillon]

we publish them here: https://github.com/wcandillon/react-native-webgpu/actions/workflows/build-dawn.yml

[sjg-wdw]

It's interesting to revisit this discussion a year later. The Native Development team implemented a direct Metal renderer and is partway through direct Vulkan support. Having been through that experience I'm glad we didn't try a translation layer.

WebGPU support is probably even easier now if someone wants to add it to the Native version. The argument for it is more complex now, though, since we've got Metal, OpenGL and soon Vulkan.

[birkskyum]

Metal/Vulkan/OpenGL definitely changed the picture, which is great! I think a WebGPU target could be worthwhile still, mainly for long-term exploring our options in the MapLibre Native for Web space. It's obviously very experimental, but we should always keep exploring and pushing the boundaries. Current efforts (here, here) can help us track current limiting factors, even if it's not likely to bring viable solutions - for now that is. Basically I hope WebGPU can improve the performance of GL JS and portability of Native, and thus level the field a bit.

[sjg-wdw]

The Vulkan implementation is just about to ship for Android (pre-release) so I think we can say more about the difficulty of adding a new rendering SDK.

Adding WebGPU is probably now within the realm of possibility for a personal, volunteer project.