propose: js modular crypto

[vasco-santos]

No description provided.

[vasco-santos]

[rvagg]

I think that we can learn from the work we ended up doing in js-multiformats which was designed specifically with this problem in mind. It's built for composability and designed to let bundlers do nice things with only including the pieces that you actually use. When consuming multiformats and the newer IPLD stack, you only pull in the narrow stack that you want to use for your application. So if you want dag-cbor + blake2b then you get just the pieces needed for that and strictly nothing else. Which is really nice to use. It's different when bundled in js-ipfs of course but this is all part of the #47 story too.

[hugomrdias]

LGTM
Maybe we can add a note about the big number libs that are duplicated in each of these crypto libs. It's a hard problem to solve but i think we should try to figure it out.

[Gozala]

One challenge I see which might be worth calling out / articulating in the proposal is how do individual components communicate dependence on availability of certain cryptographic algorithm. Unilke with js-multiformats library it may not be all that obvious what libp2p component may require what.

[Gozala]

Are there any ideas how are we going to deal with third party libraries ? My understanding was that quite a few pieces were pulled by dependencies as opposed to libp2p itself.

[Gozala]

I would suggest to put crypto interface definition that can be evaluated fairly cheaply as a first step. I think js-multiformats does a good job doing it for multibase, multihash, multicodec and I would love similar thing for crypto.

[olizilla]

A tip from @carsonfarmer

I'd recommend checking out stablelib for pure js/ts implementations of just about everything crypto: https://github.com/StableLib/stablelib
– multiformats/js-multiformats#59 (comment)

[jacobheun]

It looks like leveraging stablelib could help us drastically reduce our dependency footprint. If there are any missing crypto items there that we need for our protocols we could reach out to Dmitry (owner) and fund that work. libp2p-crypto could then just become a light, libp2p spec compliant api on top of that.

[carsonfarmer]

I actually had started creating a stablelib libp2p-crypto implementation a while back, and should still have the code lying around if anyone wants to take it over. Probably about 50% done? My goal was to simply have a drop-in replacement for libp2p-crypto, which is the biggest culprit for bundle size for me right now. Very much happy to "jam" on any efforts here with the libp2p team, or help contribute if an externally funded project gets going.

I'd be happy to be a point of contact for an external team on the proposal/funded work if that is of use?

[hugomrdias]

i found an interesting repo for wasm based hashing https://github.com/Daninet/hash-wasm

[Gozala]

I have proposed a section to alternatives #71 (comment). I think it is worth considering what would it take to get necessary crypto APIs in browsers themself as opposed to bundling various user space implementations. It also might be worth looking into crowdfunding options for this as I expect there are plenty of other groups who'd benefit from having necessary crypto APIs in browsers.

P.S.: Last time I've checked there was no opposition to extending web crypto, it just lacked champion to drive the effort.

[mikeal]

There’s a lot of information that has been surfaced since the last edit, I’d like to see it addressed or worked into the proposal before we 👎👍

[vasco-santos]

Thanks for all the feedback! I added the discussed Alternatives and a brief plan of attack based on the previous discussion of leveraging stablelib.

From this and other project discussions, it seems that we should create a project spike on WASM integration to have a better idea of the challenges that we have there, in order to properly compare alternatives

[olizilla]

An old map appears! arewedistributedyet/arewedistributedyet#19 (comment)

2 years ago, the webcrypto guardians were a little more "nope" about streaming APIs but it now sounds like it really needs a champion to guide the work. w3c/webcrypto#73 (comment)

wasm crypto libs that do what we need has felt like a useful path to explore for a while. We should support (and sponsor) WebCrypto spec improvements, but to fully unlock that work we'd need to make sure that we could also get browsers to recognise ipfs://<cid> as a secure context (@Gozala is that an issue, do you have any insight on that?)

Also an off-topic reminder that https://github.com/arewedistributedyet/arewedistributedyet/issues is a repository of issues that could benefit the dweb beyond IPFS, and if you see anything missing, please add it!

[Gozala]

but to fully unlock that work we'd need to make sure that we could also get browsers to recognise ipfs://<cid> as a secure context (@Gozala is that an issue, do you have any insight on that?)

Depends what do you mean exactly. Navigating to ipfs://bafkreic4wacjgff32iqwgip5zolur3dnhiqku2xctmng4vezdqxaxxjkti/ suggests brave does treat it as secure context, while opera on my iphone is not. I'm not sure why Opera arrived to that choice or whether it is an accident, @lidel or @autonome might know more.

As of other browsers today best we can do is registerProtocolHandler which will effectively redirect ipfs://.

Primary challenges when pursuing making ipfs:// a thing at Mozilla had been:

1. The fact that CIDv0 was case sensitive and browsers treat host part of URL as case insensitive. Browsers also don't really do URIs. I think CIDv1 is the most effective way to overcome this challenge.
2. How does content from ipfs:// interacts with content on https:// and/or http:// is complex and challenging question. I'm unaware anyone tackling this and I think it's going to be a major roadblock in making ipfs:// be more mainstream. If you look at the spec that defines interaction model between https:// and http:// it is massive and really complex, I expect it to be lot more challenging to define interactions with ipfs://. Approach I was going for at Mozilla was not to allow any references across ipfs:// and http(s):// while treating ipfs:// as secure context.

[Stebalien]

How much would being able to opt out of some crypto modules actually help? Can we walk through some use-cases, pick the modules we'd be able to drop, then compute the savings?

For example:

• Libp2p probably doesn't need any hash function but blake2 for noise and sha2.
• Libp2p probably does need ed25519, RSA, x509, and sekp (maybe?). That's pretty big... Some applications might not need, e.g., sekp, but they might want to include it anways "just in case".

[rvagg]

Can we walk through some use-cases

I don't quite have the expertise to speak to this, but we should try really hard to extract ourselves from IPFS as it's instantiated today and think about narrow verticals of applications that might consume libp2p (or even just parts of it). What if I want an application that just talks to Filecoin nodes? What about if I have an application that just talks to specific Filecoin nodes (no discovery) purely for the purpose of transferring data for storage or retrieval (pretending graphsync+selectors is cool in JS for a moment)? Or more generically, what if I'm building my own p2p application that just uses a small part of libp2p to throw data back and forth and I have strong opinions on the various protocols that need to align in my stack to make that work.

This proposal should be about unlocking use-cases that currently pull in far too many modules and add overhead and complexity to a build pipeline (including bundling).

[jacobheun]

Closing this as it is not on our immediate roadmap. We will reopen when this work intersects with our future timelines and priorities.

[Gozala]

I have proposed a section to alternatives #71 (comment). I think it is worth considering what would it take to get necessary crypto APIs in browsers themself as opposed to bundling various user space implementations. It also might be worth looking into crowdfunding options for this as I expect there are plenty of other groups who'd benefit from having necessary crypto APIs in browsers.

@autonome do you think this is something worth having conversation about with igalia ? Seems like something whole web3 space could highly benefit from.