Common Node API

[ansermino]

This is for discussion around the FIP we are proposing to establish a common API for node implementations.

A draft PR has been created: #1027

The current abstract is:

Lacking a common API for all node implementations, client diversity is hindered as applications are unlikely to be interoperable with different node implementations. Lotus is the defacto reference implementation, but lacks a sufficiently specified interface for other implementers to replicate.

In this FIP a subset of existing RPC methods are formally specified to enable consistency across implementations, and provide clarity on the methods users can expect when interacting with a node.

This effort is partly inspired by our challenges implementing the current Lotus API in Forest. Most notably, the sheer number of methods, a lack of documentation, an incomplete OpenRPC document, and inconsistencies in how types and errors are handled across methods make replicating it especially difficult.

We have chosen to take a path of least resistance (ie. no breaking changes) and directly include a subset of the current interface of Lotus, choosing to omit error handling or access control to reduce complexit. We intend to continue to refine the interface beyond this FIP, with more significant changes in a future V2 via an additional FIP.

One discussion point is whether this should be an FIP or FRC. Any guidance or commentary here would be welcomed.

[rvagg]

One discussion point is whether this should be an FIP or FRC.

FRC's continue to be under-documented and under-specified and my take on why they exist is that there's a desire for a smoother pathway to publishing something in a central location that may encourage collaboration, coordination and compatibility without requiring the whole ecosystem to agree that it should be adopted.

The new proposed FIP-0001 text has a few more words regarding FRCs that are probably helpful: https://github.com/filecoin-project/FIPs/pull/850/files

IMO this is an FRC effort, we don't need to achieve consensus, what we do in the RPC implementations is not going to break network consensus per se, although it is fairly critical to builders on-top of RPCs, so coordinating in public and seeking input from consumers is good; it just doesn't need everyone to agree and the implementers are in the best position to decide both how much work they want to sign up for as a result of this and the practicalities of the APIs themselves. We should also prefer (I think) to not block on the heavy-weight FIP process to get this in. Let's get a basic agreement, land it, then iterate if we discover problems. Ideally the next step would be to do a v2 API together that's much more focused and consistent.

[BigLep]

For visibility, there was a recent FIL Dev Summit talk on this topic: https://www.youtube.com/watch?v=R_sFU97iMzU

[BigLep]

I think it's great that FRC / spec for this is being developed. We'll also want to have a way to test/show conformance. I'm dropping some notes here from the IPFS community.

IPFS made a push over the last couple of years to have more implementations. IPFS has a set of HTTP Gateway specs and they built a conformance test suite and tooling to support it. Here are some relevant links:

• https://github.com/ipfs/gateway-conformance
• https://conformance.ipfs.tech/
• https://www.youtube.com/watch?v=PmIf77thO_c

While written in Go, it uses a DSL that is very understandable and usable for anyone I think (look at some of the tests). This was important as this conformance test suite would be authored and reviewed by IPFS implementers, some of which don't know/use Go.t

The creators and maintainers of this work are still around (e.g., IPDX, IP Shipyard). I expect they would be happy to answer questions if we have any.

(That said, if there are better alternatives - that's great. I'm just passing on a recent option I'm aware of that has worked well for that community.)

I understand there is a precondition to run a Forest, Lotus, etc. node an "offline mode" serving RPC requests from a snapshot. I heard Forest has this (great!). Here's an issue to add this capability to Lotus: filecoin-project/lotus#12226