Introducing Hierarchical Consensus #419
Replies: 3 comments 3 replies
-
|
It's great. I have no comments on this FIP at this moment, but one thing came to my mind, I think we should consider once we implement this. That is the circulation supply calculation. We may need another thread to discuss this, but I'd like to bring it up here first. Our current circulation supply is defined as This is fine before the user programmable actors introduced, since we have all locked fil included into FIL_Locked. However, the user programmable actors could lock more tokens for particular purposes, the HC is an example since it has collateral and slashing mechanism. That means, it will be hard to have a pre-defined FIL_Locked formula. This may not be a problem for other chains since TVL can be defined and revised off-chain, however, in Filecoin, the on-chain economics is depending on it, specifically, the initial pledge calculation uses circulation as an input, and it impacts the whole system. |
Beta Was this translation helpful? Give feedback.
-
|
I've thought a bit about this problem in the context of user-programmable markets locking funds: https://www.notion.so/pl-strflt/Built-in-locked-funds-actor-for-accurate-supply-calculations-ab61bf312d3f43ef8f7acc5587b588d7 I don't have a good solution unless we're willing to relax the definition of "locked". In the meanwhile I propose removing deal collateral from the calculation. |
Beta Was this translation helpful? Give feedback.
-
|
Q：HC is the enablement of layer 2 (subnet) solutions but not necessarily increasing the throuhput of mainnet. Is this a correct assessment? |
Beta Was this translation helpful? Give feedback.
-
|
Correct. It will increase the throughput of the overall system, but it won't increase the throughput of mainnet's consensus layer. We have other projects actively exploring improvements over this layer (we'll share more when we have some results). |
Beta Was this translation helpful? Give feedback.
-
|
FYI this proposal's f4 address is relevant: #459. It should provide a general enough address type for HC subnets to use. |
Beta Was this translation helpful? Give feedback.
-
|
I do think that the f4 proposal will help here. Also note: Embedding numbers encoded as a base10 string inside a binary value (an HC address) isn't a good idea. This address should probably just be a sequence of varints where each varint is the "id" of the subnet relative to the previous varint. We can fit 7 subnets (at least) in 64 bytes (including the address class). I'm also a bit confused about the "raw address". Cryptographic keys shouldn't come into the equation here. |
Beta Was this translation helpful? Give feedback.
-
The goal of this discussion is to present the FIP draft we are working on for Hierarchical Consensus, and to start gathering feedback that can help us improve the protocol in our "road to production". Feel free to propose improvements over the spec by directly opening a PR here, or to open focused/topical discussions about parts of the protocol in this thread or here.
Related discussions
Simple Summary
To increase the range of use cases that the Filecion network can support, and to overcome future scalability limitations in terms of throughput and finality of the Filecoin blockchain, this FIP introduces Hierarchical Consensus (HC), a framework to scale Filecoin horizontally.
HC enables Filecoin users to spawn new subnets from the Filecoin network. Subnets instantiate new independent state and are able to run their own consensus algorithm. Subnets interact seamlessly with the Filecoin mainnet while parallelizing the validation (and execution) of messages. Our framework enables application developers to host their applications in networks with the appropriate security-performance trade-off.
This FIP introduces:
Consensusinterface that decouples the specific consensus implementation used for the different networks, and allows the Filecoin stack to interact and use any consensus implementation that follows the interface. This interface is shared by all HC-compatible subnets and consensus algorithms.Subnet Coordinator Actor (SCA), which implements the core logic of HC and governs the interaction of subnets with other networks in the hierarchy.CIDstored in the state of other subnets.We foresee proposing future FIPs for HC-related topics, including its gas and cryptoeconomic model.
Abstract
Consensus poses a major scalability bottleneck in blockchain networks. This is particularly the case when all validators are required to process all transactions, as is the case for Filecoin, which renders the network unable to increase its performance by adding more participants (scale-out). Even more, not every application benefits from using the same consensus algorithm: different applications may have different performance and security requriements, making it difficult for a single blockchain network with a single consensus layer to accommodate any type of web3 application.
With Hierarchical Consensus (HC), we implement a framework to enable on-demand horizontal scalability of Filecoin through the deployment of subnets (self-governing chains) that spawn their own state, validate messages in parallel, and seamlessly interact with any network in the hierarchy, as well as the Filecoin mainnet. Subnets can run different consensus algorithms, adjusting to the needs of the application.
This FIP introduces the architecture and essential building blocks for the operation of the protocol. It can be seen as complementary to the programmability introduced by FVM, in that it provides a framework to further program the Filecoin network, accommodating a wide variety of use cases while overcoming potential consensus bottlenecks.
Change Motivation
With the release of FVM, we should expect additional load and more diverse use cases in the Filecoin network. These use cases may push the Filecoin network's throughput to the limit. Even more, some use cases may not be compatible with the current block finality times of the Filecoin consensus. Providing a way to horizontally scale the Filecoin network while letting users determine the security-performance trade-off they want for their application can attract innovative use cases to the network.
This framework not only unlocks unlimited scalability for the Filecoin network through the on-demand horizontal scaling of the network, but it also provides additional benefits:
Specification
The spec for HC has been drafted here. Instead of copy-pasting the spec in this discussion, we refer to the link for the spec of the proposed changes for the FIP. The sections in bold are more closely attached to the current Filecoin consensus and should warrant more care. A brief explanation of the reasoning is included below the link. We'll follow the feedback of this discussion but if you want to give feedback or propose a change over the spec feel free to directly open a PR in the aforementioned repo.
SubnetMintmethod in theRewardActorto mint new native tokens (FIL) into subnets after a top-down message. We can use a custom builtin-actors bundle in subnets to avoid including this method in theRewardActorof the root network.Design Rationale
In traditional distributed computing, one possible approach to overcoming scalability limitations is to resort to the partitioning, or sharding, of state processing and transaction ordering (like other projects in the space are doing). In a sharded system, the blockchain stack is divided into different groups called shards, each operated by its own set of nodes, which keep a subset of the state and are responsible for processing a part of the transactions sent to the system.
The main challenge with applying traditional sharding to the Byzantine fault-tolerant context of the blockchain lies in the security/performance trade-off. As miners are assigned to shards, there is a danger of diluting security when compared to the original single-chain (single-shard) solution. In both proof-of-work and proof-of-stake (PoS) blockchains, sharding may give the attacker the ability to compromise a single shard with only a fraction of the mining power, potentially compromising the system as a whole. Such attacks are often referred to as 1% attacks. To circumvent them, sharding systems need to periodically reassign miners to shards in an unpredictable way, so as to cope with a semi-dynamic adversary. We believe that this traditional approach to scaling, which considers the system as a monolith, is not suitable for decentralized blockchains due to their complexity and the fact that sharded systems reshuffle state without the consent of its owners, disrupting use cases that benefit from finer control over the system topology.
With Hierarchical Consensus, we depart from the traditional sharding approach and, instead of algorithmically assigning node membership and load balancing the distribution of the state, we follow an approach where users and miners are grouped into subnets in which they can freely partake. Users (i.e. network participants) can spawn new blockchain networks, or child subnets, from the one they are operating in, according to their needs and become miners there if they fulfill all the requirements set by the protocol. Each subnet can run its own independent consensus algorithm and set its own security and performance guarantees. Subnets in the system are organized hierarchically: each has one parent subnet and any number of child subnets, except for the root subnet (called root network or rootnet), which has no parent and is the initial anchor of trust. To circumvent the 1% attacks pertinent to traditional sharding, subnets in hierarchical consensus are firewalled, in the sense that a security violation in a given subnet is limited, in effect, to that particular subnet and its children, with bounded economic impact on its ancestors. This bounded impact of an attack is, at most, the circulating supply of the parent token in the child subnet. Moreover, ancestor subnets help secure their descendant subnets through checkpointing, which helps alleviate attacks on a child subnet, such as long-range and related attacks in the case of a PoS-based subnet.
Scalability proposals explored by other projects and used to inspire this work include:
Original paper with a high-level overivew of the proposal:
Backwards Compatibility
All the changes in this FIP are incremental. There shouldn't be any issues of backward compatibility.
Test Cases
Security Considerations
Incentive Considerations
This FIP will greatly increase the capacity of the Filecoin network, by enabling the spawning of subnets, as well as message execution within these subnets. This can simultaneously result in a large increase to the overall transaction volume and in a decrease of utilisation of the main chain, thereby potentially impacting the base fee. Moreover, collateral used to secure the subnets will impact the FIL circulating supply.
Note, however, that all subnets are required to checkpoint on their parents and that cross-net transactions will also traverse the network hierarchy, thereby maintaining coupling and generating activity on the root network.
Work on cryptoeconomic modelling for subnets is currently ongoing, and incentive considerations related thereto will be introduced in an additional FIP. The relevant parts of the spec are:
Product Considerations
As described in the Change Motivation section, HC offers newfound flexibility and the potential of unlimited scalability for the Filecoin network. This will foster innovation at the consensus layer and beyond, and it may attract brand new and existing use cases to the network.
This change also opens the door to deeper innovation in the blockchain space (like the support for partitioned networks, the complex routing of messages between networks, the deployment of use-case-specific network interoperable "by-design" with existing ones, etc.).
It is impossible to foresee all future uses of HC, but we will monitor its usage once it becomes part of the network, in order to prioritize work on future evolutions of the protocol.
Implementation
A reference implementation of the HC over a Lotus fork using the Legacy VM (including the SCA and a reference subnet actor implementation as specs-actors) can be found in this repo: https://github.com/filecoin-project/eudico/
LegacyMVAn MVP to test the compatibility of the HC framework targeting FVM can be found in the following forks (these forks may not be feature complete; we are targeting feature completeness for FVM-M2. With the recent updates in all FVM-related repos, these forks may be temporarily broken. We'll fix them and post them in the final FIP submission):
And we are working on an implementation of HC over a fork of Forest (some of the changes and refactors are also being upstreamed for
ChainSafe/forest): https://github.com/aakoshh/forest/Copyright
Copyright and related rights waived via CC0.
Beta Was this translation helpful? Give feedback.
All reactions