Merge multiple transactions into one

[Edison-Yeah]

Summary

Aggregating multiple transactions into a single transaction optimizes the user experience and reduces user overhead as paying for each transaction is expensive and cumbersome.

Abstract

In the process of transaction creation, multiple transactions are aggregated into one transaction, and they are processed separately in the execution,
thus reducing the cost of transaction execution and saving the cost of users.

Change Motivation

Chain bandwidth is a valuable resource, necessary for both critical network operations and product use.
Avoiding frequent and high-gas-cost operations reduces contention, costs, and chain validation latency.

Given the state of the network, users have to pay a fee for each transaction, and the cost of calling multiple methods in a period of time is a significant cost to the user. Imagine a scenario where a user needs to transfer money to multiple addresses at the same time, but each transaction can only be transferred to one address,
which is extremely inconvenient for the user. On the other hand, it is also a very troublesome thing when the user needs to check the fund flow of his account.

Specification

1. Aggregating information from multiple calls into the parameters of a transaction.
2. Mark transactions in a specific way, route them to a specific processing method for execution, and reduce execution costs.

Design Rationale

The handling of messages in the official project is quite mature, so changing the logic of the actor part of the code is not desirable. Therefore, in the stage of message routing, the message is intercepted, and then the message is split, and the transaction cycle execution after splitting is the most appropriate way. The advantage of this is that it does not affect the native logic, but only extends the functionality, ensuring the security and compatibility of the code.

The data structure of the message itself has jSON-formatted parameters, which can be used to carry multiple transactions. Multiple transactions are serialized into parameters and parsed at execution time.

Most of the cost of aggregating multiple transactions goes to tipping miners, whose job is simply to package transactions, so cutting tipping for miners can significantly reduce transaction costs for users.

Backwards Compatibility

None of these changes are intrusive changes to the official native code, but simply extend functionality to accommodate all operations.

Incentive Considerations

Aggregated transactions can greatly save users' expenses and facilitate users to check the flow of funds in their personal accounts.

Product Considerations

This proposal will greatly reduce the cost of user transactions, generally making use of the Filecoin blockchain cheaper.

Implementation

Implemented in lotus in filecoin-project/lotus#7727.

The preliminary implementation

1. The realization idea of transaction aggregation

The execution of transactions is implemented in the spec-actor layer. If intrusive changes are made to the execution logic of the transaction itself and the official code is changed significantly, the Lotus layer needs to work with the spec-actor to make the changes. So after thinking about it, try to intercept the transactions during the VM execution phase, route the aggregated transactions to specific processing methods, and break the aggregated transactions into small ordinary transactions to be executed by the VM.

2. Implementation method

2.1 Parameters and Methods

In the format of Message, the property params is []byte:

type Message struct {
	Version uint64

	To   address.Address
	From address.Address

	Nonce uint64

	Value abi.TokenAmount

	GasLimit   int64
	GasFeeCap  abi.TokenAmount
	GasPremium abi.TokenAmount

	Method abi.MethodNum
	Params []byte
}

The parameters of a particular data structure can then be serialized through JSON into Params. The data structure of parameters is shown as follows:

type Receive struct {
	To   address.Address   	`json:"to"`
	Value abi.TokenAmount   `json:"value"`
	Method abi.MethodNum    `json:"method"`
	Params []byte           `json:"params"`
}

type ClassicalParams struct {
	Params   []Receive   `json:"params"`
}

You only need to define the recipient address (to), the amount of transfer (value), the method to call (method), the parameter to call the method (Params), multiple Receive tokens to call multiple times, and then serialize it into the Params property of Message via JSON.
To indicate that Message is an aggregated transaction without changing the structure of the Message, the only way to do this is through the Message's Method, by specifying a special Method.

const (
	MultiMsgMethod = 100
)

Message's Method must be MultiMsgMethod as long as it is an aggregate transaction.

2.2 Perform

HandleMultiMsg() breaks down transactions into small transactions and executes them in VM. Each small transaction is handled by the corresponding Method. The State of the entire transaction is shared, that is, the execution of each small transaction has an impact on the State of other small transactions, ensuring that the change of State is correct after the completion of the entire transaction.
The failure of one small transaction can cause the failure of the entire large transaction, and State does not record the changes made by that transaction. Each small transaction has its own Method, so, in theory, this Method can be used for aggregate calls to multiple different methods.

2.3 Thinking about how to reduce user costs

In the execution of the VM, the message execution logic of the actor is not changed, so we cannot reduce the execution cost in the specific execution of the aggregate transaction by breaking it into smaller transactions. However, in the simulation calculation of Gas consumption for aggregated transactions, the calculated GasLimit will become very large. As a result, after the final transaction is executed, the MinerTip for miners will become very high, which is much higher than the sum of MinerTip for executing multiple ordinary transactions.

In this case, the idea is to cut miners' consumption. Here's why:

1. A miner's aggregated transaction is no different from a normal money transfer transaction, though the size of the transaction may be slightly larger.
2. Mine tips for aggregate transactions are higher than ordinary transactions, but lower on average than individual ordinary transactions.

2.4 implementation

To calculate the final Gas consumption, we have chosen to adjust the MinerTip, currently halving the consumption.

func (gsout *GasOutputs) SaleMinerTip() {
	sale := big.Div(gsout.MinerTip, big.NewInt(10))
	sale = big.Mul(sale, big.NewInt(5))

	saled := big.Sub(gsout.MinerTip, sale)

	gsout.MinerTip = big.Sub(gsout.MinerTip, saled)
	gsout.Refund = big.Add(gsout.Refund, saled)
}

That's all I have to say about trade aggregation.I've implemented the base code and tested it, and the results are as expected. This is the reference implementation link: https://github.com/IversonHan/lotus/tree/testhan

[anorth]

The primary objective of this proposal appears to be to reduce gas costs (as a proxy for network resource usage). Could you quantify how much is saved by batches of various sizes?

[Edison-Yeah]

The primary objective of this proposal appears to be to reduce gas costs (as a proxy for network resource usage). Could you quantify how much is saved by batches of various sizes?
Yeah, I have did some tests, and the result looks like this:

number of transactions    Overestimaton Burn      Miner Tip    Total Message Cost   averageMinerTip        averageMessageCost
       1                  1482200              49204936980      49245945980       49245945980            49245945980
       2                  2957000              49610249605      49692060205       24846030102.5          24846030102.5
       7                  10349500             173504779505     173791116605      24786397072.142857     24827302372.142857
       10                 14785000             246087300550     246496353550      24608730055.0          24649635355.0
       20                 29570100             491144575950     491962682050      24557228797.5          24598134102.5
       21                 31048600             516690185340     517549196740      24604294540.0          24645199844.761906
       25                 36962600             619641445500     620664078100      24785657820.0          24826563124.0
       50                 73925200             1238666847250    1240712112450     24773336945.0          24814242249.0
       100                147850500            2464076433000    2468166963500     24640764330.0          24681669635.0

Obviously, you will find that this saves a lot of money. For miners, they still only need to package one transaction, and the reward is greater; for users, it is faster and more convenient to complete multiple transfers in one transaction, only need to query one transaction, you can get full information.

[Edison-Yeah]

The primary objective of this proposal appears to be to reduce gas costs (as a proxy for network resource usage). Could you quantify how much is saved by batches of various sizes?
Yeah, I have did some tests, and the result looks like this:

number of transactions    Overestimaton Burn      Miner Tip    Total Message Cost   averageMinerTip        averageMessageCost
       1                  1482200              49204936980      49245945980       49245945980            49245945980
       2                  2957000              49610249605      49692060205       24846030102.5          24846030102.5
       7                  10349500             173504779505     173791116605      24786397072.142857     24827302372.142857
       10                 14785000             246087300550     246496353550      24608730055.0          24649635355.0
       20                 29570100             491144575950     491962682050      24557228797.5          24598134102.5
       21                 31048600             516690185340     517549196740      24604294540.0          24645199844.761906
       25                 36962600             619641445500     620664078100      24785657820.0          24826563124.0
       50                 73925200             1238666847250    1240712112450     24773336945.0          24814242249.0
       100                147850500            2464076433000    2468166963500     24640764330.0          24681669635.0

Obviously, you will find that this saves a lot of money. For miners, they still only need to package one transaction, and the reward is greater; for users, it is faster and more convenient to complete multiple transfers in one transaction, only need to query one transaction, you can get full information.

Of course, the size of the gas savings is adjustable, and the contents of this list represent the results of some specific parameters.