Simple termination fee, and export method for computing miner maximum aggregate termination penalty (FIP-0098) #1036
Replies: 10 comments 19 replies
-
|
imho, we could remove termination fees all together to relieve core protocol complications. How termination fee is structured should be exposed to FEVM to allow clients and providers to dictate their own storage term instead of being severely limited by the core protocol defaults. |
Beta Was this translation helpful? Give feedback.
-
|
Def |
Beta Was this translation helpful? Give feedback.
-
|
#972 would only remove termination penalties for CC (non-QAP?) sectors. This would still remain a problem for FIL+ sectors. But a simpler termination fee calculation along the lines of @Schwartz10 #2 could work for all sectors. |
Beta Was this translation helpful? Give feedback.
-
i think termination fee would somehow make sense under vanilla Filecoin, but with FEVM introduced imho some of the previous protocol design choices have to be reconsidered, ie a revamped protocol master plan. I mentioned here in May 2023 about termination fee and its complications that it brought to defi. I personally advocates for removing both termination fees (CC and QAP sectors), as fundamentally imho storage terms should be between clients and providers not to be limited / interfered by core protocol. |
Beta Was this translation helpful? Give feedback.
-
|
I think the termination fee should be greatly simplified for this and other reasons. The current calculation also contributes hugely to chain state size and conceptual complexity (discussed in #691) Either a flat termination fee that doesn't depend on history (your #2) or removing termination fees in return for an unlocking delay (see #972, but only for CC sectors) would be improvements all round. |
Beta Was this translation helpful? Give feedback.
-
|
Strong agree, and flat termination fee that doesn't depend on history makes sense to me. So much simpler! |
Beta Was this translation helpful? Give feedback.
-
|
A quick update on this discussion thread - @anorth and I started some high level brainstorming, and generally we both felt excited about simplifying the mechanism for computing termination penalties to a fixed percentage of either: (a) some amount of future block rewards given current network conditions and current power or (b) the total initial pledged balance There are a number of different questions (each with their own sub-challenges) to answer when solving for this approach:
As a first step, we'll collect some data about the network of miners, their power, and their termination penalties. Will keep the discussion thread updated as we learn more information // form any stronger opinions around valid mechanisms and approaches. |
Beta Was this translation helpful? Give feedback.
-
|
Here is some analysis of the termination fee and its ratio to pledge for sectors currently live on the network. This is based on a uniform random sample of 6.8 million active sectors a couple of weeks ago. (Note that the termination fee calculation used in this analysis is slightly simplified from the network and does not take into account the effects of upgraded CC sectors.) Absolute termination fees vary widely and the variance is strongly correlated to the sectors activation epoch. This is expected since the termination fee calculation is largely based on the expected rewards at the time of onboarding. Per sector rewards decrease over time due to both the decaying block reward and the lower share of total power each new sector earns in a growing network. Multiple downward sloping trends are visible corresponding to 32 and 64 GiB sectors and to those with or without a Fil+ multiplier. 
Termination fee as a fraction of pledge also varies in the same way, but here the pledge denominator unifies sectors with different power into a single trend line. (I think the unusually low-fee line visible along the bottom is an artefact of not computing the fee for CC-upgraded sectors properly). The current level is about 8% of pledge: 95% of sectors have a fee:pledge ratio > 8%, and 99% are >7.5%. 
General results:
|
Beta Was this translation helpful? Give feedback.
-
|
Last week at the core devs monthly meeting, I presented the ideas written in this discussion. The general sentiment seemed positive and optimistic, and the main response was to propose something more specific for the community to consider. 2 potential solutions for calculating termination penalties
The This method would be extremely simple for anyone to calculate, which comes with the following benefits:
In my opinion, this method is preferred because its the simplest.
This strategy would also significantly simplify the process of computing termination penalties. To compute the termination penalty for a sector, one could derive an expected daily reward number from a sector's power, and then use the expected daily reward number (along with some multiple or %) to calculate the termination penalty. A miner actor's aggregate termination penalty could be computed from its aggregate power. While both strategies would accomplish the intended goal, I prefer strategy 1 because strategy 2 is still somewhat confusing for outsiders to understand, difficult to do in a "napkin math" kind of way, and contingent on network conditions (so harder to forecast). It also can result in situations where the termination penalty ends up being a large % of the sector's initial pledge, if the network baseline starts increasing quickly after pledging a sector when the cost to pledge was much cheaper. In other words, with strategy 1, Filecoin should be able to erase the concept of "fee debt". With strategy 2, "fee debt" could still occur. Upgrade pathAs long as the new termination penalties computed are not massively increasing compared to the current network condition, then it should be possible and not dangerous (actually beneficial for several miners) to upgrade existing sectors to have termination penalties computed in the new strategy. In other words, if we just take the current termination penalty of the network (which is already lower than the average sector's termination penalty), it should not negatively impact any miner on the network I would strongly recommend against having differing sector types with different termination penalty calculations just to not change/touch existing sectors, however, this actually worsens the current problem that exists today (termination penalties are way too complicated to compute) Eager to hear any feedback! |
Beta Was this translation helpful? Give feedback.
-
|
Great, thanks for a very concrete recommendation. It sounds good to me. |
Beta Was this translation helpful? Give feedback.
-
|
Using magic number seems ad-hoc-ish and potentially kicking the ball down the road, ie we could tweak the magic number in the next upgrade line of thinking. imho, removing all termination fee for all sectors seems to be the simplest and most rational given we are at post-FEVM era that storage terms could be settled between clients and providers with contracts and not to be limited / interfered by core protocol. |
Beta Was this translation helpful? Give feedback.
-
|
@Fatman13 thanks for weighing in on this. I think we're on the same page about a desireable "end state" for filecoin with respect to fees/penalties being enforced by third party apps and protocols, but we may see things differently on the "steps to get there". This proposal gets us much closer to the end state that you describe while also introducing immediate benefits - i can't think of a good reason not to pass it even if it doesn't immediately fit 100% of what you're looking for. More notes below:
Think it's important to clarify what you mean by "simplest" here - from my perspective, it's much "simpler" to implement the proposed solution in this discussion, even if solely for the reason of it's (a) not contentious, (b) easy to understand, and (c) does not block us from getting to the end state you describe. I visualized this in a little venn diagram here:
I generally agree with you here too - i think the bulk of disincentives towards deterring someone from terminating any sector type should come from a specific market smart contract that can enforce its own rules and collateral types. This makes the most sense because i dont think there exists a "one size fits all" penalty for every type of deal, and this type of strategy scales flexibly to fit any market type.
This isn't kicking the ball down the road on anything. It immediately gets us the majority of the way closer to the end state that you describe from both a technical and conceptual perspective - simply set the % of pledge variable to 0, and we've achieved the state you're looking for. Easy easy easy If this FIP is passed, it immediately creates a foundation for safer Filecoin DeFi, and massively simplifies the conceptual design and code/state implementation of termination penalties. It is my strong opinion that we should not be holding up this FIP because we disagree on what the magic number is (0 or otherwise). It's way easier to lower termination penalties over time than it is to increase them. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the reply. I think there is truth in your reasoning especially when taking current FIP governance process & other factors into considerations. Hope the network could eventually reach the "desirable end state" one day. |
Beta Was this translation helpful? Give feedback.
-
|
#844 (FIP-0078) is asking "Removing Restriction to Access to Multiplier and Allowing Unrestricted Minting of Datacap", which actually is asking eliminating real data governance involved in layer 1, and of course, it can be done on upper layer. #1009 (Proof of Data Possession (PDP) [was PoCD]: Efficient Retrieval Provably Enabled) is a project which can take the role similar as Fil+ on layer 2, with decentralized provable mechanism. With these in mind, I would think we could move to the target of elimination of all termination fee or slashing fee to make things much simpler. That's why I am supporting all of these. Pass and implement FIP-0078 first, and implement the first version of PDP, which can be improved more later, and simplify the core protocol. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for providing feedback @steven004 ! I agree that it is important to simplify the core filecoin protocol, and I see this fip as a major step in the right direction. What do you mean by “pass and implement FIP-0078 first”? If you are suggesting to block this FIP until another tangential FIP is passed, I don’t think this is a good idea. I see no reason to create arbitrary blockers to pass this FIP especially when this FIP makes it easier in the future to eliminate fees all together (if that’s what the community agrees is best). This FIP will massively improve and further secure filecoin defi - why wait? |
Beta Was this translation helpful? Give feedback.
-
|
Chiming in to say that until this is solved, you can use tvx and the conformance test suite library to simulate a termination against mainnet and get an exact number. It's not on-chain, but it's the highest-fidelity off-chain solution we have today given it uses the actual actor code. |
Beta Was this translation helpful? Give feedback.
-
|
I understand the utility of having the Max Termination fee calculation be simpler, specially as more DeFI applications pick up, and sectors can be used as collateral. I think actually a large part of the difference in % of slashable pledge between sectors, is more due to the initial pledge than the termination fee. For instance, in the example you give:
I think most of the difference in the final result is due to difference in initial pledge, rather than termination fee calculation. And while yes the termination fees are different, maybe this particular example makes the problem look even worse than it actually is. So I think part of the solution to the problem here is rather simplifying initial pledge. I think my Flexible target lock proposal goes some way into simplifying this, in that sectors would be free to pledge however much they want, and can choose to make things simpler. That said, let's get back to simplifying the termination fee. Let's look back at the Filecoin spec that says what the Max termination fee is actually supposed to be doing:
Back then there were only 6 month sectors, and maximum termination fee was supposed to be capped at half of a sector's potential earning (Now there are longer sector lifetimes, but ok, let's say Max termination fee was still about 3 months of block reward). What is missing from the spec is that it doesn't specify which half of the sector's earnings we are talking about. In the implementation of this idea, this was interpreted to mean, the initial half of sector's earning, So the initial state for all sectors must be remembered, and this yields different max termination fees for different sectors that were onboarded at different times. One possible solution to simplify the termination fee calculation, is to keep the spirit of the spec's definition (3 months worth of block reward), but interpret that to mean, three months of expected future block reward, starting from present time. This calculation would be similar to the current calculation of the Storage pledge, which only requires the alpha beta filter predictions for total block rewards and network qap. So only 4 numbers need to be remembered (and they are already remembered for the storage pledge), and the same numbers are used for every sector, with everyone having the same termination fee. One can argue "3 months of a sector's potential earnings" is too high or too low, but that's at least what the current spirit of the definition is, whether we think it is too high or too low is a separate discussion that is not about simplifying the termination fee, but about changing it. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @AxCortes these are good points and help focus the discussion. Firstly, I think your proposed possible solution is the same as, or very close to, option 2 presented by @Schwartz10. It could be simplified even further to ignore the alpha/beta projections and still have the same practical effect (I'm not sure whether @Schwartz10 intended to use the projection or not). But here we can see good alignment and if this were the ultimate outcome then I think we'd all consider it a worthwhile improvement. You're right there's a further part of the discussion which is more about changing the calculation method of the fee. Given the high cost of making change, if we're going to touch it, what would be an even better outcome? I think this is worth designing and debating this at the same time. In my opinion, the spec's basis for a termination fee is outdated. I summarised a range of possible motivations for a termination fee in #691 (comment) . The spec doesn't give any justification for the capped half of rewards, but my best understanding of it is a combination of client assurance, enforcing "commitments" and stability of storage and pledge. Of these, the only one I believe to be well-conceived and justified is stability. There seems to be broad agreement that client assurance should largely be left to negotiations between providers and their clients rather than a network built-in, and "commitments" is just moralising. Stability of the bases of network security (especially pledge, which dominates), is more reasonable. Denominating termination fees in pledge would directly tie the penalty to the desired stability. This is the same conclusion reached in #972 (comment) (though that discussion then proceeds to find a way to replace penalty with an unlock delay in the case of CC sectors). I think you're right to attribute a large part of the difference in the recovery rate example to different pledges. From another point of view, the difference in recovery rates is because the TF is ignoring the pledge! If we adopted something like your flexible target lock, or an alternative that would generate more variance in pledges, I think TF based in pledge is exactly what we'd want. Otherwise SPs could engineer an extremely low cost of churning pledge by putting massive pledge against few low-fee sectors. Other conceivable future changes could be supporting zero-pledge sectors (no power/rewards, just serving clients) and no-storage pledging (rewards for network economic security) and I think again in these cases, TF aligned with pledge is closer to what we'd want (no TF for zero-pledge sectors, unstaking costs a fee or unlock delay). So, if we can't gain broad alignment that pledge is a better basis for TF, at least simplifying them based in BR is still a step forwards. But I believe the spec's conception of termination fees is flawed or at least outdated, and if we're going to change it we can do better. Note in all cases I'd favour maintaining the nominal fee for a new sector at the current levels [edit: except reducing for CC might be ok]. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for weighing in @AxCortes ! I strongly agree with @anorth opinions in the comment above. The only other thing I would add is that on top of pledge being a better basis for TF from an economic perspective, it's also a simpler conceptual design, which has positive benefits. It makes it easier for SPs and devs to reason about without having to know, understand, or make predictions about the state of filecoin's future block rewards. |
Beta Was this translation helpful? Give feedback.
-
|
I am not against the idea of a FIP doing two things: simplifying AND changing the amount of termination fee. Just that then it should be clearly stated, say in the title as "change in termination fee" and not just "simplification". I agree parameters are difficult to change in Filecoin, and if we think it should be changed, better to use the same opportunity. I think the Max Termination fee is meant to be sort of the opposite that you describe. What provides the "commitments" and stability is the definition of termination fee that says "All the block rewards the sector has earned so far". This is pretty dramatic, but ensures this incentive, that the strategy of terminating your sector before you're supposed to has negative utility. What the "Max" termination fee does is try to control in the opposite direction for how dramatic this is. The reason we have a max termination fee is "to not discourage long term sectors". Because the longer your sector has lives, the larger an amount "all the block rewards the sector has earned so far" is. So the spec states the rule, we want this rule, up to some point, when the punishment becomes too large, and now it is discouraging long term sectors. Since Termination fee in general is defined in terms of block rewards, this is why it makes sense to me that Max termination fee is also defined in terms of block rewards. Initial Pledge and block reward will continue to have diverging ratios over time, but max termination fee should be attached to block reward, as termination fee is. |
Beta Was this translation helpful? Give feedback.
-
|
Sure that's helpful feedback about you interpreting simplifying as "not changing" - i don't see how such a thing is possible (simplifying term fees without changing them), but I'll incorporate this feedback in the fip draft title. I think we're on the same page about the appetite for termination fees in general, but we may disagree on the significance of fees being denominated in block rewards. Term fees are meant to do a couple things:
We can see in practice that termination fees don't actually accomplish number 1 very well. Your point: "This is pretty dramatic, but ensures this incentive, that the strategy of terminating your sector before you're supposed to has negative utility." is not always true, even when block rewards are high. This is largely due to the fact that Storage Providers still value their cost basis in fiat terms, not FIL terms. Terminating a sector early guarantees negative economic FIL-on-FIL consequences, but it does not guarantee negative economic fiat denominated consequences. For instance, even if the termination fee is 30% of the pledge, an SP is economically better off terminating all sectors and selling if the price of FIL will fall >30%. Re point 2 - today we are operating in a network with very low termination fees as a portion of initial pledge, and the network power isn't super volatile. We know from practice that unless a Storage Provider has a major change in funding (i.e. investors decide they want to sell FIL), that the Storage Provider does not want to take any fees (even 5% is a lot to them), so they won't just magically terminate willy nilly unless there's a forced reason. This is proven even in extreme scenarios - during the stFIL incident, a storage provider I know wanted to wait out the risk of losing 100% of their miner collateral because they didnt want to terminate to lose 10% (which was their fee as a portion of pledge at that time). Regardless, I don't think denominating the fee in fixed % vs block reward terms makes a significant difference to the Storage Providers when they have to terminate sectors
This proposal is to set termination fee == max termination fee, there wouldn't be a difference. My last point, Storage Providers that are actually creating value for paying storage clients have a lot more to lose than some FIL on a sector basis when terminating an SLA - these are data center businesses, so they lose reputation, deal flow, customers...etc. Since there are so many variables for a storage provider in practice, it doesn't make sense to treat the termination fee as if its the only factor an SP considers when terminating sectors. We don't have to get too elegant with implementations to accomplish the end goal - the simpler the better |
Beta Was this translation helpful? Give feedback.
-
|
I have replicated the analysis above on a new raw sector data export provided by @rvagg (see here). The correlation charts look essentially the same, so I won't re-post them. For selecting a policy value of the ratio of termination fee to pledge.
I would propose a value of 8.5%, being just on the recent mean. It's likely this would represent a small reduction in risk to new sectors onboarded at the time this change could go live. As an aside, I observed that some of the oldest sectors have a termination fee over 200% of the initial pledge! Another sign that the current methodology is inappropriate. |
Beta Was this translation helpful? Give feedback.
-
|
The first draft of FIP-0098 specifying this is about ready at #1079 |
Beta Was this translation helpful? Give feedback.
-
|
@anorth I have a question about this: in the original design of storage fees (ie, fault fee, invalid proof fee and termination fee) we requested that TF > FF, otherwise an SP could terminate the sector to avoid paying FF for detected faults. With the new proposed formula (TF = % of IP), in the future, it could be that the condition is TF > FF is not true anymore (say for example that we will change the IP value). In this case if a sector becomes faulty, terminating it is the rational (ie cost effective) strategy, which it is not optimal imo. We would like to se SPs try to recover fault sector instead of terminating it. A solution for this would be setting TF = max{% of IP, FF}. cc @lucaniz |
Beta Was this translation helpful? Give feedback.
-
|
One way we could preserve this ordering while keeping simple termination fees is setting FF_new = max{% of TF, FF_old}. Moving complexity over from termination to fault fee would be simple to implement, efficient to calculate and I think solve everyone's problems. |
Beta Was this translation helpful? Give feedback.
-
|
sorry min{ % of TF, FF_old}. The idea being it will always preserve TF > FF |
Beta Was this translation helpful? Give feedback.
-
|
Minor note: due to the analysis we need to have TF > FF, thus we need to have some multiplier to make things work (i.e. it should not happen TF = FF ). Moreover, not sure if the max is working here right? Imagine we are in a situation where TF < FF_old, you are not meeting the ordering (i.e. FF_new = max{% of TF, FF_old} = FF_old > TF ) [sorry was typing and did not see you aready modified it] |
Beta Was this translation helpful? Give feedback.
-
|
Yes, adding some constraint here makes sense. I think either constraint could work.
Is there a strong reason to prefer one way or the other? If not, let's take the one that introduces the least complexity. Let's add a, say, 1.05 multiplier to keep TF > FF. |
Beta Was this translation helpful? Give feedback.
-
@anorth as long as I'm understanding correctly, I have a strong preference to not add multipliers to TF (keep TF to fixed % of pledge). Adding a multiplier to TF to keep TF > FF adds complexity to downstream protocols that rely on TF for computing collateral values. Even if there's only a few edge cases where TF > FF, and those edge cases would trigger a multiplier to TF, then safe defi protocols must treat every TF as TF * max multiplier (or risk being under collateralized). The consequence is this penalizes the capital efficiency of defi markets on filecoin Since FFs are once a day, as long as the protocol checks collateral values once a day, varying FF don't pose as much of a complexity challenge to defi applications |
Beta Was this translation helpful? Give feedback.
-
THE PROBLEM
In the year+ since FEVM’s launch, we’ve seen a number of protocols use Miner Actors as collateral for lending or leasing applications. Economic security of applications being built on the FEVM is important for developing trust, and understanding precise collateral values for any given miner actor is critical for a DeFi protocol’s economic security.
Today, it is both: (1) computationally expensive, (2) economically sophisticated, and (3) infeasible in a FEVM runtime to compute the maximum termination penalty for a miner actor’s sectors, which makes it infeasible to compute a "collateral value" (aka "liquidation value") for any miner actor. As a result, FEVM applications need to either (a) use heuristics to determine a collateral value for a miner (economically insecure), or (b) use off-chain solutions to compute collateral values (significantly bloating the surface area of attack vectors for any lending / leasing protocol).
Imagine running a lending business that accepts medieval coins as collateral, yet when you accept the coins as collateral, you either have to:
It's clear neither of these approaches sets the lending business up to run efficiently and without incurring unnecessary risk. It also provides unique and potentially unfair advantages to those that can accurately and quickly value miner collateral.
Exposing a method that will allow a caller to get a precise "maximum termination penalty" for any given miner on the network will allow folks to quickly and safely compute collateral values for miner actors.
Simply put:
collateralValue = availableBalance + initialPledge + lockedRewards - maxTerminationPenaltiesEXAMPLES
The collateral value of a Filecoin miner deviates significantly, making it too dangerous to use any heuristic based approach without introducing risk-of-loss for liquidity providers. One way to inspect collateral value volatility in various miner actors is to compare their "recovery rates" - the amount of FIL that will be left on the miner after terminating every sector divided by the total balance on the miner. Take a look at the recovery rates computed on these two different miners
(liquidation value / balance):In other words, assuming 100% of a miner's balance is pledged, a miner with a recovery rate of 50% will have a collateral value of half its balance, while a miner with a recovery rate of 90% will have a collateral value of 90% of its balance. There is no other easy to find signal one can use to determine these wildly different recovery rates besides actually simulating a termination of every sector.
So how is a lending business able to run with economic safety on FEVM today? Let's look at two real examples to discuss their strategies, both with pros and cons:
The oracle service itself has cost us several hundred thousand dollars in audits, an enormous amount of security research, and a deep understanding of filecoin. This is not an option for pretty much any other dapp builder on the filecoin network, as glif is uniquely positioned from our long term work in this ecosystem, benefit-of-doubt on trust assumptions, enough resources from grants and venture capital...etc.
This approach necessitates the use of a server controlled by the glif team, which puts the business in a different regulatory position. And worst of all, oracles create a large security attack surface area of defi protocols. It doesn't matter how secure a smart contract is - if you write an insecure or inaccurate oracle, it's game over for your protocol.
This is much simpler from a technical perspective, however, it introduces massive economic risk. In fact, when stfil was being shut down, our team did a deep analysis of the stfil active loans and found that many were massively under collateralized, yet the protocol itself recognized them as over collateralized. The hidden information was that the miners borrowing from stfil sometimes had unusually low recovery rates, so treating pledged collateral as 1:1 with liquid FIL was a major mistake and overestimation of collateral value. Even CeFi lenders have been using heuristic based, network wide approaches to recovery rates.
TENTATIVE SOLUTIONS
There are generally 2 approaches, both of which end with the ability to quickly and precisely compute the maximum termination penalty for a miner actor. From there, the caller can use the termination penalty to compute a collateral value for the miner.
Re option 1, here's one potential (naive) idea for the core devs, i'm sure it has many flaws. From our testing and research, the go implementation of builtin miner actor is still correct in computing termination penalties. Judging from the code, the penalty is a max between (i'm simplifying here):
The reason we must loop through every sector is to compute this max, which is different for each sector, solely because of the information that is used to compute #1. If we only had to look at #2 (current network conditions & sector power), my assumption is that we could run an algorithm that would use aggregate power on the miner (not per sector) to determine the termination penalty.
The change in data structure would be to add a new cached variable on the miner state itself that holds the aggregate
twentyDayRewardAtActivationfor all sectors, such that you could compare #1 and #2 described above on a per miner basis, rather than a per sector basis.If this worked and maintained the mathematical integrity of the granular looping approach, it would alleviate the need for a highly complex and computationally intensive liquidation value algorithm, which would then enable lotus to expose this value in a precompile so FEVM smart contracts could use it
SUMMARY
Beta Was this translation helpful? Give feedback.
All reactions