Repeated SnapDeals

[Kubuxu]

This is a proposal to allow ReplicaUpdate to be performed multiple times on the same sector.

Motivation

SnapDeals shipped with limited capability due to limitations of the larger system. These limitations are mitigated with the introduction of FIP-0045. Additionally, the ability to perform SnapDeals upgrade on the same sector multiple times would allow Storage Providers to accept a wider variety of deal terms and make sectors that went through their first SnapDeal still useful for data storage after the initial deal expires.
It is the last capability missing from enabling the full utility of all sectors in the network.

Proposal

Allow sectors which underwent the SnapDeals process in the past to repeat it. During the design of SnapDeals, this possibility was taken into account by keeping the OrginalSealedCID in SectorInfo, which is required for Repeated SnapDeals to function.

There are two possibilities for the scope of this proposal:

1. Ability to perform a SnapDeal on a sector where all deals and FIL+ claims have expired.
2. Ability to perform a SnapDeal on a sector where there are still active deals and claims by preserving active deals and claims.

The second one has the added complexity of having to verify that the data corresponding to active deals and claims was kept within the sector after a ReplicaUpdate.

Fortunately, the PieceCIDs and sizes of deals and claims are kept on-chain to facilitate such constraint.
New methods for Market and Datacap actors would have to be introduced to facilitate the status and data description access during the ReplicaUpdate process. This information is already planned to be at least partially exposed for the purpose of user-programmable actors interop.

As a way to limit the complexity of the ReplicaUpdate method, the expiration of the sector has to already suit new deals and allocations. The SP is free to perform sector extension before the ReplicaUpdate as long as it complies with constraints given by active FIL+ claims.

From the perspective of proof production and encoding, the ReSnap process decodes the existing data from the sector, a trivial operation of if an extra copy of the deal data was kept. The result of this operation is sector data corresponding to a CC sector which underwent the SnapDeals process. Then the SP packs new and existing (if we adopt scope 2) deal data into the clear sector (the position of the data within the sector can be chosen by the SP) and produces a ReplicaUpdate proof.

Repeated SnapDeals use the same proving circuit as the original SnapDeals, avoiding the need for a new proof type and trusted setup. The computation time for the proof is the same as regular SnapDeal. This also means that SnapDeal introducing small changes to the sector will have the same computational cost as SnapDeal affecting the whole sector.

During the process of the repeated ReplicaUpdate, for a short time, the SP will have to keep sector information for three copies of the sector, the currently active sector with user data, the clear sector and the new sector with new data. The clear sector can be removed after then the new sector and proof are produced. The "currently active sector" can be removed after the proof for the new sector is provided on-chain, validated and the 900 epoch delay period expires.

[ZenGround0]

This information is already planned to be at least partially exposed for the purpose of user-programmable actors interop.

👍

We should view it as a requirement on the first User Programmable markets FIP that this is supported. All markets implement a standard interface that takes in the sector identifier and the new CommD and return success/fail. I don't think we need old CommD but it's worth thinking about that and if so how we support proving that old CommD is correct( related #496). Knowing which markets to call based on a sector is a bit tricky but I think the ideas from #530 will serve us well: abstractly keep a commitment to the notifees and have SP provided data. For first implementation we can simply keep notifee list on the sector infos and carefully keep this detail out of all APIs.

[ZenGround0]

For QA power I think we can get away with no changes to verifreg actor even in scope 2. There are three different kinds of pieces with datacap: pieces being added, pieces being removed, pieces staying in place. We handle all three cases as of nv17 in extension + prove commit / replica update.

[jennijuju]

In this post #554 (reply in thread) it’s suggested that pre shark sectors cannot be snapped. Im having some trouble with understanding the complexity and the gap - @Kubuxu have you any thoughts into it? If so mind shedding me some light?

[anorth]

These limitations are mitigated with the introduction of FIP-0045

FIP-0045 fixes the main limitation, which was complex QA power calculation. But it only applies to new sectors.

An opt-in migration from old to new may be possible, but has limitations.

[jnthnvctr]

summarizing some addt'l threads and comments that may be useful in shaping discussion here:

• there's a number of paid storage on-ramps coming to market that would like to be able to resnap into sectors
• specifically this would simplify billing with SPs - for many on-ramps theyre paying SPs for capacity, and reselling that into clients
• the challenge is if you can't lifecycle deals inside of a snapped sector - the onramp needs to either (a) force customers to commit to 6 mo+ deals (fine for some class of enterprise deals, much harder for more dynamic data or for folks building saas-y products) (b) just buy more and more capacity (much less financially viable for an onramp, leads to less data directly shipping into sectors) (c) don't auto lifecycle deals into sectors (bad for the network, more expensive but more flexible for clients)

It's worth noting that there seems to be more recent discussions about deletion (intentionally overwriting data) which can help with some enterprise conversations as well. Folks from the client-wg might be able to weigh in (I will link them to this comment to thread) more details.

@anorth has put together a more encompassing revamp of the deal / sector relationship here, related to a talk he gave here.

I think there's probably two contexts that are worth discussing (and would love to help motivate):

• Short term: how do we ship something that unblocks enterprise on-ramps that are targeting bringing paid deals into the network (knowing many of these are going live in q1 and ramping) => this is important for bringing cashflows into individual providers (but also more broadly can help grow the ecosystem writ large). Would love to figure out what the MVP version of unblocking teams here are given the importance of timing. I think the challenge flagged is how we can balance client expectations around their data not being over written.

• Medium / Longer term: I think there's a lot of good insights for simplification Alex fleshed out - would be keen to see how we can orient the shorter term thing to ladder into the longer term direction (if possible)

cc @jbenet @laudiacay @nicola

(some other places discussions have been happening #sector-resnap-wg on FIL Slack, this thread in #fil-protocol )

[laudiacay]

you summarized this well @jnthnvctr

until this is fixed, we have to lock all cold data in for at least 6 months at a time (or whatever our default deal period is, determined by Banyan's network sealing capacity and burstiness of client demand- for now we expect 6 months).

this forces us to, instead of doing "rolling backups" where the latest version of customer data gets sealed to Filecoin, have the customer opt into sending data back to Filecoin by clicking "snapshot archive to Filecoin" to seal the current version. It makes things manual for the customer, which is behavioral friction that will result in less data coming onto Filecoin.

to be clear, this does not cause any problems for our hot storage, hot proofs, or ability to onboard relatively-static data onto Filecoin- we can still get (substantial) cashflows into individual providers, and we will still bring good amounts of data onto filecoin, but we will not sell as much sealed Filecoin capacity as we possibly could (especially for a substantial portion of users who will use us as drop-in replacement to S3).

we'd prefer to do "rolling/automatic backups", but that's not currently workable without either:

1. exposing high-write-frequency users to shockingly high bills (every block that ever enters the system stays stored for at least 6 months, no garbage collection until the deals expire, every file edit spawns a 6-month charge), or
2. leaving Banyan footing the bill when a user leaves a bunch of stuff on Filecoin that we can't garbage-collect for 6 months.

second, without resnapping, we can't hard-delete at all. this is extremely unusable for enterprise customers. realistically, "we deleted the encryption key and scheduled wiping it in a garbage collector within 24 hours" is acceptable (this is approximately what AWS does), but "it's going to be here for 6 more months" is not acceptable. paying miners out to get slashed is also not affordable.

adding these changes will let us flexibly garbage-collect as per user block-lifecycling policies in a way that makes sense for our bottom line and network sealing capacity, as well as be significantly more enterprise-friendly 🌈

[laudiacay]

cc @sstelfox

[f8-ptrk]

even with resnapping there will be no hard-delete possible. all the network/miners will be able to deliver is a "your data is not stored in a filecoin sector that is committed to the filecoin mainnet anymore"

better than nothing but hardly a hard-delete.

we might want to think about phrasing this properly to reflect what is actually happening instead of calling it hard-delete, even delete is wrong'ish, to avoid further confusion down the road for consumers of applications build on top of filecoin.

[stef-sf]

@f8-ptrk I agree this is a great start. This approach doesn't seem to be that much different than default behaviors in other traditional filesystems though.
I agree that some applications will require a specific level of secure deletion (aka hard-delete), though that could be an improvement in the future where the underlying filesystem is wiped multiple times.

How can we prioritize the implementation of this FIP. I believe it's crucial for all onramps.

[donkabat]

very old topic without implementetion :(
there is so many problems to implrement it, bla bla bla... but SPs have real problem with this stupid issue.
It's so easy from my point of view... if deal is expired then sector is back as possible to snapup again.

[jennijuju]

@donkabat I agree this is a nice feature to have in the network!

It's so easy from my point of view...

If you have an early idea/spec on how this could be implemented in the protocol - we would love to see it & work with you on that!

[donkabat]

no, after I read posts above I was able only to complain to get your attention and that's all what can I do. I know, that is sad, but I'm just a simple SP.

[jennijuju]

@donkabat don’t think you need to complain! Simply let us know this feature is important to you and why is useful to help us prioritize - glad you are raising your voice here 💙