No automatic cleanup for allocated-but-never-accepted blobs; expired allocations block removals #44

Raised by @alanshaw while reviewing #30: "is there any code that automatically removes a blob that is allocated but never accepted or rejected?" The answer today is no — and the gap is slightly deeper than the question implies.

Current behavior

• Removal only ever starts from an explicit /blob/release or /blob/reject. The PDPRemoveSweep task only advances the pdp_pending_piece_removals queue; nothing time-based ever queues a removal.
• Allocations carry an Expires timestamp, and the allocation store even has GetAnyNonExpired (pkg/store/allocationstore/store.go), but it is only called from tests. Nothing collects expired allocation records or the parked bytes behind them.
• Worse, expired allocations actively block removals: blobHasClaims (pkg/pdp/service/piece_remove.go) checks claims via allocationStore.Exists, which ignores expiry, and the release/reject handlers count remaining claims via ListSpaces, which also ignores expiry.

Leak scenarios

Abandoned upload (the question as asked): a space allocates, the client never uploads (or uploads but never triggers accept), and the upload service never sends /blob/reject. The allocation record and any parked bytes persist indefinitely.

Stale allocation blocks another space's removal:

1. Space A allocates + uploads + accepts digest D. Space B allocates the same D but never accepts.
2. B's allocation expires; nothing deletes the record.
3. A releases its claim via /blob/remove → /blob/release, which queues the byte release.
4. The sweep's claims re-check sees B's stale allocation record and cancels the removal. The bytes are retained forever even though no live claim remains.

Proposed direction

• Treat expired allocations as non-claims: blobHasClaims and the reject/release claim counting should ignore allocations past Expires (i.e. adopt GetAnyNonExpired semantics).
• Add a periodic sweep (or extend PDPRemoveSweep) that deletes expired allocation records and queues a removal for digests whose only claims are expired allocations and which have no acceptance in any space.

Precondition to verify first: the upload (PUT) path must refuse writes against an expired allocation — otherwise GC races a legitimate late upload of the same digest.

Pre-existing gap (the jobqueue-era code didn't collect these either); split out of #30 as a follow-up.

🤖 Generated with Claude Code