Upload fails with 404 on chunk PUT — no retry, entire upload lost

[rubenhensen]

Problem

During an upload to staging (fileshare.staging.postguard.eu), a single chunk PUT returned HTTP 404:

PUT https://fileshare.staging.postguard.eu/fileupload/e25a6d9e-35a3-404b-9bdc-2c7d8b3da904
[HTTP/2 404 121ms]

Error occurred during encryption: Error: JavaScript error: no further details

Uncaught (in promise) NetworkError: Error uploading chunk

The entire upload was lost — all progress gone, no way to resume.

Likely cause

The cryptify server expires unfinalized uploads after 15 minutes (store.rs purge task). If a large upload takes longer than that, or the connection stalls, the server-side state is purged and subsequent chunk PUTs return 404 because the UUID no longer exists.

Impact

• For large files (multiple GiB over slow connections), this is likely to happen regularly
• The user gets no warning that their upload timed out — just a generic error
• All uploaded bytes are wasted

Suggested fixes

1. Retry failed chunks — pg-js should retry transient failures (404, 5xx, network errors) with exponential backoff before giving up
2. Extend or refresh the expiry — each successful chunk PUT should reset the 15-minute expiry timer on the server
3. Better error messaging — distinguish "upload expired" (404) from other errors so the user understands what happened
4. Resume support — allow the client to resume from the last successful chunk offset instead of starting over

[dobby-coder]

Trace from a 2026-04-27 dive — leaving notes for whoever picks this up.

The "15 minutes purge" hypothesis is wrong

Cryptify stores upload sessions for 14 days, not 15 minutes. On POST /fileupload/init, src/main.rs:118 sets expires: current_time + 1_209_600 (1,209,600 s = 14 days), and the purge loop in src/store.rs:152-173 only evicts entries past that timestamp.

So a 404 on a chunk PUT during a single upload session is almost certainly not an idle-timeout eviction.

Source: src/store.rs · src/main.rs

What actually causes the 404

upload_chunk returns Ok(None) (which Rocket renders as 404) when store.get(uuid) is None (src/main.rs:284-287). The store is an in-memory HashMap (src/store.rs:37) — there is no persistence.

Two real ways the session entry can disappear mid-upload:

1. Server restart. Any deploy, OOM, panic, or container restart wipes Store.state.files clean. The on-disk file (data_dir/<uuid>) is still there but orphaned, and every subsequent chunk PUT 404s. This also leaks the on-disk file: nothing references it any more, and there is no separate disk reaper.
2. Reverse proxy / load balancer routing to a different cryptify instance if the deployment runs more than one replica. The other instance does not share the HashMap, so the chunk lookup misses.

Production / staging probably hit case 1 (single replica per the postguard-ops setup, restarts during deploys).

Implications for the suggested fixes

• Retry with backoff (suggested fix Merge cryptify with svelte version of site #1) does not help here. After a server restart the session is gone forever; retrying the same uuid keeps 404'ing. Retry would still help against transient 5xx / network blips, but the original symptom report is permanent loss.
• Reset expiry on chunk PUT (suggested fix File encrypt #2) does not help either, for the same reason — the session state is already gone, not expired.
• Better error messaging (suggested fix Styling #3) is the cheapest win. The client sees a generic 404 on the chunk endpoint and surfaces "JavaScript error: no further details". Distinguishing 404-with-known-error-body ("upload session not found, server may have restarted") from a true network 404 would let pg-js show a useful message.
• Resume support (suggested fix Design meeting #4) is the only thing that closes the underlying gap, but it requires persisting FileState (sqlite, postgres, or even a JSON sidecar in data_dir/) so the session survives restarts. Today the design assumes uploads finish within the lifetime of one process.

Suggested next step

Before any fix lands, it would be worth confirming the root cause in production by:

1. Correlating the timestamp of a 404 from the bug report with the cryptify deploy/restart log.
2. Checking data_dir/ on the cryptify host for orphan files older than the purge window — if they exist, that confirms the leak side of the same bug.

Happy to open a follow-up cryptify issue tracking the persistence gap if a maintainer wants — it's distinct from this issue, which is about the website surfacing the failure better.

[dobby-coder]

Correction to my 2026-04-27 trace above — I misread the cryptify purge mechanism. The "15 minutes" hypothesis in the original issue body is correct.

What I got wrong: I conflated the FileState.expires field (set to current_time + 1_209_600 at src/main.rs:118, i.e. 14 days) with the purge schedule. They are different things.

What actually drives the in-memory eviction is Store::create in src/store.rs:80:

let removal_instant = Instant::now() + Duration::from_secs(60 * 15);
state.expirations.insert((removal_instant, removal_id), id);

The purge_task loop walks state.expirations and removes any entry whose Instant has passed (src/store.rs:152-173). The 14-day expires field on FileState is carried in the struct but is never read by the purge loop — it appears to be intended for the recipient/download-link side.

So: an upload session is evicted from the in-memory HashMap exactly 15 minutes after init, full stop, regardless of activity. After that, every subsequent chunk PUT 404s and the on-disk file is orphaned.

This makes the implications more nuanced than my earlier note suggested:

• Retry with backoff still doesn't recover from the actual eviction (the entry is gone), but the window is much shorter than 14 days, so a fast retry inside the window can help against transient blips.
• Reset expiry on chunk PUT (suggested fix File encrypt #2) is a meaningful change — extending removal_instant on each chunk PUT would let large/slow uploads survive past the 15-minute mark. Cheap fix on the cryptify side.
• Server restart is still a separate, concurrent failure mode (HashMap is in-memory only) and a fix for that requires persistence.
• Better error messaging and resume support points stand.

Apologies for the noise — the misread happened because I grepped for expires in main.rs and stopped at the first hit instead of tracing the purge loop end-to-end. Filed cryptify#116 still tracks the persistence gap for the restart case; the eviction-window question is separate and lighter-weight to fix.

[rubenhensen]

/dobby there have been a lot of PRs mentioning this issue. Is it fixed now or is there more to do?

[dobby-coder]

Dobby is on it! Dobby will check all the PRs mentioning issue #117 and report back on what's been fixed and what still needs work.

[dobby-coder]

The original symptom — a single chunk PUT returning 404 and torching the whole upload — is fixed. The remaining gap (server-restart wiping in-flight sessions) is tracked separately.

What shipped

Server (cryptify):

• #132 — sliding 60-min idle TTL: each chunk PUT resets the expiry, so slow uploads stop tripping the old hard 15-min deadline.
• #144 — structured 404 body (UploadSessionNotFound) and configurable TTL, so the client can distinguish expired-session from a real 404.
• #145 — idempotent retry of the last committed chunk, so a lost response no longer desyncs the rolling token chain.

Client (postguard-js):

• #47 — chunk PUTs and downloads retry transient failures with exponential backoff + jitter (5 attempts, configurable).
• #52 — resumable downloads via Range on mid-stream failure.

Website:

• #121 — beforeunload warning during encrypt/upload so accidental navigation no longer silently nukes progress.
• #145 — pg-js 1.4 bump + UX surface for retries and the distinct "upload session expired" message.

Mapping back to the original suggested fixes

1. Retry failed chunks — done (pg-js#47 + cryptify#145).
2. Extend / refresh the expiry — done (cryptify#132).
3. Better error messaging — done (cryptify#144 + website#145).
4. Resume support — done for downloads (pg-js#52). Not done for uploads across server restarts, which requires session persistence on cryptify (in-memory HashMap today). That's tracked in cryptify#116 — still open, distinct scope from this issue.

Recommendation

This issue can be closed. The reporter-visible failure (404 mid-upload with no recovery) is addressed end-to-end. The restart-persistence gap is real but lives in cryptify#116 — closing this one keeps scope clean and avoids two issues tracking overlapping work.