perf: cache /v1/data/pubTree and manifest resolve in Redis (prod cherry-pick of #1286)#1287
Merged
Merged
Conversation
Same pattern as #1281 (which cached /v1/pub/versions). The dpid SSR path in nodes-web-v2 hits both endpoints on every cold render: resolvePublishedManifest → GET /<uuid>/<version> (handled by raw/resolve.ts) loadDriveTree → GET /v1/data/pubTree/... (handled by data/retrieve.ts) The first is the slow one — it calls getIndexedResearchObjects (theGraph, ~5-8s) plus an IPFS gateway fetch. The second varies but is typically 1-3s. Together they dominate the dpid cold-load TTFB. Both responses are content-addressed: - pubTree by (uuid, manifestCid, rootCid, dataPath, depth) — manifestCid is itself a content hash, so the tuple is fully immutable. No invalidation needed; new publishes mint a new manifestCid and write a fresh cache entry. - resolve by (uuid, firstParam) where firstParam is "" / index / CID. Index- and CID-keyed entries are immutable. The "latest" key is invalidated in publish.ts alongside the existing `indexed-versions` invalidation. Cache safety: only success paths cache. Component (PDF/code) responses in resolve.ts and 4xx/5xx responses are NOT cached. Combined with the Vercel edge cache shipped in nodes-web-v2#1540, cold SSR drops from 12-16s → ~1.5s; warm SSR is unchanged at ~150ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-pick of #1286 (already merged to develop, verified live on
nodes-api-dev— see metrics below) directly to main so prod gets the win without waiting for the full develop → main promotion.Same surgical 14-line addition as #1286. Identical pattern to #1281/#1285 (the
versionscache).Verified live on dev
Test ran against real dev uuid
XSwfV4AYF5qHfmu0AOLKnGNEGkjlitU4tbl_W2DsMwc.(dpid 1812).Combined stack on prod after this lands
Test plan
nodes-api.desci.com/<uuid>/0twice. First hit ~5s, second hit <200ms./v1/data/pubTree/<uuid>/<manifestCid>./<uuid>(no version) returns the new latest on the very next request (latest key invalidated bypublish.ts)./<uuid>/0(specific index) is unaffected by publish — it's immutable, stays cached.Safety
pubTreekeys are content-addressed (manifestCid is a content hash), so per-version entries are inherently immutable. No invalidation needed beyond whatpublish.tsalready does.resolveper-index and per-CID keys are immutable. Only:latestis invalidated, hooked into the existingdelFromCacheblock inpublish.ts.🤖 Generated with Claude Code