fix(npm): stale metadata cache issue #6101
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's the problem this PR addresses?
We now keep the package metadata in cache. To avoid missing new packages being released we have a check so that we only accept the cached metadata if 1/ the request asks for a semver version (not a range), and 2/ the requested version is found inside the cached metadata. In theory this means that whenever a dependency asks for a version we didn't cache, we assume something new got published, and we refetch it.
However, to prevent fetching the package metadata many times for many different versions or ranges, we also have an in-memory metadata cache where we store the cached metadata once we extracted them from either the disk or the network.
This may lead to memory cache corruption issues when two versions from the same package are resolved if one exists in the cached metadata but the other doesn't. In that case, the first package will pass the check for "is this version inside the cached metadata", get stored in the in-memory cache, and be reused for further resolutions (even if those resolutions would have failed this check). This is because the disk cache and the memory cache are the same.
Fixes #5989
How did you fix it?
I separated the in-memory cache into two buckets: the disk cache, and the network cache. This ensures that the disk cache gets properly ignored when retrieving versions we don't know, rather than be mistakenly assumed to be what the network fetched.
Checklist