Improve caching behavior around Chrome disk cache

[kylebarron]

According to docs about Chrome HTTP caching behavior: https://www.chromium.org/developers/design-documents/network-stack/http-cache/

The cache implements a single writer - multiple reader lock so that only one network request for the same resource is in flight at any given time.

So it seems that other requests for the same file will be handled serially by Chrome, which is very bad.

I think ideally we want

fetch(url, {
  headers: { Range: "bytes=..." },
  cache: "no-store",
});

which chatgpt says

• Bypasses browser cache
• Avoids lock
• Keeps server/CDN caching intact

But to do that we need access to more fetch properties than just headers, which is what https://github.com/blacha/chunkd/blob/9b7e1480d2435d06b87a0f716e3d35ab94111f80/packages/source-http/src/index.ts#L35-L38 currently exposes.

blacha/cogeotiff#1446 should hopefully make it easier to build source interfaces

[kylebarron]

We should learn from what PMTiles does here: https://github.com/protomaps/PMTiles/blob/bbeda97ed44d10b785e99f0dfe430b5728baec95/js/index.ts#L393-L482

They have an issue protomaps/PMTiles#442 where they're hitting a chromium bug https://issues.chromium.org/issues/40542704 that's problematic with range requests.

[kylebarron]

Apparently cache isn't a good thing to set globally in SourceHttp. I get this error trying to fetch from an EOxCloudless image.

Access to fetch at 'https://s2downloads.eox.at/demo/EOxCloudless/2020/rgb_corrected_geodetic/3/0/0.tif' from origin 'https://developmentseed.org' has been blocked by CORS policy: Request header field cache is not allowed by Access-Control