Adding caching to ServeFile and ServeDir

[12bitentropy]

• I have looked for existing issues (including closed) about this
  ServeFile is slower than simply reading a file #480

Feature Request

ServeFile does not use an in memory cache for "hot" files like css and js. Serving static files would be considerably slower for users that has to run their website on hard disks.

Motivation

I need to reduce the bottleneck of my slow aging hardisk from choking the performance of my app

Proposal

Implement a Read-Through cache using cloudflare's pingora crate and store a radix trie or balanced tree for precompressed files. Caching of the directory structure is useful to prevent constant checks for the file.

The balanced tree is used to determine if the precompressed files can be served, which is cheaper than checking the disk. It should not be that clients have to get the uncompressed file simply because the brotli compressed file could not be found while the other files are compressed. It might happen when files are being compressed on the go and some files have not been compressed.

There are also cases where brotli may not have the best compression ratio which makes serving files based off the smallest file size is better.

Caching invalidation is done when the time to live for the cache has expired. The filesystem tree is updated when a file cache is invalidated and file updates are checked to update the internal tree. It can use notify to check.

Some issue might appear where users are served outdated cache objects. The developer can change this using the ttl and not_found_ttl options, or their custom lookup function. Defaults to cloudflare's edge ttl defaults

Memory may be a issue to developers. The Read-Through cache can be configured based on the developer's choices to hold their choice of bytes before the cache has to evict files.

The proposal also introduces ETags as hashing is done to confirm the file hash not changed.

[12bitentropy]

If this is accepted, when is the release coming out? I would like to use it in my project

[12bitentropy]

Can this be built on top of #313

[seanmonstar]

I suspect that caching could be more general and exist as a separate middleware service.

[12bitentropy]

I suspect that caching could be more general and exist as a separate middleware service.

This could be done, but leaving it to a separate middleware will make it be dependent on url and not file. The cached url like /file.css also have to be cached differently based on content-type which is more complex.

[jplatte]

I don't see #313 or this happening anytime soon. This crate is not actively developed, only passively maintained and I don't see any of the current maintainers spending time on reviewing large-scale additions like that.

[12bitentropy]

Is this something that we want, which is caching in the ServeDir layer or do we want to move this into another layer?
I'm working on this now.