Add off-heap memory cache architecture analysis for ZipDrive’s memory tier

[Copilot]

The current memory tier stores cache entries as large managed byte[], which is a poor fit for a long-lived content cache because it increases GC pressure and makes memory usage harder to shape. Simply switching to per-entry unmanaged aligned allocations would move the problem off-heap, but would not address fragmentation or allocator control.

• Problem framing

  • Documents why the current MemoryStorageStrategy design is simple but weak for long-lived cache residency.
  • Separates the real design question — cache-shaped allocation policy — from the narrower “managed vs unmanaged” question.

• Option analysis

  • Compares four approaches:
    • keep managed byte[]
    • allocate one unmanaged block per entry
    • use an off-heap slab/arena allocator
    • reserve large virtual regions and commit on demand
  • Calls out why per-entry AlignedAlloc is only a partial improvement.

• Recommended memory-tier architecture

  • Proposes an off-heap, non-moving arena/slab allocator behind the existing IStorageStrategy<Stream> boundary.
  • Recommends:
    • size classes + slabs for small entries
    • page-run allocation for medium entries
    • dedicated extents or earlier disk routing for near-cutoff large entries
  • Preserves the current GenericCache invariants: lock-free hits, per-key materialization dedupe, ref-counted borrow/return, and deferred orphan cleanup.

• ZipDrive integration shape

  • Sketches the proposed storage model around:
    • ArenaMemoryStorageStrategy
    • IUnmanagedMemoryAllocator
    • ArenaBackedBuffer
    • a read-only stream over unmanaged memory
  • Defines allocator metrics and admission rules around reserved/committed bytes rather than logical cache size alone.

• Industry references

  • Anchors the proposal against established cache systems:
    • Memcached for slab classes
    • Meta CacheLib for allocator/cache co-design
    • Redis as a reminder that unmanaged memory alone does not solve allocator fragmentation
    • Caffeine for the separation between eviction quality and memory-allocation strategy

• Documentation wiring

  • Adds the new design note at:
    • src/Docs/MEMORY_CACHE_ARCHITECTURE_DESIGN.md
  • Links it from:
    • src/Docs/CACHING_DESIGN.md
    • src/ZipDrive.Infrastructure.Caching/README.md

Example direction proposed in the design:

< 1 MB         -> slab size classes in unmanaged arenas
1 MB - 16 MB   -> page-run allocator in unmanaged arenas
> 16 MB        -> prefer disk tier unless policy explicitly allows dedicated extents

This keeps ZipDrive’s current cache concurrency model intact while shifting the memory-tier discussion from “replace byte[] with unmanaged blocks” to “introduce an allocator designed for long-lived cache workloads.”

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• cachelib.org
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
• docs.memcached.org
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
• redis.io
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)