Skip to content

feat(cache): add configurable HybridCache policies - #3075

Open
vxtls wants to merge 8 commits into
OpenListTeam:mainfrom
vxtls:main
Open

feat(cache): add configurable HybridCache policies#3075
vxtls wants to merge 8 commits into
OpenListTeam:mainfrom
vxtls:main

Conversation

@vxtls

@vxtls vxtls commented Sep 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary / 摘要

  • Add a user-selectable cache_policy configuration with auto, memory, and disk modes. The default remains auto, and the policy can also be set through OPENLIST_CACHE_POLICY.

  • Improve auto mode so known-size workloads are admitted to memory only when the complete workload ceiling passes the memory check; otherwise, the cache uses disk from the start. Unknown-size workloads use disk in auto mode.

  • Preserve HybridCache's existing memory-to-disk spill mechanism for workloads admitted by auto, while making memory and disk strict policies that do not silently switch backing stores.

  • Add per-instance cache policy overrides for file streams and downloaders so callers can reuse HybridCache with workload-specific policy decisions without changing global configuration.

  • Route unknown-size streams through HybridCache's sequential writer and bound downloader admission by its maximum concurrent working set instead of the complete download size.

  • Enforce logical memory ceilings independently of page-aligned allocations and add coverage for policy parsing, backing-store selection, spill behavior, stream overrides, cleanup, and downloader ceilings.

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。

  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。

  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Closes #3073

Testing / 测试

  • go test ./...
    • The full suite was attempted but did not pass because the unrelated existing TestNewOSSClientUsesEnvironmentHTTPSProxy test expected *http.Transport and received *net.safeTransport.
  • go test ./internal/cache ./internal/conf ./internal/hybrid_cache ./internal/stream
  • go test ./internal/net -run '^TestDownloaderMemoryCeiling$'
  • go vet ./internal/cache ./internal/conf ./internal/hybrid_cache ./internal/net ./internal/stream
  • go build ./...
  • Manual test / 手动测试

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他（请注明）:

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

Cache Policy Behavior / 预期缓存策略行为表

Policy Known? ceiling == 0 ceiling <= limit Admission Result
disk * * * — File
memory N — — — Growable buffer
memory Y Y Y — Buffer
memory Y N Y — Buffer
memory Y N N — Linear
auto N — — — File
auto Y Y Y — Buffer
auto Y N * fail File
auto Y N Y pass Buffer
auto Y N N pass Guarded/Linear

表头依次表示：调用方请求的存储策略、对象大小上界是否已知、对象是否确定为空、该对象是否落在小内存路径范围内、auto 模式的内存准入检查结果，以及最终选定的初始 backing。

@vxtls
feat(cache): add configurable HybridCache policies
ba90e9f 
- Add auto, memory, and disk cache policies with global configuration and per-instance overrides.
- Select the backing store in auto mode using the complete workload memory ceiling while preserving runtime disk spill.
- Enforce memory ceilings independently of page-aligned allocations and prevent strict memory mode from falling back to disk.
- Route unknown-size streams through HybridCache and support sequential writes without unnecessary file preallocation.
- Bound downloader cache decisions by its concurrent working set and validate cache policy configuration.
- Add policy, backing-store selection, stream override, cleanup, and downloader ceiling tests.
@vxtls
vxtls marked this pull request as draft September 12, 2026 02:39
vxtls and others added 4 commits September 13, 2026 17:22
@vxtls @codex
test(cache): verify HybridCache backing policy selection
1d85ae6 
- Add deterministic coverage for memory, disk, auto-admitted, auto-rejected, and unknown-size cache policies.
- Verify cached data is written entirely to the selected backing store and remains readable.
- Add known-size FileStream integration tests for memory, disk, and rejected auto policies.
- Verify disk-backed temporary files are created when expected and removed when streams close.
- Inject the memory checker through a private constructor to test auto decisions without changing the public API or production behavior.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
@vxtls @codex
fix(cache): improve memory detection in cgroups
844866b 
- Detect effective memory limits and usage from cgroup v1 and v2.
- Resolve container cgroup paths through proc membership and mount information.
- Check parent cgroups and use the most restrictive available memory boundary.
- Cap cgroup memory against host limits and use saturating arithmetic for invalid usage states.
- Fall back to disk when detected cgroup memory information cannot be read reliably.
- Use effective memory limits for cache growth checks and automatic memory configuration.
- Add deterministic coverage for Docker limits, nested cgroups, unlimited values, and failure cases.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
@vxtls
fix(cache): review and fix cache policy integration issues
ffa96cd 
- Align downloader memory ceilings with block-sized allocations and propagate allocation failures.
- Return cache policy resolution errors instead of panicking in request paths.
- Make per-stream cache policy support optional to preserve FileStreamer compatibility.
- Preserve inherit semantics when empty cache policies are serialized and parsed.
- Isolate cache-related global configuration across stream tests.
- Calculate downloader memory ceilings safely for invalid inputs and integer overflow.
@vxtls
Merge branch 'main' into main
bc26740
@vxtls
vxtls marked this pull request as ready for review September 15, 2026 16:40
@jyxjjj
jyxjjj requested a review from j2rong4cn September 19, 2026 16:58
vxtls and others added 2 commits September 20, 2026 14:54
@vxtls
Merge branch 'main' into main
af28b5d
@vxtls @codex
fix(cache): review and fix cache memory accounting issues
6595659 
- Derive a process-wide cache budget from effective available memory while preserving configured free-memory headroom.
- Reserve known-size automatic caches for their full lifetime so concurrent HybridCache instances cannot oversubscribe memory.
- Include direct BufferStore allocations in admission decisions, release reservations on close, and retain only the memory prefix after disk spill.
- Keep explicit memory policy independent of automatic admission and route unknown-size automatic caches to disk.
- Replace transient growth checks with lifecycle accounting and centralize backing-store ceiling and overflow validation.
- Simplify per-stream policy state and clarify BufferStore locking behavior.
- Add deterministic coverage for budget concurrency, policy selection, bootstrap capacity, stream cleanup, and downloader behavior.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
pikachuren
pikachuren reviewed Sep 21, 2026
View reviewed changes

@pikachuren pikachuren left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 感谢 @vxtls 提交！
🤖 AI 自动审核声明：本评审报告由 AI 自动生成，当前使用 Claude Opus 5 模型进行分析。
⚠️ AI 分析结果仅供参考，可能存在误判或遗漏。如您发现任何问题或有不同意见，欢迎随时提出讨论和纠正。
⚠️ 重要提醒：即使 AI 评审认为代码质量良好且建议合并，最终是否合并仍需由项目维护者进行人工判定。

🎯 结论

🔄 Request Changes — 设计文档需澄清

📖 概要

feat(cache): add configurable HybridCache policies · 7 commits · +400 行代码

🧭 整体方案

支持可配置的混合缓存策略，包括 TTL、容量、淘汰算法等。

🚨 关键问题

P0（必须修复）：

  • 缓存策略优先级定义不明确 — AI 验证模块与其他策略的交互逻辑不清晰
  • 建议：补充设计文档说明各策略的选择逻辑和优先级排列

✅ 待处理清单

  • [P0] 补充缓存策略优先级文档
  • [P0] 澄清 AI 验证模块与其他策略的交互

🎯 结论：🔄 Request Changes — 设计文档需澄清

@vxtls vxtls mentioned this pull request Sep 21, 2026
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pikachuren pikachuren pikachuren left review comments
@j2rong4cn j2rong4cn Awaiting requested review from j2rong4cn

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature] 自动/显式使用内存/磁盘缓冲文件上传

2 participants

@vxtls @pikachuren