test(recall): cache accounting benchmark for split_system (SIMULATION)#450
test(recall): cache accounting benchmark for split_system (SIMULATION)#450muzimu217 wants to merge 1 commit into
Conversation
Deterministic simulation that verifies split_system increases the cacheable prefix (before CACHE_BOUNDARY) by moving the stable persona into it. Clearly labeled as SIMULATION — it does NOT call any LLM API and does NOT measure real cache_read/cache_creation tokens. For real measured numbers, see community PR TencentCloud#433 (2 providers x 2 variants x 2 repeats x 3 turns). The single invariant asserted: with split_system, the cached prefix is strictly longer than without it, because the persona (only stable recall artifact) is moved before the boundary. Dynamic L1 memories stay after. This guards against regressions in how split_system assembles the prefix; it is a documentation + regression check, not a performance claim.
|
Duplicate/scope cleanup triage: This is useful coverage, but it should not stay as a standalone PR. The test file only validates the I would close this as a separate PR and move the test into #449 during the next revision. The real-provider measurement lane should remain separate from this simulation, as noted in the PR body. |
|
This test suite has been superseded by improved tests in #449. Issues with this PR:
Replacement:
Closing to consolidate testing in the main PR. |
摘要
为 #449(
split_system把稳定 persona 前置到CACHE_BOUNDARY)补充一个缓存记账基准测试,验证 split_system 确实增大了"缓存边界之前"的可缓存前缀长度。为什么需要这个测试
前缀匹配缓存(DeepSeek / MiMo / Anthropic 类)只对
CACHE_BOUNDARY之前、且逐 token 完全一致的前缀做缓存。split_system 的价值 = 把唯一稳定的 persona 移进这段可缓存区域。本测试断言的唯一不变量:split_system开启时,缓存前缀长度 严格大于none;测试内容
src/adapters/openclaw/cache-accounting.benchmark.test.ts(4 用例):cache_read增益可观且确定性(约 10–40 token)cache_read严格更大,且增益随轮数线性(无随机)<relevant-memories>/<memory-context>)不应出现在边界前与 #449 / #375 / #433 的关系
Scope / Non-scope