Skip to content

follow-up: compact injected recall history with stable markers#439

Open
Qiyuanqiii wants to merge 2 commits into
TencentCloud:mainfrom
Qiyuanqiii:fix/issue-120-cache-friendly-memory
Open

follow-up: compact injected recall history with stable markers#439
Qiyuanqiii wants to merge 2 commits into
TencentCloud:mainfrom
Qiyuanqiii:fix/issue-120-cache-friendly-memory

Conversation

@Qiyuanqiii

@Qiyuanqiii Qiyuanqiii commented Jul 9, 2026

Copy link
Copy Markdown

关联 Issue

Refs #120

定位说明

根据分流建议,本 PR 不再作为 #120 的主修复。#120 的运行时提示缓存/注入模式主线应由 #375 等运行时范围 PR 处理。

本 PR 现在收窄为一个可能的后续补充:只处理已经进入 session history 的 <relevant-memories> 注入块,将其压缩为稳定 marker,避免历史中保留整段过期 recall 内容。

本 PR 做了什么

  1. 新增 recall history compaction 工具:

    • 将历史中的 <relevant-memories>...</relevant-memories> 压缩为:
      <memory-omitted reason="prevent_context_bloat" />
    • 保留“曾经注入过 memory”的审计痕迹。
    • 移除 turn-specific 的动态 recall 内容。
  2. before_message_write 中压缩即将写入历史的注入块:

    • 当前轮 LLM 已经看过完整 recall。
    • 写入 session history 时只保留稳定 marker。
  3. before_prompt_build 中增加历史兼容处理:

    • 对已有历史中的 <relevant-memories> 再做一次压缩。
    • 用于覆盖旧 session 或某些 host path 中已进入 event.messages 的注入内容。
  4. 新增文档:

    • docs/RECALL-HISTORY-COMPACTION.md
    • 明确说明本 PR 是 narrow follow-up,不替代 runtime-side prompt-cache fix。
  5. 新增测试:

    • string message 中 memory block 压缩;
    • multimodal text part 中 memory block 压缩;
    • 不改变 unrelated system/assistant messages;
    • 不同 recalled-memory 历史压缩为相同 marker 形式;
    • 无 injected memory 时保持原消息对象不变。

非目标

验证

npm.cmd test -- src/utils/memory-injection-cache.test.ts
npm.cmd test
npm.cmd run build
git diff --check

结果:

  • targeted test:1 file,5 tests,passed
  • full test:5 files,72 tests,passed
  • build:passed
  • git diff --check:passed

截图

image

@Qiyuanqiii Qiyuanqiii changed the title fix: reduce memoryfix: reduce memory injection cache churn fix: 优化 memory 注入结构以提升提示词缓存命中 Jul 9, 2026
@YOMXXX

YOMXXX commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Triage note: #375 is now the canonical scoped runtime mitigation for #120, with #188/#335 already covering related visibility/test-only pieces.

This PR overlaps on prompt-cache hygiene by compacting historical <relevant-memories> blocks into a stable marker and deduping repeated system messages. That may still be useful as a follow-up, but it should be rebased after #375/#188 and scoped to the incremental history-compaction behavior only.

Recommendation: do not merge this as the primary #120 fix. Re-scope it as a focused follow-up that proves marker-based compaction is preferable to the existing default cleanup behavior, and keep the provider-comparison docs separate if needed.

@Qiyuanqiii Qiyuanqiii changed the title fix: 优化 memory 注入结构以提升提示词缓存命中 follow-up: compact injected recall history with stable markers Jul 9, 2026
@Qiyuanqiii

Copy link
Copy Markdown
Author

分流说明:#375 现在是 #120 的规范范围运行时缓解,#188/#335 已经涵盖了相关的可见性/仅测试部分。

这种PR在提示缓存净化上有重叠,通过将历史块压缩为稳定标记并对重复的系统消息进行重复处理。这仍可能作为后续有用,但应在 #375/#188 之后重新基于,并仅限于增量历史压缩行为。<relevant-memories>

建议:不要把它合并为主要的#120修正。将其重新定义为一个聚焦的后续,证明基于标记的压缩比现有默认清理行为更优,并在需要时保持提供者比较文档分开。

感谢大佬您的分流说明,我理解的大佬您的意思是这样的:#375 更适合作为 #120 的运行时范围主修复,而 #439 不应该继续声称覆盖 provider/prompt-cache 主线。所以我已经根据建议重新收窄了 #439 的作用范围:现在它只保留“基于 marker 的历史压缩”这一点,作为可能的后续补充。

以下是具体调整:

  • 移除了 system message 去重逻辑,避免和更广泛的运行时/提示处理混在一起;
  • 删除了 DeepSeek/MiMo provider 对比和 provider cache-hit 声明;
  • 将文档改为 docs/RECALL-HISTORY-COMPACTION.md,明确写明这是 narrow follow-up,不替代 fix(recall): stabilize prompt-cache memory injection #375 / runtime-side fix;
  • 现在只处理已经进入 session history 的 <relevant-memories>,将其压缩为 <memory-omitted reason="prevent_context_bloat" />
  • 保留 before_message_writebefore_prompt_build 两道压缩,用于新写入历史和旧历史兼容。

新的提交是 729ec25。如果大佬您认为这个 marker-based history compaction 仍然应该等 #375 合并后再重新基于它处理,我也可以继续调整或关闭后重开

Thank you for the triage clarification. My understanding of your point is as follows: #375 is a better fit as the scoped runtime-side primary fix for #120, while #439 should no longer claim to cover the provider / prompt-cache mainline. Based on your recommendation, I have narrowed the scope of #439 accordingly: it now only keeps the “marker-based history compaction” part as a possible follow-up enhancement.

The specific changes are:

  • Removed the system message deduplication logic to avoid mixing this PR with broader runtime / prompt handling behavior;
  • Removed the DeepSeek / MiMo provider comparison and provider cache-hit claims;
  • Renamed the documentation to docs/RECALL-HISTORY-COMPACTION.md, explicitly stating that this is a narrow follow-up and does not replace fix(recall): stabilize prompt-cache memory injection #375 or the runtime-side fix;
  • Now only handles <relevant-memories> blocks that have already entered session history, compacting them into <memory-omitted reason="prevent_context_bloat" />;
  • Kept both before_message_write and before_prompt_build compaction paths, covering newly written history as well as compatibility with existing history.

The new commit is 729ec25. If the maintainers think this marker-based history compaction should still wait until #375 is merged and then be rebased on top of it, I can continue adjusting it, or close this PR and reopen a smaller one later.

@Qiyuanqiii

Copy link
Copy Markdown
Author

分流说明:#375 现在是 #120 的规范范围运行时缓解,#188/#335 已经涵盖了相关的可见性/仅测试部分。

这种PR在提示缓存净化上有重叠,通过将历史块压缩为稳定标记并对重复的系统消息进行重复处理。这仍可能作为后续有用,但应在 #375/#188 之后重新基于,并仅限于增量历史压缩行为。<relevant-memories>

建议:不要把它合并为主要的#120修正。将其重新定义为一个聚焦的后续,证明基于标记的压缩比现有默认清理行为更优,并在需要时保持提供者比较文档分开。

大佬大佬,能不能麻烦您有空再帮忙看下这个收窄后的方向是否符合预期,如果您仍然觉得应该继续拆分,我也可以继续调整。

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program!
We have successfully received your submission. The program is currently in full swing, and we will complete the Code Review for you as soon as possible. Please keep an eye on the status notifications for this PR so you can follow up promptly once the review feedback is provided.
Thanks again for your contribution and open-source spirit! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants