Skip to content

feat(memory): add memory_summary tool for memory overview and source tracking#1124

Open
shiloong wants to merge 1 commit into
alibaba:mainfrom
shiloong:feat/memory/memory-summary
Open

feat(memory): add memory_summary tool for memory overview and source tracking#1124
shiloong wants to merge 1 commit into
alibaba:mainfrom
shiloong:feat/memory/memory-summary

Conversation

@shiloong

Copy link
Copy Markdown
Collaborator

Description

Add memory_summary tool for memory store overview and source tracking:

  • memory_summary: Returns statistics about the memory store — total entries, entries by category, entries by source (auto-consolidation, auto-capture, manual-observe), index size, and search performance metrics.
  • Supports source tracking to distinguish auto-created vs manually curated memories.

Provides observability into memory store health and composition.

Related Issue

no-issue: memory summary and source tracking tool

Scope

  • memory (agent-memory)

Checklist

  • cargo clippy --all-targets -- -D warnings passes
  • cargo test passes (138 tests)
  • TOTAL_TOOLS updated (27 → 28)

@Forrest-ly Forrest-ly left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


PR #1124 Review — feat(memory): add memory_summary tool for memory overview and source tracking

本 PR 添加 memory_summary 工具,扫描整个记忆存储并返回结构化概览:总数、自动/手动创建统计、分类分布、高频概念、高频文件引用和最近记忆列表。253 行新增,5 文件变动。

代码整体可读,frontmatter 解析和排序逻辑清晰,MemorySummary 结构设计合理。


发现

  1. src/agent-memory/src/tools/memory_summary_tool.rs:~178 — Source tracking 依赖 4 个 source 值,但当前代码库中没有任何代码写入这些值 (CONFIRMED, 中)

match source.as_str() {
"auto-consolidation" | "auto-capture" => summary.auto_created += 1,
"manual-observe" | "manual-write" => summary.manual_created += 1,
_ => summary.unknown_source += 1,
}

在当前 main 分支中 grep 确认:auto-consolidation、auto-capture、manual-observe、manual-write 这四个字符串均不被任何代码写入 frontmatter。memory_observe 不写 source 字段(PR #1123 才添加但未合并),consolidation 代码也不写。结果是:合并后 所有
记忆都会被归为 unknown_source,auto_created 和 manual_created 恒为 0。Summary 的核心卖点——区分自动/手动来源——完全无法工作,直到多个后续 PR 合并。

  1. src/agent-memory/src/tools/memory_summary_tool.rs:~142 — 扫描不排除 MEMORY.md 等非记忆文件 (CONFIRMED, 中)

PR #1122 在 mount root 创建 MEMORY.md 索引文件。本 PR 的 WalkDir 扫描包含所有 .md 文件,仅排除 meta_dir。MEMORY.md 没有 frontmatter,会被计入 total_memories(+1)、unknown_source(+1)、by_category["uncategorized"](+1),甚至可能出现在
recent_memories 列表中。同理,若 mount root 存在 README.md 或其他非记忆 .md 文件也会被错误统计。应显式跳过已知的非记忆文件(至少跳过 MEMORY.md)。

  1. src/agent-memory/src/tools/memory_summary_tool.rs:~89 — parse_frontmatter_flat 第 N 份拷贝 (CONFIRMED, 低-中)

这个函数在 PR #1120(session_history.rs)、PR #1122(memory_index.rs)、PR #1124(本 PR)中各有一份几乎完全相同的实现。如果三个 PR 都合并,代码库中将存在 3 个独立的 frontmatter 解析器。应提取为共享模块(如
crate::frontmatter::parse_flat),避免未来修复需同步多处。

  1. src/agent-memory/tests/profile_test.rs:~322 — 注释与常量值不一致 (CONFIRMED, 低)

// 11 Tier A + 3 Tier B + 3 snapshot + 2 git + 2 consolidation + 4 task + 2 export/import = 27
const TOTAL_TOOLS: usize = 28;

注释仍然写 "= 27" 但值已改为 28。应更新为 + 1 summary = 28。

  1. src/agent-memory/src/tools/memory_summary_tool.rs:~109 — parse_frontmatter_list 不支持 YAML 多行列表格式 (PLAUSIBLE, 低)

解析器假设列表值为内联 JSON 风格(["a", "b"])。若 frontmatter 使用 YAML 标准多行格式:
concepts:
- rust
- memory
flat parser 的 !key.starts_with('-') 过滤会丢弃列表项,concepts 的值变为空字符串。top_concepts 统计会遗漏这些记忆的概念。实际风险取决于 consolidation 代码输出的格式——若统一使用内联风格则无影响。

…tracking

Dreaming V3 Memory Summary page equivalent:
- Total memory count with auto-created vs manual breakdown
- Category distribution (by_category map)
- Source distribution (by_source map): auto-consolidation, manual-observe, etc.
- Top 10 concepts (from frontmatter concepts field)
- Top 10 referenced files (from frontmatter files field)
- Recent memories list (configurable limit, sorted by created_at)
- Total bytes across all memory files

Frontmatter source tracking:
- parse_frontmatter_flat() extracts key:value pairs from YAML blocks
- parse_frontmatter_list() handles ["a", "b"] style list values
- Source field distinguishes auto-consolidation from manual-observe

Tests: 15 suites, all passing
Clippy: clean
Fmt: clean
Tools: 20 total (was 19)

Signed-off-by: Shile Zhang <shile.zhang@linux.alibaba.com>
@shiloong shiloong force-pushed the feat/memory/memory-summary branch from e9c9c4d to fe4c233 Compare June 25, 2026 10:23
@shiloong

Copy link
Copy Markdown
Collaborator Author

Review 修复回复

1. source 值未被写入 — ⚠️ 跨 PR 依赖

source 字段值(auto-consolidation/auto-capture/manual-observe/manual-write)需要多个 PR 合入后才能完整工作:#1123 添加 source: manual-observe,consolidation PR 已添加 source: auto-consolidation。合入后 summary 的来源统计将自动生效。

2. 扫描不排除 MEMORY.md — ✅ 已修复

增加 if rel_path == "MEMORY.md" || rel_path == "README.md" { continue; } 跳过非记忆文件。

3. parse_frontmatter_flat 重复 — ⚠️ 后续重构

3 个 PR 各有一份拷贝。后续应提取为 crate::frontmatter::parse_flat 共享模块。

4. 注释与常量不一致 — ✅ 已修复

注释更新为 = 28

5. parse_frontmatter_list 不支持 YAML 多行 — ⚠️ 已知

consolidation 代码统一使用内联 JSON 风格,无影响。

CI: fmt ✅ clippy ✅ 135 tests ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants