diff --git a/docs/memory-mcp.md b/docs/memory-mcp.md new file mode 100644 index 000000000..a3b5d3bca --- /dev/null +++ b/docs/memory-mcp.md @@ -0,0 +1,592 @@ +# 专家记忆 MCP Server 使用文档 + +> Octop 把**专家记忆**通过标准 **MCP（Streamable HTTP）** 暴露给外部 agent（编码 agent、机器人、 +> 其他 AI 工具），能力与进程内 `MemoryService` 对齐：外部 agent 可以召回记忆、按路径下钻读全文、 +> 写入新的事件与事实，并能区分"这条记忆是谁说的"。 +> +> 当前工具面：**9 个 `memory_*` 工具**（读取 3 个 / 写入 2 个 / 提取与审核 4 个）。 +> +> 本地终端（Claude Code / Kiro / DSH / Codex / Cursor）可以用服务端自带的**一键安装器** +> 自动完成「MCP 条目 + hooks（+ DSH 共享记忆预设）」，见下方**快速上手**。 + +--- + +## 快速上手（curl 一行接入） + +前提只有两条：① 服务端已设 `OCTOP_MEMORY_MCP_TOKEN` 并重启（未设置时 `/mcp/memory` 不挂载， +见 §5）；② 有一个可用的**专家 ID**（即 `X-Octop-Agent-Id`，用 `octop agent list` 查，或用默认专家）。 + +在**要接入记忆的那台机器**上执行——`terminals` 可多选，逗号分隔： + +```bash +curl -fLsS "https:///mcp/memory/setup?agent=<专家ID>&terminals=claude-code" | sh +``` + +脚本会：检测本机装了哪些终端 → 让你勾选 → 问一次 token（部署已配 `OCTOP_MEMORY_MCP_TOKEN` +时内置，不再问）与用户标识 → 写入每个终端**自己的** hooks 目录与配置，并登记 MCP 条目。 +结尾会打一行 `octop memory 接入完成`，顺便做一次端点自检（`HTTP 200` 即通）。 + +**可重复执行**：重跑就是升级路径——先清理早期版本的 hook 文件与本安装器的陈旧登记， +同一端点只保留一条 MCP 条目，不会动你其它的 MCP / hook 条目。 + +### 零交互（CI / 批量装机 / 脚本化） + +`curl | sh` 没有 tty 时不会提问，全部走环境变量或 URL 参数： + +```bash +curl -fLsS "https:///mcp/memory/setup" \ + | OCTOP_AGENT_ID='<专家ID>' \ + OCTOP_TERMINALS='claude-code,kiro' \ + OCTOP_MEMORY_TOKEN='<共享 token>' \ + OCTOP_USER_ID='<你的域账号>' \ + sh +``` + +| 变量 | 作用 | +|---|---| +| `OCTOP_AGENT_ID` / `OCTOP_TERMINALS` | 等价于 `?agent=` / `?terminals=`；缺专家 ID 会直接报错退出 | +| `OCTOP_MEMORY_TOKEN` | 覆盖部署内置的默认凭据（即 `OCTOP_MEMORY_MCP_TOKEN` 的值） | +| `OCTOP_USER_ID` | 写进 `X-Octop-User-Id`，决定记忆里"谁说的"；**建议域账号**（HTTP 头只能 ASCII，中文会被忽略） | +| `OCTOP_TOKEN_HEADER` | hooks 用的鉴权头名，缺省 `Authorization` | + +### 装完自检 / 卸载 + +> **凭据只有一份**：**实例共享 token**，也就是服务端 `OCTOP_MEMORY_MCP_TOKEN` 的值。 +> 它由安装器内置成默认凭据，正常无需手传，要覆盖才设 `OCTOP_MEMORY_TOKEN`；hooks 与客户端 +> MCP 条目用的是同一份，没有"每人的个人令牌"这种东西。`?url=` / `OCTOP_MEMORY_PUBLIC_MCP_URL` +> 只改变**条目地址**（指向自建的公网入口），不改凭据。 + +```bash +# 1) 端点活着：不带 token 应 401（挂载了才可能 401，未挂载是 404） +curl -s -o /dev/null -w '%{http_code}\n' -X POST "https:///mcp/memory/" \ + -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' + +# 2) 重开终端/IDE（Kiro 重载 hooks；Codex 首次需在 codex 里信任该 hook）， +# 然后问一句与本项目相关的问题——模型侧应自动调到 memory_search，命中会注入上下文。 + +# 卸载（等价于把安装器写过的条目与 hook 文件清掉） +curl -fLsS "https:///mcp/memory/uninstall" | sh +``` + +> 手工接入（不想跑脚本）看 §4.3 Kiro / §4.4 Claude Code / §4.5 DSH / §4.6 其它客户端； +> 安装器具体写了哪些文件、每个终端的路径与工具清单，见 §4.7。 + +--- + +## 1. 连接信息 + +| 项 | 值 | +|---|---| +| Endpoint | `http(s):///mcp/memory`（MCP 侧实际请求路径为 `/mcp/memory/`） | +| 协议 | MCP Streamable HTTP（SSE + JSON） | +| 鉴权 | `Authorization: Bearer ` | +| 专家绑定 | `X-Octop-Agent-Id: ` 请求头 | +| 调用者标识 | `X-Octop-User-Id: ` 请求头（可选，见 §1.3） | + +### 1.1 鉴权（fail-closed） + +- 服务端通过环境变量 `OCTOP_MEMORY_MCP_TOKEN` 配置这条通道的独立 token；**未配置时整个 + `/mcp/memory` 端点不挂载**（安全默认）。 +- 每个请求都必须带 `Authorization: Bearer `，否则返回 `401`。 +- 建议该 token 与 Octop 登录凭据完全隔离——它是独立凭据，仅用于记忆通道。 + +### 1.2 专家绑定（按请求校验，无需重启） + +- 端点只有一个 `/mcp/memory`，URL 里不含专家 id；专家由请求头 `X-Octop-Agent-Id` 选择。 +- **每个请求都会用该 id 去专家注册表校验**（存在且处于启用状态），校验不通过返回 `404`。 + 因此新建或停用专家**立即生效**，不需要重启服务。 +- 所有读写都落在该专家的 `Memory` 实例上，工具调用时**不再**传 agent id。 +- 记忆是**专家级共享**的：同一专家下的多个外部调用者看到同一份记忆，不做按人硬隔离； + 需要按人定位时依赖 §1.3 的发送者归属。 + +### 1.3 调用者归属（`X-Octop-User-Id`） + +- 调用者身份通过 `X-Octop-User-Id` 头（或工具的显式 `user` 参数，优先级更高）传递。 +- `memory_capture` / `memory_save` 会把调用者 id **拼进内容前缀** + `说：…` 后再落库。原因：`AtomCard` 没有 user 列，把发送者写进正文才能随原子一起 + 落库、被全文检索命中（查询里带上发送者名字即可命中其记忆）、并在召回时直接可见。 +- 因此**不要把名字重复写进 `content`**；内容已带同名前缀时不会重复拼接；不带该头时保持原文。 +- 未匹配到调用者身份时，写入仍然成功，只是没有前缀。 + +--- + +## 2. 工具清单（9 个） + +### 2.1 读取（3 个） + +| 工具 | 参数 | 说明 | +|---|---|---| +| `memory_search` | `query: str`, `max_results: int = 5`, `corpus: str = "all"`, `user?: str` | **读入口**。跑完整召回/重排管线（分词 → 路由 → FTS → 重排 → 去重），给每条命中一个虚拟 `path`，交给 `memory_get` 下钻；只想"把背景拉进上下文"直接用命中里的 `snippet` 即可。自动注入（hook）场景用的就是它：注入块以 `[octop memory search]` 开头，`memory_capture` 按该标记丢弃回声，不会被当成新事件采回来。`corpus`：`all`/`memory`（原子+原始事件同管线）、`atom`（只要 L2 原子）、`raw`（直接走 L0 全文检索，不受"有原子命中就丢 raw"的兜底影响） | +| `memory_get` | `path: str`, `start?: int`, `lines?: int` | 把命中路径解析成完整 markdown。支持 `atom/.md`、`page/.md`、`raw//.md`；长内容用 `start`/`lines` 分页（1-based）。路径非法或过期时返回 `{error, hint}` 而不是抛栈 | +| `memory_raws` | `query?: str`, `session_id?: str`, `host?: str`, `user?: str`, `limit: int = 50` | **原始事件（证据源）**。`query` 走全文检索（写入后立即可见，提取前也能查），其余字段做结构化过滤，按时间倒序返回 | + +**按调用者偏好排序**：调用者 id 有值时（`X-Octop-User-Id` 头或显式 `user` 参数）， +`memory_search` 把该调用者的命中排在最前、其他人的紧随其后——只调顺序、 +**不过滤**（记忆是专家级共享的）。raw 命中按 raw 的 `user` 列判定，原子/主题页按正文里的 +`<调用者>说：` 前缀判定；命中带 `personal` 布尔，raw 命中还带 `user`，返回体带 `caller` +与 `personal_hits`。 + +返回形状： + +```jsonc +// memory_search +{ "hits": [{ "path", "layer", "snippet", "occurred_at", "source_id" }], "total", "corpus", "hint" } +// memory_get +{ "path", "kind", "content", "total_lines", "from_line", "to_line", "truncated", "metadata" } +// memory_raws +{ "events": [{ "event_id", "timestamp", "session_id", "user", "event_type", "source", "content" }], "count", "caller" } +``` + +### 2.2 写入（2 个，两条通道） + +| 工具 | 参数 | 说明 | +|---|---|---| +| `memory_capture` | `content: str`, `source: str`, `session_id?: str`, `assistant?: str`, `thread_id?: str`, `dry_run?: bool`, `extract?: bool`, `tool_events?: str`, `transcript_path?: str`, `user?: str` | **记录原始事件（L0）**：交给「提取 → 候选 → 晋升」流水线，适合记录对话/事件原文。写入后立即可用 `memory_raws` / `memory_search(corpus="raw")` 查；晋升成原子后才回到常规检索结果里。`source` 既是谁记录的，也决定内容形态：`claude-code` / `kiro` / `dsh` + JSONL（`H4sI` 开头自动按 gzip+base64 解码）走该终端的 transcript 解析，整份 session 文件解析成多条事件；其它 `source` 按「一轮对话」处理，`content` 是用户侧、`assistant` 是助手侧（角色由字段隐含，别把助手产出塞进 `content`）。`tool_events` 决定 transcript 形态落哪些事件：`none`（默认）只落 user/assistant，`calls` 带工具调用（丢输出），`all` 连工具输出一起落。`dry_run` 只解析不写；`extract=False` 先只落 L0，再用 `memory_extract` 受控提取。`transcript_path` 仅供终端 `mcp_tool` hook 直传本机文件（限 `~/.claude/projects` / `~/.kiro/sessions` / `~/.dsh/sessions`；远程部署读不到，请改用 hook 上报 `content`）。`session_id` 缺省派生：一轮对话为 `ext:{source}:{user}`，transcript 为 `cc:` | +| `memory_save` | `content: str`, `source: str`, `topic?: str`, `supersedes?: str`, `user?: str` | **直接保存事实（L2）**：不经过提取，立即可召回。适合已知的明确事实/约定。`supersedes` 传**旧原子 id** 即完成「更正」：旧原子标记 deprecated，新事实带 `supersedes` 关联——更正事实 = 保存新事实 + 声明它取代谁，不需要单独的更新工具 | + +`memory_capture` 是**幂等**的：同一 `session_id` + 同一内容重复写入时不会产生重复 L0 事件， +返回里带 `duplicate: true` 并复用已有 `event_id`（下游提取因此可以反复重跑）。 + +它还有**回声保护**：内容里带记忆注入标记（`[octop memory …]` / `[memory] Earlier in this workspace` / +`## Memory Recall`）时**不写入**，直接返回 `{recorded: false, skipped: "recall_echo"}`。 +原因：MCP 写路径直接调 `Memory.add_raw`，绕过了 `MemoryRuntime.capture` 的 `skip_memory_echo`； +不补这层，hook 注入的记忆块会被当作新事件采集，形成"注入 → 采集 → 再召回"的放大环。 + +transcript 形态还有**原文归档**：上传的整份 session 会另存一份可读 markdown +（`{workspace}/.octop/memory-sessions/.md`，增量追加、重发去重），返回体的 +`session_archive` 给出实际路径。L0 默认只留 user/assistant，工具调用与 thinking 在归档里 +仍可复查；transcript 形态下 hook 自身的读工具输出、注入块也会被丢弃（返回体带 +`skipped_echo` / `skipped_own_results`）。 + +### 2.3 提取与审核流水线（4 个） + +| 工具 | 参数 | 说明 | +|---|---|---| +| `memory_extract` | `session_id?: str`, `limit: int = 100`, `promote: bool = False` | 手动触发提取：取最近 L0 事件 → LLM 类型化提取 → 候选（pending）；`promote=True` 时直接跑晋升检查。复用**该专家进程内**的 `MemoryService`（含其配置的提取模型）；专家未运行/无记忆运行时时返回 `error` | +| `memory_candidates` | `status?: str`, `session_id?: str`, `limit: int = 50` | 列出 L1 候选（默认 pending）。`status` 可取 `pending` / `promoted` / `rejected` / `needs_review` / `conflict`，非法值报错 | +| `memory_promote` | `candidate_ids: list[str]` | 审核晋升：对指定候选跑 5 项晋升检查，通过则写入 L2 原子并记录 journal；逐条结果与错误分别由 `candidates` / `errors` 返回 | +| `memory_reject` | `candidate_id: str`, `reason: str = "rejected by external caller"`, `actor?: str` | 拒绝候选并记录原因（不进原子层，journal 可审计） | + +--- + +## 3. 记忆分层与选工具 + +| 层级 | 内容 | 对应工具 | +|---|---|---| +| L0 | 原始事件（原话、证据） | `memory_capture` 写入；`memory_raws` / `memory_search(corpus="raw")` 读取 | +| L1 | 候选（提取产物，待审核） | `memory_extract` 产生；`memory_candidates` 查看；`memory_promote` / `memory_reject` 裁决 | +| L2 | 原子（长期记忆，可被检索到） | `memory_save` 直写（`supersedes` 做更正）；`memory_capture` 经流水线晋升；`memory_search` 检索 | +| L3 | 主题页（实体页） | 随晋升重新生成；可用 `memory_get(page/.md)` 读取 | + +按意图选工具： + +- 想把相关背景拉进上下文 → `memory_search`（命中里的 `snippet` 直接可用） +- 要定位某条具体记忆并读全文 → `memory_search` 拿 `path`，再 `memory_get(path)` +- 要原话/证据，或内容刚写入还没晋升 → `memory_raws`（或 `memory_search(corpus="raw")`） +- 记录对话/事件 → `memory_capture`；记录明确事实/规则 → `memory_save`；纠正过时事实 → `memory_save(supersedes=<旧原子 id>)` +- 处理审核队列 → `memory_candidates` → `memory_promote` / `memory_reject` + +--- + +## 4. 使用示例 + +> 下面默认客户端**直连** Octop 的 `/mcp/memory`。如果 Octop 前面还挂了一层网关/代理（例如由代理统一校验调用人身份并注入上游 token），只需把 URL 换成代理端点、按代理要求填它需要的凭证（通常是**一个** token 头）；此时 `Authorization` / `X-Octop-User-Id` 由代理负责，**不要**在客户端重复配置。 + +> 接入任一客户端都是**两步**：① 配好 MCP 服务器（§4.3 Kiro / §4.4 Claude Code / §4.5 DSH 各自的「配置」段）——这一步只让工具**可用**；② 再配**自动化**（Kiro 与 Claude Code 挂 hooks；DSH 新建并使用「共享记忆」预设）——这一步才会**自动召回、自动沉淀**。只做第 ① 步的话，每次都得手动让模型去调工具。§4.1 / §4.2 是最简的手动自测形态；**§4.7 的一键安装器把 ① ② 一起做掉**，本地终端优先用它。 + +### 4.1 直接 HTTP（curl） + +```bash +TOKEN="" +AGENT="" +BASE="http://127.0.0.1:/mcp/memory" + +# 1) 初始化（MCP 握手） +curl -s -X POST "$BASE/" \ + -H "Authorization: Bearer $TOKEN" \ + -H "X-Octop-Agent-Id: $AGENT" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"cli","version":"1.0"}}}' + +# 2) 记录一条原始事件（L0） +curl -s -X POST "$BASE/" \ + -H "Authorization: Bearer $TOKEN" \ + -H "X-Octop-Agent-Id: $AGENT" \ + -H "X-Octop-User-Id: alice" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"memory_capture","arguments":{"content":"报告页横幅没有渲染出来","source":"review-bot"}}}' +``` + +其余工具只是换 `params.name` / `params.arguments`： + +```jsonc +// 直接保存一条事实（L2，立即可召回） +{ "name": "memory_save", + "arguments": { "content": "部署约定：记忆端点挂在 /mcp/memory", "source": "planning-agent", "topic": "octop-deploy" } } + +// 检索（拿 path 或直接读 snippet） +{ "name": "memory_search", "arguments": { "query": "部署约定", "corpus": "atom" } } + +// 读全文（path 来自上一步 hits[].path） +{ "name": "memory_get", "arguments": { "path": "atom/.md", "lines": 40 } } +``` + +### 4.2 MCP 客户端 SDK（Python） + +```python +# pip install mcp +import asyncio + +from mcp import ClientSession +from mcp.client.streamable_http import streamablehttp_client + + +async def main(): + # 注意：endpoint 以 / 结尾；鉴权与专家绑定都放在 headers 里 + async with streamablehttp_client( + "http://127.0.0.1:/mcp/memory/", + headers={ + "Authorization": "Bearer ", + "X-Octop-Agent-Id": "", + "X-Octop-User-Id": "alice", + }, + ) as (read, write, _): + async with ClientSession(read, write) as session: + await session.initialize() + tools = await session.list_tools() + print([t.name for t in tools.tools]) # 11 个 memory_* 工具 + + await session.call_tool( + "memory_capture", + {"content": "报告页横幅没有渲染出来", "source": "review-bot"}, + ) + res = await session.call_tool("memory_search", {"query": "报告页 横幅", "corpus": "atom"}) + print(res) + + +asyncio.run(main()) +``` + +### 4.3 Kiro + +Kiro 原生支持远程 MCP（`url` + `headers`）。配置分两级，同名条目以工作区为准： + +- 工作区：`.kiro/settings/mcp.json` +- 用户级：`~/.kiro/settings/mcp.json` +- 打开方式：命令面板 `Kiro: Open workspace MCP config (JSON)` / `Kiro: Open user MCP config (JSON)`；保存后自动热重载。 + +```jsonc +{ + "mcpServers": { + "octop-memory": { + "type": "streamable_http", + "url": "https:///mcp/memory/", + "headers": { + "Authorization": "Bearer ", + "X-Octop-Agent-Id": "", + "X-Octop-User-Id": "" + }, + "disabled": false + } + } +} +``` + +Kiro 侧注意事项： + +- **远程 `url` 必须是 `https`**（只有本地回环允许 `http`）。 +- 工具名带**服务器名前缀**：服务器名 `octop-memory` → 工具 `mcp_octop_memory_memory_search`；在 hooks 或提示词里必须写全名。 +- 工具默认不放行：在 `~/.kiro/settings/permissions.yaml` 里找到 `capability: mcp` 的 `match` 列表，按需加 `effect: allow`（只读先放 `memory_search` / `memory_get` / `memory_raws`，需要沉淀再放 `memory_capture` / `memory_save`）。 +- 用 `${VAR}` 引用环境变量时，要先把变量加入 Kiro 的允许列表（设置项 `Mcp Approved Env Vars`），否则不会被展开。 +- 校验：会话里执行 `/mcp`，确认服务器已连接、工具已加载。 + +#### 自动化：hooks（配好 MCP 之后必配） + +**只配 `mcp.json` 只是"工具能用"——不会自动召回、也不会自动沉淀，必须再挂 hooks。** Kiro 的 hooks 放在 `~/.kiro/hooks/*.json`，用 `trigger` + `action.type: agent` 让模型在固定时机去调对应工具。 + +提交提示词时召回 —— `~/.kiro/hooks/memory-recall-on-prompt-submit.json`： + +```jsonc +{ + "version": "v1", + "hooks": [ + { + "name": "Recall Memory on Prompt Submit", + "trigger": "UserPromptSubmit", + "action": { + "type": "agent", + "prompt": "回答前，按需主动检索相关记忆来辅助本次工作。结合用户本次输入与当前项目上下文构造查询，调用 `mcp_octop_memory_memory_search`（corpus=atom）。若本次输入与已检索记忆无关可跳过，不要重复检索同一查询。" + }, + "description": "收到消息时自动召回相关记忆，提供上下文连续性。", + "enabled": true + } + ] +} +``` + +会话结束时沉淀 —— `~/.kiro/hooks/memory-save-on-stop.json`： + +```jsonc +{ + "version": "v1", + "hooks": [ + { + "name": "Save Memory on Session End", + "trigger": "Stop", + "action": { + "type": "agent", + "prompt": "会话结束前静默沉淀：1) 调用 `mcp_octop_memory_memory_capture` 记录本次会话（source=\"kiro-session\"，session_id 形如 \"session-YYYY-MM-DD-主题\"）；2) 对值得长期保留的事实（偏好、约定、架构决策、环境配置）调用 `mcp_octop_memory_memory_save`（source=\"kiro-session\"）。只写对未来确有价值的信息；成功后一句话告知，不要复述会话内容。" + }, + "description": "会话结束时把可复用事实沉淀到 octop-memory。", + "enabled": true + } + ] +} +``` + +改完 hooks / permissions 后重载或重启 Kiro 生效。 + +### 4.4 Claude Code + +Claude Code 直接支持远程 HTTP + 自定义请求头： + +```bash +claude mcp add --transport http octop-memory https:///mcp/memory/ \ + --header "Authorization: Bearer " \ + --header "X-Octop-Agent-Id: " \ + --header "X-Octop-User-Id: " +``` + +`--header` 可重复（短写 `-H`）。也可以直接写配置文件（`.mcp.json` 项目级 / `~/.claude/settings.json` 用户级）： + +```jsonc +{ + "mcpServers": { + "octop-memory": { + "type": "http", + "url": "https:///mcp/memory/", + "headers": { + "Authorization": "Bearer ${OCTOP_MEMORY_MCP_TOKEN}", + "X-Octop-Agent-Id": "", + "X-Octop-User-Id": "" + } + } + } +} +``` + +Claude Code 侧注意事项： + +- Streamable HTTP 写 `"type": "http"`（MCP 规范名 `streamable-http` 也接受）；**带 `url` 就必须带 `type`**，否则该条目会被当作 stdio 服务器跳过。 +- 工具名带前缀：`mcp__octop-memory__memory_search`。 +- 排查：`claude mcp list`、`claude mcp get octop-memory`，会话内 `/mcp`。 + +#### 自动化：hooks（配好 MCP 之后必配） + +同样，**只加 MCP 服务器不会自动召回/沉淀**。Claude Code 在 `settings.json` 的 `hooks` 字段挂命令脚本（脚本放 `~/.claude/hook-events/`），两个事件各一个： + +| 脚本 | 事件 | 作用 | +|---|---|---| +| `octop-user-prompt-recall.mjs` | `UserPromptSubmit` | 向 stdout 注入「先召回」的指令；真正调用工具的是模型，脚本**不直连 MCP**，任何异常静默放行、不阻断本轮 | +| `octop-stop-capture.mjs` | `Stop` | 输出 `{"decision":"block","reason":"..."}`，让模型在追加的一轮里执行 `memory_capture`；`stop_hook_active` 为真时直接退出，防止召回/沉淀互相触发成死循环 | + +```jsonc +// ~/.claude/settings.json +{ + "hooks": { + "UserPromptSubmit": [ + { "hooks": [ { "type": "command", "command": "node ~/.claude/hook-events/octop-user-prompt-recall.mjs" } ] } + ], + "Stop": [ + { "hooks": [ { "type": "command", "command": "node ~/.claude/hook-events/octop-stop-capture.mjs" } ] } + ] + } +} +``` + +`octop-user-prompt-recall.mjs` 最小实现： + +```javascript +#!/usr/bin/env node +// UserPromptSubmit：把「先检索」指令注入本轮上下文；真正调用 MCP 工具的是模型。 +process.stdout.write( + "[octop-memory] 回答前先按需检索相关记忆：结合用户输入与当前项目上下文构造查询，" + + "调用 MCP 工具 mcp__octop-memory__memory_search（corpus=atom）；与本轮无关可跳过，不要重复检索同一查询。\n", +); +``` + +`octop-stop-capture.mjs` 最小实现： + +```javascript +#!/usr/bin/env node +// Stop：让模型多跑一轮，把本轮可复用事实写入 L0。 +import fs from "node:fs"; + +let payload = {}; +try { + const raw = fs.readFileSync(0, "utf8"); + if (raw.trim()) payload = JSON.parse(raw); +} catch { + // 解析失败也继续：沉淀不该因为载荷缺字段而丢掉 +} + +if (payload.stop_hook_active) process.exit(0); // 防召回/沉淀互相触发 + +process.stdout.write( + JSON.stringify({ + decision: "block", + reason: + "会话结束前静默沉淀：调用 mcp__octop-memory__memory_capture 写入本轮可复用事实" + + '（source="claude-code"）；没有可沉淀内容就不要写入，成功后一句话告知即可。', + }) + "\n", +); +``` + +### 4.5 DSH（DeepSeek Harness） + +DSH 在 **Settings → MCP** 里管理 MCP 服务器（定义持久化在 `~/.dsh/storages/mcp_servers.json`）。设置页新增一条 Streamable HTTP 服务器时，对应的字段如下： + +```jsonc +{ + "serverName": "octop-memory", + "transport": "streamable-http", + "enabled": true, + "url": "https:///mcp/memory/", + "headers": [ + { "name": "Authorization", "value": "Bearer ${OCTOP_MEMORY_MCP_TOKEN}" }, + { "name": "X-Octop-Agent-Id", "value": "" }, + { "name": "X-Octop-User-Id", "value": "" } + ], + "toolCallTimeoutMs": 60000, + "failOnStartupError": true +} +``` + +注意事项： + +- `transport` 取 `streamable-http`（另支持 `stdio`）；`enabled: false` 停用整条；`failOnStartupError` 决定连接失败是否阻断启动。 +- 设置页里的 Headers 是每行一个 `名称: 值`，值支持 `${ENV}` 替换；密钥类变量交给 DSH 的全局环境变量/凭据存储，不要写进明文配置。 +- DSH 默认**按需注入** MCP 工具：会话里先检索一次，才会把 `memory_*` 挂进当前对话——所以只加服务器**更不会**自动召回/沉淀。 +- 新增/修改服务器后按提示重连；若工具列表没有刷新，重启 `dsh web`。 + +#### 自动化：共享记忆预设（配好 MCP 之后必配） + +和 Kiro / Claude Code 要挂 hooks 一样，DSH 这边还要**新增一个「共享记忆」模式的 agent preset，并用它开会话**，自动化才会发生： + +- 会话首轮：先 `memory_search`（`corpus=atom`），把相关原子记忆拉进上下文； +- 每轮结束：若本轮产生了可沉淀的事实，调用一次 `memory_capture`（`source` 标记客户端来源、`session_id` 用工作目录派生）；寒暄或已记录内容不重复写。 + +预设是一个目录（`~/.dsh/.agent-presets/<你的预设>/`），含 `preset.yml`（名称与描述）和 `agent.cordis.yml`（该预设的完整组合）： + +```text +<你的预设>/ + preset.yml + agent.cordis.yml # 共享记忆策略写在 persona 文本末尾 +``` + +`agent.cordis.yml` 的 `persona` 里追加的策略段（等价于 Kiro / Claude Code 的 hooks）： + +```text +── 共享记忆 (shared memory) ───────────────────────────── +This preset auto-reads and auto-writes the octop-memory expert store so durable facts carry across sessions and experts: memory_search at conversation start, memory_capture at turn end. + +会话开始时(本会话首轮、尚无历史):若工具列表里没有 `memory_search`,先按需注入 octop-memory 工具,再调用 `memory_search(query=用户当前目标全文, corpus=atom)`,把相关原子记忆纳入上下文。 + +每轮结束时(给出最终回复前):若本轮产生了可沉淀的事实(需求、决定、偏好、约定、结论、关键路径),调用一次 `memory_capture`: + content: 1–3 句简洁事实摘要(不写机密、不做原始转储) + source: 'dsh:shared-memory' + session_id: 'ext:dsh:<当前工作目录名>' +寒暄或已记录内容不重复记录;没有可沉淀事实就不调用。 +``` + +- 从自带预设**拷贝到自己目录再改**，不要直接改部署自带的预设（升级会覆盖）。 +- 建好后要在会话里**选择这个预设**才生效；已经在跑的会话不会自动切换。 + +### 4.6 其它 MCP 客户端 + +多数客户端共用同一份 JSON 描述一个 Streamable HTTP server： + +```jsonc +{ + "mcpServers": { + "octop-memory": { + "type": "streamable-http", + "url": "https:///mcp/memory/", + "headers": { + "Authorization": "Bearer ", + "X-Octop-Agent-Id": "", + "X-Octop-User-Id": "" + } + } + } +} +``` + +只支持 stdio 的客户端可以用 `mcp-remote` 这类桥接工具转发到远程端点；`type` 的取值随客户端而异（有的写 `http`，有的写 `streamable-http`），以客户端文档为准。自动化同样取决于该客户端有没有 hooks / 预设机制：只挂服务器通常只解决"工具可用"。 + +### 4.7 一键安装器（Claude Code / Kiro / DSH / Codex / Cursor） + +服务端自带安装器。**命令、零交互变量、自检与卸载见开头的「快速上手」**；这一节是它的实现细节 +（写了哪些文件、地址参数、鉴权头）。 + +```bash +curl -fsSL "https:///mcp/memory/setup" | sh +# 可预填的参数（等价的环境变量见快速上手）： +# ?agent= 专家 ID +# ?terminals= claude-code,kiro,dsh,codex,cursor +# ?url= 客户端 MCP 条目的地址（缺省 = 本实例挂载点） +# ?hook_url= hooks 实际 POST 的地址（缺省 = 本实例挂载点） +``` + +- **不发放凭据**：token 由运行脚本的人当场输入（读 `/dev/tty`，不回显）；可用 + `OCTOP_MEMORY_TOKEN` 预置。部署若已设 `OCTOP_MEMORY_MCP_TOKEN`，该值会内置成脚本默认凭据。 +- 鉴权头由 `OCTOP_TOKEN_HEADER` 决定，缺省 `Authorization`（此时值写成 `Bearer `）； + hooks 与条目共用这一份凭据。 +- 每个终端各写自己的 hooks 目录与配置文件：Claude Code `~/.claude/hooks-events` + + `settings.json`、Kiro `~/.kiro/hooks`、DSH `~/.dsh/hooks/octop-hooks.json`、 + Codex `~/.codex/hooks` + `config.toml`、Cursor `~/.cursor/hooks` + `hooks.json`； + 各带一份 `config.json`（endpoint / token / 专家）。 +- **可重复执行**：重跑先清理早期版本的 hook 文件与本安装器的陈旧登记，同一端点只保留一条 + MCP 条目（这是升级路径）。 +- 卸载：`curl -fsSL "https:///mcp/memory/uninstall" | sh`。 + +--- + +## 5. 部署与配置 + +| 配置 | 说明 | +|---|---| +| `OCTOP_MEMORY_MCP_TOKEN` | 必填。未设置时 `/mcp/memory` 不挂载（fail-closed）；同时作为 §4.7 安装器脚本的默认凭据 | +| `X-Octop-Agent-Id` | 每个请求必填，指向一个存在且启用的专家 | +| `X-Octop-User-Id` | 可选；用于把发送者写进记忆正文（§1.3） | +| `OCTOP_MEMORY_PUBLIC_MCP_URL` / `config.json: memory_public_mcp_url` | 可选。把客户端 MCP **条目**指向自建的公网入口（hooks 仍打本实例挂载点，凭据同一份）；不配则条目 = 本实例挂载点。安装器也可用 `?url=` 单次覆盖 | +| `OCTOP_TOKEN_HEADER` | 可选。安装器生成的 hooks 使用的鉴权头名，缺省 `Authorization`（值写成 `Bearer `） | +| 记忆后端 | 由专家配置 `memory.backend` 决定：默认 SQLite（`memory.sqlite`）；PostgreSQL 控制面下默认复用控制面 DSN（每专家 schema） | + +开启步骤： + +1. 设置环境变量 `OCTOP_MEMORY_MCP_TOKEN=`。 +2. 重启 Octop 服务（挂载发生在启动阶段）。 +3. 验证：不带 token 请求 `/mcp/memory/` 应返回 `401`；带 token 且带合法 + `X-Octop-Agent-Id` 时应返回 MCP 协议响应（未知/停用专家返回 `404`）。 +4. 外部 agent 按 §1 / §4 接入。 + +--- + +## 6. 行为契约与边界 + +- **写入分两条通道**：`memory_capture` 走"提取 → 候选 → 晋升"（学习型记忆，质量由流水线把关）； + `memory_save`（`supersedes` 更正）是权威直写（规则/明确事实，立即可召回）。 + 晋升治理（提取触发、候选审核）保留在服务端，站内会话与外部写入走**同一套**记忆治理路径。 +- **capture 之后常规检索没有结果属预期**：内容还在 L0，需要经提取晋升成原子才会进入 + `memory_search` 的结果；想立刻看到请用 `memory_raws` 或 `memory_search(corpus="raw")`。 +- **capture 幂等**：同 `session_id` + 同内容不重复落库（见 §2.2）。 +- **召回回声有写入侧保护**：`memory_capture` 丢弃带注入标记的内容（`skipped=recall_echo`）—— + 注入块以 `[octop memory search]` 开头，所以 hook 注入 → 采回 → 再检索的放大环被写侧断掉， + 不需要读侧再传 `session_id` 配合。 + 两侧都做，才不会出现"注入 → 采集 → 再召回"的放大环（§4.3 / §4.4 的自动召回就是这个场景）。 +- **召回是专家级共享**，不做按人隔离；按发送者定位依赖正文里的 `说：` 前缀 + 全文检索。 +- **错误形态**：`memory_get` 的坏路径返回 `{error, hint}`；`memory_search` 的非法 `corpus`、 + `memory_candidates` 的非法 `status` 直接报错（参数错误不会被静默吞掉）。 diff --git a/src/octop/api/app.py b/src/octop/api/app.py index ccc5e1091..333c3c06d 100644 --- a/src/octop/api/app.py +++ b/src/octop/api/app.py @@ -298,6 +298,24 @@ async def acme_http01_challenge(token: str) -> PlainTextResponse: ], ) + # 专家记忆 MCP server（对外暴露，独立 token 鉴权，未配置 OCTOP_MEMORY_MCP_TOKEN 时不挂载） + from octop.infra.agents.memory_mcp import mount_memory_mcp + + memory_mcp_managers = mount_memory_mcp(app, server) + if memory_mcp_managers: + from collections.abc import AsyncIterator + from contextlib import AsyncExitStack, asynccontextmanager + + @asynccontextmanager + async def _memory_mcp_lifespan(application: FastAPI) -> AsyncIterator[None]: + # streamable_http_app 的 task group 依赖 lifespan，挂载后须手动并入 + async with AsyncExitStack() as stack: + for mgr in memory_mcp_managers: + await stack.enter_async_context(mgr.run()) + yield + + app.router.lifespan_context = _memory_mcp_lifespan + if enable_api_docs: @app.get("/api/docs", include_in_schema=False) diff --git a/src/octop/infra/agents/agent_presets/dsh-memory-share/cordis.patch.yml b/src/octop/infra/agents/agent_presets/dsh-memory-share/cordis.patch.yml new file mode 100644 index 000000000..023f486ad --- /dev/null +++ b/src/octop/infra/agents/agent_presets/dsh-memory-share/cordis.patch.yml @@ -0,0 +1,164 @@ +# DSH agent preset「记忆共享模式」: the shipped `cordis` (创造模式) child plugin +# list, plus the octop memory hooks. Copied from DSH's `@deepseek-ai/dsh-web-app` +# preset `cordis` — refresh this copy when that preset changes (见 client_setup.py +# 的 `_DSH_PRESET_FILES`), and keep `configPath` as the installer's placeholder. +- insert: + - id: preset-octop-shared-memory + name: '@deepseek-ai/dsh-agent-preset' + config: + id: octop-shared-memory + name: 记忆共享模式 + description: 基于创造模式的全能力副本，并接入 octop 共享记忆： + user prompt submit 自动检索相关记忆注入上下文，session stop 自动上报本会话的对话。 + order: 5 + plugins: + # Same persona as `standard`: tool descriptions and the skill catalog carry + # the operating detail, and the skills own the preset-authoring rules. + - id: persona + name: '@deepseek-ai/dsh-persona' + config: + suffix: Your working directory is {{cwd}}. + prefix: >- + You are a coding agent powered by the {{model}} model. + - id: agent-instructions + name: '@deepseek-ai/dsh-agent-instructions' + config: + maxBytes: 65536 + - id: tool-bash + name: '@deepseek-ai/dsh-tool-bash' + disabled: !!js process.platform === 'win32' + - id: tool-pwsh + name: '@deepseek-ai/dsh-tool-pwsh' + disabled: !!js process.platform !== 'win32' + - id: tool-fs + name: '@deepseek-ai/dsh-tool-fs' + - id: tool-fs-search + name: '@deepseek-ai/dsh-tool-fs-search' + config: + sampleOverCapGlobResults: false + - id: tool-jobs + name: '@deepseek-ai/dsh-tool-jobs' + - id: command-goal + name: '@deepseek-ai/dsh-command-goal' + - id: tool-goal + name: '@deepseek-ai/dsh-tool-goal' + - id: planning + name: cordis:group + group: true + isolate: + planMode: true + config: + - id: plan-mode + name: '@deepseek-ai/dsh-plan-mode' + config: + section: | + You are in plan mode. Stay in plan mode until exit_plan_mode succeeds or the user switches the session mode. Imperative language to implement changes means plan the implementation, not execute it. A user's conversational agreement — including an answer confirming something you asked — approves nothing and does not end plan mode; fold the confirmed decision into the plan and submit it through exit_plan_mode. + + Explore first. Use non-mutating reads, searches, static analysis, and checks to ground the plan in the actual repository. Do not edit or write files, change configuration, run formatters or code generation that rewrites tracked files, commit, or otherwise carry out the plan. Prefer existing functions and patterns over new machinery. + + The tool catalog stays the same across modes for request-cache stability. These plan-mode rules override any later tool description or guidance that suggests using mutation tools; those tools remain listed to keep the tool catalog unchanged. Do not use todo_write to track this planning phase: it tracks implementation after an approved plan, while the plan itself belongs in exit_plan_mode. + + Resolve discoverable facts by inspection. Use ask_user_question only for user-owned choices or material ambiguity that inspection cannot answer. Do not ask the user where code lives or how current behavior works when you can find out. + + Make the plan decision-complete: state the goal and success criteria; group implementation changes by subsystem; identify public API, schema, and data-flow changes; cover edge cases, failure modes, tests, acceptance criteria, and explicit assumptions. Keep it concise enough to review but detailed enough that another engineer can implement it without making design decisions. + + When ready, call exit_plan_mode with the complete plan markdown, starting with a # title. Make exit_plan_mode the only and final tool call in that assistant response: it presents the plan for approval, and implementation begins only in a later step after approval. Do not paste the final plan as a plain reply or ask "should I proceed?" through prose or ask_user_question. If review rejects it, incorporate the feedback and present again. If the review channel is unavailable or aborted, stay in plan mode and ask the user to switch modes manually; do not proceed with implementation. + - id: compaction + name: cordis:group + group: true + isolate: + compaction: true + toolResultPruner: true + config: + - id: compaction-basic + name: '@deepseek-ai/dsh-compaction-basic' + - id: command-compact + name: '@deepseek-ai/dsh-command-compact' + - id: tool-result-pruner + name: '@deepseek-ai/dsh-compaction-tool-result-pruner' + config: + thresholdChars: 8192 + headChars: 4096 + tailChars: 1024 + - id: delegation + name: cordis:group + group: true + isolate: + workflowEngine: true + config: + - id: tool-subagent-control + name: '@deepseek-ai/dsh-tool-subagent-control' + - id: tool-subagent-list-agents + name: '@deepseek-ai/dsh-tool-subagent-control/list-agents' + - id: tool-subagent + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: spawn + toolName: subagent + modelSelectionSettings: true + backgroundMode: continuable + - id: tool-subagent-fork + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: fork + toolName: subagent_fork + backgroundMode: continuable + - id: tool-subagent-codex + name: '@deepseek-ai/dsh-tool-subagent' + disabled: true + config: + provider: codex + toolName: subagent_codex + backgroundMode: one-shot + maxDepth: provider-managed + - id: tool-subagent-claude-code + name: '@deepseek-ai/dsh-tool-subagent' + disabled: true + config: + provider: claude-code + toolName: subagent_claude_code + backgroundMode: one-shot + maxDepth: provider-managed + - id: workflow-ptc + name: '@deepseek-ai/dsh-workflow-ptc' + config: + provider: spawn + - id: tool-workflow + name: '@deepseek-ai/dsh-tool-workflow' + - id: tool-ralph + name: '@deepseek-ai/dsh-tool-ralph' + disabled: true + config: + subagentProvider: spawn + maxRounds: 64 + - id: tool-ask-user + name: '@deepseek-ai/dsh-tool-ask-user' + - id: tool-todo + name: '@deepseek-ai/dsh-tool-todo' + config: + allowParallelInProgress: true + - id: tool-web + name: '@deepseek-ai/dsh-tool-web' + config: + fetch: true + searchTimeoutMs: 60000 + - id: tool-cordis + name: '@deepseek-ai/dsh-tool-cordis' + - id: skill-filesystem + name: '@deepseek-ai/dsh-skill-filesystem' + config: + customSkillDirs: + - !!js process.getBuiltinModule('node:path').join(process.getBuiltinModule('node:path').dirname(process.getBuiltinModule('node:module').createRequire(baseUrl).resolve('@deepseek-ai/dsh-agent-preset/package.json')), 'skills') + - id: tool-skill + name: '@deepseek-ai/dsh-tool-skill' + - id: present + name: '@deepseek-ai/dsh-tool-present' + - id: tool-plugin-manager + name: '@deepseek-ai/dsh-plugin-manager/tools' + disabled: !!js "!ctx.get('profileContext')" + + # 共享记忆：UserPromptSubmit 检索注入，Stop 上报（hook 脚本见 ~/.dsh/hooks/dsh）。 + - id: hooks-octop-memory + name: '@deepseek-ai/dsh-hooks-claude-code' + config: + configPath: __OCTOP_HOOKS_JSON__ diff --git a/src/octop/infra/agents/agent_presets/dsh-memory-share/package.json b/src/octop/infra/agents/agent_presets/dsh-memory-share/package.json new file mode 100644 index 000000000..2f113d55f --- /dev/null +++ b/src/octop/infra/agents/agent_presets/dsh-memory-share/package.json @@ -0,0 +1,12 @@ +{ + "name": "octop-dsh-memory-share", + "version": "0.1.0", + "private": true, + "description": "DSH agent preset 记忆共享模式: the shipped cordis toolset plus octop memory hooks (UserPromptSubmit recall, Stop capture).", + "type": "module", + "dsh": { + "bundle": { + "patch": "./cordis.patch.yml" + } + } +} diff --git a/src/octop/infra/agents/claude_transcript.py b/src/octop/infra/agents/claude_transcript.py new file mode 100644 index 000000000..4d486fe3d --- /dev/null +++ b/src/octop/infra/agents/claude_transcript.py @@ -0,0 +1,1100 @@ +"""Replay terminal session transcripts (Claude Code, Kiro) into an agent's memory (L0). + +Claude Code already writes a complete session transcript to +``~/.claude/projects//.jsonl``: every user prompt, assistant +reply, tool call and tool result, each with its own timestamp. Re-capturing that +session through ``memory_capture`` cannot reproduce it — a real session has +hundreds of turns and dozens of tool calls — so this module replays the +transcript straight into L0 instead, where the normal extraction pipeline +(``extract`` → candidate → promote → atom) picks it up unchanged. + +Record mapping (record types that carry no conversation content — ``progress`` / +``queue-operation`` / ``last-prompt`` / ``system`` — are ignored): + +============================ ===================== ========================== +Claude Code record L0 ``event_type`` payload +============================ ===================== ========================== +user, ``content`` str/text ``user_message`` — +user, ``tool_result`` block ``tool_result`` tool_use_id, is_error +assistant, ``text`` block ``assistant_message`` — +assistant, ``tool_use`` block ``tool_call`` tool_name, input +assistant, ``thinking`` — skipped (scratchpad) +============================ ===================== ========================== + +Every event carries ``payload["src_key"] = ":"``, which +makes the import idempotent: keys already present in the target session are +skipped, so re-running a file (or a transcript that grew with new turns) only +appends what is missing. +""" + +from __future__ import annotations + +import hashlib +import json +import logging +import uuid +from collections.abc import Callable, Iterator +from dataclasses import dataclass, field +from datetime import UTC, datetime +from pathlib import Path +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from octop_memory.types import RawEventType + +logger = logging.getLogger(__name__) + +DEFAULT_HOST = "claude-code" +"""``RawEvent.host`` stamped on imported events (``memory_raws(host=…)`` filter).""" + +SESSION_PREFIX = "cc:" +"""L0 session id prefix; one Claude Code session maps onto one octop session.""" + +_MAX_CONTENT_CHARS = 4000 +"""Cap on a stored L0 body. + +A ``tool_result`` can be an entire file or build log, and the extractor prompt +receives every event's ``content`` in one call — so bodies have to stay bounded. +""" + +_MAX_PAYLOAD_CHARS = 4000 +"""Cap on a single serialized payload value (tool inputs are stored verbatim).""" + +_MESSAGE_EVENT_TYPES: frozenset[str] = frozenset({"user_message", "assistant_message"}) +"""The two event types a whole-message marker check may drop.""" + +_TOOL_EVENT_POLICIES: dict[str, frozenset[str]] = { + "all": frozenset(), + "calls": frozenset({"tool_result"}), + "none": frozenset({"tool_call", "tool_result"}), +} +"""``tool_events`` value → the tool event types to drop from L0. + +This is a *landing* policy, not an upload filter: the terminal always ships the whole +transcript (byte-delta, gzip+base64) and the parser reads every record — the policy +only decides which records become L0 rows. The product entry points (the +``memory_capture`` tool and ``octop memory import-claude``) default to ``none``: +L0 raw events feed the extractor and act as the recall fallback layer, and tool +traffic is 79% of a transcript's bytes (``tool_result`` alone 61%), so keeping it +dilutes what the session's memories look like. ``calls`` keeps what the agent did +and drops the output; ``all`` keeps the record whole for callers that want it. +""" + + +def _dropped_tool_types(tool_events: str) -> frozenset[str]: + try: + return _TOOL_EVENT_POLICIES[tool_events] + except KeyError: + raise ValueError( + f"unknown tool_events {tool_events!r}; expected one of {sorted(_TOOL_EVENT_POLICIES)}" + ) from None + + +_EXISTING_SCAN_LIMIT = 100_000 +"""How many existing L0 rows to scan for ``cc_key`` when deduplicating.""" + +_OWN_READ_TOOLS: frozenset[str] = frozenset( + { + "memory_recall", + "memory_search", + "memory_get", + "memory_raws", + "memory_candidates", + } +) +"""Our own memory *read* tools. + +Their tool results restate what is already stored, so importing them back into L0 +would close the "inject → capture → recall again" loop. They are dropped at parse +time — before the marker-based check — because a hook may format its own +injection block and lose the marker. + +``memory_recall`` is no longer exposed by the MCP server; it stays listed so +transcripts recorded by earlier installs still get filtered. +""" + + +def _is_own_read_tool(tool_name: str) -> bool: + """True for our read tools under any client naming. + + The MCP entry may be called ``octop-memory-local`` / ``-prod`` / ``-oauth`` or + anything the user renamed it to, so only the tool name decides: the bare + ``memory_search``, Claude Code's ``mcp____memory_search`` and Kiro's + ``mcp__memory_search``. + """ + tail = tool_name.rsplit("__", 1)[-1] + return tail in _OWN_READ_TOOLS or any(tool_name.endswith(name) for name in _OWN_READ_TOOLS) + + +@dataclass(frozen=True) +class TranscriptEvent: + """One L0 event mapped out of a Claude Code transcript record.""" + + key: str + event_type: RawEventType + content: str + timestamp: datetime + payload: dict[str, Any] + + +@dataclass(frozen=True) +class ParsedSession: + """Result of parsing one ``.jsonl`` transcript.""" + + session_id: str | None + """The Claude Code ``sessionId`` (``None`` when no record carried one).""" + + events: list[TranscriptEvent] + + skipped_own_results: int = 0 + """Tool results dropped because they came from our own memory read tools.""" + skipped_tools: int = 0 + """Tool events dropped by the caller's ``tool_events`` policy.""" + + +@dataclass +class ImportStats: + """Per-file import counters, reported back to the caller.""" + + session_id: str + parsed: int = 0 + imported: int = 0 + skipped_existing: int = 0 + skipped_echo: int = 0 + """Events dropped because their content carried a recall-injection marker.""" + skipped_own_results: int = 0 + """Tool results dropped because they came from our own memory read tools.""" + skipped_tools: int = 0 + """Tool events dropped by the caller's ``tool_events`` policy.""" + sidechain: int = 0 + dry_run: bool = False + warnings: list[str] = field(default_factory=list) + by_type: dict[str, int] = field(default_factory=dict) + """Parsed-event histogram (``event_type`` -> count), for caller verification.""" + + +# --------------------------------------------------------------------------- +# parsing (pure — no writes, no memory instance) +# --------------------------------------------------------------------------- + + +def _iter_records(text: str) -> Iterator[Any]: + for line in text.splitlines(): + line = line.strip() + if not line: + continue + try: + yield json.loads(line) + except json.JSONDecodeError: + logger.debug("claude_transcript: skipping malformed transcript line") + + +def _parse_timestamp(value: Any) -> datetime: + if isinstance(value, (int, float)) and value: + # DSH writes epoch milliseconds (``time``), older hosts seconds. + seconds = value / 1000 if value > 1e11 else float(value) + try: + return datetime.fromtimestamp(seconds, tz=UTC) + except (OverflowError, OSError, ValueError): + return datetime.now(UTC) + if isinstance(value, str) and value: + text = value[:-1] + "+00:00" if value.endswith("Z") else value + try: + parsed = datetime.fromisoformat(text) + except ValueError: + parsed = None + if parsed is not None: + return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC) + return datetime.now(UTC) + + +def _block_text(value: Any) -> str: + """Flatten a tool-result body (string, or a list of content blocks) to text.""" + if isinstance(value, str): + return value + if isinstance(value, list): + parts = [ + str(block.get("text") or "") + for block in value + if isinstance(block, dict) and block.get("type") == "text" + ] + return "\n".join(part for part in parts if part) + return "" + + +def _cap(text: str, limit: int = _MAX_CONTENT_CHARS) -> str: + body = (text or "").strip() + return body if len(body) <= limit else body[: limit - 1].rstrip() + "…" + + +def _bounded(value: Any) -> Any: + """Keep a payload value structured but bounded (payload is stored verbatim).""" + if isinstance(value, str): + return _cap(value, _MAX_PAYLOAD_CHARS) + try: + encoded = json.dumps(value, ensure_ascii=False, default=str) + except (TypeError, ValueError): + return str(value)[:_MAX_PAYLOAD_CHARS] + if len(encoded) <= _MAX_PAYLOAD_CHARS: + return value + return {"truncated_json": encoded[:_MAX_PAYLOAD_CHARS]} + + +def _blocks(content: Any) -> list[dict[str, Any]]: + """Normalize a message body to a list of content blocks.""" + if isinstance(content, str): + return [{"type": "text", "text": content}] + if isinstance(content, list): + return [block for block in content if isinstance(block, dict)] + return [] + + +def _map_block( + block: dict[str, Any], *, role: str +) -> tuple[RawEventType, str, dict[str, Any]] | None: + """Map one content block to ``(event_type, content, payload extras)``.""" + block_type = block.get("type") + if block_type == "text": + text = str(block.get("text") or "") + if not text.strip(): + return None + event_type: RawEventType = "user_message" if role == "user" else "assistant_message" + return event_type, _cap(text), {} + if block_type == "tool_use": + name = str(block.get("name") or "tool") + args = block.get("input") + summary = json.dumps(args, ensure_ascii=False, default=str) if args else "" + return ( + "tool_call", + _cap(f"{name} {summary}".strip()), + {"tool_use_id": block.get("id"), "tool_name": name, "input": _bounded(args)}, + ) + if block_type == "tool_result": + body = _cap(_block_text(block.get("content"))) + return ( + "tool_result", + body or "(empty tool result)", + {"tool_use_id": block.get("tool_use_id"), "is_error": bool(block.get("is_error"))}, + ) + # ``thinking`` (model scratchpad), images and unknown blocks are not L0 material. + return None + + +def parse_text(text: str) -> ParsedSession: + """Map transcript JSONL *content* to L0 events (pure, no writes). + + This is the entry point for callers that receive the transcript over the + wire (a cloud deployment cannot read the client's ``~/.claude`` directory). + """ + session_id: str | None = None + events: list[TranscriptEvent] = [] + tool_names: dict[str, str] = {} + skipped_own = 0 + for index, record in enumerate(_iter_records(text)): + if not isinstance(record, dict): + continue + record_type = record.get("type") + if record_type not in ("user", "assistant"): + continue + raw_session = record.get("sessionId") + if session_id is None and raw_session: + session_id = str(raw_session) + message = record.get("message") + if not isinstance(message, dict): + continue + role = str(message.get("role") or record_type) + timestamp = _parse_timestamp(record.get("timestamp")) + record_uuid = str(record.get("uuid") or f"line{index}") + sidechain = bool(record.get("isSidechain")) + for block_index, block in enumerate(_blocks(message.get("content"))): + if block.get("type") == "tool_use": + call_id = block.get("id") + if isinstance(call_id, str): + tool_names[call_id] = str(block.get("name") or "") + elif block.get("type") == "tool_result": + call_id = block.get("tool_use_id") + origin = tool_names.get(str(call_id) or "", "") + if origin and _is_own_read_tool(origin): + skipped_own += 1 + continue + mapped = _map_block(block, role=role) + if mapped is None: + continue + event_type, content, extra = mapped + key = f"{record_uuid}:{block_index}" + events.append( + TranscriptEvent( + key=key, + event_type=event_type, + content=content, + timestamp=timestamp, + payload={ + "source": DEFAULT_HOST, + "src_key": key, + "role": role, + "sidechain": sidechain, + **{name: value for name, value in extra.items() if value is not None}, + }, + ) + ) + return ParsedSession(session_id=session_id, events=events, skipped_own_results=skipped_own) + + +def parse_session(path: Path) -> ParsedSession: + """Map a transcript *file* to L0 events (same-host callers).""" + return parse_text(path.read_text(encoding="utf-8")) + + +# --------------------------------------------------------------------------- +# Kiro (``~/.kiro/sessions///messages.jsonl``) +# --------------------------------------------------------------------------- + +_KIRO_KINDS: dict[str, str] = { + "user": "user_message", + "assistant": "assistant_message", + "tool_call": "tool_call", + "tool_result": "tool_result", +} +"""Kiro ``payload.type`` → L0 ``event_type``. + +Everything else Kiro writes (``turn_start`` / ``turn_end`` / ``session_metadata`` / +``usage_summary`` / ``session_event`` / ``pending_interaction`` / +``interaction_resolved``) carries no conversation content. ``ContextualHookInvoked`` +is deliberately absent: that is our own hook invocation — it even carries the hook's +``output`` — i.e. exactly the plumbing that must not become memory. +""" + +_KIRO_SCRATCHPAD: frozenset[str] = frozenset({"Reasoning"}) +"""``assistant.operationType`` that is the model's scratchpad (Kiro's thinking).""" + + +def _map_kiro_payload(payload: dict[str, Any]) -> tuple[RawEventType, str, dict[str, Any]] | None: + kind = payload.get("type") + event_type = _KIRO_KINDS.get(kind) if isinstance(kind, str) else None + if event_type is None: + return None + if event_type in ("user_message", "assistant_message"): + if str(payload.get("operationType") or "") in _KIRO_SCRATCHPAD: + return None + text = str(payload.get("content") or "") + if not text.strip(): + return None + return event_type, _cap(text), {} + if event_type == "tool_call": + name = str(payload.get("toolName") or "tool") + args = payload.get("args") + summary = json.dumps(args, ensure_ascii=False, default=str) if args else "" + return ( + "tool_call", + _cap(f"{name} {summary}".strip()), + { + "tool_use_id": payload.get("toolCallId"), + "tool_name": name, + "input": _bounded(args), + }, + ) + body = _cap(str(payload.get("content") or "")) + return ( + "tool_result", + body or "(empty tool result)", + {"tool_use_id": payload.get("toolCallId"), "success": payload.get("success")}, + ) + + +def parse_kiro_text(text: str) -> ParsedSession: + """Map a Kiro ``messages.jsonl`` to L0 events (pure, no writes). + + The Kiro session id is the **directory** name, not a record field, so callers + pass it explicitly (the hook does: ``session_id="kiro:"``). + """ + events: list[TranscriptEvent] = [] + skipped_own = 0 + for index, record in enumerate(_iter_records(text)): + if not isinstance(record, dict): + continue + payload = record.get("payload") + if not isinstance(payload, dict): + continue + if payload.get("type") == "ContextualHookInvoked": + skipped_own += 1 + continue + mapped = _map_kiro_payload(payload) + if mapped is None: + continue + event_type, content, extra = mapped + record_id = str(record.get("id") or f"line{index}") + key = f"{record_id}:0" + events.append( + TranscriptEvent( + key=key, + event_type=event_type, + content=content, + timestamp=_parse_timestamp(record.get("timestamp")), + payload={ + "source": DEFAULT_HOST, + "src_key": key, + "role": "user" + if event_type in ("user_message", "tool_result") + else "assistant", + "sidechain": False, + **{name: value for name, value in extra.items() if value is not None}, + }, + ) + ) + return ParsedSession(session_id=None, events=events, skipped_own_results=skipped_own) + + +# --------------------------------------------------------------------------- +# DSH (``~/.dsh/sessions///session.jsonl.zstd``) +# --------------------------------------------------------------------------- + +_DSH_TOOL_BLOCK_TYPES: frozenset[str] = frozenset({"tool-call"}) +"""Block types that restate a call already emitted as its own ``tool/call`` record.""" + + +def _dsh_text_blocks(blocks: Any) -> str: + """Join the ``text`` blocks of a DSH content array (images / attachments dropped).""" + if not isinstance(blocks, list): + return "" + parts = [ + str(block.get("text") or "") + for block in blocks + if isinstance(block, dict) and block.get("type") == "text" + ] + return "\n".join(part for part in parts if part) + + +def _dsh_event( + key: str, + event_type: RawEventType, + content: str, + timestamp: datetime, + role: str, + extra: dict[str, Any] | None = None, +) -> TranscriptEvent: + return TranscriptEvent( + key=key, + event_type=event_type, + content=content, + timestamp=timestamp, + payload={ + "source": DEFAULT_HOST, + "src_key": key, + "role": role, + "sidechain": False, + **(extra or {}), + }, + ) + + +def parse_dsh_text(text: str) -> ParsedSession: + """Map a DSH ``session.jsonl`` to L0 events (pure, no writes). + + DSH streams its answer in ``assistant/chunk`` / ``reasoning-chunks`` / + ``text-chunks`` records and then writes the finished message — only the finished + records are mapped, otherwise every token would land in L0 twice. A call also + appears twice (as a ``tool/call`` record and as a ``tool-call`` block inside + ``assistant/message``), so the record wins and the block is skipped by call id. + """ + records = [record for record in _iter_records(text) if isinstance(record, dict)] + canonical_calls = { + str((record.get("data") or {}).get("callId") or "") + for record in records + if record.get("type") == "tool/call" + } + session_id: str | None = None + events: list[TranscriptEvent] = [] + skipped_own = 0 + for index, record in enumerate(records): + record_type = record.get("type") + raw_data = record.get("data") + data: dict[str, Any] = raw_data if isinstance(raw_data, dict) else {} + timestamp = _parse_timestamp(record.get("time")) + seq = record.get("seq", f"line{index}") + if record_type == "session": + session_id = session_id or str(record.get("id") or "") or None + elif record_type == "user/message": + raw_source = data.get("source") + kind = str(raw_source.get("kind") or "") if isinstance(raw_source, dict) else "" + if kind and kind != "user": + skipped_own += 1 + continue + body = _cap(_dsh_text_blocks(data.get("content"))) + if body.strip(): + events.append(_dsh_event(f"{seq}:0", "user_message", body, timestamp, "user")) + elif record_type == "assistant/message": + raw_message = data.get("message") + message: dict[str, Any] = raw_message if isinstance(raw_message, dict) else {} + for block_index, block in enumerate(message.get("content") or []): + if not isinstance(block, dict): + continue + block_type = block.get("type") + if block_type == "text": + body = _cap(str(block.get("text") or "")) + if body.strip(): + events.append( + _dsh_event( + f"{seq}:{block_index}", + "assistant_message", + body, + timestamp, + "assistant", + ) + ) + elif ( + block_type in _DSH_TOOL_BLOCK_TYPES + and str(block.get("id") or "") not in canonical_calls + ): + events.append(_dsh_tool_call(block, f"{seq}:{block_index}", timestamp)) + elif record_type == "tool/call": + events.append( + _dsh_tool_call( + { + "id": data.get("callId"), + "name": data.get("name"), + "arguments": data.get("arguments"), + }, + f"{seq}:0", + timestamp, + ) + ) + elif record_type == "tool/result": + raw_result = data.get("message") + result_message: dict[str, Any] = raw_result if isinstance(raw_result, dict) else {} + for block_index, block in enumerate(result_message.get("content") or []): + if not isinstance(block, dict) or block.get("type") != "tool-result": + continue + body = _cap(_dsh_text_blocks(block.get("content"))) + events.append( + _dsh_event( + f"{seq}:{block_index}", + "tool_result", + body or "(empty tool result)", + timestamp, + "user", + { + "tool_use_id": block.get("toolCallId"), + "is_error": block.get("isError"), + }, + ) + ) + # every other record type is streaming delta / turn-step / metadata noise + return ParsedSession(session_id=session_id, events=events, skipped_own_results=skipped_own) + + +def _dsh_tool_call(block: dict[str, Any], key: str, timestamp: datetime) -> TranscriptEvent: + name = str(block.get("name") or "tool") + raw_args = block.get("arguments") + parsed: Any = raw_args + if isinstance(raw_args, str) and raw_args.strip(): + try: + parsed = json.loads(raw_args) + except json.JSONDecodeError: + parsed = raw_args + summary = json.dumps(parsed, ensure_ascii=False, default=str) if parsed else "" + return _dsh_event( + key, + "tool_call", + _cap(f"{name} {summary}".strip()), + timestamp, + "assistant", + {"tool_use_id": block.get("id"), "tool_name": name, "input": _bounded(parsed)}, + ) + + +# --------------------------------------------------------------------------- +# Codex CLI (``~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl``) +# --------------------------------------------------------------------------- + +_CODEX_TOOL_ITEM_TYPES: frozenset[str] = frozenset( + {"function_call", "local_shell_call", "custom_tool_call", "web_search_call"} +) +_CODEX_TOOL_OUTPUT_TYPES: frozenset[str] = frozenset( + {"function_call_output", "local_shell_call_output", "custom_tool_call_output"} +) +_CODEX_TEXT_ITEM_TYPES: frozenset[str] = frozenset({"input_text", "output_text", "text"}) +"""Codex rollout records: ``event_msg`` carries the conversation, ``response_item`` +the wire-level items (developer instructions, tool calls, and a copy of the +messages). Both are folded here with a text-level dedupe so a turn is never stored +twice.""" + + +def _codex_item_text(content: Any) -> str: + if isinstance(content, str): + return content + if not isinstance(content, list): + return "" + parts: list[str] = [] + for block in content: + if isinstance(block, dict) and block.get("type") in _CODEX_TEXT_ITEM_TYPES: + text = block.get("text") + if isinstance(text, str) and text.strip(): + parts.append(text) + elif isinstance(block, str) and block.strip(): + parts.append(block) + return "\n".join(parts) + + +def parse_codex_text(text: str) -> ParsedSession: + """Map a Codex rollout JSONL to L0 events (pure, no writes).""" + events: list[TranscriptEvent] = [] + skipped_own = 0 + seen_messages: set[str] = set() + for index, record in enumerate(_iter_records(text)): + if not isinstance(record, dict): + continue + payload = record.get("payload") + if not isinstance(payload, dict): + continue + record_type = str(record.get("type") or "") + payload_type = str(payload.get("type") or "") + record_id = str(payload.get("id") or record.get("id") or f"line{index}") + key = f"{record_id}:0" + event_type = "" + content = "" + extra: dict[str, Any] = {} + + if record_type == "event_msg": + if payload_type == "user_message": + event_type, content = "user_message", str(payload.get("message") or "") + elif payload_type == "agent_message": + event_type, content = "assistant_message", str(payload.get("message") or "") + else: + continue # task_started / token_count / turn_aborted … plumbing + if not content.strip(): + continue + seen_messages.add(content.strip()) + elif record_type == "response_item": + if payload_type in _CODEX_TOOL_ITEM_TYPES: + event_type = "tool_call" + name = str(payload.get("name") or payload_type) + content = f"{name} {json.dumps(payload.get('arguments') or payload.get('action') or {}, ensure_ascii=False)}" + elif payload_type in _CODEX_TOOL_OUTPUT_TYPES: + event_type = "tool_result" + output = payload.get("output") + content = ( + output + if isinstance(output, str) + else json.dumps(output or "", ensure_ascii=False) + ) + elif payload_type == "message": + role = str(payload.get("role") or "") + if role not in ("user", "assistant"): + skipped_own += 1 # developer/system instructions are host plumbing + continue + body = _codex_item_text(payload.get("content")) + if not body.strip(): + continue + event_type = "user_message" if role == "user" else "assistant_message" + content = body + if content.strip() in seen_messages: + skipped_own += 1 # same turn already taken from event_msg + continue + seen_messages.add(content.strip()) + else: + continue # reasoning / other wire items + else: + continue # session_meta / turn_context / world_state + + events.append( + TranscriptEvent( + key=key, + event_type=event_type, + content=content, + timestamp=_parse_timestamp(record.get("timestamp")), + payload={ + "source": DEFAULT_HOST, + "src_key": key, + "role": "user" + if event_type in ("user_message", "tool_result") + else "assistant", + "sidechain": False, + **{name: value for name, value in extra.items() if value is not None}, + }, + ) + ) + return ParsedSession(session_id=None, events=events, skipped_own_results=skipped_own) + + +# --------------------------------------------------------------------------- +# Cursor (``~/.cursor/hooks.json`` -> ``transcript_path``) +# --------------------------------------------------------------------------- + +_CURSOR_ROLE_KEYS: tuple[str, ...] = ("role", "type", "author", "sender") +_CURSOR_TEXT_KEYS: tuple[str, ...] = ("text", "content", "message", "value") + + +def _cursor_role(record: dict[str, Any]) -> str: + for name in _CURSOR_ROLE_KEYS: + value = record.get(name) + if isinstance(value, str) and value.lower() in ( + "user", + "human", + "assistant", + "ai", + "agent", + "bot", + ): + return "user" if value.lower() in ("user", "human") else "assistant" + return "" + + +def _cursor_text(value: Any) -> str: + if isinstance(value, str): + return value + if isinstance(value, list): + return _codex_item_text(value) + if isinstance(value, dict): + for name in _CURSOR_TEXT_KEYS: + if name in value: + nested = _cursor_text(value[name]) + if nested.strip(): + return nested + return "" + + +def _cursor_stable_key(record: dict[str, Any], *, role: str, body: str) -> str: + """Idempotency key that stays stable across byte-delta uploads. + + Cursor agent-transcript lines usually have no ``id`` / ``uuid``. The old + fallback ``line{index}:0`` renumbered from 0 inside every delta the stop + hook ships, so the user turn at the head of each chunk reused ``line0:0`` + and was dropped as ``skipped_existing`` while later assistant lines + (higher indices) still imported. Prefer a real id when present; otherwise + hash ``role`` + body so re-sends dedupe but distinct turns do not collide. + """ + for name in ("id", "uuid"): + value = record.get(name) + if isinstance(value, str) and value.strip(): + return f"{value.strip()}:0" + if value is not None and not isinstance(value, (dict, list)): + text = str(value).strip() + if text: + return f"{text}:0" + digest = hashlib.sha256(f"{role}\0{body}".encode()).hexdigest()[:16] + return f"c{digest}:0" + + +def parse_cursor_text(text: str) -> ParsedSession: + """Map a Cursor transcript to L0 events (pure, no writes). + + The ``stop`` hook's ``transcript_path`` points at + ``~/.cursor/projects//agent-transcripts//.jsonl``. + Each line is ``{"role","message":{"content":[{"type":"text","text":…}]}}``; + tool blocks in that array are dropped. A JSON array or + ``{"messages":[…]}`` / ``{"turns":[…]}`` is still accepted. + + ``src_key`` is content-stable (see :func:`_cursor_stable_key`) so the hook's + byte-delta uploads stay idempotent without colliding across chunks. + """ + events: list[TranscriptEvent] = [] + skipped_own = 0 + records: list[Any] = [] + stripped = text.lstrip() + if stripped.startswith("["): + loaded = json.loads(text) + records = loaded if isinstance(loaded, list) else [] + elif stripped.startswith("{"): + try: + loaded = json.loads(text) + except json.JSONDecodeError: + loaded = None + if isinstance(loaded, dict): + for name in ("messages", "turns", "items", "conversation"): + if isinstance(loaded.get(name), list): + records = loaded[name] + break + else: + records = [loaded] + else: + records = list(_iter_records(text)) # JSONL fallback + else: + records = list(_iter_records(text)) + + for record in records: + if not isinstance(record, dict): + continue + role = _cursor_role(record) + if not role: + skipped_own += 1 # tool/system entries carry no conversational role + continue + body = "" + for name in _CURSOR_TEXT_KEYS: + if name in record: + body = _cursor_text(record[name]) + if body.strip(): + break + if not body.strip(): + continue + key = _cursor_stable_key(record, role=role, body=body) + events.append( + TranscriptEvent( + key=key, + event_type="user_message" if role == "user" else "assistant_message", + content=body, + timestamp=_parse_timestamp( + record.get("timestamp") or record.get("createdAt") or record.get("created_at") + ), + payload={ + "source": DEFAULT_HOST, + "src_key": key, + "role": role, + "sidechain": False, + }, + ) + ) + return ParsedSession(session_id=None, events=events, skipped_own_results=skipped_own) + + +_PARSERS: dict[str, Callable[[str], ParsedSession]] = { + "claude-code": parse_text, + "kiro": parse_kiro_text, + "dsh": parse_dsh_text, + "codex": parse_codex_text, + "cursor": parse_cursor_text, +} +"""``source`` → transcript parser. A terminal's name picks its parser.""" + +SESSION_PREFIXES: dict[str, str] = { + "claude-code": "cc:", + "kiro": "kiro:", + "dsh": "dsh:", + "codex": "codex:", + "cursor": "cursor:", +} +"""L0 session-id prefix per terminal, so one terminal's sessions never collide.""" + +# --------------------------------------------------------------------------- +# import (writes L0 through the target agent's memory store) +# --------------------------------------------------------------------------- + + +USER_MESSAGE = "user_message" +"""The user side of a transcript; used to tell injected context from a reply.""" + + +def attribute_user(content: str, caller: str | None) -> str: + """Prefix ``说：`` so the sender survives into the extracted atom. + + harness-memory serialises only ``event_id / event_type / content / timestamp / + session_id`` for the extractor prompt (``pipeline/extractor/__init__.py``), and + ``AtomCard`` has no user column — so the raw event's ``user`` field never reaches + the model. Writing the sender into the content is the only way "who said it" + survives extraction, stays FTS-searchable and shows up verbatim on recall. + Already-prefixed content is left alone, so a re-send cannot double it. + """ + name = (caller or "").strip() + if not name: + return content + prefix = f"{name}说：" + return content if content.startswith(prefix) else prefix + content + + +def _is_echo(content: str, event_type: str, markers: tuple[str, ...]) -> bool: + """This event is our own injected text coming back, not the user's words. + + Two shapes, because the two roles differ: + + * assistant side — only a block that *starts* with the marker is ours. A reply + that merely quotes the marker (explaining how the hooks work) is real content + and must survive; + * user side — the marker anywhere means it is the context we injected (Claude + Code also prefixes it, e.g. ``Stop hook feedback:``), so it is dropped. + """ + if any(content.lstrip().startswith(marker) for marker in markers): + return True + return event_type == USER_MESSAGE and any(marker in content for marker in markers) + + +def _persist( + memory: Any, + parsed: ParsedSession, + *, + fallback_session: str, + session_id: str | None, + user: str | None, + host: str, + dry_run: bool, + skip_markers: tuple[str, ...] = (), + skip_message_markers: tuple[str, ...] = (), + tool_events: str = "all", + session_prefix: str = SESSION_PREFIX, +) -> ImportStats: + """Deduplicate a parsed session against L0 and append only the missing events. + + Accounting: ``parsed = imported + skipped_existing + skipped_echo + skipped_tools`` + plus the ``skipped_own_results`` the parser already dropped. + """ + from octop_memory.types import RawEvent # noqa: PLC0415 + + resolved_session = session_id or f"{session_prefix}{parsed.session_id or fallback_session}" + stats = ImportStats( + session_id=resolved_session, + parsed=len(parsed.events), + dry_run=dry_run, + skipped_own_results=parsed.skipped_own_results, + ) + + candidates = parsed.events + if skip_markers or skip_message_markers: + kept = [ + event + for event in candidates + # 我们的注入块以标记**开头**；模型回答里只是引用该标记（例如解释接入方式） + # 时必须保留——否则会把正常对话当回声丢掉。 + if not _is_echo(event.content, event.event_type, skip_markers) + # Injected *messages* are plumbing; the same string inside tool output is + # just data (a file being read may quote it), so that check is narrower. + and not ( + event.event_type in _MESSAGE_EVENT_TYPES + and any(marker in event.content for marker in skip_message_markers) + ) + ] + stats.skipped_echo = len(candidates) - len(kept) + candidates = kept + + drop_types = _dropped_tool_types(tool_events) + if drop_types: + kept = [event for event in candidates if event.event_type not in drop_types] + stats.skipped_tools = len(candidates) - len(kept) + candidates = kept + + stats.sidechain = sum(1 for event in candidates if event.payload.get("sidechain")) + for event in candidates: + stats.by_type[event.event_type] = stats.by_type.get(event.event_type, 0) + 1 + + seen: set[str] = set() + try: + for existing in memory.list_raw(session_id=resolved_session, limit=_EXISTING_SCAN_LIMIT): + # ``cc_key`` is the pre-Kiro name of the same field; old rows keep it. + key = (existing.payload or {}).get("src_key") or (existing.payload or {}).get("cc_key") + if isinstance(key, str): + seen.add(key) + except Exception: # noqa: BLE001 - an audit scan must not lose the import + logger.warning("claude_transcript: existing-event scan failed", exc_info=True) + stats.warnings.append("existing-event scan failed; duplicate check degraded") + + pending = [event for event in candidates if event.key not in seen] + stats.skipped_existing = len(candidates) - len(pending) + if dry_run or not pending: + return stats + + memory.add_raw_batch( + [ + RawEvent( + id=str(uuid.uuid4()), + host=host, + session_id=resolved_session, + thread_id=resolved_session, + user=user, + timestamp=event.timestamp, + event_type=event.event_type, + # Only the human side is attributed (``event_type`` already marks the + # other), and never a sidechain record — those "user" messages are a + # subagent's synthetic prompt, not something the caller said. + content=( + attribute_user(event.content, user) + if event.event_type == USER_MESSAGE and not event.payload.get("sidechain") + else event.content + ), + payload={**event.payload, "source": host}, + ) + for event in pending + ] + ) + stats.imported = len(pending) + return stats + + +def import_content( + memory: Any, + content: str, + *, + session_id: str | None = None, + user: str | None = None, + host: str = DEFAULT_HOST, + dry_run: bool = False, + skip_markers: tuple[str, ...] = (), + skip_message_markers: tuple[str, ...] = (), + tool_events: str = "all", + parser: str = "claude-code", +) -> ImportStats: + """Replay transcript *content* (JSONL text) into L0. + + This is the upload path: a cloud deployment cannot read the client's + ``~/.claude`` directory, so the terminal sends the file body instead. + ``skip_markers`` drops events whose content carries one of those strings + (e.g. this service's own recall-injection markers). + """ + if parser not in _PARSERS: + raise ValueError( + f"unknown transcript parser {parser!r}; expected one of {sorted(_PARSERS)}" + ) + return _persist( + memory, + _PARSERS[parser](content), + fallback_session="upload", + session_id=session_id, + user=user, + host=host, + dry_run=dry_run, + skip_markers=skip_markers, + skip_message_markers=skip_message_markers, + tool_events=tool_events, + session_prefix=SESSION_PREFIXES.get(parser, SESSION_PREFIX), + ) + + +def import_session( + memory: Any, + path: Path, + *, + session_id: str | None = None, + user: str | None = None, + host: str = DEFAULT_HOST, + dry_run: bool = False, + skip_markers: tuple[str, ...] = (), + skip_message_markers: tuple[str, ...] = (), + tool_events: str = "all", + parser: str = "claude-code", +) -> ImportStats: + """Replay one transcript *file* into L0 (same-host callers). + + ``session_id`` defaults to ``cc:`` so one Claude Code + session maps onto one octop session (extraction groups by session). + """ + if parser not in _PARSERS: + raise ValueError( + f"unknown transcript parser {parser!r}; expected one of {sorted(_PARSERS)}" + ) + # Kiro names every file ``messages.jsonl``; the session id is its parent dir. + fallback = path.parent.name if path.name.startswith("messages") else path.stem + return _persist( + memory, + _PARSERS[parser](path.read_text(encoding="utf-8")), + fallback_session=fallback, + session_id=session_id, + user=user, + host=host, + dry_run=dry_run, + skip_markers=skip_markers, + skip_message_markers=skip_message_markers, + tool_events=tool_events, + session_prefix=SESSION_PREFIXES.get(parser, SESSION_PREFIX), + ) + + +__all__ = [ + "DEFAULT_HOST", + "SESSION_PREFIX", + "ImportStats", + "ParsedSession", + "TranscriptEvent", + "SESSION_PREFIXES", + "import_content", + "import_session", + "parse_dsh_text", + "parse_kiro_text", + "parse_session", + "parse_text", +] diff --git a/src/octop/infra/agents/client_setup.py b/src/octop/infra/agents/client_setup.py new file mode 100644 index 000000000..3a44e6bf9 --- /dev/null +++ b/src/octop/infra/agents/client_setup.py @@ -0,0 +1,1349 @@ +"""Bootstrap a local terminal so it talks to this memory service. + +``GET /mcp/memory/setup`` streams the interactive installer this module renders — no +authentication, because it only hands out a script: the credentials are typed by the +person running it, never fetched:: + + curl -fsSL "https:///mcp/memory/setup" | sh + # → 检测到：1) Claude Code 2) Kiro 3) DSH 4) Codex 5) Cursor + # → 要配置哪些？ token: ****（不回显） 专家 ID: XXXX 用户标识（建议域账号）: xxxx + +``GET /mcp/memory/uninstall`` is the counterpart — removes hooks / MCP entries this +installer wrote (does not delete server-side memory):: + + curl -fsSL "https:///mcp/memory/uninstall" | OCTOP_TERMINALS=cursor sh + +Nothing is assumed about where an installer runs. The script detects which terminals +exist, asks which to configure, prompts for the token and the expert id (reading +``/dev/tty`` — a piped script must not consume its own stdin), then installs a copy of +both hook scripts next to *each* terminal, in the directory that terminal uses, and +registers them in that terminal's own hook config: + +=========================== ========================================== ============================== +Claude Code ``~/.claude/hooks-events/`` ``~/.claude/settings.json`` +Kiro ``~/.kiro/hooks/`` ``~/.kiro/hooks/*.json`` +DSH ``~/.dsh/hooks/`` ``~/.dsh/hooks/octop-hooks.json`` +Codex ``~/.codex/hooks/`` ``~/.codex/config.toml`` +Cursor ``~/.cursor/hooks/`` ``~/.cursor/hooks.json`` +=========================== ========================================== ============================== + +Each copy carries its own ``config.json`` (endpoint, token, expert). The hook roles are +fixed: **UserPromptSubmit / beforeSubmitPrompt → memory_search** (pull background in) +and **Stop / stop → memory_capture** (ship the session transcript). +""" + +from __future__ import annotations + +import base64 +import json +from pathlib import Path + +_HOOKS_SRC = Path(__file__).with_name("hook_scripts") +# Every host gets the host-agnostic transport module next to its own entry scripts. +_LIB_FILES = ("lib/octop-memory-client.mjs",) + +HOST_FILES: dict[str, tuple[str, ...]] = { + "claude-code": ("claude-code/stop-capture.mjs", "claude-code/user-prompt-recall.mjs"), + "kiro": ("kiro/stop-capture.mjs",), + "dsh": ("dsh/stop-capture.mjs", "dsh/user-prompt-recall.mjs", "dsh/memory-target.mjs"), + "codex": ("codex/stop-capture.mjs", "codex/user-prompt-recall.mjs"), + "cursor": ("cursor/stop-capture.mjs", "cursor/user-prompt-recall.mjs"), +} +"""Which hook scripts a terminal gets. Each host has its own entry script — no +runtime host detection — so changing one host cannot affect another; the only +shared file is the host-agnostic transport module.""" + +_DSH_PRESET_SRC = Path(__file__).with_name("agent_presets") / "dsh-memory-share" +_DSH_PRESET_FILES = ("package.json", "cordis.patch.yml") +"""DSH ≥0.1.7 declares presets in a bundle patch, so the installer ships one: +``记忆共享模式`` = the shipped ``cordis`` plugin list + the memory hooks row.""" + +TERMINALS: tuple[str, ...] = ("claude-code", "kiro", "dsh", "codex", "cursor") +"""Terminals this installer knows how to configure.""" + +# Owned by each terminal, so no cross-terminal directory is invented. +HOOKS_DIRS: dict[str, str] = { + "claude-code": "~/.claude/hooks-events", + "kiro": "~/.kiro/hooks", + "dsh": "~/.dsh/hooks", + "codex": "~/.codex/hooks", + "cursor": "~/.cursor/hooks", +} + +_SELECT_PY = r""" +import sys + +# 交互式终端清单：↑/↓ 移动 · 空格 选择 · a 全选 · 回车 确认。 +# 键位与绘制都走 /dev/tty，stdout 只用来回传结果（shell 捕获）。 +# 任何一步不可用（无 tty / 无 termios / 空选择）就返回非 0，让 shell 回退到数字输入。 + +def main() -> int: + items = [item.split("|", 1) for item in sys.argv[1:] if "|" in item] + if not items: + return 3 + try: + import termios + import tty as ttymod + + tty_in = open("/dev/tty", "rb", buffering=0) + tty_out = open("/dev/tty", "w") + fd = tty_in.fileno() + saved = termios.tcgetattr(fd) + except Exception: + return 3 + + chosen = [False] * len(items) + cursor = 0 + + def render(first: bool = False) -> None: + if not first: + tty_out.write("\033[%dA" % (len(items) + 1)) + for index, (_, label) in enumerate(items): + box = "◉" if chosen[index] else "◯" + pointer = "❯" if index == cursor else " " + tty_out.write("\033[2K%s %s %s\r\n" % (pointer, box, label)) + hint = " ↑/↓ 移动 · 空格 选择 · a 全选 · 回车 确认" + if not any(chosen): + hint = " ↑/↓ 移动 · 空格 选择 · a 全选 · 回车 确认（还没选）" + tty_out.write("\033[2K%s\r\n" % hint) + tty_out.flush() + + def read_key() -> str: + key = tty_in.read(1) + if key == b"\x1b": + rest = tty_in.read(2) + return {b"[A": "up", b"[B": "down"}.get(rest, "other") + if key in (b"\r", b"\n"): + return "enter" + if key == b" ": + return "space" + if key in (b"a", b"A"): + return "all" + if key == b"\x03": + return "abort" + if key in (b"k", b"j"): + return "up" if key == b"k" else "down" + return "other" + + try: + ttymod.setraw(fd) + tty_out.write("要配置哪些终端？（可多选）\r\n") + render(first=True) + while True: + key = read_key() + if key == "up": + cursor = (cursor - 1) % len(items) + elif key == "down": + cursor = (cursor + 1) % len(items) + elif key == "space": + chosen[cursor] = not chosen[cursor] + elif key == "all": + turn_on = not all(chosen) + chosen = [turn_on] * len(items) + elif key == "enter": + if not any(chosen): + continue # 一个都没选时不确认，提示留在屏幕上 + break + elif key == "abort": + return 130 + render() + except Exception: + return 3 + finally: + try: + termios.tcsetattr(fd, termios.TCSADRAIN, saved) + except Exception: + pass + tty_out.write("\r\n") + + print(",".join(value for (value, _), on in zip(items, chosen) if on)) + return 0 + + +sys.exit(main()) +""".strip() + +_MERGE_PY = ''' +import base64 +import json +import os +import pathlib +import shlex +import shutil +import subprocess + + +SEARCH_PROMPT = ( + "回答前，先用 memory_search 检索与本轮相关的记忆（query 用用户本次输入，corpus=atom，" + "max_results=5），把命中内容当作背景；与本次工作无关就忽略，不要重复检索同一个查询。" +) +"""Kiro 的召回提示词——Kiro 不会把 runCommand 的 stdout 注进上下文，只能让模型自己调工具。 + +上报**没有**对应的提示词：让模型"总结这段会话"只会把复述写进 L0，原始会话要由 hook 脚本 +（整段 ``messages.jsonl``）提交，见 ``do_kiro``。 +""" + + +def _load(path): + try: + return json.loads(path.read_text(encoding="utf-8")) + except FileNotFoundError: + return {} + except Exception as exc: # noqa: BLE001 - a broken config must not break setup + print(f" ! {path} 不是合法 JSON（{exc}）；跳过该文件") + return None + + +def _dump(path, data): + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(data, ensure_ascii=False, indent=2) + "\\n", encoding="utf-8") + + +def _install_hooks(target, host): + """把共享传输模块 + 该宿主的入口脚本写进这个终端自己的 hooks 目录。""" + target.mkdir(parents=True, exist_ok=True) + for rel, blob in HOOK_FILES[host].items(): + path = target / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(base64.b64decode(blob)) + config = {"url": HOOK_URL, "token": TOKEN, "tokenHeader": TOKEN_HEADER, "agentId": AGENT} + if USER_ID: + config["userId"] = USER_ID # 直连端点时「谁问的」只靠这个头 + _dump(target / "config.json", config) + print(f" ✓ hook 脚本（{host} 专用，含共享传输模块）+ config.json → {target}") + print( + f" · 日志/增量状态：{target}/octop-hook.log、{target}/octop-transcript-state.json" + "（该目录不可写时退到 $TMPDIR/octop-hooks 并打印告警）" + ) + + +def _entry(transport=None): + # 条目与 hooks 共用同一份凭据：**实例共享 token**；header 名由 ``OCTOP_TOKEN_HEADER`` + # 决定（缺省 ``Authorization`` → ``Bearer ``）。「谁问的」靠 ``X-Octop-User-Id`` 头。 + headers = {"X-Octop-Agent-Id": AGENT} + headers[TOKEN_HEADER] = f"Bearer {TOKEN}" if TOKEN_HEADER == "Authorization" else TOKEN + if USER_ID: + headers["X-Octop-User-Id"] = USER_ID # 「谁问的」靠这个头，不靠 token 解析 + entry = {"type": transport or "http", "url": ENTRY_URL, "headers": headers} + return entry + + +def _octop_servers(servers): + """配置里已有的 octop memory 服务名——``local`` / ``prod`` / ``test`` 这类后缀都算。""" + return [name for name in servers if str(name).replace("_", "-").startswith("octop-memory")] + + +def _merge_mcp(path, transport=None): + """登记 MCP 服务：端点一致就复用已有条目，已有别的 octop 条目就不重复添加。 + + 服务名由客户端的配置说了算（同一个 ``octop-memory`` 可以带不同环境后缀、也可以被改名）， + 这里只按端点判断；召回/上报 hook 调的地址来自 ``config.json``，与名字无关。 + """ + entry = _entry(transport) + cfg = _load(path) + if cfg is None: + return + servers = cfg.setdefault("mcpServers", {}) + ours = _octop_servers(servers) + reused = next( + (name for name in ours if (servers.get(name) or {}).get("url") == entry["url"]), "" + ) + if reused: + print(f" ✓ MCP 服务 {reused}（已存在且端点一致，复用）→ {path}") + return + if ours: + print(f" · {path} 已有 octop 服务：{', '.join(ours)}（端点与本次 {URL} 不同，保持原样）") + print(" · 模型侧用哪个服务由你决定；召回/上报 hook 用 config.json 里的地址") + return + servers[NAME] = entry + _dump(path, cfg) + print(f" ✓ MCP 服务 {NAME} → {path}") + + +def _claude_hooks(hooks_dir): + """Claude Code 的两条 command hook。 + + - ``UserPromptSubmit`` → 召回脚本：**该事件不支持 ``mcp_tool``**（实测 2.1.263 静默跳过）， + 所以用 command hook 跑脚本，脚本直接调 MCP ``memory_search``，stdout 被注入上下文。 + - ``Stop`` → 上报脚本：``Stop`` 触发时本轮还没落盘，脚本会**后台脱离**、等几秒再读 + transcript（见脚本头注释），hook 本身立即返回，不给每轮加延迟。 + ``SessionEnd`` 不能用：那里没有 MCP 客户端上下文（``mcp_tool hooks are not available + for the 'SessionEnd' hook event``）。 + """ + search = {"type": "command", "command": hook_command(f"{hooks_dir}/claude-code/user-prompt-recall.mjs")} + stop = {"type": "command", "command": hook_command(f"{hooks_dir}/claude-code/stop-capture.mjs")} + # **简写形态**：事件直接挂 handler 数组，不套 matcher 组那层。我们的 hook 对所有调用都生效， + # 本来就不需要 matcher；而某些终端的 SDK 侧不保证 ``hooks`` 数组按顺序执行， + # 少一层嵌套更稳（老配置里的嵌套形态仍会被识别并替换，见 _handlers_of）。 + return {"UserPromptSubmit": [search], "Stop": [stop]} + + +_OUR_SCRIPTS = ( + "user-prompt-recall.mjs", + "stop-capture.mjs", + # 包装脚本（配置里引用的是 .sh） + "user-prompt-recall.sh", + "stop-capture.sh", + # 早期版本的文件名，重装时一并替换掉 + "octop-user-prompt-recall.mjs", + "octop-stop-capture.mjs", +) + +_OUR_MCP_TOOLS = ("memory_search", "memory_capture") +"""旧版 mcp_tool hook 用的工具名——按工具认，不按 MCP 服务名认。""" + + +def _is_our_handler(handler): + """认我们自己留下的 hook：command 形态按脚本名，旧版 mcp_tool 按工具名。 + + 同一个 ``octop-memory`` 条目可以带不同环境后缀、也可能被用户改名，所以不能拿服务名 + 当判据；工具名才是这段接入的契约。 + """ + handler = handler or {} + if handler.get("type") == "mcp_tool": + return handler.get("tool") in _OUR_MCP_TOOLS + command = handler.get("command") or "" + return any(script in command for script in _OUR_SCRIPTS) + + +def _handlers_of(entry): + """一个条目里的 handler 列表：可能是 matcher 组（``{"hooks": [...]}``）或简写形态的单条。 + + 两种都认 —— 简写是我们自己写的（某些终端的 SDK 侧不保证 ``hooks`` 数组按顺序执行，见 + ``_claude_hooks``），嵌套形态是别人或早期版本写的。 + """ + entry = entry or {} + nested = entry.get("hooks") + if isinstance(nested, list): + return nested + return [entry] + + +def _is_our_group(entry): + """这个 hook 条目是不是我们写的（任一 handler 命中就算）。""" + return any(_is_our_handler(handler) for handler in _handlers_of(entry)) + + +def _upsert_groups(settings, event, groups): + """Drop earlier entries that are ours (either形态), keep every other hook, add ours.""" + hooks = settings.setdefault("hooks", {}) + kept = [entry for entry in (hooks.get(event) or []) if not _is_our_group(entry)] + hooks[event] = kept + list(groups) + + +def do_claude_code(): + # 两条 hook 都是 command 脚本（召回 / 上报），任何模式都要装脚本 + config.json + _install_hooks(CC_DIR, "claude-code") + _merge_mcp(home / ".claude.json") + settings = home / ".claude" / "settings.json" + st = _load(settings) + if st is not None: + for event, groups in _claude_hooks(CC_DIR).items(): + _upsert_groups(st, event, groups) + # 早期版本挂过 SessionEnd（mcp_tool 在那里不可用，会持续报错）：清掉我们留下的那条 + _upsert_groups(st, "SessionEnd", []) + if not st["hooks"].get("SessionEnd"): + st["hooks"].pop("SessionEnd", None) + _dump(settings, st) + print(" ✓ Claude Code hooks（UserPromptSubmit → 召回脚本；Stop → 上报脚本，后台延迟等落盘）") + print(f" · → {settings}") + + +def do_kiro(): + """Kiro 的 hook 只有 runCommand / askAgent 两种动作，没有 mcp_tool。 + + 因此： + + * 召回（UserPromptSubmit）用**提示词**（``agent``）：Kiro 不会把 runCommand 的 stdout + 注入上下文，只能让模型经 Kiro 的 MCP 客户端调 ``memory_search``； + * 上报（Stop）**始终用 command hook** 跑脚本，把整段 + ``~/.kiro/sessions///messages.jsonl`` 提交进 L0 —— Kiro 侧 + 最有价值的就是**原始的用户/助手消息**，绝不能让模型自己总结（早期版本在共享 token + 下退化为 ``agent`` 提示词，结果是"AI 复述"进 L0）。共享 token 下"谁问的"由脚本带的 + ``X-Octop-User-Id``（安装时的 ``OCTOP_USER_ID``）表达，不存在第二份额外凭据。 + """ + _merge_mcp(home / ".kiro" / "settings" / "mcp.json", transport="streamable_http") + _install_hooks(KIRO_DIR, "kiro") + + recall = KIRO_DIR / "memory-recall-on-start.json" + _dump( + recall, + { + "version": "v1", + "hooks": [ + { + "name": "octop memory (UserPromptSubmit)", + "trigger": "UserPromptSubmit", + "enabled": True, + "timeout": 30, + "action": {"type": "agent", "prompt": SEARCH_PROMPT}, + } + ], + }, + ) + print(f" ✓ Kiro 召回 hook（agent 提示词）已写入 {recall}") + + save = KIRO_DIR / "memory-save-on-stop.json" + action = {"type": "command", "command": hook_command(f"{KIRO_DIR}/kiro/stop-capture.mjs")} + _dump( + save, + { + "version": "v1", + "hooks": [ + { + "name": "octop memory (Stop)", + "trigger": "Stop", + "enabled": True, + "timeout": 120, + "action": action, + } + ], + }, + ) + print(f" ✓ Kiro 上报 hook（command 脚本提交整段 messages.jsonl）已写入 {save}") + + +_CODEX_HOOKS = (("UserPromptSubmit", "user-prompt-recall"), ("Stop", "stop-capture")) +_CURSOR_HOOKS = (("beforeSubmitPrompt", "user-prompt-recall"), ("stop", "stop-capture")) + + +def _toml_string(value): + """TOML 字面串：hook 命令里有双引号、没有单引号，用单引号串就不必转义。""" + return "'" + value + "'" + + +def _toml_blocks(text): + """按表头行列把 TOML 切成块，保留原始文本（首块是文件头/杂项）。""" + blocks = [[]] + for line in text.splitlines(keepends=True): + if line.lstrip().startswith("[") and not line.lstrip().startswith("#"): + blocks.append([]) + blocks[-1].append(line) + return blocks + + +def _codex_upsert(path): + """把两条 hook 挂进 ``~/.codex/config.toml``（幂等）。 + + 结构是 ``[[hooks.]]`` 容器 + ``[[hooks..hooks]]`` 处理项，先删掉我们自己 + 写过的处理项（连它那个空容器），再把新处理项插到该事件最后一组之后 —— 不动已有 hook 的 + 先后位置，因为 codex 的 hook 信任状态按 ``:::`` 记索引。 + """ + text = path.read_text(encoding="utf-8") if path.exists() else "" + blocks, kept = _toml_blocks(text), [] + for block in blocks: + body = "".join(block) + header = block[0].strip() if block else "" + ours = str(CODEX_DIR) in body + if ours and header.startswith("[[hooks.") and header.endswith(".hooks]]"): + event = header[len("[[hooks.") : -len(".hooks]]")] + # 我们自己写的空容器也一起清掉（重装不堆空表） + if kept and kept[-1][0].strip() == f"[[hooks.{event}]]" and not _codex_container_body( + kept[-1] + ): + kept.pop() + continue + kept.append(block) + + additions = {} + for event, script in _CODEX_HOOKS: + handler = ( + f"[[hooks.{event}.hooks]]\\n" + f"command = {_toml_string(hook_command(f'{CODEX_DIR}/codex/{script}.mjs'))}\\n" + f'type = "command"\\n' + ) + insert_at = None + for index, block in enumerate(kept): + header = block[0].strip() if block else "" + if header in (f"[[hooks.{event}]]", f"[[hooks.{event}.hooks]]"): + insert_at = index + 1 + if insert_at is None: + additions[event] = f"[[hooks.{event}]]\\n\\n{handler}" + else: + kept.insert(insert_at, [handler, "\\n"]) + + body = "".join("".join(block) for block in kept).rstrip("\\n") + tail = "\\n\\n".join(additions[event] for event, _ in _CODEX_HOOKS if event in additions) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text((body + "\\n\\n" if body else "") + tail + "\\n", encoding="utf-8") + + +def _codex_container_body(block): + """容器块是否只有表头（没有处理项）。""" + return not "".join(block[1:]).strip() + + +def _cursor_upsert(path): + """把两条 hook 挂进 ``~/.cursor/hooks.json``（幂等，其它事件与别人的 hook 一律保留）。""" + cfg = _load(path) + if cfg is None: + return + cfg.setdefault("version", 1) + hooks = cfg.setdefault("hooks", {}) + for event, script in _CURSOR_HOOKS: + entries = [ + entry + for entry in hooks.get(event) or [] + if str(CURSOR_DIR) not in json.dumps(entry, ensure_ascii=False) + ] + entries.append({"command": hook_command(f"{CURSOR_DIR}/cursor/{script}.mjs")}) + hooks[event] = entries + _dump(path, cfg) + + +def do_codex(): + """Codex：hook 写在 ``~/.codex/config.toml``（``[[hooks.]]`` + ``type="command"``）。 + + codex 对 hook 有信任机制：新写入的 hook 需要用户信任一次才会执行，所以这里只写配置、 + 不伪造 ``hooks.state`` 里的 ``trusted_hash``。 + """ + _install_hooks(CODEX_DIR, "codex") + config = home / ".codex" / "config.toml" + _codex_upsert(config) + print(f" ✓ Codex hooks（UserPromptSubmit → 召回；Stop → 上报）→ {config}") + print(" · 首次需在 codex 里信任该 hook（否则不执行）；自动化可加 --dangerously-bypass-hook-trust") + print(" · Codex 的 MCP 服务由它自己的配置管理，这里不动") + + +def do_cursor(): + """Cursor：hook 写在 ``~/.cursor/hooks.json``（``{"version":1,"hooks":{:[{command}]}}``）。""" + _install_hooks(CURSOR_DIR, "cursor") + hooks = home / ".cursor" / "hooks.json" + _cursor_upsert(hooks) + print(f" ✓ Cursor hooks（beforeSubmitPrompt → 召回；stop → 上报）→ {hooks}") + print(" · Cursor 的 MCP 服务由它自己的配置管理，这里不动") + + +def _dsh_bridge(hooks_dir): + """DSH 的 hooks 桥接只执行 command 钩子（mcp_tool / prompt 会被跳过），故写死 command。""" + return { + "hooks": { + "UserPromptSubmit": [ + { + "hooks": [ + {"type": "command", "command": hook_command(f"{hooks_dir}/dsh/user-prompt-recall.mjs")} + ] + } + ], + "Stop": [ + {"hooks": [{"type": "command", "command": hook_command(f"{hooks_dir}/dsh/stop-capture.mjs")}]} + ], + } + } + + +def _dsh_profile(): + """装哪个 profile：OCTOP_DSH_PROFILE，否则按 web / desktop / default 取第一个存在的。""" + profiles = home / ".dsh" / "profiles" + wanted = os.environ.get("OCTOP_DSH_PROFILE", "") + for name in ([wanted] if wanted else ["web", "desktop", "default"]): + if name and (profiles / name / "package.json").exists(): + return name, profiles / name + return "", pathlib.Path() + + +def _install_dsh_preset(bridge): + """把「记忆共享模式」bundle 写到 ~/.dsh/bundles，configPath 换成刚写好的桥接文件。""" + bundle = DSH_DIR.parent / "bundles" / "dsh-memory-share" + for name, blob in PRESET_FILES.items(): + target = bundle / name + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text( + base64.b64decode(blob).decode("utf-8").replace("__OCTOP_HOOKS_JSON__", str(bridge)), + encoding="utf-8", + ) + return bundle + + +def _select_dsh_bundle(profile_dir, name): + """把 bundle 加进 profile 的 dsh.profile.bundles（幂等）——DSH 只按那里的名字加载 bundle。""" + package = profile_dir / "package.json" + config = _load(package) + if config is None: + return False + bundles = config.setdefault("dsh", {}).setdefault("profile", {}).setdefault("bundles", []) + if name in bundles: + return True + bundles.append(name) + package.write_text(json.dumps(config, ensure_ascii=False, indent=2) + "\\n", encoding="utf-8") + return True + + +def _run_dsh_install(bundle, profile, profile_dir): + """把 bundle 装进 profile：优先 dsh CLI，其次直接 pnpm（源码 checkout 常没有 dsh 命令）。 + + 两条路都在 profile 目录里跑包管理器——DSH 的 CLI 也只是把参数转给 profile 里的 pnpm。 + """ + if shutil.which("dsh"): + command = ["dsh", "plugin", "--profile", profile, "add", str(bundle)] + elif shutil.which("pnpm"): + command = ["pnpm", "add", str(bundle)] + else: + print(" · 没找到 dsh / pnpm 命令，无法自动装依赖") + return None + done = subprocess.run(command, cwd=str(profile_dir), capture_output=True, text=True) + if done.returncode != 0: + tail = (done.stderr or done.stdout or "").strip().splitlines() + print(f" ! 装依赖失败（{command[0]} 退出码 {done.returncode}）" + (f"：{tail[-1]}" if tail else "")) + return False + return True + + +def do_dsh(): + """装 hook 脚本 + 桥接文件（只有 command 钩子），并把 preset bundle 装进 profile。 + + DSH 0.1.7 起 preset 由 bundle patch 声明，旧版「在 agent preset 文件里加一行」已经 + 不再生效，所以这里直接生成 bundle 并装进 profile 的 dsh.profile.bundles。 + + 脚本直传不再挑 token 方言：共享 token 下"谁问的"由脚本带的 ``X-Octop-User-Id`` + （安装时的 ``OCTOP_USER_ID``）表达，所以任何模式都装 hooks。 + """ + _install_hooks(DSH_DIR, "dsh") + bridge = DSH_DIR / "octop-hooks.json" + _dump(bridge, _dsh_bridge(DSH_DIR)) + print(f" ✓ DSH hooks 桥接（UserPromptSubmit → 召回；Stop → 上报）→ {bridge}") + bundle = _install_dsh_preset(bridge) + name = json.loads((bundle / "package.json").read_text(encoding="utf-8"))["name"] + print(f" ✓ DSH 模式 bundle「记忆共享模式」→ {bundle}") + profile, profile_dir = _dsh_profile() + if not profile: + print(" ! 没找到 DSH profile（~/.dsh/profiles/），跳过自动安装") + elif _run_dsh_install(bundle, profile, profile_dir) and _select_dsh_bundle(profile_dir, name): + print(f" ✓ 已装进 profile「{profile}」：重启 DSH 后，新会话的模式里选「记忆共享模式」") + else: + print(" ! 没自动装进 profile，手动两步：") + print(f" 1) 在 {profile_dir} 执行：pnpm add {bundle}") + print(f" 2) 把 {name} 加进该目录 package.json 的 dsh.profile.bundles，然后重启 DSH") + print(" · DSH 的 MCP 服务请在它的 MCP 面板/CLI 里添加（该条目由 DSH 自己管理）：") + print(f" url: {URL}") + print(f" headers: {json.dumps(_entry()['headers'], ensure_ascii=False)}") + + +home = pathlib.Path(os.path.expanduser("~")) +URL = os.environ["OCTOP_URL"] +HOOK_URL = os.environ.get("OCTOP_HOOK_URL") or URL # hooks 实际 POST 的地址（直连） +TOKEN = os.environ["OCTOP_TOKEN"] +TOKEN_HEADER = os.environ["OCTOP_TOKEN_HEADER"] +NODE = os.environ.get("OCTOP_NODE") or "node" # 安装时解析出的绝对路径（非 login shell 也能跑） +# 用户标识：token 的唯一作用是让服务端知道「谁问的」，只能靠这个头表达。 +# 建议填域账号：HTTP header 只能是 ASCII，Node 的 fetch 对非 ASCII 直接抛错（hook 会崩）。 +USER_ID = os.environ.get("OCTOP_USER_ID", "").strip() +if USER_ID and not USER_ID.isascii(): + print(f" ! 用户标识「{USER_ID}」含非 ASCII 字符：header 传不了（hook 会直接报错），已忽略") + print(" · 请填域账号（如 alice）；要中文显示名需服务端支持 percent 解码") + USER_ID = "" + +# 条目地址默认 = 实例挂载点；部署可用 ?url= / OCTOP_MEMORY_PUBLIC_MCP_URL 把它指向公网 +# ingress（hooks 仍打 HOOK_URL）。凭据不分模式：条目与 hooks 都用实例共享 token。 +ENTRY_URL = URL +NAME = os.environ["OCTOP_SERVER_NAME"] + + +def hook_command(script_path): + """一条 command hook 的 ``command``：**独立包装脚本**的路径（写它、返回它）。 + + 不把 node 绝对路径 / ``NODE_OPTIONS`` / 失败兜底那一长串直接塞进 JSON-TOML 字符串 —— + 里面的引号与 ``$`` 很容易把配置写坏，也难读（用户反馈过这一点）。包装脚本负责： + + * 用安装时解析出的 node **绝对路径**跑 ``.mjs``（hook 由终端派生，非 login shell 里 + 没有 nvm 的 node，裸 ``node`` 会找不到）； + * ``OCTOP_USE_SYSTEM_CA=1`` 时导出 ``NODE_OPTIONS=--use-system-ca``（Node 22.15+）： + Node 自带的证书库不认企业根（macOS 钥匙串 / Windows 证书库里有、Node 没有）， + 不加这行 hook 里的 fetch 会失败；已有 ``NODE_OPTIONS`` 时追加，不覆盖； + * 失败时把原因打到 stderr —— command hook 的非 0 退出是**静默**的。 + """ + mjs = pathlib.Path(str(script_path)) + wrapper = mjs.with_suffix(".sh") + lines = [ + "#!/bin/sh", + "# octop memory hook（安装器生成，勿手改）：配置里只引用本文件路径", + ] + if os.environ.get("OCTOP_USE_SYSTEM_CA") == "1": + lines.append('export NODE_OPTIONS="${NODE_OPTIONS:+$NODE_OPTIONS }--use-system-ca"') + lines.append( + f"{shlex.quote(str(NODE))} {shlex.quote(str(mjs))} " + '|| echo "octop-memory: hook 执行失败（node 路径不对或脚本报错）" >&2' + ) + wrapper.parent.mkdir(parents=True, exist_ok=True) + wrapper.write_text("\\n".join(lines) + "\\n", encoding="utf-8") + wrapper.chmod(0o755) + return shlex.quote(str(wrapper)) +AGENT = os.environ["OCTOP_AGENT"] +CC_DIR = pathlib.Path(os.environ["OCTOP_CC_DIR"]).expanduser() +KIRO_DIR = pathlib.Path(os.environ["OCTOP_KIRO_DIR"]).expanduser() +DSH_DIR = pathlib.Path(os.environ["OCTOP_DSH_DIR"]).expanduser() +CODEX_DIR = pathlib.Path(os.environ["OCTOP_CODEX_DIR"]).expanduser() +CURSOR_DIR = pathlib.Path(os.environ["OCTOP_CURSOR_DIR"]).expanduser() +HOOK_FILES = { + host: json.loads(os.environ["OCTOP_FILES_" + host.upper().replace("-", "_")]) + for host in ("claude-code", "kiro", "dsh", "codex", "cursor") +} +PRESET_FILES = json.loads(os.environ["OCTOP_FILES_PRESET"]) + +for terminal in os.environ["OCTOP_TERMINALS"].split(","): + terminal = terminal.strip() + if not terminal: + continue + print(f"· {terminal}") + { + "claude-code": do_claude_code, + "kiro": do_kiro, + "dsh": do_dsh, + "codex": do_codex, + "cursor": do_cursor, + }[terminal]() + +print(" · hook 角色：用户提交时召回（CC / DSH / Codex 的 UserPromptSubmit，Cursor 的 beforeSubmitPrompt）；") +print(" 回合结束时上报（Stop / Cursor 的 stop）") +print(f" · hooks 打 {HOOK_URL}；客户端 MCP 条目指向 {ENTRY_URL}（实例共享 token：写 {TOKEN_HEADER}）") +if USER_ID: + print(f" · 用户标识：{USER_ID}（写进条目 X-Octop-User-Id 与 hooks）") +print(" · octop memory 接入完成（可重复执行，不会影响其它 MCP/hook 条目）") +'''.strip() + + +def _b64(text: str) -> str: + return base64.b64encode(text.encode("utf-8")).decode("ascii") + + +def _host_manifest(host: str) -> str: + """``{relative path: base64}`` for one host: its entries + the shared lib modules.""" + files = (*_LIB_FILES, *HOST_FILES[host]) + payload = {name: _b64((_HOOKS_SRC / name).read_text(encoding="utf-8")) for name in files} + return json.dumps(payload, ensure_ascii=False, separators=(",", ":")) + + +def _dsh_preset_manifest() -> str: + """``{relative path: base64}`` for the DSH preset bundle the installer writes.""" + payload = { + name: _b64((_DSH_PRESET_SRC / name).read_text(encoding="utf-8")) + for name in _DSH_PRESET_FILES + } + return json.dumps(payload, ensure_ascii=False, separators=(",", ":")) + + +def _server_name(mcp_url: str) -> str: + """``octop-memory-local`` for a loopback endpoint, ``octop-memory`` otherwise.""" + host = mcp_url.split("//", 1)[-1].split("/", 1)[0].split(":", 1)[0].lower() + return "octop-memory-local" if host in ("127.0.0.1", "localhost", "::1") else "octop-memory" + + +def _sh_single_quote(value: str) -> str: + """Always return a single-quoted shell literal (safe for script embedding). + + Unlike ``shlex.quote`` (which may leave safe tokens unquoted), this keeps the + ``VAR='…'`` form used throughout the installer and escapes embedded quotes. + """ + return "'" + str(value).replace("'", "'\"'\"'") + "'" + + +def render_setup_script( + *, + mcp_url: str, + hook_url: str = "", + agent_id: str = "", + terminals: str = "", + default_token: str = "", +) -> str: + """Render the interactive installer. + + ``mcp_url`` 是登记进客户端 MCP 条目的地址（唯一对外地址）；``hook_url`` 是 hooks 真正 + POST 的地址，默认同 ``mcp_url``。两者不同时，条目走部署声明的公网 ingress、hooks 打实例 + 挂载点，但凭据仍是同一个**实例共享 token**。 + + ``default_token`` 是部署内置的默认凭据（实例发布时设置了 ``OCTOP_MEMORY_MCP_TOKEN``）： + 脚本把它当 ``OCTOP_MEMORY_TOKEN`` 的兜底，于是用户只需要专家 ID（可选用户标识）。 + 为空时在安装时问一次；header 名由 ``OCTOP_TOKEN_HEADER`` 决定，缺省 ``Authorization`` + （此时值写成 ``Bearer ``）。 + ``agent_id`` 和 ``terminals``（逗号清单）只是给自动化预填提示用。 + """ + hook_url = hook_url or mcp_url + token_literal = _sh_single_quote(default_token) if default_token else "''" + # Query-string prefills must be shell-safe (same escaping as DEFAULT_TOKEN). + url_literal = _sh_single_quote(mcp_url) + hook_url_literal = _sh_single_quote(hook_url) + agent_literal = _sh_single_quote(agent_id) if agent_id else "''" + terminals_literal = _sh_single_quote(terminals) if terminals else "''" + default_token_line = f"DEFAULT_TOKEN={token_literal}" if default_token else "" + default_token_block = ( + f"""if [ -z "$TOKEN" ] && [ -n {token_literal} ]; then + TOKEN={token_literal} + echo " · token：用本实例部署时配置的默认凭据（要覆盖设 OCTOP_MEMORY_TOKEN）" +fi""" + if default_token + else "" + ) + return f"""#!/bin/sh +# octop memory — 本地终端接入（hooks 发往 {hook_url}；可重复执行） +set -eu + +URL={url_literal} +HOOK_URL={hook_url_literal} +DEFAULT_AGENT={agent_literal} +{default_token_line} +DEFAULT_TERMINALS={terminals_literal} +CC_DIR='{HOOKS_DIRS["claude-code"]}' +KIRO_DIR='{HOOKS_DIRS["kiro"]}' +DSH_DIR='{HOOKS_DIRS["dsh"]}' +CODEX_DIR='{HOOKS_DIRS["codex"]}' +CURSOR_DIR='{HOOKS_DIRS["cursor"]}' + +# ---- 1) 检测本机装了哪些终端（目录或命令任一命中） +have_claude=0; have_kiro=0; have_dsh=0; have_codex=0; have_cursor=0 +{{ [ -e "$HOME/.claude.json" ] || [ -d "$HOME/.claude" ] || command -v claude >/dev/null 2>&1; }} && have_claude=1 +{{ [ -d "$HOME/.kiro" ] || command -v kiro >/dev/null 2>&1; }} && have_kiro=1 +{{ [ -d "$HOME/.dsh" ] || command -v dsh >/dev/null 2>&1; }} && have_dsh=1 +{{ [ -d "$HOME/.codex" ] || command -v codex >/dev/null 2>&1; }} && have_codex=1 +{{ [ -d "$HOME/.cursor" ] || command -v cursor >/dev/null 2>&1; }} && have_cursor=1 + +DETECTED='' +[ "$have_claude" = 1 ] && DETECTED="$DETECTED claude-code" +[ "$have_kiro" = 1 ] && DETECTED="$DETECTED kiro" +[ "$have_dsh" = 1 ] && DETECTED="$DETECTED dsh" +[ "$have_codex" = 1 ] && DETECTED="$DETECTED codex" +[ "$have_cursor" = 1 ] && DETECTED="$DETECTED cursor" +[ -n "$DETECTED" ] || DETECTED='claude-code' + +echo "octop memory 接入 → {mcp_url}" +echo "检测到：" +[ "$have_claude" = 1 ] && echo " 1) Claude Code　（hook 装到 ~/.claude/hooks-events）" +[ "$have_kiro" = 1 ] && echo " 2) Kiro　　　　（hook 装到 ~/.kiro/hooks）" +[ "$have_dsh" = 1 ] && echo " 3) DSH　　　　（hook 装到 ~/.dsh/hooks）" +[ "$have_codex" = 1 ] && echo " 4) Codex　　　（hook 装到 ~/.codex/hooks，配置写 ~/.codex/config.toml）" +[ "$have_cursor" = 1 ] && echo " 5) Cursor　　　（hook 装到 ~/.cursor/hooks，配置写 ~/.cursor/hooks.json）" + +# 有 controlling tty 才交互：curl | sh 时 stdin 是脚本本身，读它会把脚本吃掉； +# 没有 tty（CI/cron）时依赖环境变量，缺什么就明确报错而不是盲读 stdin。 +HAS_TTY=0 +if (true /dev/null; then HAS_TTY=1; fi + +ask() {{ # ask <提示> <默认值> + if [ "$HAS_TTY" = 1 ]; then + printf '%s' "$1" > /dev/tty 2>/dev/null || true + read -r _answer < /dev/tty || _answer='' + else + _answer='' + fi + if [ -n "$_answer" ]; then printf '%s' "$_answer"; else printf '%s' "$2"; fi +}} + +# ---- 2) 选终端：交互式清单（↑↓ 移动 · 空格 选择 · a 全选 · 回车 确认）； +# 没有 tty 或没有 python3 时回退到数字输入。 +TERMINALS="${{OCTOP_TERMINALS:-$DEFAULT_TERMINALS}}" +if [ -z "$TERMINALS" ] && [ "$HAS_TTY" = 1 ]; then + _picked='' + if command -v python3 >/dev/null 2>&1; then + set -- + [ "$have_claude" = 1 ] && set -- "$@" "claude-code|Claude Code　（hook 装到 ~/.claude/hooks-events）" + [ "$have_kiro" = 1 ] && set -- "$@" "kiro|Kiro　（hook 装到 ~/.kiro/hooks）" + [ "$have_dsh" = 1 ] && set -- "$@" "dsh|DSH　（hook 装到 ~/.dsh/hooks）" + [ "$have_codex" = 1 ] && set -- "$@" "codex|Codex　（hook 装到 ~/.codex/hooks）" + [ "$have_cursor" = 1 ] && set -- "$@" "cursor|Cursor　（hook 装到 ~/.cursor/hooks）" + _picked="$(python3 - "$@" 2>/dev/null <<'OCTOP_SELECT' +{_SELECT_PY} +OCTOP_SELECT +)" || _picked='' + fi + if [ -n "$_picked" ]; then + TERMINALS="$_picked" + else + # 回退：数字输入（1,2 / 1 3 / 12 / a=全部；回车 = 全部检测到的；选不中会重问） + _attempt=0 + while [ -z "$TERMINALS" ] && [ "$_attempt" -lt 3 ]; do + _attempt=$((_attempt + 1)) + printf '要配置哪些？(可多选：1,2 / 1 3 / 12 / a=全部，回车=全部检测到的): ' > /dev/tty + read -r _pick < /dev/tty || _pick='' + _chosen='' + case "$_pick" in + ''|a|A|all|ALL|全部) _chosen="$DETECTED" ;; + *) + for _n in $(printf '%s' "$_pick" | tr ',' ' '); do + case "$_n" in + [0-9][0-9]*) _items="$(printf '%s' "$_n" | fold -w1 | tr '\n' ' ')" ;; + *) _items="$_n" ;; + esac + for _i in $_items; do + case "$_i" in + 1) _add="$([ "$have_claude" = 1 ] && echo claude-code)" ;; + 2) _add="$([ "$have_kiro" = 1 ] && echo kiro)" ;; + 3) _add="$([ "$have_dsh" = 1 ] && echo dsh)" ;; + 4) _add="$([ "$have_codex" = 1 ] && echo codex)" ;; + 5) _add="$([ "$have_cursor" = 1 ] && echo cursor)" ;; + *) _add='' ;; + esac + if [ -z "$_add" ]; then + echo " ? 已忽略：${{_i}}（未检测到或无法识别）" >&2 + else + case ",$_chosen," in *",$_add,"*) ;; *) _chosen="$_chosen,$_add" ;; esac + fi + done + done + ;; + esac + TERMINALS="$(printf '%s' "$_chosen" | sed 's/^,//')" + [ -n "$TERMINALS" ] || echo " ! 没有选中任何已检测到的终端，请重选（1-5，或 a=全部）" >&2 + done + fi +fi + +[ -n "$TERMINALS" ] || {{ echo "没有可配置的终端" >&2; exit 1; }} + +# 本机实例（loopback）的共享 token 走 Authorization: Bearer；判据用 hooks 真正要打的地址。 +URL_HOST="$(printf '%s' "$HOOK_URL" | sed -e 's#^[^/]*//##' -e 's#/.*##' -e 's#:.*##')" +IS_LOCAL=0 +case "$URL_HOST" in + 127.0.0.1|localhost) IS_LOCAL=1 ;; +esac + +# ---- 3) 凭据：hooks 与客户端 MCP 条目共用同一个「实例共享 token」 +TOKEN="${{OCTOP_MEMORY_TOKEN:-}}" +{default_token_block} +if [ -z "$TOKEN" ] && [ "$HAS_TTY" = 1 ]; then + stty -echo < /dev/tty 2>/dev/null || true + if [ "$IS_LOCAL" = 1 ]; then + printf '实例共享 token（本机实例，走 Authorization: Bearer）: ' > /dev/tty + else + printf '实例共享 token（云上实例）: ' > /dev/tty + fi + read -r _t < /dev/tty || _t='' + stty echo < /dev/tty 2>/dev/null || true + printf '\n' > /dev/tty + [ -n "$_t" ] && TOKEN="$_t" +fi +[ -n "$TOKEN" ] || {{ echo "缺少 token：非交互运行时请设 OCTOP_MEMORY_TOKEN=" >&2; exit 1; }} +# header 名可配（缺省 Authorization）；header 是 Authorization 时值写成 ``Bearer ``。 +TOKEN_HEADER="${{OCTOP_TOKEN_HEADER:-Authorization}}" +echo " · 客户端 MCP 条目与 hooks：${{URL}}（实例共享 token：写 ${{TOKEN_HEADER}}）" + +AGENT="${{OCTOP_AGENT_ID:-}}" +if [ -z "$AGENT" ]; then + if [ -n "$DEFAULT_AGENT" ]; then + AGENT="$(ask "专家 ID [$DEFAULT_AGENT]: " "$DEFAULT_AGENT")" + else + AGENT="$(ask "专家 ID: " "")" + fi +fi +[ -n "$AGENT" ] || {{ echo "缺少专家 ID：非交互运行时请设 OCTOP_AGENT_ID=" >&2; exit 1; }} + +# 用户标识（建议域账号）：「谁问的」只能靠 X-Octop-User-Id，条目和 hooks 都写它。 +USER_ID="${{OCTOP_USER_ID:-}}" +if [ -z "$USER_ID" ] && [ "$HAS_TTY" = 1 ]; then + USER_ID="$(ask "用户标识（建议填域账号，如 alice；写进 X-Octop-User-Id 标记谁问的）: " "")" +fi + +command -v node >/dev/null 2>&1 || echo " ! 未找到 node：hook 需要它，请先安装 Node.js" >&2 +command -v python3 >/dev/null 2>&1 || {{ echo " ! 未找到 python3：无法合并终端配置" >&2; exit 1; }} + +# ---- 3.5) 解析 node 绝对路径：hook 由终端派生，常在非 login shell 里跑， +# nvm 的 node 不在 PATH —— 写裸 node 会静默失败（command hook 非 0 退出不报错）。 +NODE_BIN="${{OCTOP_NODE:-}}" +if [ -z "$NODE_BIN" ] && command -v node >/dev/null 2>&1; then + NODE_BIN="$(command -v node)" +fi +if [ -z "$NODE_BIN" ] && [ -d "$HOME/.nvm/versions/node" ]; then + _ver="$(ls -1 "$HOME/.nvm/versions/node" 2>/dev/null | sort | tail -n 1)" + [ -n "$_ver" ] && [ -x "$HOME/.nvm/versions/node/$_ver/bin/node" ] && NODE_BIN="$HOME/.nvm/versions/node/$_ver/bin/node" +fi +for _cand in /opt/homebrew/bin/node /usr/local/bin/node "$HOME/.volta/bin/node" /usr/bin/node; do + if [ -z "$NODE_BIN" ] && [ -x "$_cand" ]; then NODE_BIN="$_cand"; fi +done +if [ -z "$NODE_BIN" ] || [ ! -x "$NODE_BIN" ]; then + NODE_BIN=node + echo " ! 找不到 node 的绝对路径：hook 里只能写裸 node，非 login shell 下可能静默失败" >&2 +else + echo " · node: $NODE_BIN" +fi +export OCTOP_NODE="$NODE_BIN" + +# Node 22.15+ 的 --use-system-ca 读操作系统证书库。企业根不在 Node 自带库里时， +# 不加这个前缀，hook 和下面的端点自检都会 TypeError: fetch failed。 +USE_SYSTEM_CA=0 +if "$NODE_BIN" --use-system-ca -e 'process.exit(0)' >/dev/null 2>&1; then + USE_SYSTEM_CA=1 + echo " · node 使用系统证书库（--use-system-ca）" +else + echo " ! 当前 node 不支持 --use-system-ca（需要 22.15+）：企业根证书下 hook 的 fetch 可能失败" >&2 +fi +export OCTOP_USE_SYSTEM_CA="$USE_SYSTEM_CA" +if [ "$USE_SYSTEM_CA" = 1 ]; then + NODE_OPTIONS="${{NODE_OPTIONS:+$NODE_OPTIONS }}--use-system-ca" + export NODE_OPTIONS +fi + +# ---- 4) 先验证这份凭据在该端点可用（免得装完才发现 401） +if [ "$NODE_BIN" != node ] || command -v node >/dev/null 2>&1; then + OCTOP_URL="$HOOK_URL" OCTOP_TOKEN="$TOKEN" OCTOP_TOKEN_HEADER="$TOKEN_HEADER" \\ + OCTOP_AGENT="$AGENT" "$NODE_BIN" --input-type=module -e ' +const cfg = {{ url: process.env.OCTOP_URL, token: process.env.OCTOP_TOKEN }}; +const headers = {{ "Content-Type": "application/json", Accept: "application/json, text/event-stream", + "X-Octop-Agent-Id": process.env.OCTOP_AGENT }}; +const header = process.env.OCTOP_TOKEN_HEADER || "Authorization"; +headers[header] = header === "Authorization" ? `Bearer ${{cfg.token}}` : cfg.token; +try {{ + const r = await fetch(cfg.url, {{ method: "POST", headers, signal: AbortSignal.timeout(8000), body: JSON.stringify({{ jsonrpc: "2.0", id: 1, method: "tools/list" }}) }}); + console.log(` · 端点自检（hook 直连）：HTTP ${{r.status}}${{r.status === 200 ? "" : "（凭据或地址不对，请检查）"}}`); +}} catch (e) {{ + console.log(` · 端点自检失败：${{e}}（安装仍继续，请自行确认地址）`); +}} +' 2>/dev/null || true +fi + +# ---- 5) 各终端各自安装 hook 脚本 + 合并其配置 +OCTOP_URL="$URL" OCTOP_HOOK_URL="$HOOK_URL" OCTOP_TOKEN="$TOKEN" OCTOP_TOKEN_HEADER="$TOKEN_HEADER" \\ +OCTOP_AGENT="$AGENT" OCTOP_SERVER_NAME='{_server_name(mcp_url)}' \\ +OCTOP_USER_ID="$USER_ID" \\ +OCTOP_CC_DIR="$CC_DIR" OCTOP_KIRO_DIR="$KIRO_DIR" OCTOP_DSH_DIR="$DSH_DIR" \\ +OCTOP_CODEX_DIR="$CODEX_DIR" OCTOP_CURSOR_DIR="$CURSOR_DIR" \\ +OCTOP_TERMINALS="$TERMINALS" \\ +OCTOP_FILES_CLAUDE_CODE='{_host_manifest("claude-code")}' \\ +OCTOP_FILES_KIRO='{_host_manifest("kiro")}' \\ +OCTOP_FILES_DSH='{_host_manifest("dsh")}' \\ +OCTOP_FILES_CODEX='{_host_manifest("codex")}' \\ +OCTOP_FILES_CURSOR='{_host_manifest("cursor")}' \\ +OCTOP_FILES_PRESET='{_dsh_preset_manifest()}' \\ +python3 - <<'OCTOP_PY' +{_MERGE_PY} +OCTOP_PY + +echo " · 重开终端/IDE（Kiro 重载 hooks；Codex 首次需在 codex 里信任该 hook）后生效" +""" + + +_UNINSTALL_PY = r""" +import json +import os +import pathlib +import shutil + + +home = pathlib.Path(os.path.expanduser("~")) +CC_DIR = pathlib.Path(os.environ["OCTOP_CC_DIR"]).expanduser() +KIRO_DIR = pathlib.Path(os.environ["OCTOP_KIRO_DIR"]).expanduser() +DSH_DIR = pathlib.Path(os.environ["OCTOP_DSH_DIR"]).expanduser() +CODEX_DIR = pathlib.Path(os.environ["OCTOP_CODEX_DIR"]).expanduser() +CURSOR_DIR = pathlib.Path(os.environ["OCTOP_CURSOR_DIR"]).expanduser() + +_OUR_SCRIPTS = ( + "user-prompt-recall.mjs", + "stop-capture.mjs", + "octop-user-prompt-recall.mjs", + "octop-stop-capture.mjs", +) +_OUR_MCP_TOOLS = ("memory_search", "memory_capture") + + +def _load(path): + if not path.exists(): + return None + try: + return json.loads(path.read_text(encoding="utf-8")) + except Exception as exc: + print(f" ! 读不了 {path}：{exc}") + return None + + +def _dump(path, data): + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(data, ensure_ascii=False, indent=2) + "\n", encoding="utf-8") + + +def _octop_servers(servers): + return [name for name in servers if str(name).replace("_", "-").startswith("octop-memory")] + + +def _is_our_handler(handler): + handler = handler or {} + if handler.get("type") == "mcp_tool": + return handler.get("tool") in _OUR_MCP_TOOLS + command = handler.get("command") or "" + return any(script in command for script in _OUR_SCRIPTS) + + +def _rm_tree(path, label): + if not path.exists(): + print(f" · {label} 不存在，跳过 → {path}") + return + shutil.rmtree(path) + print(f" ✓ 已删除 {label} → {path}") + + +def _strip_mcp(path): + cfg = _load(path) + if cfg is None: + return + servers = cfg.get("mcpServers") or {} + ours = _octop_servers(servers) + if not ours: + print(f" · {path} 无 octop-memory* MCP 条目") + return + for name in ours: + servers.pop(name, None) + cfg["mcpServers"] = servers + _dump(path, cfg) + print(f" ✓ 已移除 MCP 服务 {', '.join(ours)} → {path}") + + +def undo_claude_code(): + settings = home / ".claude" / "settings.json" + st = _load(settings) + if st is not None: + hooks = st.setdefault("hooks", {}) + changed = False + for event in list(hooks): + kept = [ + group + for group in (hooks.get(event) or []) + if not any(_is_our_handler(h) for h in (group or {}).get("hooks") or []) + ] + if len(kept) != len(hooks.get(event) or []): + changed = True + if kept: + hooks[event] = kept + else: + hooks.pop(event, None) + changed = True + if changed: + _dump(settings, st) + print(f" ✓ 已从 Claude Code settings 去掉 octop hooks → {settings}") + else: + print(f" · Claude Code settings 无 octop hooks → {settings}") + _strip_mcp(home / ".claude.json") + _rm_tree(CC_DIR, "Claude Code hooks 目录") + + +def undo_kiro(): + for name in ("memory-recall-on-start.json", "memory-save-on-stop.json"): + path = KIRO_DIR / name + if path.exists(): + path.unlink() + print(f" ✓ 已删除 Kiro hook 文件 → {path}") + else: + print(f" · Kiro hook 文件不存在 → {path}") + _strip_mcp(home / ".kiro" / "settings" / "mcp.json") + _rm_tree(KIRO_DIR, "Kiro hooks 目录") + + +def undo_dsh(): + bridge = DSH_DIR / "octop-hooks.json" + if bridge.exists(): + bridge.unlink() + print(f" ✓ 已删除 DSH 桥接 → {bridge}") + bundle = DSH_DIR.parent / "bundles" / "dsh-memory-share" + if bundle.exists(): + shutil.rmtree(bundle) + print(f" ✓ 已删除 DSH preset bundle → {bundle}") + profiles = home / ".dsh" / "profiles" + for name in ("web", "desktop", "default"): + package = profiles / name / "package.json" + cfg = _load(package) + if cfg is None: + continue + bundles = cfg.get("dsh", {}).get("profile", {}).get("bundles") + if not isinstance(bundles, list): + continue + kept = [b for b in bundles if b != "dsh-memory-share"] + if len(kept) != len(bundles): + cfg["dsh"]["profile"]["bundles"] = kept + _dump(package, cfg) + print(f" ✓ 已从 DSH profile「{name}」去掉 dsh-memory-share") + _rm_tree(DSH_DIR, "DSH hooks 目录") + + +def _toml_blocks(text): + blocks = [[]] + for line in text.splitlines(keepends=True): + if line.lstrip().startswith("[") and not line.lstrip().startswith("#"): + blocks.append([]) + blocks[-1].append(line) + return blocks + + +def undo_codex(): + path = home / ".codex" / "config.toml" + if path.exists(): + text = path.read_text(encoding="utf-8") + blocks, kept = _toml_blocks(text), [] + removed = 0 + for block in blocks: + body = "".join(block) + header = block[0].strip() if block else "" + ours = str(CODEX_DIR) in body + if ours and header.startswith("[[hooks.") and header.endswith(".hooks]]"): + event = header[len("[[hooks.") : -len(".hooks]]")] + if kept and kept[-1][0].strip() == f"[[hooks.{event}]]" and not "".join(kept[-1][1:]).strip(): + kept.pop() + removed += 1 + continue + kept.append(block) + if removed: + path.write_text("".join("".join(block) for block in kept), encoding="utf-8") + print(f" ✓ 已从 Codex config.toml 去掉 octop hooks → {path}") + else: + print(f" · Codex config.toml 无 octop hooks → {path}") + else: + print(f" · Codex config.toml 不存在 → {path}") + _rm_tree(CODEX_DIR, "Codex hooks 目录") + + +def undo_cursor(): + path = home / ".cursor" / "hooks.json" + cfg = _load(path) + if cfg is not None: + hooks = cfg.setdefault("hooks", {}) + marker = str(CURSOR_DIR) + changed = False + for event in list(hooks): + entries = hooks.get(event) or [] + kept = [ + entry + for entry in entries + if marker not in json.dumps(entry, ensure_ascii=False) + ] + if len(kept) != len(entries): + changed = True + if kept: + hooks[event] = kept + else: + hooks.pop(event, None) + changed = True + if changed: + cfg["hooks"] = hooks + _dump(path, cfg) + print(f" ✓ 已从 Cursor hooks.json 去掉 octop hooks → {path}") + else: + print(f" · Cursor hooks.json 无 octop hooks → {path}") + else: + print(f" · Cursor hooks.json 不存在 → {path}") + # setup 不为 Cursor 写 MCP，但检索常靠 ~/.cursor/mcp.json 里的 octop-memory* + _strip_mcp(home / ".cursor" / "mcp.json") + _rm_tree(CURSOR_DIR, "Cursor hooks 目录") + + +for terminal in os.environ["OCTOP_TERMINALS"].split(","): + terminal = terminal.strip() + if not terminal: + continue + print(f"· {terminal}") + { + "claude-code": undo_claude_code, + "kiro": undo_kiro, + "dsh": undo_dsh, + "codex": undo_codex, + "cursor": undo_cursor, + }[terminal]() + +print(" · octop memory 卸载完成（可重复执行；服务端已有记忆不会删除）") +print(" · 请重开终端/IDE 后生效") +""".strip() + + +def render_uninstall_script(*, terminals: str = "") -> str: + """Render the interactive uninstaller (counterpart of ``render_setup_script``). + + No token / agent prompts — only chooses which terminals to scrub. Safe to + re-run; leaves non-octop hooks and MCP servers alone. + """ + terminals_literal = _sh_single_quote(terminals) if terminals else "''" + return f"""#!/bin/sh +# octop memory — 卸载本地终端接入（与 /mcp/memory/setup 相对；可重复执行） +set -eu + +DEFAULT_TERMINALS={terminals_literal} +CC_DIR='{HOOKS_DIRS["claude-code"]}' +KIRO_DIR='{HOOKS_DIRS["kiro"]}' +DSH_DIR='{HOOKS_DIRS["dsh"]}' +CODEX_DIR='{HOOKS_DIRS["codex"]}' +CURSOR_DIR='{HOOKS_DIRS["cursor"]}' + +have_claude=0; have_kiro=0; have_dsh=0; have_codex=0; have_cursor=0 +{{ [ -e "$HOME/.claude.json" ] || [ -d "$HOME/.claude" ] || command -v claude >/dev/null 2>&1; }} && have_claude=1 +{{ [ -d "$HOME/.kiro" ] || command -v kiro >/dev/null 2>&1; }} && have_kiro=1 +{{ [ -d "$HOME/.dsh" ] || command -v dsh >/dev/null 2>&1; }} && have_dsh=1 +{{ [ -d "$HOME/.codex" ] || command -v codex >/dev/null 2>&1; }} && have_codex=1 +{{ [ -d "$HOME/.cursor" ] || command -v cursor >/dev/null 2>&1; }} && have_cursor=1 + +DETECTED='' +[ "$have_claude" = 1 ] && DETECTED="$DETECTED claude-code" +[ "$have_kiro" = 1 ] && DETECTED="$DETECTED kiro" +[ "$have_dsh" = 1 ] && DETECTED="$DETECTED dsh" +[ "$have_codex" = 1 ] && DETECTED="$DETECTED codex" +[ "$have_cursor" = 1 ] && DETECTED="$DETECTED cursor" +[ -n "$DETECTED" ] || DETECTED='cursor' + +echo "octop memory 卸载" +echo "检测到：" +[ "$have_claude" = 1 ] && echo " 1) Claude Code　（清 ~/.claude/hooks-events + settings/MCP）" +[ "$have_kiro" = 1 ] && echo " 2) Kiro　　　　（清 ~/.kiro/hooks + MCP）" +[ "$have_dsh" = 1 ] && echo " 3) DSH　　　　（清 ~/.dsh/hooks + preset）" +[ "$have_codex" = 1 ] && echo " 4) Codex　　　（清 ~/.codex/hooks + config.toml 中的 octop hooks）" +[ "$have_cursor" = 1 ] && echo " 5) Cursor　　　（清 ~/.cursor/hooks + hooks.json；顺带去掉 mcp.json 里的 octop-memory*）" + +HAS_TTY=0 +if (true /dev/null; then HAS_TTY=1; fi + +TERMINALS="${{OCTOP_TERMINALS:-$DEFAULT_TERMINALS}}" +if [ -z "$TERMINALS" ] && [ "$HAS_TTY" = 1 ]; then + _picked='' + if command -v python3 >/dev/null 2>&1; then + set -- + [ "$have_claude" = 1 ] && set -- "$@" "claude-code|Claude Code" + [ "$have_kiro" = 1 ] && set -- "$@" "kiro|Kiro" + [ "$have_dsh" = 1 ] && set -- "$@" "dsh|DSH" + [ "$have_codex" = 1 ] && set -- "$@" "codex|Codex" + [ "$have_cursor" = 1 ] && set -- "$@" "cursor|Cursor" + _picked="$(python3 - "$@" 2>/dev/null <<'OCTOP_SELECT' +{_SELECT_PY} +OCTOP_SELECT +)" || _picked='' + fi + if [ -n "$_picked" ]; then + TERMINALS="$_picked" + else + _attempt=0 + while [ -z "$TERMINALS" ] && [ "$_attempt" -lt 3 ]; do + _attempt=$((_attempt + 1)) + printf '要卸载哪些？(可多选：1,2 / 1 3 / 12 / a=全部，回车=全部检测到的): ' > /dev/tty + read -r _pick < /dev/tty || _pick='' + _chosen='' + case "$_pick" in + ''|a|A|all|ALL|全部) _chosen="$DETECTED" ;; + *) + for _n in $(printf '%s' "$_pick" | tr ',' ' '); do + case "$_n" in + [0-9][0-9]*) _items="$(printf '%s' "$_n" | fold -w1 | tr '\\n' ' ')" ;; + *) _items="$_n" ;; + esac + for _i in $_items; do + case "$_i" in + 1) _add="$([ "$have_claude" = 1 ] && echo claude-code)" ;; + 2) _add="$([ "$have_kiro" = 1 ] && echo kiro)" ;; + 3) _add="$([ "$have_dsh" = 1 ] && echo dsh)" ;; + 4) _add="$([ "$have_codex" = 1 ] && echo codex)" ;; + 5) _add="$([ "$have_cursor" = 1 ] && echo cursor)" ;; + *) _add='' ;; + esac + if [ -z "$_add" ]; then + echo " ? 已忽略：${{_i}}" >&2 + else + case ",$_chosen," in *",$_add,"*) ;; *) _chosen="$_chosen,$_add" ;; esac + fi + done + done + ;; + esac + TERMINALS="$(printf '%s' "$_chosen" | sed 's/^,//')" + [ -n "$TERMINALS" ] || echo " ! 没有选中任何终端，请重选" >&2 + done + fi +fi + +[ -n "$TERMINALS" ] || {{ echo "没有可卸载的终端：非交互请设 OCTOP_TERMINALS=cursor" >&2; exit 1; }} +command -v python3 >/dev/null 2>&1 || {{ echo " ! 未找到 python3" >&2; exit 1; }} + +OCTOP_CC_DIR="$CC_DIR" OCTOP_KIRO_DIR="$KIRO_DIR" OCTOP_DSH_DIR="$DSH_DIR" \\ +OCTOP_CODEX_DIR="$CODEX_DIR" OCTOP_CURSOR_DIR="$CURSOR_DIR" \\ +OCTOP_TERMINALS="$TERMINALS" \\ +python3 - <<'OCTOP_PY' +{_UNINSTALL_PY} +OCTOP_PY +""" diff --git a/src/octop/infra/agents/hook_scripts/claude-code/stop-capture.mjs b/src/octop/infra/agents/hook_scripts/claude-code/stop-capture.mjs new file mode 100644 index 000000000..1e01f76cf --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/claude-code/stop-capture.mjs @@ -0,0 +1,114 @@ +#!/usr/bin/env node +// Claude Code ``Stop`` hook: ship this session's transcript into octop memory. +// +// Claude-Code-specific in exactly two ways: the payload carries +// ``transcript_path``, and ``Stop`` fires *before* the current turn is flushed — +// so the hook hands the work to a detached worker that polls until an assistant +// record shows up after the last user record. The hook itself returns at once. +import { spawn } from "node:child_process"; +import fs from "node:fs"; +import { fileURLToPath } from "node:url"; +import { capture, isConfigured, loadConfig, makeLogger, sleep } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("stopcap.claude-code"); +const DELAY_MS = Number(process.env.OCTOP_CAPTURE_DELAY_MS ?? 3000); +const POLL_MS = Number(process.env.OCTOP_CAPTURE_POLL_MS ?? 1000); +const MAX_WAIT_MS = Number(process.env.OCTOP_CAPTURE_MAX_WAIT_MS ?? 20000); +const IS_WORKER = process.argv.includes("--worker"); + +let raw = ""; +if (IS_WORKER) { + raw = Buffer.from(process.env.OCTOP_HOOK_PAYLOAD_B64 || "", "base64").toString("utf8"); +} else { + try { + raw = fs.readFileSync(0, "utf8"); + } catch { + raw = ""; + } + if (process.env.OCTOP_CAPTURE_SYNC !== "1") { + try { + const worker = spawn(process.execPath, [fileURLToPath(import.meta.url), "--worker"], { + detached: true, + stdio: "ignore", + env: { ...process.env, OCTOP_HOOK_PAYLOAD_B64: Buffer.from(raw).toString("base64") }, + }); + worker.unref(); + log(`deferred: 交给后台进程，等本轮落盘（${DELAY_MS}ms 起轮询）`); + process.exit(0); // the hook returns at once; the worker waits, then uploads + } catch (exc) { + log(`detach failed, running inline: ${exc}`); + } + } +} +let hookInput = {}; +try { + if (raw.trim()) hookInput = JSON.parse(raw); +} catch { + log("stdin-not-json"); +} + +const sessionId = hookInput.session_id || ""; +const event = hookInput.hook_event_name || "Stop"; +const transcriptPath = hookInput.transcript_path || ""; +if (!transcriptPath) { + log(`${event} skip: payload has no transcript_path`); + process.exit(0); +} + +/** The turn landed when an assistant record follows the last user record. */ +function turnLanded(path) { + try { + let lastUser = -1; + let lastAssistant = -1; + const lines = fs.readFileSync(path, "utf8").split("\n"); + lines.forEach((line, index) => { + if (!line.trim()) return; + let record; + try { + record = JSON.parse(line); + } catch { + return; + } + if (record.type === "user") lastUser = index; + else if (record.type === "assistant") lastAssistant = index; + }); + return lastAssistant > lastUser; + } catch { + return true; // unreadable -> do not wait + } +} + +if (IS_WORKER) { + if (DELAY_MS > 0) await sleep(DELAY_MS); + const deadline = Date.now() + MAX_WAIT_MS; + let waited = 0; + while (Date.now() < deadline && !turnLanded(transcriptPath)) { + await sleep(POLL_MS); + waited += POLL_MS; + } + if (waited) log(`${event} waited ${waited}ms for the turn to land`); +} + +const config = loadConfig(); +if (!isConfigured(config)) { + log("skip: not configured (need OCTOP_MEMORY_MCP_URL / _TOKEN / _AGENT_ID or config.json)"); + process.exit(0); +} +let buffer; +try { + buffer = fs.readFileSync(transcriptPath); +} catch (exc) { + log(`${event} read failed: ${exc} (${transcriptPath})`); + process.exit(0); +} +await capture({ + config, + log, + event, + source: "claude-code", + sessionId, + transcriptPath, + buffer, + sessionPrefix: "cc:", +}); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/claude-code/user-prompt-recall.mjs b/src/octop/infra/agents/hook_scripts/claude-code/user-prompt-recall.mjs new file mode 100644 index 000000000..4db3d5099 --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/claude-code/user-prompt-recall.mjs @@ -0,0 +1,51 @@ +#!/usr/bin/env node +// Claude Code ``UserPromptSubmit`` hook: recall before the model answers. +// +// ``mcp_tool`` hooks are silently skipped for UserPromptSubmit (2.1.263), so this +// is a command hook that calls ``memory_search`` itself and prints the block — +// Claude Code injects the stdout as extra context for this turn. +import fs from "node:fs"; +import { callTool, isConfigured, loadConfig, makeLogger } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("urecall.claude-code"); +const MARKER = "[octop memory search]"; +const payload = (() => { + try { + const raw = fs.readFileSync(0, "utf8"); + return raw.trim() ? JSON.parse(raw) : {}; + } catch { + return {}; + } +})(); + +async function main() { + const prompt = String(payload.prompt || "").trim(); + if (!prompt) { + log("skip: empty prompt"); + return; + } + const config = loadConfig(); + if (!isConfigured(config)) { + log("skip: not configured"); + return; + } + const maxResults = Number(process.env.OCTOP_RECALL_MAX_RESULTS ?? 5); + const result = await callTool(config, "memory_search", { + query: prompt, + corpus: "atom", + max_results: maxResults, + }); + const hits = result.detail?.hits ?? []; + log(`prompt="${prompt.slice(0, 60)}" hits=${hits.length} http=${result.status}`); + if (!hits.length) return; + const lines = [MARKER]; + for (const hit of hits) { + const when = String(hit.occurred_at || "").slice(0, 19); + lines.push(`- [${hit.layer}] (${hit.role_hint ?? hit.layer} @ ${when}) ${hit.snippet}`); + } + lines.push("若需要原文/证据，用 memory_get(path=…) 读全文；与本次工作无关则忽略。"); + process.stdout.write(lines.join("\n") + "\n"); +} + +await main(); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/codex/stop-capture.mjs b/src/octop/infra/agents/hook_scripts/codex/stop-capture.mjs new file mode 100644 index 000000000..8f89a97b1 --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/codex/stop-capture.mjs @@ -0,0 +1,102 @@ +#!/usr/bin/env node +// Codex CLI ``Stop`` hook: ship this session's rollout transcript into octop memory. +// +// Codex-specific: hooks are declared in ``~/.codex/config.toml`` +// (``[[hooks.Stop]]`` with ``type = "command"``), the payload carries +// ``transcript_path`` (the ``rollout-*.jsonl`` under ``~/.codex/sessions``) and +// ``session_id``, and — like Claude Code — Stop can fire before the turn is +// flushed, so the hook defers to a detached worker and waits for the file to stop +// growing. +import { spawn } from "node:child_process"; +import fs from "node:fs"; +import { fileURLToPath } from "node:url"; +import { capture, isConfigured, loadConfig, makeLogger, sleep } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("stopcap.codex"); +const DELAY_MS = Number(process.env.OCTOP_CAPTURE_DELAY_MS ?? 3000); +const POLL_MS = Number(process.env.OCTOP_CAPTURE_POLL_MS ?? 1000); +const MAX_WAIT_MS = Number(process.env.OCTOP_CAPTURE_MAX_WAIT_MS ?? 20000); +const IS_WORKER = process.argv.includes("--worker"); + +let raw = ""; +if (IS_WORKER) { + raw = Buffer.from(process.env.OCTOP_HOOK_PAYLOAD_B64 || "", "base64").toString("utf8"); +} else { + try { + raw = fs.readFileSync(0, "utf8"); + } catch { + raw = ""; + } + if (process.env.OCTOP_CAPTURE_SYNC !== "1") { + try { + const worker = spawn(process.execPath, [fileURLToPath(import.meta.url), "--worker"], { + detached: true, + stdio: "ignore", + env: { ...process.env, OCTOP_HOOK_PAYLOAD_B64: Buffer.from(raw).toString("base64") }, + }); + worker.unref(); + log(`deferred: 交给后台进程，等本轮落盘（${DELAY_MS}ms 起轮询）`); + process.exit(0); + } catch (exc) { + log(`detach failed, running inline: ${exc}`); + } + } +} +let hookInput = {}; +try { + if (raw.trim()) hookInput = JSON.parse(raw); +} catch { + log("stdin-not-json"); +} + +const sessionId = hookInput.session_id || ""; +const event = hookInput.hook_event_name || "Stop"; +const transcriptPath = hookInput.transcript_path || ""; +if (!transcriptPath) { + log(`${event} skip: payload has no transcript_path`); + process.exit(0); +} + +if (IS_WORKER) { + if (DELAY_MS > 0) await sleep(DELAY_MS); + const deadline = Date.now() + MAX_WAIT_MS; + let previousSize = -1; + let waited = 0; + while (Date.now() < deadline) { + let size; + try { + size = fs.statSync(transcriptPath).size; + } catch { + break; + } + if (size === previousSize) break; + previousSize = size; + await sleep(POLL_MS); + waited += POLL_MS; + } + if (waited) log(`${event} waited ${waited}ms for the rollout to settle`); +} + +const config = loadConfig(); +if (!isConfigured(config)) { + log("skip: not configured"); + process.exit(0); +} +let buffer; +try { + buffer = fs.readFileSync(transcriptPath); +} catch (exc) { + log(`${event} read failed: ${exc} (${transcriptPath})`); + process.exit(0); +} +await capture({ + config, + log, + event, + source: "codex", + sessionId, + transcriptPath, + buffer, + sessionPrefix: "codex:", +}); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/codex/user-prompt-recall.mjs b/src/octop/infra/agents/hook_scripts/codex/user-prompt-recall.mjs new file mode 100644 index 000000000..7e657a65f --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/codex/user-prompt-recall.mjs @@ -0,0 +1,55 @@ +#!/usr/bin/env node +// Codex CLI ``UserPromptSubmit`` hook: recall before the model answers. +// +// Codex hook output understands ``additionalContext`` (same contract as Claude +// Code), so the recalled block is injected for this turn. +import fs from "node:fs"; +import { callTool, isConfigured, loadConfig, makeLogger } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("urecall.codex"); +const MARKER = "[octop memory search]"; + +async function main() { + let payload = {}; + try { + const raw = fs.readFileSync(0, "utf8"); + if (raw.trim()) payload = JSON.parse(raw); + } catch { + log("stdin-not-json"); + } + const prompt = String(payload.prompt || "").trim(); + if (!prompt) { + log("skip: empty prompt"); + return; + } + const config = loadConfig(); + if (!isConfigured(config)) { + log("skip: not configured"); + return; + } + const result = await callTool(config, "memory_search", { + query: prompt, + corpus: "atom", + max_results: Number(process.env.OCTOP_RECALL_MAX_RESULTS ?? 5), + }); + const hits = result.detail?.hits ?? []; + log(`prompt="${prompt.slice(0, 60)}" hits=${hits.length} http=${result.status}`); + if (!hits.length) return; + const lines = [MARKER]; + for (const hit of hits) { + const when = String(hit.occurred_at || "").slice(0, 19); + lines.push(`- [${hit.layer}] (${hit.role_hint ?? hit.layer} @ ${when}) ${hit.snippet}`); + } + lines.push("若需要原文/证据，用 memory_get(path=…) 读全文；与本次工作无关则忽略。"); + process.stdout.write( + JSON.stringify({ + hookSpecificOutput: { + hookEventName: "UserPromptSubmit", + additionalContext: lines.join("\n"), + }, + }) + "\n", + ); +} + +await main(); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/cursor/stop-capture.mjs b/src/octop/infra/agents/hook_scripts/cursor/stop-capture.mjs new file mode 100644 index 000000000..de38e7d0e --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/cursor/stop-capture.mjs @@ -0,0 +1,65 @@ +#!/usr/bin/env node +// Cursor ``stop`` hook: ship this session's transcript into octop memory. +// +// Cursor-specific: hooks live in ``~/.cursor/hooks.json`` (``{"version":1, +// "hooks":{"stop":[{"command":…}]}}``) and the payload carries ``transcript_path`` +// plus ``conversation_id`` / ``session_id``. That path is +// ``~/.cursor/projects//agent-transcripts//.jsonl`` +// (one JSON object per line: ``{role, message:{content:[{type,text}]}}``). +// ``~/.cursor/chats/**/store.db`` is protobuf and is not read — if a payload has +// no ``transcript_path``, this logs and skips instead of guessing. +import fs from "node:fs"; +import { capture, isConfigured, loadConfig, makeLogger, readStdinJson, sleep } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("stopcap.cursor"); +const hookInput = readStdinJson(log); +const event = hookInput.hook_event_name || "stop"; +const sessionId = hookInput.conversation_id || hookInput.session_id || ""; +const transcriptPath = hookInput.transcript_path || ""; +if (!transcriptPath) { + log(`${event} skip: payload has no transcript_path (session=${sessionId})`); + process.exit(0); +} + +const POLL_MS = Number(process.env.OCTOP_CAPTURE_POLL_MS ?? 1000); +const MAX_WAIT_MS = Number(process.env.OCTOP_CAPTURE_MAX_WAIT_MS ?? 20000); +const deadline = Date.now() + MAX_WAIT_MS; +let previousSize = -1; +let waited = 0; +while (Date.now() < deadline) { + let size; + try { + size = fs.statSync(transcriptPath).size; + } catch { + break; + } + if (size === previousSize) break; + previousSize = size; + await sleep(POLL_MS); + waited += POLL_MS; +} +if (waited) log(`${event} waited ${waited}ms for the transcript to settle`); + +const config = loadConfig(); +if (!isConfigured(config)) { + log("skip: not configured"); + process.exit(0); +} +let buffer; +try { + buffer = fs.readFileSync(transcriptPath); +} catch (exc) { + log(`${event} read failed: ${exc} (${transcriptPath})`); + process.exit(0); +} +await capture({ + config, + log, + event, + source: "cursor", + sessionId, + transcriptPath, + buffer, + sessionPrefix: "cursor:", +}); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/cursor/user-prompt-recall.mjs b/src/octop/infra/agents/hook_scripts/cursor/user-prompt-recall.mjs new file mode 100644 index 000000000..8b662856e --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/cursor/user-prompt-recall.mjs @@ -0,0 +1,56 @@ +#!/usr/bin/env node +// Cursor ``beforeSubmitPrompt`` hook: recall before the model answers. +// +// Cursor's payload carries ``prompt`` (and ``conversation_id``/``cwd``); its hook +// output understands ``additionalContext``, so the recalled block is injected for +// this turn. +import fs from "node:fs"; +import { callTool, isConfigured, loadConfig, makeLogger } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("urecall.cursor"); +const MARKER = "[octop memory search]"; + +async function main() { + let payload = {}; + try { + const raw = fs.readFileSync(0, "utf8"); + if (raw.trim()) payload = JSON.parse(raw); + } catch { + log("stdin-not-json"); + } + const prompt = String(payload.prompt || payload.user_message || "").trim(); + if (!prompt) { + log("skip: empty prompt"); + return; + } + const config = loadConfig(); + if (!isConfigured(config)) { + log("skip: not configured"); + return; + } + const result = await callTool(config, "memory_search", { + query: prompt, + corpus: "atom", + max_results: Number(process.env.OCTOP_RECALL_MAX_RESULTS ?? 5), + }); + const hits = result.detail?.hits ?? []; + log(`prompt="${prompt.slice(0, 60)}" hits=${hits.length} http=${result.status}`); + if (!hits.length) return; + const lines = [MARKER]; + for (const hit of hits) { + const when = String(hit.occurred_at || "").slice(0, 19); + lines.push(`- [${hit.layer}] (${hit.role_hint ?? hit.layer} @ ${when}) ${hit.snippet}`); + } + lines.push("若需要原文/证据，用 memory_get(path=…) 读全文；与本次工作无关则忽略。"); + process.stdout.write( + JSON.stringify({ + hookSpecificOutput: { + hookEventName: "beforeSubmitPrompt", + additionalContext: lines.join("\n"), + }, + }) + "\n", + ); +} + +await main(); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/dsh/memory-target.mjs b/src/octop/infra/agents/hook_scripts/dsh/memory-target.mjs new file mode 100644 index 000000000..c5b9feaad --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/dsh/memory-target.mjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +// DSH-only target resolution for the memory hooks. +// +// The installer writes `config.json` next to the scripts, but a DSH whose MCP +// panel already knows the octop server needs no second copy of the credentials. +// Precedence: environment -> `config.json` -> the `octop-memory*` entry of this +// DSH's own MCP server table (`$DSH_HOME/storages/mcp_servers.json`), enabled +// servers first, then prod, local, oauth. `OCTOP_MEMORY_SERVER` names one entry +// explicitly and may pick a disabled one. +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { loadConfig } from "../lib/octop-memory-client.mjs"; + +const ENV_VAR_NAME = /^[A-Z][A-Z0-9_]*$/; + +function serverTablePath() { + const home = process.env.DSH_HOME || path.join(os.homedir(), ".dsh"); + return path.join(home, "storages", "mcp_servers.json"); +} + +/** A bare env-var name resolves from the environment. */ +function headerValue(value) { + const text = String(value ?? "").trim(); + if (ENV_VAR_NAME.test(text) && process.env[text]) return process.env[text]; + return text; +} + +function headerMap(server) { + const headers = {}; + for (const item of Array.isArray(server.headers) ? server.headers : []) { + if (item && typeof item.name === "string" && item.name) headers[item.name] = headerValue(item.value); + } + return headers; +} + +function rank(server) { + const name = String(server.serverName ?? "").toLowerCase(); + if (name === "octop-memory-prod") return 10; + if (name === "octop-memory" || name === "octop-memory-local") return 20; + if (name === "octop-memory-oauth") return 30; + return 40; +} + +function configFromServer(server) { + const headers = headerMap(server); + const names = Object.keys(headers); + const tokenHeader = + names.find((name) => name.toLowerCase() === "authorization") ?? + names.find((name) => name.toLowerCase() === "x-octop-memory-token") ?? + ""; + return { + endpoint: String(server.url ?? "").trim(), + token: tokenHeader ? headers[tokenHeader].replace(/^Bearer\s+/i, "") : "", + tokenHeader, + agentId: headers["X-Octop-Agent-Id"] ?? "", + userId: headers["X-Octop-User-Id"] ?? "", + }; +} + +/** The usable octop entry of this DSH's MCP server table, or null. */ +export function discoverConfig() { + let servers; + try { + const parsed = JSON.parse(fs.readFileSync(serverTablePath(), "utf8")); + servers = Object.values(parsed?.tables?.servers ?? {}); + } catch { + return null; // no DSH storage here -> nothing to discover + } + const wanted = (process.env.OCTOP_MEMORY_SERVER || "").trim().toLowerCase(); + const candidates = servers + .filter((server) => server && typeof server === "object" && !server.stale) + .filter((server) => /^octop[-_]?memory/i.test(String(server.serverName ?? ""))) + .filter((server) => !wanted || String(server.serverName ?? "").toLowerCase() === wanted) + .sort((a, b) => (wanted ? 0 : Number(!a.enabled) - Number(!b.enabled)) || rank(a) - rank(b)); + for (const server of candidates) { + const config = configFromServer(server); + if (config.endpoint && config.token) return config; + } + return null; +} + +/** What these hooks should talk to. */ +export function resolveConfig() { + const configured = loadConfig(); + if (configured.endpoint && configured.token) return configured; + return discoverConfig() ?? configured; +} diff --git a/src/octop/infra/agents/hook_scripts/dsh/stop-capture.mjs b/src/octop/infra/agents/hook_scripts/dsh/stop-capture.mjs new file mode 100644 index 000000000..1c24326e7 --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/dsh/stop-capture.mjs @@ -0,0 +1,223 @@ +#!/usr/bin/env node +// DSH ``Stop`` hook (through the hooks bridge): ship this session's +// ``session*.jsonl.zstd`` into octop memory. +// +// Two shipping modes, chosen by probing the endpoint (force one with +// ``OCTOP_CAPTURE_MODE=text|transcript``): +// +// text ``memory_capture`` takes ``assistant`` -> one call per turn with +// the roles in their own fields, so the assistant's answer is not +// stored as a user message. +// transcript no ``assistant`` field -> gzip the raw JSONL delta and let the +// server parse typed events (its DSH transcript path). +// +// DSH-specific: the payload has only ``session_id``, the file is versioned +// (``session.jsonl.zstd`` -> ``session.v3.jsonl.zstd`` -> …) so the newest one +// inside the session directory is used, and it is zstd-compressed — decoded here +// so the server needs no zstd dependency. +import fs from "node:fs"; +import os from "node:os"; +import { + capture, + captureTurns, + decompressZstdContainer, + isConfigured, + loadState, + makeLogger, + newestFile, + readStdinJson, + saveState, + sleep, + supportsAssistant, +} from "../lib/octop-memory-client.mjs"; +import { resolveConfig } from "./memory-target.mjs"; + +const log = makeLogger("stopcap.dsh"); +const event = "Stop"; +const sessionId = readStdinJson(log).session_id || ""; +if (!sessionId) { + log(`${event} skip: payload has no session_id`); + process.exit(0); +} + +const root = process.env.OCTOP_DSH_SESSIONS_ROOT || `${os.homedir()}/.dsh/sessions`; +function findSessionFile(id) { + try { + for (const entry of fs.readdirSync(root, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const found = newestFile(`${root}/${entry.name}/${id}`, /^session.*\.jsonl\.zstd$/); + if (found) return found; + } + } catch { + // no DSH install / unreadable root + } + return ""; +} + +const transcriptPath = findSessionFile(sessionId); +if (!transcriptPath) { + log(`${event} skip: no transcript for ${sessionId} under ${root}`); + process.exit(0); +} + +const POLL_MS = Number(process.env.OCTOP_CAPTURE_POLL_MS ?? 1000); +const MAX_WAIT_MS = Number(process.env.OCTOP_CAPTURE_MAX_WAIT_MS ?? 20000); +const deadline = Date.now() + MAX_WAIT_MS; +let previousSize = -1; +let waited = 0; +while (Date.now() < deadline) { + let size; + try { + size = fs.statSync(transcriptPath).size; + } catch { + break; + } + if (size === previousSize) break; + previousSize = size; + await sleep(POLL_MS); + waited += POLL_MS; +} +if (waited) log(`${event} waited ${waited}ms for the file to settle`); + +const config = resolveConfig(); +if (!isConfigured(config)) { + log("skip: not configured"); + process.exit(0); +} +let buffer; +try { + buffer = decompressZstdContainer(fs.readFileSync(transcriptPath)); +} catch (exc) { + log(`${event} zstd decompress failed: ${exc} (${transcriptPath})`); + process.exit(0); +} + +// --- transcript mode: the server parses the records into typed events --- +const forced = (process.env.OCTOP_CAPTURE_MODE || "auto").trim().toLowerCase(); +const asTurns = + forced === "text" ? true : forced === "transcript" ? false : await supportsAssistant(config); +if (!asTurns) { + await capture({ + config, + log, + event, + source: "dsh", + sessionId, + transcriptPath, + buffer, + sessionPrefix: "dsh:", + }); + process.exit(0); +} + +// --- text mode: one call per turn, roles in their own fields --- + +function textBlocks(content) { + if (!Array.isArray(content)) return ""; + return content + .filter((block) => block && block.type === "text" && typeof block.text === "string") + .map((block) => block.text.trim()) + .filter(Boolean) + .join("\n"); +} + +/** Bytes of ``buffer`` that end on a complete JSONL record; a torn tail waits for the next Stop. */ +function completeRecordsEnd() { + const lastNewline = buffer.lastIndexOf(0x0a); + if (lastNewline + 1 >= buffer.length) return lastNewline + 1; + const tail = buffer.subarray(lastNewline + 1).toString("utf8").trim(); + if (!tail) return buffer.length; + try { + JSON.parse(tail); + return buffer.length; + } catch { + return lastNewline + 1; + } +} + +/** + * Group the delta into turns with each turn's end offset, so the offset advances + * one shipped turn at a time. A ``user/message`` starts a turn only when its + * source kind is ``user`` — plugin snapshots, system reminders and hook + * injections share that record type; ``assistant/message`` keeps text blocks only. + */ +function turnBatches(start, end) { + const batches = []; + const closeTurn = (offset) => { + if (batches.length) batches[batches.length - 1].endOffset = offset; + }; + let cursor = start; + while (cursor < end) { + const newline = buffer.indexOf(0x0a, cursor); + const lineEnd = newline === -1 || newline >= end ? end : newline + 1; + const line = buffer + .subarray(cursor, lineEnd - (lineEnd === newline + 1 ? 1 : 0)) + .toString("utf8"); + const lineStart = cursor; + cursor = lineEnd; + if (!line.trim()) continue; + let record; + try { + record = JSON.parse(line); + } catch { + continue; // torn or non-JSON line + } + if (record?.type === "user/message") { + const kind = String(record?.data?.source?.kind || ""); + if (kind && kind !== "user") continue; + closeTurn(lineStart); + const text = textBlocks(record?.data?.content); + batches.push({ user: text ? [text] : [], assistant: [], endOffset: end }); + } else if (record?.type === "assistant/message") { + const text = textBlocks(record?.data?.message?.content); + if (!text) continue; + if (!batches.length) batches.push({ user: [], assistant: [], endOffset: end }); + batches[batches.length - 1].assistant.push(text); + } + } + closeTurn(end); + return batches.filter((batch) => batch.user.length || batch.assistant.length); +} + +const state = loadState(); +const previous = state[sessionId]; +let start = 0; +if (previous && previous.path === transcriptPath && previous.offset <= buffer.length) { + start = previous.offset; + // Only a boundary that still follows a newline is safe to resume from; a + // rewritten/compacted file falls back to a full re-send. + if (start > 0 && start < buffer.length && buffer[start - 1] !== 0x0a) start = 0; +} +const end = completeRecordsEnd(); +if (end <= start) { + log(`${event} session=${sessionId} no new records (offset=${start}/${end}) -> skip`); + process.exit(0); +} +let turns = turnBatches(start, end); +const maxTurns = Number(process.env.OCTOP_CAPTURE_MAX_TURNS ?? 25); +if (turns.length > maxTurns) turns = turns.slice(0, maxTurns); + +if (!turns.length) { + // Tool traffic only: consume it so the next Stop starts after it. + state[sessionId] = { path: transcriptPath, offset: end }; + saveState(state); + log(`${event} session=${sessionId} no user/assistant text in ${end - start} bytes -> offset advanced`); + process.exit(0); +} + +const result = await captureTurns({ + config, + log, + event, + source: "dsh", + sessionId, + transcriptPath, + turns, + sessionPrefix: "dsh:", + startOffset: start, +}); +log( + `${event} session=${sessionId} source=dsh http=${result.status} mode=text ` + + `from=${start} bytes=${end - start} turns=${turns.length} sent=${result.sent} offset=${result.offset}`, +); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/dsh/user-prompt-recall.mjs b/src/octop/infra/agents/hook_scripts/dsh/user-prompt-recall.mjs new file mode 100644 index 000000000..f40dd4398 --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/dsh/user-prompt-recall.mjs @@ -0,0 +1,97 @@ +#!/usr/bin/env node +// DSH ``UserPromptSubmit`` hook: search memory for the incoming prompt and inject +// the hits as extra context. +// +// The bridge reads context only from structured output, so the answer must be the +// ``hookSpecificOutput`` object below — plain stdout is silently dropped (unlike +// Claude Code, where the claude-code entry script's stdout becomes context). +// +// The store's full-text search is conjunctive: every term must match, so a long +// prompt whose tail is an instruction ("请只回答…") widens the query until nothing +// matches. Try the first sentence, then its shorter prefixes, then the whole +// prompt; only widen the layer to raw events as a last resort. +import { callTool, isConfigured, makeLogger, readStdinJson } from "../lib/octop-memory-client.mjs"; +import { resolveConfig } from "./memory-target.mjs"; + +const MARKER = "[octop memory search] related memories from this workspace, most relevant first:"; +const FOOTER = "若需要原文/证据，用 memory_get(path=…) 读全文；与本次工作无关则忽略。"; + +function candidateQueries(text) { + const candidates = []; + const push = (value) => { + const candidate = String(value || "").trim(); + if (candidate && !candidates.includes(candidate)) candidates.push(candidate); + }; + const firstSentence = + text.split(/[。！？!?；;\n\r]+/).map((part) => part.trim()).find(Boolean) ?? text; + push(firstSentence); + push(firstSentence.slice(0, 60)); + push(firstSentence.slice(0, 30)); + push(text.slice(0, 60)); + return candidates; +} + +async function main() { + const log = makeLogger("urecall.dsh"); + const prompt = String(readStdinJson(log).prompt || "").trim(); + if (!prompt) { + log("skip: empty prompt"); + return; + } + const config = resolveConfig(); + if (!isConfigured(config)) { + log("skip: not configured"); + return; + } + + const maxResults = Number(process.env.OCTOP_RECALL_MAX_RESULTS ?? 5); + const candidates = candidateQueries(prompt); + let hits = []; + let query = candidates[0]; + let corpus = "atom"; + for (const candidate of candidates) { + const result = await callTool(config, "memory_search", { + query: candidate, + corpus: corpus, + max_results: maxResults, + }); + hits = result.detail?.hits ?? []; + if (hits.length) { + query = candidate; + break; + } + } + if (!hits.length) { + // Fresh facts stay in the raw layer until extraction promotes them. + corpus = "all"; + const result = await callTool(config, "memory_search", { + query: candidates[0], + corpus: corpus, + max_results: maxResults, + }); + hits = result.detail?.hits ?? []; + } + log( + `prompt="${prompt.slice(0, 60)}" query="${query.slice(0, 40)}" ` + + `corpus=${corpus} hits=${hits.length}`, + ); + if (!hits.length) return; + + const lines = [MARKER]; + for (const hit of hits) { + const when = String(hit.occurred_at || "").slice(0, 19); + lines.push(`- [${hit.layer}] (${hit.role_hint ?? hit.layer}${when ? ` @ ${when}` : ""}) ${hit.snippet}`); + } + lines.push(FOOTER); + process.stdout.write( + JSON.stringify({ + hookSpecificOutput: { + hookEventName: "UserPromptSubmit", + additionalContext: lines.join("\n"), + }, + }), + ); +} + +await main(); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/kiro/stop-capture.mjs b/src/octop/infra/agents/hook_scripts/kiro/stop-capture.mjs new file mode 100644 index 000000000..58b78ad9b --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/kiro/stop-capture.mjs @@ -0,0 +1,74 @@ +#!/usr/bin/env node +// Kiro ``Stop`` hook: ship this session's ``messages.jsonl`` into octop memory. +// +// Kiro-specific: the payload has no file path (only ``sess_…``), so the session +// file is located as ``///messages.jsonl``. +import fs from "node:fs"; +import os from "node:os"; +import { capture, isConfigured, loadConfig, makeLogger, readStdinJson, sleep } from "../lib/octop-memory-client.mjs"; + +const log = makeLogger("stopcap.kiro"); +const event = "Stop"; +const sessionId = readStdinJson(log).session_id || ""; +if (!sessionId) { + log(`${event} skip: payload has no session_id`); + process.exit(0); +} + +const root = process.env.OCTOP_KIRO_SESSIONS_ROOT || `${os.homedir()}/.kiro/sessions`; +function findSessionFile(id) { + try { + for (const entry of fs.readdirSync(root, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const candidate = `${root}/${entry.name}/${id}/messages.jsonl`; + if (fs.existsSync(candidate)) return candidate; + } + } catch { + // no Kiro install / unreadable root + } + return ""; +} + +const transcriptPath = findSessionFile(sessionId); +if (!transcriptPath) { + log(`${event} skip: no transcript for ${sessionId} under ${root}`); + process.exit(0); +} + +// Kiro's Stop also fires before the file settles: wait until its size stops growing. +const POLL_MS = Number(process.env.OCTOP_CAPTURE_POLL_MS ?? 1000); +const MAX_WAIT_MS = Number(process.env.OCTOP_CAPTURE_MAX_WAIT_MS ?? 20000); +const deadline = Date.now() + MAX_WAIT_MS; +let previousSize = -1; +let waited = 0; +while (Date.now() < deadline) { + let size; + try { + size = fs.statSync(transcriptPath).size; + } catch { + break; + } + if (size === previousSize) break; + previousSize = size; + await sleep(POLL_MS); + waited += POLL_MS; +} +if (waited) log(`${event} waited ${waited}ms for the file to settle`); + +const config = loadConfig(); +if (!isConfigured(config)) { + log("skip: not configured"); + process.exit(0); +} +const buffer = fs.readFileSync(transcriptPath); +await capture({ + config, + log, + event, + source: "kiro", + sessionId, + transcriptPath, + buffer, + sessionPrefix: "kiro:", +}); +process.exit(0); diff --git a/src/octop/infra/agents/hook_scripts/lib/octop-memory-client.mjs b/src/octop/infra/agents/hook_scripts/lib/octop-memory-client.mjs new file mode 100644 index 000000000..915c3ab16 --- /dev/null +++ b/src/octop/infra/agents/hook_scripts/lib/octop-memory-client.mjs @@ -0,0 +1,362 @@ +// Shared transport for every octop-memory host hook. +// +// Host-independent on purpose: this module knows how to talk to the memory MCP +// (config, auth, incremental offsets, logging, gzip, DSH's zstd container) and +// nothing about *which* terminal called it. Each host gets its own entry script +// under ``/`` that decides what to read and which ``source`` / +// ``session_id`` prefix to use — so changing one host never touches another. +// +// Config precedence: environment variable -> ``config.json`` next to the entry +// script (the /mcp/memory/setup installer writes that file, so a hooked-up +// terminal needs no secret in its hook command line). +import fs from "node:fs"; +import os from "node:os"; +import zlib from "node:zlib"; +import { fileURLToPath } from "node:url"; +import path from "node:path"; + +const ENTRY_DIR = path.dirname(fileURLToPath(import.meta.url)); +const HOOK_DIR = path.dirname(ENTRY_DIR); // .../hook_scripts (or the host dir) + +// Where this hook may write. Claude Code runs hooks unsandboxed, so the hook +// directory next to the scripts works. DSH runs the same command hooks inside a +// workspace-write sandbox whose writable roots are the session workspace, /tmp +// and the process temp dir: a $HOME hook directory stays readable there but every +// log/state write is denied — silently, because both are best-effort. Probe once +// and fall back to a private temp directory so the incremental offset and the +// diagnostic log survive in both worlds. +function resolveScratchDir() { + for (const dir of [HOOK_DIR, `${os.tmpdir()}/octop-hooks`]) { + const probe = `${dir}/.octop-write-probe`; + try { + fs.mkdirSync(dir, { recursive: true, mode: 0o700 }); + fs.writeFileSync(probe, ""); + fs.unlinkSync(probe); + return dir; + } catch { + // not writable here -> try the next candidate + } + } + return HOOK_DIR; +} + +export const paths = (() => { + const scratchDir = resolveScratchDir(); + if (scratchDir !== HOOK_DIR) { + process.stderr.write( + `octop-memory: ${HOOK_DIR} 不可写，日志与增量状态改写到 ${scratchDir}\n`, + ); + } + return { + hookDir: HOOK_DIR, + configPath: `${HOOK_DIR}/config.json`, + scratchDir, + logPath: `${scratchDir}/octop-hook.log`, + statePath: `${scratchDir}/octop-transcript-state.json`, + }; +})(); + +export function loadConfig() { + let fromFile = {}; + try { + fromFile = JSON.parse(fs.readFileSync(paths.configPath, "utf8")); + } catch { + // no config file (env-only setups keep working) + } + return { + endpoint: process.env.OCTOP_MEMORY_MCP_URL || fromFile.url || "", + token: process.env.OCTOP_MEMORY_MCP_TOKEN || fromFile.token || "", + agentId: process.env.OCTOP_AGENT_ID || fromFile.agentId || "", + userId: process.env.OCTOP_USER_ID || fromFile.userId || "", + // Configurable header name; ``Authorization`` (the default) takes a bearer. + tokenHeader: process.env.OCTOP_TOKEN_HEADER || fromFile.tokenHeader || "Authorization", + }; +} + +function baseHeaders(config) { + const headers = { + "X-Octop-Agent-Id": config.agentId, + "Content-Type": "application/json", + Accept: "application/json, text/event-stream", + }; + if (config.userId) headers["X-Octop-User-Id"] = config.userId; + return headers; +} + +export function authHeaders(config) { + // One configurable header carries the token: ``Authorization`` (the default) + // takes ``Bearer ``, any other header name is written verbatim. + const headers = baseHeaders(config); + const header = config.tokenHeader || "Authorization"; + headers[header] = header.toLowerCase() === "authorization" ? `Bearer ${config.token}` : config.token; + return headers; +} + +/** + * Whether these hooks have somewhere to send to: an endpoint, an agent, and a token. + */ +export function isConfigured(config) { + return !!config.endpoint && !!config.agentId && !!config.token; +} + +/** One logger per host: ``tag`` is what shows up in the shared log file. */ +export function makeLogger(tag) { + return (line) => { + try { + fs.appendFileSync(paths.logPath, `[${new Date().toISOString()}] ${tag}: ${line}\n`, { + mode: 0o600, + }); + } catch { + // logging must never break the hook + } + }; +} + +export function loadState() { + try { + return JSON.parse(fs.readFileSync(paths.statePath, "utf8")); + } catch { + return {}; + } +} + +export function saveState(state) { + try { + fs.writeFileSync(paths.statePath, JSON.stringify(state), { mode: 0o600 }); + } catch { + // a lost offset only costs one redundant (idempotent) upload + } +} + +/** Read the hook payload from stdin (the host hands it over as JSON). */ +export function readStdinJson(log) { + try { + const raw = fs.readFileSync(0, "utf8"); + return raw.trim() ? JSON.parse(raw) : {}; + } catch { + log("stdin-not-json"); + return {}; + } +} + +// Decode DSH's session container: it is a concatenation of independently +// decodable Zstandard frames (one header frame, then one frame per durable +// batch). Node's zstd API stops after the first frame, so a whole-file +// ``zstdDecompressSync`` silently returns only the session header. Walk the +// frames structurally and decode each complete one; a torn final frame is left +// for the next Stop to pick up. +export function decompressZstdContainer(buffer) { + const ZSTD_MAGIC = 0xfd2fb528; + const parts = []; + let offset = 0; + while (offset < buffer.length) { + const start = offset; + if (buffer.length - offset < 4) break; + if (buffer.readUInt32LE(offset) !== ZSTD_MAGIC) { + throw new Error(`invalid Zstandard frame magic at byte ${offset}`); + } + offset += 4; + if (offset === buffer.length) break; + const descriptor = buffer.readUInt8(offset); + offset += 1; + if ((descriptor & 0x18) !== 0) { + throw new Error(`reserved Zstandard frame-header bit at byte ${offset - 1}`); + } + const contentSizeFlag = descriptor >>> 6; + const singleSegment = (descriptor & 0x20) !== 0; + const checksum = (descriptor & 0x04) !== 0; + const dictionaryFlag = descriptor & 0x03; + const dictionaryBytes = dictionaryFlag === 3 ? 4 : dictionaryFlag; + const contentSizeBytes = contentSizeFlag === 0 ? (singleSegment ? 1 : 0) : 1 << contentSizeFlag; + const remainingHeaderBytes = (singleSegment ? 0 : 1) + dictionaryBytes + contentSizeBytes; + if (buffer.length - offset < remainingHeaderBytes) break; + offset += remainingHeaderBytes; + + let torn = false; + for (;;) { + if (buffer.length - offset < 3) { + torn = true; + break; + } + const blockHeader = buffer.readUIntLE(offset, 3); + offset += 3; + const lastBlock = (blockHeader & 1) !== 0; + const blockType = (blockHeader >>> 1) & 0x03; + const blockSize = blockHeader >>> 3; + if (blockType === 0x03) throw new Error(`reserved Zstandard block type at byte ${offset - 3}`); + const payloadBytes = blockType === 0x01 ? 1 : blockSize; + if (buffer.length - offset < payloadBytes) { + torn = true; + break; + } + offset += payloadBytes; + if (lastBlock) break; + } + if (torn) break; + if (checksum) { + if (buffer.length - offset < 4) break; + offset += 4; + } + parts.push(zlib.zstdDecompressSync(buffer.subarray(start, offset))); + } + return Buffer.concat(parts); +} + +/** Call one memory MCP tool; returns ``{ok, status, detail, raw}`` (never throws). */ +export async function callTool(config, name, args) { + return rpc(config, "tools/call", { name, arguments: args }); +} + +/** One raw JSON-RPC call; returns ``{ok, status, detail, error, raw}`` (never throws). */ +export async function rpc(config, method, params) { + const response = await fetch(config.endpoint, { + method: "POST", + headers: authHeaders(config), + body: JSON.stringify({ jsonrpc: "2.0", id: 1, method, params }), + }); + const text = await response.text(); + let payload = text; + for (const line of text.split("\n")) { + if (line.startsWith("data: ")) { + payload = line.slice(6); + break; + } + } + let parsed = {}; + try { + parsed = JSON.parse(payload.slice(payload.indexOf("{"), payload.lastIndexOf("}") + 1)); + } catch { + // keep the raw body for diagnostics + } + return { + ok: response.ok, + status: response.status, + detail: parsed?.result?.structuredContent ?? parsed?.result ?? {}, + error: parsed?.error?.message ?? "", + raw: text, + }; +} + +/** + * Whether this endpoint's ``memory_capture`` takes the assistant side as its own + * field (``assistant``) instead of expecting one merged ``content``. + */ +export async function supportsAssistant(config) { + const result = await rpc(config, "tools/list", {}); + const tool = (result.detail?.tools ?? []).find((entry) => entry?.name === "memory_capture"); + return Object.prototype.hasOwnProperty.call(tool?.inputSchema?.properties ?? {}, "assistant"); +} + +/** + * Ship already-grouped conversation turns with the roles in their own fields: + * ``content`` = the user side, ``assistant`` = the assistant side. + * + * Each accepted turn advances the per-``session_id`` offset, so a failure or a + * cap resumes at the next turn instead of re-sending the whole session. + */ +export async function captureTurns(ctx) { + const { config, log, event, source, sessionId, transcriptPath, turns, sessionPrefix } = ctx; + const state = loadState(); + let sent = 0; + let status = 0; + let offset = ctx.startOffset ?? 0; + for (const turn of turns) { + const args = { content: turn.user.join("\n\n"), source }; + if (sessionId) args.session_id = `${sessionPrefix}${sessionId}`; + if (turn.assistant.length) args.assistant = turn.assistant.join("\n\n"); + const result = await callTool(config, "memory_capture", args); + status = result.status; + if (!result.ok) { + log( + `${event} session=${sessionId} turn ${sent + 1} http=${status} ` + + `err=${result.detail?.error ?? result.error ?? ""} -> offset kept at ${offset}`, + ); + break; + } + sent += 1; + offset = turn.endOffset; + if (sessionId) { + state[sessionId] = { path: transcriptPath, offset }; + saveState(state); + } + } + return { sent, status, offset }; +} + +/** + * Ship the *new* bytes of a session file to ``memory_capture``. + * + * The offset is per ``session_id`` and only advances after a 2xx; a failed + * upload is retried in full next time, and the server dedupes on record keys, so + * re-sending is free. + */ +export async function capture(ctx) { + const { config, log, event, source, sessionId, transcriptPath, buffer, sessionPrefix } = ctx; + const state = loadState(); + const previous = sessionId ? state[sessionId] : undefined; + + let start = 0; + if (previous && previous.path === transcriptPath && previous.offset <= buffer.length) { + start = previous.offset; + // Only a boundary that still follows a newline is safe to resume from; a + // rewritten/compacted file falls back to a full (idempotent) re-send. + if (start > 0 && start < buffer.length && buffer[start - 1] !== 0x0a) start = 0; + } + if (start >= buffer.length) { + log(`${event} session=${sessionId} no new bytes (offset=${start}) -> skip`); + return { skipped: true }; + } + + const delta = buffer.subarray(start); + const whole = start === 0; + let blob; + try { + blob = zlib.gzipSync(delta).toString("base64"); + } catch (exc) { + log(`${event} gzip failed: ${exc}`); + return { failed: true }; + } + + const args = { content: blob, source }; + if (sessionId) args.session_id = `${sessionPrefix}${sessionId}`; + // Opt-in only: the server default is "none" (tool traffic is most of a + // transcript's bytes and distorts both extraction and raw-layer recall), so set + // this to "all"/"calls" only for a terminal that needs the full record. + if (process.env.OCTOP_TOOL_EVENTS) args.tool_events = process.env.OCTOP_TOOL_EVENTS; + + try { + const result = await callTool(config, "memory_capture", args); + const detail = result.detail ?? {}; + log( + `${event} session=${sessionId} source=${source} http=${result.status} ` + + `mode=${whole ? "full" : "delta"} from=${start} bytes=${delta.length} gz=${blob.length} ` + + `parsed=${detail.parsed} imported=${detail.imported} skipped=${detail.skipped_existing} ` + + `extract=${detail.extract_scheduled} err=${detail.error ?? result.error ?? ""}`, + ); + if (result.ok && sessionId) { + state[sessionId] = { path: transcriptPath, offset: buffer.length }; + saveState(state); + } + return { status: result.status, detail }; + } catch (exc) { + log(`${event} upload failed: ${exc}`); + return { failed: true }; + } +} + +/** Latest mtime order helper for hosts that discover their session file. */ +export function newestFile(dir, pattern) { + try { + return fs + .readdirSync(dir) + .filter((name) => pattern.test(name)) + .map((name) => ({ name, path: `${dir}/${name}`, mtime: fs.statSync(`${dir}/${name}`).mtimeMs })) + .sort((a, b) => b.mtime - a.mtime)[0]?.path ?? ""; + } catch { + return ""; + } +} + +export function sleep(ms) { + return new Promise((resolve) => setTimeout(resolve, ms)); +} diff --git a/src/octop/infra/agents/memory_mcp.py b/src/octop/infra/agents/memory_mcp.py new file mode 100644 index 000000000..a65998c38 --- /dev/null +++ b/src/octop/infra/agents/memory_mcp.py @@ -0,0 +1,1878 @@ +"""Expose Octop expert memory as an MCP server for external agents. + +External agents (coding agents, bots) can read/write Octop expert memory over +MCP (Streamable HTTP), aligned with the in-process ``MemoryService`` +capabilities. Every write stamps a ``source`` marker that can be traced back +on recall. + +Expert binding: the endpoint is a single ``/mcp/memory`` mount; the expert is +selected at connect time via the ``X-Octop-Agent-Id`` header (one connection +binds one expert — the caller never passes an agent id per tool call). + +raw vs atom (aligned with ``MemoryService``): + +* ``memory_capture`` -> ``add_raw``: writes an **L0 raw event**, which goes + through the extraction pipeline (extract -> candidate -> promote -> atom). + Use it to record raw conversations / events. The record is visible + immediately via ``memory_raws`` / ``memory_search(corpus="raw")``; the + regular search returns it only after extraction promotes it to an atom. +* ``memory_save`` -> ``store``: persists a structured fact directly into the + canonical atom/tree (durable, no extraction). Use it when you already know + the exact fact to remember. + +Read surface (three tools, same storage as the in-process ``memory_search`` / +``memory_get`` exposed to Octop's own agents): + +* ``memory_search`` -> ``MemoryRuntime.memory_search`` (``corpus=raw`` uses + ``Memory.search_raw``): ranked hits that carry a virtual ``path`` instead of + rendered markdown. L2 atoms first; ``page`` headlines are folded into atom + hits; L0 raw is only a fallback (dropped as soon as any atom matches). +* ``memory_raws`` -> L0 inspection: the raw events themselves, newest first. +* ``memory_get`` -> ``MemoryRuntime.memory_get``: resolve that path to the full + markdown (``atom/.md`` / ``page/.md`` / ``raw//.md``). + +Recall echo guard: ``memory_capture`` drops content carrying a marker from +``_SKIP_CONTENT_MARKERS``, the same rule ``MemoryRuntime.capture`` applies via +``skip_memory_echo`` — the MCP write path calls ``Memory.add_raw`` directly and +would otherwise capture a hook's own injected memory block as a new event. + +Content markers: everything this service puts back into a conversation is tagged +``[octop memory ]`` — today ``search`` / ``get`` / ``capture`` (the retired +``recall`` tool shares the prefix), emitted as the ``marker`` field of the tool +response and as the first line of the blocks our hooks inject (the hook prefixes +``[octop memory search]``). Filtering is +therefore one rule: drop anything containing ``_OWN_CONTENT_PREFIX``. Older markers +(``## Memory Recall`` / ``[memory] Earlier in this workspace`` / ``[octop-memory +hook]``) stay in the list because transcripts already on disk carry them. + +Record shape: ``source`` decides what ``memory_capture`` received. A terminal name +(``claude-code``, later ``kiro`` / ``dsh``) means ``content`` is a whole session +transcript, parsed record by record. Anything else means one conversation turn: +``content`` is the user side, ``assistant`` the reply — mirroring in-process +``MemoryService.capture_turn``, so the extractor sees both sides in one batch. +The extractor prompt only receives ``event_id / event_type / content / timestamp / +session_id``, so the role has to come from the data: in transcript mode each record +carries it, in one-turn mode the field the text was passed in does. Role-less +``manual`` content would be read as a user-stated fact — which is why the old +hard-coded ``manual`` had to go. + +Sender attribution: ``X-Octop-User-Id`` identifies the caller, and the one-turn +write path (``memory_capture`` / ``memory_save``) prefixes the content with +``说：``. harness-memory's ``AtomCard`` has no user column, and the extractor +prompt only serialises ``event_id / event_type / content / timestamp / session_id`` +— never the event's ``user`` — so putting the sender into the text is what makes it +reach the atom, stay FTS-searchable (a query naming the sender matches), and remain +visible on recall. + +Naming: the ``user`` parameter is the caller **identity** (kept from the original +tool surface), *not* the user's utterance — the human side is ``content``. Upstream +``MemoryService.capture_turn`` names them the other way round (``user`` = message +content, ``user_id`` = identity); aligning with it would be a breaking rename, so +this tool deliberately keeps the historical names. + +Only the *user* side is prefixed. An ``assistant`` side is the assistant's own +words and already identified by its ``event_type``; stamping it with ``说：`` +would record the assistant's reply as if the user had said it. A whole transcript +takes the terminal path instead, where every record keeps its own role and no +prefix is added at all. + +Auth: independent token via ``OCTOP_MEMORY_MCP_TOKEN`` (fail-closed when +unset), enforced by the ASGI middleware in ``mount_memory_mcp``. +""" + +from __future__ import annotations + +import logging +import os +from collections.abc import Callable +from contextvars import ContextVar +from typing import Any, get_args + +from mcp.server.fastmcp import FastMCP +from mcp.server.fastmcp.server import Context +from mcp.server.transport_security import TransportSecuritySettings + +from octop.infra.agents.memory.backend import open_memory_kwargs +from octop.infra.server import OctopServer + +logger = logging.getLogger(__name__) + +# 当前 MCP HTTP 请求的绑定状态（由 _AgentRouter 中间件写入，工具读取）。 +# stateless streamable HTTP 下 mcp SDK 不提供 ctx.request_context，故用 contextvar +# 跨 ASGI 中间件 → 工具传递： +# - _current_agent_id: 本次请求绑定的 expert（X-Octop-Agent-Id 校验后写入） +# - _current_caller_user: 调用者 user id（供 memory_capture/save 做 per-user 追溯） +_current_agent_id: ContextVar[str] = ContextVar("octop_mcp_agent_id", default="") +_current_caller_user: ContextVar[str] = ContextVar("octop_mcp_caller_user", default="") + +_SEARCH_CORPORA: frozenset[str] = frozenset({"all", "memory", "atom", "raw"}) +"""Corpora ``memory_search`` accepts. ``atom``/``raw`` are layer filters over the +recall pipeline; ``all``/``memory`` are the same pipeline without a filter.""" + +_SNIPPET_CHARS = 200 + +_TRANSCRIPT_SOURCES: dict[str, str] = { + "claude-code": "claude-code", + "kiro": "kiro", + "dsh": "dsh", + "codex": "codex", + "cursor": "cursor", +} +_TRANSCRIPT_PATH_ROOTS: tuple[str, ...] = ( + "~/.claude/projects", + "~/.kiro/sessions", + "~/.dsh/sessions", + "~/.codex/sessions", + "~/.cursor", +) +"""Where ``memory_capture(transcript_path=…)`` may read from. + +A terminal's ``mcp_tool`` hook can only pass the path (Claude Code substitutes +``${transcript_path}``), and that path is readable only by an MCP server running on +the same machine. The allow-list keeps that convenience from becoming a general +arbitrary-file read: anything outside these terminal session stores is refused. +""" + + +"""``source`` values whose payload is a whole session transcript, not one turn. + +``source`` names the terminal, and the terminal decides the parser — so callers +pass only one of the two. Unknown sources mean plain text (one turn per call). +DSH sessions are zstd-compressed, so the client decompresses before uploading. +""" + +_GZIP_B64_PREFIX = "H4sI" +"""``base64(gzip(…))`` always starts with this; a JSONL body always starts with ``{``. + +Only consulted in transcript mode, so a plain-text capture can never be misread +as a compressed payload. +""" + +_OWN_CONTENT_PREFIX = "[octop memory " +"""Uniform marker for anything this service puts back into a conversation. + +Emitted as the ``marker`` field of the read/write tool responses a client may quote +back (``[octop memory search|get|capture|recall]``) and as the first line of blocks +our hooks inject. One substring rule — this prefix — recognizes all of them, +including tools added later. +""" + +_LEGACY_ECHO_MARKERS: tuple[str, ...] = ( + "## Memory Recall", + "[memory] Earlier in this workspace", + "[octop-memory hook]", +) +"""Older markers still present in transcripts already on disk. + +``## Memory Recall`` / ``[memory] Earlier in this workspace`` are what the upstream +recall render emits (and what ``MemoryRuntime.capture`` filters on); +``[octop-memory hook]`` is the pre-2026 hooks' instruction text. Kept so backfilling +old sessions filters exactly like the one-turn rule always did. +""" + +_SKIP_CONTENT_MARKERS: tuple[str, ...] = (_OWN_CONTENT_PREFIX, *_LEGACY_ECHO_MARKERS) +"""Content carrying any of these is our own plumbing, never L0 evidence.""" + +_HOST_PLUMBING_MARKERS: tuple[str, ...] = ( + "", + "", + "Base directory for this skill:", +) +"""Text the terminal injects on its own account: instructions to itself and the +bundled skill documentation it pastes in. Not something the user said, and not this +service's output either — but the same kind of noise, so a transcript import drops it +**when it appears in a message event**. The same string inside a tool result is data +(the file that was read may quote it), so that check stays narrower. + +Slash-command wrappers (```` / ```` / ````) +and ```` are deliberately **kept**: they record a user action and +its effect (``/model`` → which model got selected), which is worth remembering. + +``_TRANSCRIPT_SKIP_MARKERS`` is what a transcript import actually applies.""" + +_TRANSCRIPT_SKIP_MARKERS: tuple[str, ...] = _SKIP_CONTENT_MARKERS + _HOST_PLUMBING_MARKERS +"""Everything a transcript import drops — kept for callers that filter themselves.""" + + +def _looks_like_transcript(text: str) -> bool: + """True when *text* plausibly is JSONL rather than one piece of prose. + + Terminal names are also free-form ``source`` values, so ``source="claude-code"`` + with a sentence must keep recording that sentence (as it did before transcripts + existed) instead of being parsed into zero events and dropped. + """ + return text.lstrip().startswith("{") + + +USER_ROLE = "user_message" +ASSISTANT_ROLE = "assistant_message" +"""The two roles one-turn capture writes. Only ``user_message`` carries the sender +prefix — the assistant side is already identified by its role.""" + + +def _read_transcript_file(raw_path: str) -> str: + """Read a transcript file handed over by a local ``mcp_tool`` hook. + + Raises ``ValueError`` when the path is outside the terminal session stores, is + compressed (a hook script ships those as gzip+base64 instead), or is unreadable. + """ + from pathlib import Path as _Path # noqa: PLC0415 + + candidate = _Path(raw_path).expanduser() + try: + resolved = candidate.resolve(strict=True) + except OSError as exc: + raise ValueError(f"transcript_path is not readable: {raw_path!r} ({exc})") from exc + allowed = [(_Path(root).expanduser().resolve()) for root in _TRANSCRIPT_PATH_ROOTS] + if not any(resolved == root or root in resolved.parents for root in allowed): + raise ValueError( + "transcript_path must live under a terminal session store " + f"({', '.join(_TRANSCRIPT_PATH_ROOTS)}); got {raw_path!r}" + ) + if resolved.suffix == ".zstd": + raise ValueError( + "compressed transcripts must be sent as gzip+base64 content " + "(the terminal's hook script does this); transcript_path supports plain JSONL" + ) + return resolved.read_text(encoding="utf-8", errors="replace") + + +def _resolve_agent_scope(server: OctopServer, agent_id: str) -> tuple[Any, dict[str, Any]]: + """The agent's workspace directory + its parsed ``config_json``. + + Shared by the memory backend resolution and the raw-session archive (both live + under that workspace; the archive under its system dir, ``{workspace}/.octop/``). + """ + services = server.services + assert services is not None, "server.services required for memory backend" + runtime = getattr(server, "app_runtime", None) + registry = getattr(runtime, "agent_registry", None) if runtime is not None else None + if registry is not None and hasattr(registry, "resolve_workspace_dir"): + workspace = registry.resolve_workspace_dir(agent_id) + else: + paths = getattr(server, "paths", None) or services.paths + workspace = paths.ensure_agent_workspace(agent_id) + + row = services.agent_repo.get(agent_id) + cfg: dict[str, Any] = {} + if row is not None and row.config_json: + import json # noqa: PLC0415 + + try: + parsed = json.loads(row.config_json) + if isinstance(parsed, dict): + cfg = parsed + except json.JSONDecodeError: + cfg = {} + return workspace, cfg + + +def _archive_uploaded_session( + server: OctopServer, + agent_id: str, + *, + text: str, + source: str, + session_id: str, + user: str | None, +) -> str | None: + """Append this upload to the agent's markdown archive of the raw session. + + L0 only keeps user / assistant messages, so the tool traffic and the scratchpad + would otherwise be gone; the archive keeps the whole uploaded record for review at + ``{workspace}/.octop/memory-sessions/.md``. Best-effort by design — a + read-only or full disk must not fail the capture. + """ + try: + from octop.infra.agents.memory_session_archive import ( # noqa: PLC0415 + ARCHIVE_DIRNAME, + append_session, + ) + from octop.infra.agents.workspace import host_system_dir # noqa: PLC0415 + + workspace, cfg = _resolve_agent_scope(server, agent_id) + session_dir = host_system_dir(workspace, cfg) / ARCHIVE_DIRNAME + path = append_session(session_dir, session_id=session_id, text=text, host=source, user=user) + return str(path) if path else None + except Exception: # noqa: BLE001 - the archive must never break capture + logger.warning("memory_capture: session archive failed", exc_info=True) + return None + + +def _open_memory(server: OctopServer, agent_id: str) -> Any: + """Open the agent's ``Memory`` instance (sqlite by default, postgres opt-in). + + Mirrors ``api.common.memory_client._open_memory_for_agent`` but stays in + ``infra/`` (no api dependency). Workspace is resolved from the agent + registry, falling back to the Octop default layout. + """ + from octop_memory.core import Memory # noqa: PLC0415 + + services = server.services + assert services is not None, "server.services required for memory backend" + workspace, cfg = _resolve_agent_scope(server, agent_id) + + ns, backend, backend_config = open_memory_kwargs( + agent_id=agent_id, + cfg=cfg, + octop_config=services.config, + workspace_dir=workspace, + ) + return Memory(namespace=ns, backend=backend, backend_config=backend_config) + + +def _dashboard_bridge(memory: Any) -> Any: + """Open the harness-memory JSON-RPC bridge over ``memory``. + + This is the same adapter the memory panel reaches through + ``api/common/memory_client.call_memory_rpc`` (``Bridge`` → + ``octop_memory.application.dashboard_data``). Routing the MCP write tools + through it keeps one implementation behind both entry points, so the MCP and + the dashboard can never drift apart on candidate review semantics. The + import stays in ``infra/`` on purpose: ``octop_memory`` directly, never + ``octop.api``. + """ + from octop_memory.adapters.bridge.handlers import Bridge # noqa: PLC0415 + + return Bridge(memory) + + +def _bridge_rpc( + bridge: Any, + method: str, + params: dict[str, Any], +) -> dict[str, Any]: + """Call one dashboard RPC and return its result, raising ``ValueError`` on error. + + The message is the bridge's own, i.e. the very text the memory panel shows + when the identical call fails there. + """ + + def _invoke() -> dict[str, Any]: + response = bridge.handle({"jsonrpc": "2.0", "id": 1, "method": method, "params": params}) + error = response.get("error") + if error: + raise ValueError(error.get("message") or f"memory rpc {method!r} failed") + return response.get("result") or {} + + return _invoke() + + +def _snippet(text: str) -> str: + """Cap a raw event body so ``memory_search`` hits stay small (``memory_get`` reads the rest).""" + body = (text or "").strip() + return body if len(body) <= _SNIPPET_CHARS else body[: _SNIPPET_CHARS - 1].rstrip() + "…" + + +def _is_recall_echo(content: str) -> bool: + """True when ``content`` is our own memory block coming back as a new event. + + Mirrors ``MemoryRuntime.capture``'s anti-feedback rule; without it a hook that + injects ``memory_search`` output and then captures the turn via MCP would feed the + injected block back in, and each round would surface (and re-capture) more of it. + """ + return any(marker in content for marker in _SKIP_CONTENT_MARKERS) + + +def _attributed(content: str, caller: str) -> str: + """Prefix the sender so the caller id lives inside the text. + + harness-memory's ``AtomCard`` has no user column, so the sender is stamped by + writing ``说：`` into the content itself: it then reaches the atom's + assertion (manual writes) or its raw event (captured events), stays + FTS-searchable, and shows up verbatim on recall. One implementation, also used by + the transcript importer (``claude_transcript.attribute_user``). + """ + from octop.infra.agents.claude_transcript import attribute_user # noqa: PLC0415 + + return attribute_user(content, caller) + + +def _lookup_raw_users(memory: Any, hits: list[dict[str, Any]]) -> dict[str, str]: + """``{raw event id: user}`` for the raw hits — the raw layer is where the sender lives. + + Atoms / pages have no user column (the sender is written into the body as + ``说：`` when it is captured), so a batched raw lookup is how a raw hit gets + attributed. Best-effort: a failure only costs the ordering hint. + """ + ids = [ + str(hit.get("source_id")) + for hit in hits + if hit.get("layer") == "raw" and hit.get("source_id") + ] + if not ids: + return {} + try: + events = memory.get_raw_events_by_ids(list(dict.fromkeys(ids))) + except Exception: # noqa: BLE001 - attribution must never break search + logger.warning("memory_search: raw attribution lookup failed", exc_info=True) + return {} + return {str(event.id): str(event.user or "") for event in events} + + +def _is_personal_hit(hit: dict[str, Any], caller: str, raw_users: dict[str, str]) -> bool: + """Whether this hit looks like it belongs to *caller*.""" + if hit.get("layer") == "raw": + return raw_users.get(str(hit.get("source_id")), "") == caller + # Atoms / pages: the capture-time ``说：`` prefix is the only sender signal. + return f"{caller}说：" in str(hit.get("snippet") or "") + + +def _prefer_personal(memory: Any, hits: list[dict[str, Any]], caller: str) -> list[dict[str, Any]]: + """Stable-partition hits: *caller*'s first, then everyone else's. + + A preference, not a filter — memory is team-shared, so another person's hit is + still returned, just after this caller's own. Each hit is annotated with + ``personal``; raw hits also carry the raw event's ``user``. + """ + raw_users = _lookup_raw_users(memory, hits) + mine: list[dict[str, Any]] = [] + rest: list[dict[str, Any]] = [] + for hit in hits: + personal = _is_personal_hit(hit, caller, raw_users) + hit["personal"] = personal + if hit.get("layer") == "raw" and str(hit.get("source_id")) in raw_users: + hit["user"] = raw_users[str(hit.get("source_id"))] or None + (mine if personal else rest).append(hit) + return [*mine, *rest] + + +def _prefer_personal_events(events: list[Any], caller: str) -> list[Any]: + """Same preference for raw *events* (``corpus=raw``): own rows first, stable order.""" + mine = [event for event in events if str(getattr(event, "user", "") or "") == caller] + rest = [event for event in events if str(getattr(event, "user", "") or "") != caller] + return [*mine, *rest] + + +def _pipeline_hits( + memory: Any, + query: str, + max_results: int, + *, + atom_only: bool, + caller: str = "", +) -> list[dict[str, Any]]: + """Run the in-process multi-source search and return its path-carrying hits. + + ``atom_only`` keeps just L2 atoms, so the request is widened first — + otherwise raw hits could crowd the atoms out before the filter runs. A *caller* + widens it too: the caller's hits are pulled to the front after the fetch, so the + pool must be bigger than the final page. + """ + from octop_memory.application.runtime import MemoryRuntime # noqa: PLC0415 + + runtime = MemoryRuntime(memory) + result = runtime.memory_search( + { + "query": query, + "maxResults": max_results * 4 if (atom_only or caller) else max_results, + "corpus": "memory", + } + ) + hits = list(result.get("hits") or []) + if atom_only: + hits = [hit for hit in hits if hit.get("layer") == "atom"] + if caller: + hits = _prefer_personal(memory, hits, caller) + return hits[:max_results] + + +def build_memory_mcp(server: OctopServer) -> FastMCP: + """Build the shared memory MCP app (expert bound per request, not per build). + + The expert is selected at request time by ``X-Octop-Agent-Id`` (validated + against the agent repo by ``_AgentRouter``) and carried to the tools via the + ``_current_agent_id`` contextvar. A single app is shared by every expert, so + agents created or disabled after process start are honored immediately — + no process restart is needed to pick up new agents. + """ + mcp = FastMCP( + "octop-memory", + # Stateless streamable HTTP: every request gets a fresh transport, no + # Mcp-Session-Id tracking. Session state is in-memory per process, so a + # server restart silently orphans every client session id and the next + # tool call fails with -32600 "Session not found". Stateless mode + # eliminates that failure class entirely (clients re-initialize per + # request); the cost is one extra initialize per tool call. + stateless_http=True, + # Octop runs behind a reverse proxy (Host is the public domain, forwarded + # by nginx), not a localhost dev scenario — the mcp SDK's localhost + # DNS-rebinding protection does not apply and would reject the Host + # with 421 unless the domain is allow-listed. + transport_security=TransportSecuritySettings(enable_dns_rebinding_protection=False), + ) + # Collapse the streamable-HTTP path to "/" so the endpoint is exactly + # /mcp/memory (the default "/mcp" would make it /mcp/memory/mcp). + mcp.settings.streamable_http_path = "/" + + def _agent_id() -> str: + """Agent bound to this request (set by ``_AgentRouter`` from the header).""" + agent_id = _current_agent_id.get() + if not agent_id: + raise RuntimeError("X-Octop-Agent-Id header not bound to this request") + return agent_id + + def _memory() -> Any: + return _open_memory(server, _agent_id()) + + def _caller_user(ctx: Any | None) -> str: + """读取当前 MCP 请求的调用者 user id。 + + 优先级：显式 ``user`` 参数 → ``X-Octop-User-Id`` header（由 + ``_AgentRouter`` 中间件写入 contextvar）。stateless HTTP 下 mcp SDK + 不提供 ``ctx.request_context``，故不依赖它。 + """ + try: + return _current_caller_user.get() or "" + except Exception: # noqa: BLE001 + return "" + + def _derive_session(source: str, user: str) -> str: + """外部调用缺省 session_id 时派生稳定会话键。 + + 规则 ``ext:{source}:{user}``：同 source 同 user 的多次 capture 落入 + 同一分组，harness 提取管线能聚合蒸馏成 atom；不同 source / 不同 user + 分开分组，避免混入彼此上下文。 + """ + return f"ext:{source or 'mcp'}:{user or 'anon'}" + + def _observe_tool( + tool: str, + *, + session_id: str | None = None, + input_payload: dict[str, Any] | None = None, + metadata: dict[str, Any] | None = None, + fn: Callable[[], dict[str, Any]], + ) -> dict[str, Any]: + """Run one tool body. + + This fork ships no tracing backend, so *tool*, *session_id*, + *input_payload* and *metadata* are accepted (and ignored) to keep every + tool call site shaped the same as the upstream implementation. + """ + return fn() + + @mcp.tool() + def memory_search( + query: str, + max_results: int = 5, + corpus: str = "all", + user: str | None = None, + ctx: Context | None = None, # type: ignore[type-arg] + ) -> dict[str, Any]: + """**检索记忆（读入口）**：全文检索记忆，返回带虚拟路径的命中列表。 + + 每条命中带一个 ``path``，交给 ``memory_get`` 读全文；只想"把背景拉进上下文" + 就直接用命中里的 ``snippet``，要"原话/证据"用 ``memory_raws``。自动注入（hook） + 场景用的就是本工具：注入块以 ``[octop memory search]`` 开头，``memory_capture`` + 按该标记丢弃回声，所以不会被当成新事件采回来。 + + **按调用者偏好排序**：调用者 id 有值（``X-Octop-User-Id`` 头或显式 ``user``）时， + 这个人的命中排在最前、其他人的紧随其后 —— 只调顺序，**不过滤**：记忆是团队共享的， + 别人踩过的坑照样返回。归属判定：raw 命中看 raw 的 ``user`` 列；原子/主题页看正文里的 + ``<调用者>说：`` 前缀（那句话在 capture 时打进去的）。命中带 ``personal`` 布尔， + raw 命中还带 ``user``；返回体有 ``caller`` 与 ``personal_hits``。 + + Args: + query: 自然语言问题/关键词（中文会做 n-gram 分词）。 + max_results: 最多返回命中数，默认 5。 + corpus: 检索范围： + ``all``（默认）/``memory`` = 原子(L2)+原始事件(L0)同一套管线； + ``atom`` = 只要 L2 原子命中； + ``raw`` = 直接走 L0 全文检索，不受"有原子命中就丢 raw"的兜底策略影响， + 适合找刚 ``memory_capture``、还没晋升成原子的内容。 + user: 可选调用者 id（覆盖 ``X-Octop-User-Id`` 头）。 + ctx: MCP 注入的上下文（读取 ``X-Octop-User-Id`` 头）。 + """ + corpus_value = (corpus or "all").strip().lower() + if corpus_value not in _SEARCH_CORPORA: + raise ValueError( + f"invalid corpus {corpus!r}; expected one of {sorted(_SEARCH_CORPORA)}" + ) + caller = user or _caller_user(ctx) + + def _run() -> dict[str, Any]: + memory = _memory() + if corpus_value == "raw": + from octop_memory.application.path_projection import ( # noqa: PLC0415 + raw_to_path, + ) + + events = list( + memory.search_raw(query, limit=max_results * 4 if caller else max_results) + ) + if caller: + events = _prefer_personal_events(events, caller) + hits = [ + { + "path": raw_to_path(event), + "layer": "raw", + "snippet": _snippet(event.content), + "occurred_at": event.timestamp.isoformat(), + "source_id": event.id, + "user": str(event.user or "") or None, + **({"personal": str(event.user or "") == caller} if caller else {}), + } + for event in events[:max_results] + ] + else: + hits = _pipeline_hits( + memory, + query, + max_results, + atom_only=corpus_value == "atom", + caller=caller, + ) + return { + "marker": "[octop memory search]", + "hits": hits, + "total": len(hits), + "corpus": corpus_value, + "caller": caller or None, + "personal_hits": sum(1 for hit in hits if hit.get("personal")), + "hint": ( + "每条命中自带 path，可交给 memory_get(path) 读全文；" + "命中已按「本次调用者相关」优先排序（personal=true 的那批在前，团队共享内容不过滤）" + ), + } + + return _observe_tool( + "search", + input_payload={ + "query": query, + "max_results": max_results, + "corpus": corpus_value, + }, + metadata={"caller": caller or None}, + fn=_run, + ) + + @mcp.tool() + def memory_get( + path: str, + start: int | None = None, + lines: int | None = None, + ) -> dict[str, Any]: + """**读取记忆全文**：把 ``memory_search`` 命中的虚拟路径解析成 markdown。 + + 支持的路径形态：``atom/.md``（L2 原子）、``page/.md`` + （L3 主题页）、``raw//.md``（L0 原始事件）。长内容用 + ``start`` / ``lines`` 分页（配合返回的 ``total_lines`` / ``truncated``）。 + + Args: + path: 虚拟路径，取自 ``memory_search`` 的 ``hits[].path``。 + start: 可选起始行号（1-based）。 + lines: 可选返回行数。 + """ + from octop_memory.application.runtime import MemoryRuntime # noqa: PLC0415 + + def _run() -> dict[str, Any]: + params: dict[str, Any] = {"path": path} + if start is not None: + params["from"] = start + if lines is not None: + params["lines"] = lines + try: + result = MemoryRuntime(_memory()).memory_get(params) + except Exception as exc: # stale / mistyped path from a previous call + return { + "path": path, + "error": f"{exc.__class__.__name__}: {exc}", + "hint": ( + "path 形如 atom/.md / page/.md / " + "raw//.md，取自 memory_search 的 hits[].path" + ), + } + return { + "marker": "[octop memory get]", + "path": result.get("path", path), + "kind": result.get("kind"), + "content": result.get("excerpt") or "", + "total_lines": result.get("total_lines"), + "from_line": result.get("from_line"), + "to_line": result.get("to_line"), + "truncated": result.get("truncated"), + "metadata": result.get("metadata") or {}, + } + + return _observe_tool( + "get", + input_payload={"path": path, "start": start, "lines": lines}, + fn=_run, + ) + + @mcp.tool() + def memory_save( + content: str, + source: str, + topic: str | None = None, + supersedes: str | None = None, + user: str | None = None, + ctx: Context | None = None, # type: ignore[type-arg] + ) -> dict[str, Any]: + """**直接保存事实**：把一条已知事实写入原子层（跳过提取，立即可召回）。 + + 用于明确、需长期记住的事实（如用户偏好、项目约定）。日常对话内容请用 + ``memory_capture`` 交给提取管线。 + + 旧事实过时了就用 ``supersedes`` 传旧原子 id：旧原子标记 deprecated，新事实 + 立即可被 ``memory_search`` 检索到并带上 ``supersedes`` 关联——"更正事实"就是 + 保存新事实 + 声明它取代谁，不需要另一个工具。 + + 来源写入 ``metadata.source``，调用者写入 ``metadata.user``；调用者 id 还会自动 + 拼进内容前缀（``说：…``），这样它随原子一起落库、可被检索、召回时直接 + 可见——不要把名字重复写进 ``content``。 + + Args: + content: 要记住（或用来替代旧事实）的内容。 + source: 谁记录的（如 "coding-agent"），用于追溯。 + topic: 可选主题标签。 + supersedes: 可选——被这条新事实取代的旧原子 id；给了就先把它标记 deprecated。 + user: 可选调用者 id（覆盖 ``X-Octop-User-Id`` 头）。 + ctx: MCP 注入的上下文（读取 ``X-Octop-User-Id`` 头）。 + """ + caller = user or _caller_user(ctx) + + def _run() -> dict[str, Any]: + memory = _memory() + metadata: dict[str, Any] = { + "source": source, + **({"user": caller} if caller else {}), + } + deprecated: bool | None = None + if supersedes: + deprecated = memory.deprecate_atom( + supersedes, actor="user", note="superseded by memory_save" + ) + metadata["supersedes"] = supersedes + node = memory.store( + _attributed(content, caller), + topic=topic, + metadata=metadata, + ) + result: dict[str, Any] = { + "node_id": node.id, + "content": node.content, + "source": source, + "user": caller or None, + } + if supersedes: + result["deprecated"] = deprecated + result["superseded_atom_id"] = supersedes + return result + + return _observe_tool( + "save", + input_payload={ + "source": source, + "topic": topic, + "supersedes": supersedes, + "content": content, + "content_chars": len(content or ""), + }, + metadata={"caller": caller or None}, + fn=_run, + ) + + @mcp.tool() + def memory_capture( + content: str, + source: str, + session_id: str | None = None, + user: str | None = None, + assistant: str | None = None, + thread_id: str | None = None, + dry_run: bool = False, + extract: bool = True, + tool_events: str = "none", + transcript_path: str | None = None, + ctx: Context | None = None, # type: ignore[type-arg] + ) -> dict[str, Any]: + """**写入原始事件（L0）**：一轮对话或整段会话记录，交给提取流水线。 + + 记什么，由 ``source`` 决定——它既是谁记录的，也说明这批内容是哪种形态： + + - **终端 transcript**（``source="claude-code"`` / ``"kiro"``，将来还有 dsh）： + ``content`` 是整份 session 文件，服务端按该终端的记录格式解析成多条 L0 事件 + （``user_message`` / ``assistant_message`` / ``tool_call`` / ``tool_result``）。 + 真实会话 1~2MB，模型产不出这个参数，**由终端 hook 脚本调用**；以 ``H4sI`` + 开头时自动按 gzip+base64 解码。靠 ``src_key`` 幂等，可每轮上传整份文件、 + 只补新增轮次。**上传的原文另存一份可读归档**： + ``{workspace}/.octop/memory-sessions/.md``（增量追加、重发去重）， + 那里保留 L0 会丢掉的工具调用 / 结果与 thinking，返回体的 ``session_archive`` + 给出实际路径。 + - **其它 source**：``content`` 是本轮用户侧内容，``assistant`` 是助手回复 + （两条事件落在同一批里，提取器才能判断"助手建议是否被用户接受"）； + 只记助手结论时 ``content`` 留空字符串。 + + 鉴权、专家路由与用户归属都由 MCP 接入层按连接配置自动带上，工具本身不感知。 + 记录后立即可用 ``memory_raws`` / ``memory_search(corpus="raw")`` 查询， + 晋升成原子后才回到常规检索结果里。 + + 调用者 id（``X-Octop-User-Id``）会自动拼成内容前缀 ``说：…``：发送者由此进入 + 原子正文，既能被 FTS 直接搜到，也能在召回时一眼看出是谁说的——**不要**自己再 + 写一遍名字。两种形态都加：一轮对话形态的用户侧；transcript 形态的 ``user_message`` + （跳过 sidechain 记录——那是子代理的合成提示；助手侧与工具事件不加，role 已由 + ``event_type`` 表达）。 + + 回声保护（两层，都是为了断掉"注入 → 采集 → 再召回"的放大环）： + + 1. 带记忆注入标记（``[octop memory …]`` / ``[memory] Earlier in this workspace`` / + ``## Memory Recall``）的事件**不写入**；一轮对话形态下两条都是回声才整体 + 返回 ``skipped=recall_echo``，transcript 形态下逐条丢弃并计入 + ``skipped_echo``。transcript 形态还会一并丢弃**我们自己的 hook 指令文本** + （``[octop-memory hook]``）——那是提示词管道，不是用户说过的话。 + 2. transcript 形态下还会丢弃**我们自己的记忆读工具**（``memory_search`` / + ``memory_get`` / ``memory_raws`` / ``memory_candidates``，以及已下线的 + ``memory_recall``）产生的 ``tool_result``——那些内容本来就来自库，且调用方 + hook 自己拼注入块时未必带标记。丢弃条数见返回体的 ``skipped_own_results``。 + 3. transcript 形态下同样丢弃**终端自己注入的文本**：````、 + ````、``Base directory for this skill:``（自带技能文档）。 + 斜杠命令包装（```` / ```` / ````） + 与 ```` **保留**——那是用户动作及其结果。这一层只作用于 + **消息事件**：工具输出（读到的文件内容）里出现同样字符串只是数据，不能当注入丢。 + + **写得越详细，流水线产出的记忆越多。** 提取是一次 LLM 调用：它只把"自洽、 + 具体、能独立读懂"的段落拆成候选（Fact / Decision / Task / Preference / + ConflictCandidate），再经晋升检查落成原子。标题式、抽象或只有结论词的记录 + 通常产出 0 条候选——不是流水线丢了，而是里面没有可提取的事实。 + + 一条记录应尽量写全（有多少写多少）： + + - **谁说的**：一轮对话**一次调用**写完——``content`` 写用户原话，``assistant`` + 写助手自己的结论/建议。角色由字段隐含，**不要把助手产出塞进 ``content``**， + 那会让提取器按高置信的"用户事实"入账。 + - **发生了什么**：背景 + 动作，而不是结论词（"修好了" → "把 X 文件的 Y 分支 + 改成 Z，因为…"）。 + - **结论与决定**：做/不做什么，**保留否定与限定词原文**（"先不做 Replace" + 不能写成 "决定做 Augment"）。 + - **具体标识**：文件路径、表名、接口、MR/issue 号、版本、端口、配置项、 + 数字与阈值——它们是后续召回和实体锚定的抓手。 + - **理由与权衡**：为什么这样定，放弃了什么。 + - **未决项**：遗留 TODO、待确认的问题。 + - 一次涉及多个互不相关的事实就**分多次 capture**，一次一个主题。 + + 正例（一轮一次，可产出 Fact / Decision 候选）:: + + memory_capture( + content="记忆这块先按上次说的，把 MCP 写入路径补齐", # 用户原话 + assistant=( + "octop 记忆写入路径排查：memory_capture 直接调 Memory.add_raw，" + "绕过了 MemoryRuntime.capture 的隐私层，redact_secrets 不生效；" + "决定把 transcript 上传并进 memory_capture、角色由字段隐含，" + "涉及 src/octop/infra/agents/memory_mcp.py。未决：脱敏是否同分支一起修。" + ), + source="coding-agent", + session_id="ext:dsh:demo-octop", + ) + + 反例（几乎必然产出 0 条候选）:: + + memory_capture(content="记忆有问题", source="coding-agent") + + 两个角色的写法（**谁说的由字段决定**）： + + - ``content`` 是**人这一侧**的内容（用户原话，或一条提炼过的事实/结论）。落库为 + ``user_message``，并自动带 ``<调用者>说：`` 前缀——提取器只能看到 ``content``， + 前缀是"谁记录的"进入原子正文、可被 FTS 命中的唯一途径。 + - ``assistant`` 是**助手这一侧**的内容。落库为 ``assistant_message``，**不加前缀**： + 角色已由 ``event_type`` 表达，助手的话不能记成用户说的。 + - **整段会话（有人也有助手）别走这里**：用终端形态（``source="claude-code"`` / + ``"kiro"`` / ``"dsh"``）上传 session 文件，每条记录自带 ``event_type``、不加任何 + 前缀。纯文本形态的整段对话目前没有入口。 + + Args: + content: 一轮对话时的**人这一侧**内容（只记助手结论时留空字符串）；``source`` 是终端时 + 是整份 transcript（以 ``H4sI`` 开头则自动按 gzip+base64 解码）。 + source: 谁记录的，用于追溯；同时说明内容形态——``claude-code`` / ``kiro`` / ``dsh`` + 且内容确实是 JSONL（或以 ``H4sI`` 开头的 gzip+base64）时走该终端的 transcript + parser；**终端名 + 普通文本会退回一轮对话记录**（老调用方不改也能用），其它 + ``source`` 一律按一轮对话处理；并作为 L0 的 ``host`` + （``memory_raws(host=…)`` 可筛）。 + assistant: 一轮对话时的**助手这一侧**回复（不加发送者前缀）；留空则该次只写人这一侧。 + session_id: 可选会话 id，用于提取分组；一轮对话缺省派生为 ``ext:{source}:{user}``， + transcript 缺省为 ``cc:``。同一段工作请保持同一个 + id，跨调用稳定。 + thread_id: 可选线程 id，透传给 L0。 + dry_run: 只解析不写入，返回将要写入的条数 / 事件列表。 + extract: 写入后是否立即触发提取（默认 ``True``）。批量回灌传 ``False``， + 先只落 L0，再用 ``memory_extract`` 受控提取。 + transcript_path: 仅用于终端的 ``mcp_tool`` hook：Claude Code 会把 + ``${transcript_path}`` 原样传进来，由**同机的** MCP 服务端读取该文件 + （只允许终端会话目录：``~/.claude/projects`` / ``~/.kiro/sessions`` / + ``~/.dsh/sessions``；``.zstd`` 不支持，需按 gzip+base64 传 content）。 + 远程部署读不到调用方的文件，请改用 hook 脚本上报 content。 + tool_events: transcript 形态下**只决定落库哪些事件，不影响上传内容**——``content`` + 始终是整份 transcript（hook 按增量上传，服务端解压后逐条解析）。 + ``none``（默认）只落 ``user_message`` / ``assistant_message``：L0 原始素材是 + 提取流水线的输入与召回兜底层，工具流量占体积约 79%（``tool_result`` 一项 61%）， + 留着会挤占每批候选预算、稀释召回内容。要保真复查再显式放开：``calls`` 只记 + ``tool_call``（知道 agent 做了什么、丢输出），``all`` 连 ``tool_result`` 一起记。 + user: 可选调用者 id（覆盖 ``X-Octop-User-Id`` 头）。 + ctx: MCP 注入的上下文（读取 ``X-Octop-User-Id`` 头）。 + """ + caller = user or _caller_user(ctx) + + def _run() -> dict[str, Any]: + nonlocal content + memory = _memory() + + # ---- terminal transcript: ``source`` names the terminal, it names the parser + # A real session is 1~2MB, so it is shipped by the terminal's hook script + # (no model could pass that as a tool argument) and gzipped+base64-encoded. + if transcript_path and source in _TRANSCRIPT_SOURCES: + content = _read_transcript_file(transcript_path) + if source in _TRANSCRIPT_SOURCES and ( + content.startswith(_GZIP_B64_PREFIX) or _looks_like_transcript(content) + ): + from octop.infra.agents.claude_transcript import import_content # noqa: PLC0415 + + text = content + if text.startswith(_GZIP_B64_PREFIX): + import base64 # noqa: PLC0415 + import gzip # noqa: PLC0415 + + try: + text = gzip.decompress(base64.b64decode(text)).decode("utf-8", "replace") + except Exception as exc: # noqa: BLE001 - surface a usable message + raise ValueError( + f"content starts like gzip+base64 but cannot be decoded: {exc}" + ) from exc + + stats = import_content( + memory, + text, + session_id=session_id, + user=caller or None, + host=source, + dry_run=dry_run, + skip_markers=_SKIP_CONTENT_MARKERS, + skip_message_markers=_HOST_PLUMBING_MARKERS, + tool_events=tool_events, + parser=_TRANSCRIPT_SOURCES[source], + ) + archive = None + if not dry_run: + archive = _archive_uploaded_session( + server, + _agent_id(), + text=text, + source=source, + session_id=stats.session_id, + user=caller or None, + ) + extract_scheduled = bool(extract and not dry_run) and _trigger_extract( + server, stats.session_id + ) + return { + "marker": "[octop memory capture]", + "session_id": stats.session_id, + "session_archive": archive, + "parsed": stats.parsed, + "imported": stats.imported, + "skipped_existing": stats.skipped_existing, + "skipped_echo": stats.skipped_echo, + "skipped_own_results": stats.skipped_own_results, + "skipped_tools": stats.skipped_tools, + "tool_events": tool_events, + "events_by_type": stats.by_type, + "sidechain": stats.sidechain, + "dry_run": dry_run, + "extract_scheduled": extract_scheduled, + "warnings": stats.warnings, + "note": ( + "transcript parsed into L0 raw events; extraction runs off the request path, " + "so atoms show up only after promotion" + if extract_scheduled or dry_run + else ( + "transcript parsed into L0 raw events; extraction skipped (extract=false) — " + "run memory_extract(session_id=…) in a paced sweep to promote them" + ) + ), + } + + # ---- one turn: the role is implied by which field the text is in --------- + outgoing: list[tuple[str, str]] = [] + if content.strip(): + outgoing.append((USER_ROLE, content)) + if assistant: + outgoing.append((ASSISTANT_ROLE, assistant)) + if not outgoing: + raise ValueError("content and assistant are both empty; nothing to record") + kept = [(role, body) for role, body in outgoing if not _is_recall_echo(body)] + if not kept: + logger.info("memory_capture: dropped recall echo for agent %s", _agent_id()) + return { + "marker": "[octop memory capture]", + "recorded": False, + "skipped": "recall_echo", + "reason": ( + "every event carries a memory injection marker " + f"(e.g. {_OWN_CONTENT_PREFIX!r}); " + "dropped so our own injected block is not captured as a new event" + ), + "user": caller or None, + } + effective_session = session_id or _derive_session(source, caller) + + # Idempotent capture: reuse the existing raw event whenever the same + # session + content + role is already present, so re-ingesting the same + # conversation does not duplicate L0 events. Keeps extraction re-runnable. + existing: dict[tuple[Any, Any], tuple[str, str]] = {} + try: + for ev in memory.list_raw(session_id=effective_session, limit=1000): + existing.setdefault( + (getattr(ev, "content", None), getattr(ev, "event_type", None)), + (ev.id, str(getattr(ev, "content", ""))), + ) + except Exception: # noqa: BLE001 + # If duplicate detection fails, fall back to recording (safe). + existing = {} + + recorded: list[dict[str, Any]] = [] + for role, body in kept: + # Only the human side is attributed. ``event_type`` already marks the other + # side as the assistant, and the extractor prompt only ever sees + # event_id / event_type / content / timestamp / session_id — never the + # ``user`` column — so the prefix is how a sender reaches the atom text. + # Prefixing the assistant side would record the assistant's own words as + # if the user had said them. + stored_content = _attributed(body, caller) if role == USER_ROLE else body + duplicate = existing.get((stored_content, role)) + if duplicate is not None: + event_id, event_content = duplicate + recorded.append( + { + "event_id": event_id, + "event_type": role, + "content": event_content, + "duplicate": True, + } + ) + continue + if dry_run: + recorded.append( + { + "event_id": None, + "event_type": role, + "content": stored_content, + "duplicate": False, + } + ) + continue + raw = memory.add_raw( + stored_content, + event_type=role, + host="mcp-external", + session_id=effective_session, + thread_id=thread_id, + user=caller or None, + payload={"source": source}, + ) + recorded.append( + { + "event_id": raw.id, + "event_type": role, + "content": raw.content, + "duplicate": False, + } + ) + + extract_scheduled = bool(extract and not dry_run) and _trigger_extract( + server, effective_session + ) + result: dict[str, Any] = { + "marker": "[octop memory capture]", + "event_id": recorded[0]["event_id"], + "content": recorded[0]["content"], + "events": recorded, + "source": source, + "user": caller or None, + "session_id": effective_session, + "event_type": kept[0][0], + "thread_id": thread_id, + "recorded": not dry_run, + "dry_run": dry_run, + "extract_scheduled": extract_scheduled, + "note": ( + "raw (L0) events recorded; visible now via memory_raws, " + "searchable via memory_search after the extraction pipeline " + "promotes them to atom(s)" + ), + } + if source in _TRANSCRIPT_SOURCES: + result["note"] = ( + f"source {source!r} names a terminal, but content is not a transcript " + "(it does not start with a JSON record); recorded as one turn instead. " + "Pass the terminal's session file, or use a non-terminal source for text." + ) + if dry_run: + result["note"] = "dry run: parsed only, nothing written" + elif all(item["duplicate"] for item in recorded): + result["duplicate"] = True + result["note"] = "raw (L0) event already present; skipped (idempotent capture)" + return result + + return _observe_tool( + "capture", + session_id=session_id, + input_payload={ + "source": source, + "session_id": session_id, + "thread_id": thread_id, + "dry_run": dry_run, + "extract": extract, + "tool_events": tool_events, + "content": content, + "assistant": assistant, + "content_chars": len(content or ""), + "assistant_chars": len(assistant or "") if assistant else 0, + "has_transcript_path": bool(transcript_path), + }, + metadata={"caller": caller or None}, + fn=_run, + ) + + # ─── 记忆分层查询 + 流水线调度（L0/L1/L2）───────────────────────── + # 参考 DSH 记忆工具集（memory_raws/candidates/atoms/extract/promote/reject）， + # 把记忆生产流水线的每个环节暴露为 MCP 工具，供外部调度。 + + @mcp.tool() + def memory_raws( + query: str | None = None, + session_id: str | None = None, + host: str | None = None, + user: str | None = None, + limit: int = 50, + ctx: Context | None = None, # type: ignore[type-arg] + ) -> dict[str, Any]: + """**查原始事件**：FTS 搜索或结构化过滤 L0 原始事件（证据源）。 + + ``query`` 走全文搜索（capture 后立即可见），``session_id``/``host``/``user`` + 做结构化过滤，按时间倒序返回。 + + Args: + query: FTS 关键词（可选）。 + session_id: 按会话过滤（如 ``ext:review-bot:user-alice``）。 + host: 按记录主机过滤（如 ``mcp-external``）。 + user: 按调用者过滤。 + limit: 最多返回条数，默认 50。 + ctx: MCP 注入的上下文。 + """ + caller = user or _caller_user(ctx) + + def _run() -> dict[str, Any]: + memory = _memory() + events = ( + memory.search_raw(query, limit=limit) + if query + else memory.list_raw( + session_id=session_id, + host=host, + user=user or (caller or None), + limit=limit, + ) + ) + return { + "events": [ + { + "event_id": e.id, + "timestamp": e.timestamp.isoformat(), + "session_id": e.session_id, + "user": e.user, + "event_type": e.event_type, + "source": (e.payload or {}).get("source") if e.payload else None, + "content": e.content, + } + for e in events + ], + "count": len(events), + "caller": caller or None, + } + + return _observe_tool( + "raws", + session_id=session_id, + input_payload={ + "query": query, + "session_id": session_id, + "host": host, + "limit": limit, + }, + metadata={"caller": caller or None}, + fn=_run, + ) + + @mcp.tool() + def memory_candidates( + status: str | None = None, + session_id: str | None = None, + limit: int = 50, + ) -> dict[str, Any]: + """**查候选记忆**：列出 L1 候选（默认 pending 队列）。 + + 候选由 ``memory_capture``/``memory_extract`` 生成。可用 ``status`` 过滤 + （pending / promoted / rejected / needs_review / conflict）。 + + Args: + status: 候选状态过滤，默认 pending。 + session_id: 按来源会话过滤。 + limit: 最多返回条数，默认 50。 + """ + from octop_memory.core import CandidateStatus # noqa: PLC0415 + + # ``CandidateStatus`` is a ``typing.Literal`` (not an Enum), so it cannot + # be instantiated: ``CandidateStatus(status)`` raises ``TypeError``. The + # backend stores status as a plain string, so validate the caller's value + # against the literal and pass the string straight through. + status_value: str | None = None + if status: + allowed = set(get_args(CandidateStatus)) + if status not in allowed: + raise ValueError( + f"invalid candidate status {status!r}; expected one of {sorted(allowed)}" + ) + status_value = status + + def _run() -> dict[str, Any]: + memory = _memory() + candidates = memory.list_candidates( + status=status_value, + session_id=session_id, + limit=limit, + ) + return { + "candidates": [ + { + "candidate_id": c.id, + "status": c.status.value if hasattr(c.status, "value") else str(c.status), + "candidate_type": getattr(c, "candidate_type", None), + "assertion": getattr(c, "assertion", None), + "session_id": getattr(c, "session_id", None), + "confidence": getattr(c, "confidence", None), + } + for c in candidates + ], + "count": len(candidates), + } + + return _observe_tool( + "candidates", + session_id=session_id, + input_payload={ + "status": status_value, + "session_id": session_id, + "limit": limit, + }, + fn=_run, + ) + + @mcp.tool() + def memory_extract( + session_id: str | None = None, + limit: int = 100, + promote: bool = False, + ) -> dict[str, Any]: + """**手动触发提取**：把最近 L0 原始事件提取为候选（可选直达原子）。 + + 取最近 ``limit`` 条 L0 事件 → LLM 类型化提取 → 候选（pending）； + ``promote=True`` 时对候选执行晋升检查（L1 → L2），跳过人工审核。 + + Args: + session_id: 仅提取该会话的事件；缺省提取最近全部。 + limit: 提取的最近原始事件数，默认 100。 + promote: 是否对候选直接晋升，默认 False。 + """ + runtime_server = server.app_runtime + assert runtime_server is not None, "app_runtime required for memory extract" + agent = runtime_server.agent_registry.get_agent(_agent_id()) + runtime = getattr(agent, "_memory_runtime", None) + service = getattr(runtime, "service", None) if runtime else None + if service is None: + return {"error": "MemoryService unavailable (agent not running / no memory runtime)"} + + eff_session = session_id or "manual" + # Prefer the unwrapped extract so MCP owns the root ``memory.extract`` + # span (and nested stages) without nesting another identical root. + extract_fn = getattr(service, "_octop_memory_extract_inner", None) or service.extract + + def _run() -> dict[str, Any]: + result = extract_fn(eff_session, incremental=True, promote=promote, regen_pages=False) + if not isinstance(result, dict): + return {"session_id": eff_session, "candidates": 0, "promoted": 0} + return { + "session_id": eff_session, + "events_considered": result.get("events_considered", 0), + "candidates": result.get("candidates", 0), + "promoted": result.get("promoted", 0) + if isinstance(result.get("promotion"), dict) + else 0, + "error": result.get("failure_reason"), + "promotion": result.get("promotion"), + } + + return _observe_tool( + "extract", + session_id=eff_session, + input_payload={ + "session_id": eff_session, + "limit": limit, + "promote": promote, + }, + fn=_run, + ) + + @mcp.tool() + def memory_promote(candidate_ids: list[str]) -> dict[str, Any]: + """**采纳候选**（与记忆面板「采纳」是同一套实现）。 + + 逐条走 dashboard 的 ``promote_candidate``：``pending`` 候选执行 5 项晋升 + 检查（通过则写入原子，也可能落为 merge / conflict / needs_review）， + ``needs_review`` / ``conflict`` 候选按人工裁决强制转正（取决于所装的 + harness-memory 版本：0.9.9 起会同时 supersede 被顶替的旧原子）。状态不 + 允许（已 promoted / rejected）或候选不存在时，该条以 ``errors`` 返回，不 + 影响同批其它候选。 + + Args: + candidate_ids: 要采纳的候选 id 列表（来自 ``memory_candidates``）。 + + Returns: + 与面板一致的聚合计数（promoted / merged / conflicts / needs_review / + dropped / llm_calls）、逐条结果 ``candidates`` 与逐条错误 ``errors``； + ``errors`` 非空即表示并非全部采纳成功，调用方必须检查。 + """ + bridge = _dashboard_bridge(_memory()) + + def _run() -> dict[str, Any]: + totals = dict.fromkeys( + ("promoted", "merged", "conflicts", "needs_review", "dropped", "llm_calls"), + 0, + ) + candidates: list[dict[str, Any]] = [] + errors: list[dict[str, Any]] = [] + for candidate_id in candidate_ids: + try: + detail = _bridge_rpc( + bridge, + "promote_candidate", + {"candidate_id": candidate_id}, + ) + except Exception as exc: # noqa: BLE001 - 逐条隔离，错误原样回给调用方 + errors.append({"candidate_id": candidate_id, "error": str(exc)}) + continue + candidates.append({"candidate_id": candidate_id, **detail}) + for key in totals: + totals[key] += int(detail.get(key) or 0) + return {**totals, "candidates": candidates, "errors": errors} + + return _observe_tool( + "promote", + input_payload={"candidate_ids": list(candidate_ids)}, + fn=_run, + ) + + @mcp.tool() + def memory_reject( + candidate_id: str, + reason: str = "rejected by external caller", + actor: str | None = None, + ) -> dict[str, Any]: + """**忽略候选**（与记忆面板「忽略」是同一套实现）。 + + 走 dashboard 的 ``reject_candidate``：只允许 pending / needs_review / + conflict 的候选；置为 rejected 并记录 decided_by / decided_at 与一条 + ``action="reject"`` 的 journal（可审计），不触碰原子层。状态不允许或候选 + 不存在时抛错，错误信息与面板提示一致。 + + Args: + candidate_id: 候选 id。 + reason: 忽略原因，写入 journal 的 note。 + actor: 决策者，缺省与面板一致取 ``user``；可选 ``auto`` / ``rule``。 + """ + params: dict[str, Any] = {"candidate_id": candidate_id} + if reason: + params["reason"] = reason + if actor: + params["actor"] = actor + + def _run() -> dict[str, Any]: + result = _bridge_rpc( + _dashboard_bridge(_memory()), + "reject_candidate", + params, + ) + return {"ok": True, **result} + + return _observe_tool( + "reject", + input_payload=params, + fn=_run, + ) + + return mcp + + +def _extract_failure_reason(result: Any) -> str | None: + """Return the failure reason reported by ``MemoryService.extract``, if any. + + ``harness-memory`` degrades instead of raising: the returned dict carries + ``failure_reason`` when the extraction LLM call failed. Older / foreign + shapes may expose it as ``error``. + """ + if not isinstance(result, dict): + return None + for key in ("failure_reason", "error"): + value = result.get(key) + if isinstance(value, str) and value.strip(): + return value.strip() + return None + + +def _memory_service_of(agent: Any) -> Any | None: + """``MemoryService`` on a live harness agent, or ``None`` when memory is off.""" + runtime = getattr(agent, "_memory_runtime", None) + return getattr(runtime, "service", None) if runtime is not None else None + + +def _live_memory_service(registry: Any, agent_id: str) -> Any | None: + """Return the in-process ``MemoryService`` when the expert is already loaded.""" + get_agent = getattr(registry, "get_agent", None) + if not callable(get_agent): + return None + try: + agent = get_agent(agent_id) + except Exception: # noqa: BLE001 — not running / not found / harness down + return None + return _memory_service_of(agent) + + +async def _ensure_memory_service(registry: Any, agent_id: str) -> Any | None: + """Resolve ``MemoryService``, starting the expert when it is stopped. + + Terminal hooks (Cursor / Claude Code / …) capture into L0 even while the + expert is unloaded; extraction still needs the agent's configured LLM via + the live ``MemoryService``. Start failures are logged at WARNING. + """ + service = _live_memory_service(registry, agent_id) + if service is not None: + return service + start = getattr(registry, "start", None) + if not callable(start): + logger.warning( + "memory extract: agent %s has no MemoryService and registry cannot start", + agent_id, + ) + return None + try: + await start(agent_id) + except Exception: + logger.warning( + "memory extract: failed to start agent %s for extract", + agent_id, + exc_info=True, + ) + return None + service = _live_memory_service(registry, agent_id) + if service is None: + logger.warning( + "memory extract: agent %s started but MemoryService still unavailable", + agent_id, + ) + return service + + +def _trigger_extract(server: OctopServer, session_id: str | None) -> bool: + """Asynchronously extract + promote after MCP ``memory_capture``. + + MCP-written L0 events are outside the harness extractor's tracked sessions, + so capture must schedule extract explicitly. If the expert is stopped, start + it first so the in-process ``MemoryService`` (with the agent's extraction + LLM) can run. Failures are logged at WARNING — never silently skipped. + Returns whether an extract task was scheduled. + """ + import asyncio + + agent_id = _current_agent_id.get() + if not session_id or not agent_id: + if agent_id and not session_id: + logger.warning( + "memory extract skipped for agent %s: capture returned no session_id", + agent_id, + ) + return False + + runtime_server = getattr(server, "app_runtime", None) + if runtime_server is None: + logger.warning( + "memory extract skipped for agent %s session %s: app_runtime unavailable", + agent_id, + session_id, + ) + return False + registry = getattr(runtime_server, "agent_registry", None) + if registry is None: + logger.warning( + "memory extract skipped for agent %s session %s: agent_registry unavailable", + agent_id, + session_id, + ) + return False + + async def _extract() -> None: + try: + service = await _ensure_memory_service(registry, agent_id) + if service is None: + return + result = await asyncio.to_thread( + service.extract, + session_id, + incremental=True, + promote=True, + regen_pages=True, + ) + except Exception: + logger.warning( + "memory extract failed for agent %s session %s", + agent_id, + session_id, + exc_info=True, + ) + return + # ``extract`` reports LLM / promotion failures inside the result + # instead of raising, so a silently-dead model chain would otherwise + # leave no trace above INFO. Surface it as a warning. + reason = _extract_failure_reason(result) + if reason: + logger.warning( + "memory extract failed for agent %s session %s: %s", + agent_id, + session_id, + reason, + ) + return + logger.info( + "memory extract done for agent %s session %s: %s", + agent_id, + session_id, + result, + ) + + try: + asyncio.create_task(_extract()) + return True + except Exception: + logger.warning( + "memory extract trigger failed for agent %s session %s", + agent_id, + session_id, + exc_info=True, + ) + return False + + +def _memory_mcp_token() -> str | None: + """Read the MCP auth token (empty string treated as unconfigured).""" + return (os.environ.get("OCTOP_MEMORY_MCP_TOKEN") or "").strip() or None + + +class _TokenAuthMiddleware: + """ASGI middleware enforcing ``Authorization: Bearer`` or ``X-Octop-Memory-Token``.""" + + def __init__(self, app: Any, token: str) -> None: + self._app = app + self._token = token + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + if scope.get("type") != "http": + await self._app(scope, receive, send) + return + + headers = { + k.decode("latin-1").lower(): v.decode("latin-1") for k, v in scope.get("headers", []) + } + auth = headers.get("authorization", "") + provided = auth[7:].strip() if auth.startswith("Bearer ") else "" + if not provided: + provided = headers.get("x-octop-memory-token", "").strip() + + if provided != self._token: + body = b'{"error":"unauthorized"}' + await send( + { + "type": "http.response.start", + "status": 401, + "headers": [ + (b"content-type", b"application/json"), + (b"content-length", str(len(body)).encode()), + ], + } + ) + await send({"type": "http.response.body", "body": body}) + return + + await self._app(scope, receive, send) + + +class _AgentRouter: + """ASGI dispatcher validating ``X-Octop-Agent-Id`` against the agent repo and + forwarding to the single shared memory MCP app. + + The agent set is NOT snapshotted at startup: every request is checked against + the agent repo (existence + ``enabled``), so agents created or disabled after + process start take effect immediately (no restart required). + """ + + def __init__(self, app: Any, server: OctopServer) -> None: + self._app = app + self._server = server + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + if scope.get("type") != "http": + return # lifespan is wired into the host FastAPI manually; http only here + + headers = { + k.decode("latin-1").lower(): v.decode("latin-1") for k, v in scope.get("headers", []) + } + agent_id = headers.get("x-octop-agent-id", "").strip() + services = self._server.services + row = services.agent_repo.get(agent_id) if (services is not None and agent_id) else None + if row is None or not row.enabled: + body = b'{"error":"missing or unknown agent_id (X-Octop-Agent-Id)"}' + await send( + { + "type": "http.response.start", + "status": 404, + "headers": [ + (b"content-type", b"application/json"), + (b"content-length", str(len(body)).encode()), + ], + } + ) + await send({"type": "http.response.body", "body": body}) + return + # 把本次请求绑定的 expert 与调用者 user id 写入 contextvar，供工具读取 + # （stateless HTTP 下 mcp SDK 不提供 ctx.request_context）。 + agent_cv = _current_agent_id.set(agent_id) + user = headers.get("x-octop-user-id", "").strip() + token_cv = _current_caller_user.set(user) + try: + await self._app(scope, receive, send) + finally: + _current_caller_user.reset(token_cv) + _current_agent_id.reset(agent_cv) + + +async def _send_text( + send: Any, status: int, body: str, *, content_type: str = "text/plain; charset=utf-8" +) -> None: + payload = body.encode("utf-8") + await send( + { + "type": "http.response.start", + "status": status, + "headers": [ + (b"content-type", content_type.encode("ascii")), + (b"content-length", str(len(payload)).encode("ascii")), + ], + } + ) + await send({"type": "http.response.body", "body": payload}) + + +def _instance_config() -> dict[str, Any]: + """The instance ``config.json`` as a dict (empty when missing/unreadable).""" + import json # noqa: PLC0415 + + from octop.infra.utils.paths import PathLayout # noqa: PLC0415 + + try: + raw = PathLayout.from_env().config.read_text(encoding="utf-8") + loaded = json.loads(raw) + except Exception: # noqa: BLE001 - a missing/odd config must not break /setup + return {} + return loaded if isinstance(loaded, dict) else {} + + +_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "::1"}) +"""本机地址：只有它没有 TLS 前置，按请求的 scheme 走 http。""" + + +def _host_without_port(host: str) -> str: + """``host``（可能带 ``:port`` 或 IPv6 方括号）里的裸主机名，小写。""" + value = host.strip() + if value.startswith("["): + return value[1:].split("]", 1)[0].lower() + return value.split(":", 1)[0].lower() + + +def _host_mount_url(scheme: str, host: str) -> str: + """本实例自己的挂载点（hooks 永远打它）。 + + 本机照请求 scheme 走 http（没有 TLS 前置）；其余一律 **https** —— 反代/ingress 通常在 + 这一层终结 TLS，``http://`` 打不通，而 ``X-Forwarded-Proto`` 不可靠（网关不透传或透传成 http）。 + """ + if _host_without_port(host) in _LOOPBACK_HOSTS: + return f"{scheme or 'http'}://{host}/mcp/memory/" + return f"https://{host}/mcp/memory/" + + +def _default_mcp_url(scheme: str, host: str) -> str: + """客户端条目默认该连的地址：本实例自己的挂载点。 + + ``?url=`` / ``OCTOP_MEMORY_PUBLIC_MCP_URL`` / ``memory_public_mcp_url`` 仍是**显式覆盖**， + 部署可以用它把客户端条目指向自建的公网入口。 + """ + return _host_mount_url(scheme, host) + + +def _public_mcp_url() -> str | None: + """**显式覆盖**的客户端条目地址（部署可选）： + + * ``OCTOP_MEMORY_PUBLIC_MCP_URL`` (environment), or + * ``memory_public_mcp_url`` in the instance's ``config.json``. + + 没配也照样能用 —— 默认地址就是本实例自己的挂载点。 + """ + from_env = os.environ.get("OCTOP_MEMORY_PUBLIC_MCP_URL", "").strip() + if from_env: + return from_env + return _config_public_mcp_url() + + +def _config_public_mcp_url() -> str | None: + """``memory_public_mcp_url`` from the instance ``config.json`` (None when absent).""" + value = str(_instance_config().get("memory_public_mcp_url") or "").strip() + return value or None + + +class _InstallerApp: + """ASGI app for ``/setup``: the unauthenticated local-client installer. + + It hands out no credential — the script asks for the token at run time — so it + deliberately sits outside the memory MCP token check; the only artifact is the + script. Deleting this route (or the whole mount) removes nothing else. + """ + + def __init__(self, server: OctopServer) -> None: + self._server = server + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + from urllib.parse import parse_qsl # noqa: PLC0415 + + if scope.get("type") != "http": + return + if str(scope.get("method", "GET")).upper() not in ("GET", "HEAD"): + await _send_text(send, 405, "GET only\n") + return + + from octop.infra.agents.client_setup import render_setup_script # noqa: PLC0415 + + query = dict(parse_qsl(scope.get("query_string", b"").decode("utf-8"))) + headers = { + k.decode("latin-1").lower(): v.decode("latin-1") for k, v in scope.get("headers", []) + } + scheme = headers.get("x-forwarded-proto", "").split(",")[0].strip().lower() + host = headers.get("host", "127.0.0.1") + # hooks 走本实例的直连挂载点；mcp_url 是登记进客户端 MCP 条目的地址，缺省就是本实例 + # 的挂载点（部署可用 ?url= / memory_public_mcp_url 指向自建的公网入口）。两份地址共用 + # 同一份实例共享 token。 + instance_url = _host_mount_url(scheme, host) + script = render_setup_script( + mcp_url=query.get("url") or _public_mcp_url() or _default_mcp_url(scheme, host), + hook_url=query.get("hook_url") or instance_url, + agent_id=(query.get("agent") or "").strip(), + terminals=(query.get("terminals") or "").strip(), + # 实例发布时的 OCTOP_MEMORY_MCP_TOKEN 直接内置成脚本默认凭据：用户只需专家 ID + # （+ 可选用户标识）。注意 /setup 本身不鉴权 —— 谁连得到它就拿到这个 token。 + default_token=_memory_mcp_token() or "", + ) + await _send_text(send, 200, script, content_type="text/x-shellscript; charset=utf-8") + + +class _UninstallerApp: + """ASGI app for ``/uninstall``: counterpart of ``/setup`` (also unauthenticated).""" + + def __init__(self, server: OctopServer) -> None: + self._server = server + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + from urllib.parse import parse_qsl # noqa: PLC0415 + + if scope.get("type") != "http": + return + if str(scope.get("method", "GET")).upper() not in ("GET", "HEAD"): + await _send_text(send, 405, "GET only\n") + return + + from octop.infra.agents.client_setup import render_uninstall_script # noqa: PLC0415 + + query = dict(parse_qsl(scope.get("query_string", b"").decode("utf-8"))) + script = render_uninstall_script(terminals=(query.get("terminals") or "").strip()) + await _send_text(send, 200, script, content_type="text/x-shellscript; charset=utf-8") + + +class _SetupFirstApp: + """Serve ``/setup`` and ``/uninstall`` before the token check, then MCP. + + Both scripts hand out no credential, so they sit in front of + ``_TokenAuthMiddleware``. Keeping them under the same mount means a + deployment that only exposes ``/mcp…`` needs no extra ingress rule. + """ + + def __init__(self, installer: Any, uninstaller: Any, inner: Any) -> None: + self._installer = installer + self._uninstaller = uninstaller + self._inner = inner + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + if scope.get("type") == "http": + path = str(scope.get("path", "")) + root = str(scope.get("root_path", "")) + relative = path[len(root) :] if root and path.startswith(root) else path + cleaned = relative.rstrip("/") + if cleaned == "/setup": + await self._installer(scope, receive, send) + return + if cleaned == "/uninstall": + await self._uninstaller(scope, receive, send) + return + await self._inner(scope, receive, send) + + +def mount_memory_mcp(app: Any, server: OctopServer) -> list[Any]: + """Mount the memory MCP endpoint at ``/mcp/memory``; the expert is selected + per request via the ``X-Octop-Agent-Id`` header (validated at request time; + the URL stays uniform and does not leak expert ids). + + + Does not mount when ``OCTOP_MEMORY_MCP_TOKEN`` is unset (fail-closed). + Returns the session managers that must be initialized in the host FastAPI + lifespan (``streamable_http_app`` task groups depend on it). + """ + token = _memory_mcp_token() + if token is None: + return [] + + services = server.services + assert services is not None, "server.services required for memory MCP mount" + mcp = build_memory_mcp(server) + # IMPORTANT: _session_manager is created lazily by streamable_http_app(). + # Read it only AFTER building the ASGI app and drop None entries — in + # stateless HTTP mode there is no session manager to keep alive, and reading + # mcp._session_manager before streamable_http_app() yields None, which then + # crashes the host FastAPI lifespan with "NoneType has no attribute 'run'". + streamable_app = mcp.streamable_http_app() + managers = [mgr for mgr in (mcp._session_manager,) if mgr is not None] + + app.mount( + "/mcp/memory", + _SetupFirstApp( + _InstallerApp(server), + _UninstallerApp(server), + _TokenAuthMiddleware(_AgentRouter(streamable_app, server), token), + ), + ) + return managers + + +__all__ = ["build_memory_mcp", "mount_memory_mcp"] diff --git a/src/octop/infra/agents/memory_recall_patch.py b/src/octop/infra/agents/memory_recall_patch.py new file mode 100644 index 000000000..c180ee0c6 --- /dev/null +++ b/src/octop/infra/agents/memory_recall_patch.py @@ -0,0 +1,356 @@ +"""Persistent recall-quality patches for ``octop_memory``. + +The upstream ``harness-memory`` package (PyPI) ships a recall pipeline whose +retrieval quality is poor for multi-person shared-conversation datasets like +the evaluation corpus. Three defects were found during eval work and are +patched here at startup (idempotent), so the fixes survive +``uv sync`` / redeploys instead of living only in the venv: + + 1. ``router.route`` — entity hints are matched against stored aliases with + an exact string lookup, but natural-language queries attach Chinese + suffixes ("张小明的deadline" → hint "张小明的", stored alias "张小明"). + Result: no entity anchor is resolved, recall degrades to topical FTS + and cross-entity topics (everyone's deadlines) drown the requested + person's. Fix: progressively strip common suffixes before alias lookup. + + 2. ``multi_source._per_token_atom_search`` — merges per-token FTS hits + ordered by token position, so an early generic token ("李小" n-gram of + "李小婉") outranks a later *relevant* token ("recurring"). Result: + recall returns the entity's generic facts (age / company) instead of + the topical memory. Fix: rank by count of matched *strong* tokens + (full Latin words + complete Han runs + resolved anchor names), with + n-grams contributing to recall but not to the relevance count. + + 3. ``multi_source._gather_atoms`` anchor branch — when the router resolves + an entity, its "recent atoms" fill the candidate list before FTS + topical hits, so a query-relevant atom outside the recency window is + truncated before rerank. Fix: run FTS first, then anchor atoms as + fallback, and pass anchor entity names into the strong-token set so + "张小明 + deadline" outranks someone else's "deadline". + +Each patch wraps the original symbol (preserving the upstream signature and +behaviour as the fallback); ``apply_memory_recall_patch()`` is idempotent +and is invoked from ``octop.infra.server._boot_runtime`` before any agent +memory service is created. +""" + +from __future__ import annotations + +import logging +import re +from collections.abc import Sequence +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from octop_memory.pipeline.recall.router import RoutingDecision + +logger = logging.getLogger(__name__) + +_PATCHED = False + +# ── shared helpers ──────────────────────────────────────────────────────── + +_LATIN_WORD_RE = re.compile(r"[A-Za-z][A-Za-z0-9_-]{1,}") +_HAN_RUN_RE = re.compile(r"[\u4e00-\u9fff]{2,}") + +# Chinese possessive / copular suffixes that attach to a name in queries. +_SUFFIXES = ("的", "是", "在", "了", "与", "和") + + +def _strong_query_tokens(text: str) -> set[str]: + """Full-word query tokens — Latin words + complete Han runs (no n-grams).""" + out: set[str] = set() + for m in _LATIN_WORD_RE.finditer(text): + out.add(m.group(0).lower()) + for m in _HAN_RUN_RE.finditer(text): + out.add(m.group(0)) + return out + + +def _alias_candidates(alias: str) -> tuple[str, ...]: + """Ordered alias lookup candidates: exact match, then suffix-stripped.""" + out = [alias] + if alias and not alias.isascii(): + stripped = alias + for _ in range(3): + if stripped and stripped[-1] in _SUFFIXES: + stripped = stripped[:-1] + if stripped: + out.append(stripped) + else: + break + return tuple(out) + + +# ── 4. substring entity anchoring (cross-entity / conflict phrasing) ───── +# +# ``parser._entity_hints`` produces *fragmented* hints for compound +# natural-language queries: "在多位同事都提供了age的情况下，周晓东本人的age" +# yields hints like "周晓东本人的" (exact alias lookup misses "周晓东"), +# "和赵晓磊同住…王小明" yields one long blob, and "当吴晓强的…" keeps the +# leading "当". Exact-then-suffix-stripped resolution cannot recover the +# entity, so the router resolves nothing and recall degrades to topical FTS. +# Fix: also match any known entity name/alias as a *substring* of the query +# text, so the target person (王小明/周晓东/吴晓强/赵晓磊) is resolved and flows +# into the anchor branch of ``_gather_atoms`` (fixes cross-entity + conflict +# "以哪个值为准" phrasing misses). + +_ENTITY_INDEX: dict[str, dict[str, str]] = {} + + +def _entity_index_key(memory: Any) -> str: + """Key the entity-index cache by memory identity so daily / dev (two + separate agents) never share one index.""" + return str(getattr(memory, "namespace", None) or id(getattr(memory, "_backend", memory))) + + +def _entity_index_for(memory: Any) -> dict[str, str]: + key = _entity_index_key(memory) + if key not in _ENTITY_INDEX: + _ENTITY_INDEX[key] = _build_entity_index(memory) + return _ENTITY_INDEX[key] + + +def _build_entity_index(memory: Any, *, limit: int = 800) -> dict[str, str]: + """Map every entity canonical name + alias -> entity id.""" + idx: dict[str, str] = {} + try: + for e in memory.list_entities(limit=limit): + names = [e.canonical_name, *(getattr(e, "aliases", None) or [])] + for name in names: + name = str(name).strip() + if len(name) >= 2: + idx.setdefault(name, e.id) + # The alias table may hold names not denormalized onto the entity. + for a in memory.list_aliases(limit=limit * 2): + al = str(getattr(a, "alias", "")).strip() + if len(al) >= 2: + idx.setdefault(al, a.entity_id) + except Exception: # noqa: BLE001 + # A bare/mock backend may lack these; fall back to an empty index. + pass + return idx + + +def _substring_entity_hits(text: str, idx: dict[str, str]) -> list[str]: + """Entity ids whose name/alias appears as a substring of ``text``. + + De-duped by entity id, order-preserving. + """ + hits: dict[str, str] = {} + for name, eid in idx.items(): + if len(name) >= 2 and name in text: + hits.setdefault(eid, name) + return list(dict.fromkeys(hits)) + + +# ── 1. router.route: suffix-tolerant entity resolution ──────────────────── + + +def _patched_route( + memory: Any, + parsed: Any, + *, + thread_id: str | None = None, +) -> RoutingDecision: + from octop_memory.pipeline.recall.router import _orig_route # noqa: PLC0415 + + # Re-resolve hints with suffix stripping: try each candidate alias in + # order, resolve the first hit, and inject it back into the parsed + # query so the original route() picks it up. + resolved: list[str] = [] + for alias in parsed.entity_hints or (): + for candidate in _alias_candidates(alias): + try: + entity = memory.find_entity_by_alias(candidate) + except Exception: # noqa: BLE001 + continue + if entity is not None: + resolved.append(entity.id) + break + + # Substring anchoring: the parser's hint fragments (e.g. "当吴晓强的", + # "周晓东本人的", "和赵晓磊同住…王小明") fail exact alias lookup, so the + # router resolves nothing for these compound queries. Match any known + # entity name/alias as a substring of the whole query text to recover the + # target entity. + try: + for eid in _substring_entity_hits(parsed.text, _entity_index_for(memory)): + if eid not in resolved: + resolved.append(eid) + except Exception: # noqa: BLE001 + pass + + decision = _orig_route(memory, parsed, thread_id=thread_id) + # Union our anchors (suffix + substring) with whatever the original route + # resolved; never drop the original resolution. Only rebuild the decision + # when we actually resolved something extra. + union = list(dict.fromkeys(resolved + list(decision.resolved_entity_ids))) + if union and tuple(union) != tuple(decision.resolved_entity_ids): + from dataclasses import replace # noqa: PLC0415 + + decision = replace(decision, resolved_entity_ids=tuple(union)) + return decision + + +# ── 2. multi_source._per_token_atom_search: strong-token ranking ────────── + + +def _patched_per_token_atom_search( + memory: Any, + parsed: Any, + *, + limit: int, + anchor_names: Sequence[str] = (), +) -> list[Any]: + """Per-token FTS merge ranked by matched *strong* token count. + + Mirrors the upstream function but counts only strong tokens (full words + + anchor entity names), so a hit matching "张小明" + "deadline" outranks + a single-token "deadline" hit on a different entity. + """ + strong = _strong_query_tokens(parsed.text) + for name in anchor_names: + norm = str(name).strip() + if len(norm) >= 2: + strong.add(norm.lower() if norm.isascii() else norm) + seen: dict[str, dict[str, Any]] = {} + tokens = parsed.raw_tokens or (parsed.text,) + for token_idx, token in enumerate(tokens): + if not str(token).strip(): + continue + try: + # Search deep: FTS5 rank is a global corpus score, so the + # relevant hit for a common token (e.g. "recurring" shared by + # many entities) can sit far past the top-N. A shallow pool + # truncates it before our strong-token ranking can promote it. + atoms = memory.search_atoms(token, limit=limit * 8) + except Exception: + continue + is_strong = token in strong + for atom_idx, atom in enumerate(atoms): + entry = seen.get(atom.id) + if entry is None: + seen[atom.id] = { + "count": 1 if is_strong else 0, + "first": (token_idx, atom_idx), + "atom": atom, + } + elif is_strong: + entry["count"] += 1 + ranked = sorted( + seen.values(), + key=lambda e: (-e["count"], e["first"][0], e["first"][1]), + ) + return [e["atom"] for e in ranked[:limit]] + + +# ── 3. multi_source._gather_atoms anchor branch: FTS first + anchor names ── + +# item 3: decision/cause/pitfall/experience intent reorder. The corpus stores +# "技术决策" atoms as e.g. "「X」做过一个技术决策：用 G6 而非 D3.js：因为…" and +# "踩坑/经验复用" atoms as "「X」的pitfalls有更新…" / "可复用经验…". For a query +# asking "技术决策及原因 / 为什么 / 经验复用 / 踩坑", those atoms are added as +# *anchor fillers after* the FTS "技术选型" matches and get truncated by +# `limit`. Reorder them ahead of generic selection atoms when the query shows +# that intent. +_DECISION_INTENT_RE = re.compile( + r"技术决策|决策及原因|原因|为什么|若非|为何|经验|复用|踩|坑|规避|如何解决" +) +_DECISION_MARK_RE = re.compile(r"技术决策|而非|因为|坑|pitfall|经验|可复用|规避|决策|N\+1") + + +def _decision_first(atoms: list[Any]) -> list[Any]: + """Stable-partition: decision/cause/pitfall atoms first, others after.""" + dec = [a for a in atoms if _DECISION_MARK_RE.search(getattr(a, "assertion", "") or "")] + rest = [a for a in atoms if not _DECISION_MARK_RE.search(getattr(a, "assertion", "") or "")] + return dec + rest + + +def _patched_gather_atoms( + memory: Any, + parsed: Any, + *, + limit: int, + anchor_ids: Sequence[str] = (), +) -> list[Any]: + """Anchor-narrowed atom gather: FTS topical hits first, anchor fillers + after, with anchor entity names promoted into the strong-token set.""" + + if anchor_ids: + anchor_names: list[str] = [] + for eid in anchor_ids: + ent = memory.get_entity(eid) + if ent is not None and ent.canonical_name: + anchor_names.append(ent.canonical_name) + out: dict[str, Any] = {} + for atom in _patched_per_token_atom_search( + memory, parsed, limit=limit, anchor_names=anchor_names + ): + out.setdefault(atom.id, atom) + if parsed.time_window is not None: + for eid in anchor_ids: + for atom in memory.search_atoms_by_time_range( + start=parsed.time_window.start, + end=parsed.time_window.end, + entity_id=eid, + limit=limit, + ): + out.setdefault(atom.id, atom) + else: + for eid in anchor_ids: + for atom in memory.list_atoms(entity_id=eid, limit=limit): + out.setdefault(atom.id, atom) + atoms = list(out.values()) + # Reorder the WHOLE gathered set (FTS hits + anchor fillers) so the + # decision/cause/pitfall atom is not truncated by `limit` before it + # can be promoted ahead of generic "技术选型" atoms. + if _DECISION_INTENT_RE.search(parsed.text or ""): + atoms = _decision_first(atoms) + return atoms[:limit] + + if parsed.time_window is not None: + return list( + memory.search_atoms_by_time_range( + start=parsed.time_window.start, + end=parsed.time_window.end, + limit=limit, + ) + ) + return _patched_per_token_atom_search(memory, parsed, limit=limit) + + +# ── apply ───────────────────────────────────────────────────────────────── + + +def apply_memory_recall_patch() -> None: + """Install the recall-quality patches (idempotent).""" + global _PATCHED + if _PATCHED: + return + + try: + import octop_memory.pipeline.recall.multi_source as ms # noqa: PLC0415 + import octop_memory.pipeline.recall.router as router # noqa: PLC0415 + + if not hasattr(router, "_orig_route"): + router._orig_route = router.route + router.route = _patched_route + + if not hasattr(ms, "_orig_per_token_atom_search"): + ms._orig_per_token_atom_search = ms._per_token_atom_search + ms._per_token_atom_search = _patched_per_token_atom_search + + if not hasattr(ms, "_orig_gather_atoms"): + ms._orig_gather_atoms = ms._gather_atoms + ms._gather_atoms = _patched_gather_atoms + + _PATCHED = True + logger.info("memory recall patches applied (router suffix + strong-token ranking)") + except Exception: # noqa: BLE001 + # Never fail startup because a recall patch could not be installed — + # upstream behaviour is the safe fallback. + logger.warning( + "memory recall patch installation failed; using upstream recall", + exc_info=True, + ) diff --git a/src/octop/infra/agents/memory_session_archive.py b/src/octop/infra/agents/memory_session_archive.py new file mode 100644 index 000000000..2ae219bd9 --- /dev/null +++ b/src/octop/infra/agents/memory_session_archive.py @@ -0,0 +1,210 @@ +"""Server-side markdown archive of the raw sessions uploaded to ``memory_capture``. + +L0 keeps only what the extractor and recall need (user / assistant messages), so the +tool traffic and the model's scratchpad are dropped there. This writer keeps the +*uploaded* session as readable markdown next to the agent's memory store, so the full +record stays available for later review: + + {workspace}/.octop/memory-sessions/.md + +The client uploads byte-deltas of the transcript, so the file is **append-only**: each +upload renders its new records and adds them; re-sends (``mode=full`` after a lost +offset state) are deduplicated by the record key written as an HTML comment, which is +invisible when rendered. Failures must never break capture — callers treat them as +best-effort and log. +""" + +from __future__ import annotations + +import json +import re +from collections.abc import Iterator +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +# Reuse the transcript parser's record iteration and block normalization: the archive +# must agree with what the parser sees, and a second copy of those rules would drift. +from octop.infra.agents.claude_transcript import _block_text, _blocks, _iter_records + +ARCHIVE_DIRNAME = "memory-sessions" +"""Subdirectory of the agent system dir (``{workspace}/.octop/``) holding the archives.""" + +_KEY_MARKER = "" +_KEY_PATTERN = re.compile(r"") + +_MAX_BODY_CHARS = 20000 +"""Per-record cap in the markdown body — much looser than L0's 4000. + +The archive is for human inspection, so it keeps far more of a long tool result than +the L0 row does; it still bounds a single record so one 5MB build log cannot dominate +the file. +""" + + +def _now() -> str: + return datetime.now(UTC).astimezone().strftime("%Y-%m-%d %H:%M:%S") + + +def _cap(text: str, limit: int = _MAX_BODY_CHARS) -> str: + body = (text or "").strip() + return body if len(body) <= limit else body[: limit - 1].rstrip() + "…" + + +def _fence(body: str, language: str) -> str: + """Fence *body*, widening the fence when the body itself contains backticks.""" + fence = "```" + while fence in body: + fence += "`" + return f"{fence}{language}\n{body}\n{fence}" + + +def _record_role(record: dict[str, Any]) -> str: + return str(record.get("type") or "record") + + +def _render_record(key: str, record: dict[str, Any], index: int) -> str: + """One transcript record → one markdown section (all blocks, nothing dropped).""" + message = record.get("message") + message = message if isinstance(message, dict) else {} + role = str(message.get("role") or _record_role(record)) + stamp = record.get("timestamp") + when = "" + if isinstance(stamp, str) and stamp: + when = stamp.replace("T", " ").replace("Z", "")[:19] + lines: list[str] = [f"## {index} · {when or _now()} · {role}"] + lines.append(_KEY_MARKER.format(key)) + lines.append("") + sidechain = "（sidechain）" if record.get("isSidechain") else "" + if sidechain: + lines.append(sidechain) + lines.append("") + + blocks = _blocks(message.get("content")) + if not blocks: + raw = record.get("content") + if isinstance(raw, str) and raw.strip(): + blocks = [{"type": "text", "text": raw}] + for block in blocks: + kind = block.get("type") + if kind == "text": + body = _cap(str(block.get("text") or "")) + if body: + lines.append(body) + lines.append("") + elif kind == "thinking": + body = _cap(str(block.get("thinking") or block.get("text") or "")) + if body: + lines.append("*（thinking）*") + lines.append("") + lines.append("\n".join(f"> {line}" for line in body.splitlines()) or "> ") + lines.append("") + elif kind == "tool_use": + name = str(block.get("name") or "tool") + args = block.get("input") + body = json.dumps(args, ensure_ascii=False, indent=2, default=str) if args else "" + lines.append(f"**tool_call · {name}**") + lines.append("") + if body: + lines.append(_fence(_cap(body), "json")) + lines.append("") + elif kind == "tool_result": + name = str(block.get("name") or "") + body = _cap(_block_text(block.get("content"))) + flag = "（error）" if block.get("is_error") else "" + lines.append(f"**tool_result{flag}{' · ' + name if name else ''}**") + lines.append("") + if body: + lines.append(_fence(body, "text")) + lines.append("") + else: + lines.append(f"*（{kind or 'unknown'} block）*") + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +def iter_conversation_records(text: str) -> Iterator[tuple[str, dict[str, Any]]]: + """Yield ``(record key, record)`` for the conversation records in *text*. + + Only ``user`` / ``assistant`` records — the transcript's bookkeeping records + (``mode`` / ``cost-state`` / ``attachment`` / ``file-history-snapshot``) are not + conversation. The key is the record's ``uuid`` — the same identity L0 + deduplicates on. + """ + for index, record in enumerate(_iter_records(text), start=1): + if not isinstance(record, dict): + continue + if str(record.get("type") or "") not in ("user", "assistant"): + continue + key = str(record.get("uuid") or f"line{index}") + yield key, record + + +def iter_sections(text: str) -> Iterator[tuple[str, str]]: + """Yield ``(record key, markdown section)`` for every conversation record in *text*. + + Every block is kept: text, thinking, ``tool_use`` and ``tool_result``. + """ + for index, (key, record) in enumerate(iter_conversation_records(text), start=1): + yield key, _render_record(key, record, index) + + +def _header(*, session_id: str, host: str, user: str | None) -> str: + rows = [f"# {host} session {session_id}", "", f"- host: {host}", f"- session: {session_id}"] + if user: + rows.append(f"- user: {user}") + rows += [ + f"- created: {_now()}", + "- source: `memory_capture` 上传的原始 transcript（含工具调用与结果、thinking）", + "", + "---", + "", + ] + return "\n".join(rows) + + +def archive_path(session_dir: Path, session_id: str) -> Path: + safe = re.sub(r"[^0-9A-Za-z._-]+", "_", session_id).strip("_") or "session" + return session_dir / f"{safe}.md" + + +def append_session( + session_dir: Path, + *, + session_id: str, + text: str, + host: str, + user: str | None = None, +) -> Path | None: + """Append the records of one uploaded chunk to the session's markdown archive. + + Returns the file path when something was written, ``None`` when the chunk carried + no new record (a re-send) — so callers can skip without special-casing. + """ + path = archive_path(session_dir, session_id) + existing = "" + if path.exists(): + existing = path.read_text(encoding="utf-8", errors="replace") + seen = set(_KEY_PATTERN.findall(existing)) + index = len(seen) + + sections: list[str] = [] + for key, record in iter_conversation_records(text): + if key in seen: + continue + seen.add(key) + index += 1 + sections.append(_render_record(key, record, index)) + if not sections: + return None + + session_dir.mkdir(parents=True, exist_ok=True) + body = "\n".join(section.rstrip() + "\n" for section in sections) + if existing: + with path.open("a", encoding="utf-8") as handle: + handle.write("\n" + body) + else: + path.write_text( + _header(session_id=session_id, host=host, user=user) + body, encoding="utf-8" + ) + return path diff --git a/src/octop/infra/server.py b/src/octop/infra/server.py index 3f6075b92..311e32bcc 100644 --- a/src/octop/infra/server.py +++ b/src/octop/infra/server.py @@ -415,6 +415,14 @@ async def _boot_runtime(self, config: OctopConfig) -> None: configure_browser_idle_timeout(config.browser_idle_timeout_minutes) + # octop_memory recall 质量补丁（实体后缀解析 + 强 token 排序）： + # 在任意 MemoryService / MCP 记忆工具创建前应用，保证评测与生产召回一致。 + from octop.infra.agents.memory_recall_patch import ( + apply_memory_recall_patch, + ) + + apply_memory_recall_patch() + registry = AgentManager( repos=self.services.repos, paths=self.paths, diff --git a/tests/unit/agents/test_claude_transcript.py b/tests/unit/agents/test_claude_transcript.py new file mode 100644 index 000000000..dd8d38f9a --- /dev/null +++ b/tests/unit/agents/test_claude_transcript.py @@ -0,0 +1,830 @@ +"""Unit tests for the Claude Code transcript importer (infra/agents/claude_transcript).""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +from octop_memory.core import Memory + +from octop.infra.agents.claude_transcript import ( + import_content, + import_session, + parse_dsh_text, + parse_kiro_text, + parse_session, +) + +_SESSION = "sess-abc" +_TS = "2026-09-22T05:00:00.000Z" + + +def _record(**kwargs): + base = {"sessionId": _SESSION, "timestamp": _TS, "isSidechain": False} + base.update(kwargs) + return base + + +def _write(tmp_path: Path, records: list[dict]) -> Path: + path = tmp_path / "session.jsonl" + path.write_text( + "\n".join(json.dumps(record, ensure_ascii=False) for record in records), encoding="utf-8" + ) + return path + + +def _turn_records() -> list[dict]: + return [ + {"type": "queue-operation", "sessionId": _SESSION, "operation": "enqueue"}, + _record(type="progress", uuid="p1"), + _record(type="user", uuid="u1", message={"role": "user", "content": "帮我加一个导入命令"}), + _record( + type="assistant", + uuid="a1", + message={ + "role": "assistant", + "content": [ + {"type": "thinking", "thinking": "model scratchpad"}, + {"type": "text", "text": "好的，先看现有 CLI 结构"}, + { + "type": "tool_use", + "id": "toolu_1", + "name": "Read", + "input": {"file_path": "src/octop/cli/main.py"}, + }, + ], + }, + ), + _record( + type="user", + uuid="u2", + message={ + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_1", + "content": [{"type": "text", "text": "@click.group ..."}], + "is_error": False, + } + ], + }, + ), + ] + + +@pytest.fixture +def agent_memory(tmp_path: Path) -> Memory: + return Memory( + namespace="agent_TEST", + backend="sqlite", + backend_config={"db_path": str(tmp_path / "memory.sqlite")}, + ) + + +def test_parse_maps_conversation_records(tmp_path: Path) -> None: + parsed = parse_session(_write(tmp_path, _turn_records())) + + assert parsed.session_id == _SESSION + # thinking / progress / queue-operation carry no L0 content + assert [event.event_type for event in parsed.events] == [ + "user_message", + "assistant_message", + "tool_call", + "tool_result", + ] + # keys use the ORIGINAL block index (thinking keeps index 0) so they stay + # stable if the importer later changes which block types it maps + assert [event.key for event in parsed.events] == ["u1:0", "a1:1", "a1:2", "u2:0"] + + user, assistant, tool_call, tool_result = parsed.events + assert user.content == "帮我加一个导入命令" + assert assistant.content == "好的，先看现有 CLI 结构" + assert tool_call.content.startswith("Read ") + assert tool_call.payload["tool_name"] == "Read" + assert tool_call.payload["tool_use_id"] == "toolu_1" + assert tool_call.payload["input"] == {"file_path": "src/octop/cli/main.py"} + assert tool_result.content == "@click.group ..." + assert tool_result.payload["tool_use_id"] == "toolu_1" + assert tool_result.payload["is_error"] is False + assert user.timestamp.year == 2026 and user.timestamp.tzinfo is not None + # role/session markers survive on every event + assert {event.payload["role"] for event in parsed.events} == {"user", "assistant"} + assert all(event.payload["src_key"] == event.key for event in parsed.events) + + +def test_parse_skips_non_conversation_records(tmp_path: Path) -> None: + parsed = parse_session(_write(tmp_path, _turn_records())) + assert not [event for event in parsed.events if event.payload["src_key"].startswith("p1")] + + +def test_import_writes_l0_and_is_idempotent(tmp_path: Path, agent_memory: Memory) -> None: + path = _write(tmp_path, _turn_records()) + + first = import_session(agent_memory, path) + assert first.session_id == f"cc:{_SESSION}" + assert first.parsed == 4 + assert first.imported == 4 + assert first.skipped_existing == 0 + + stored = agent_memory.list_raw(session_id=f"cc:{_SESSION}", limit=100) + assert len(stored) == 4 + assert {raw.event_type for raw in stored} == { + "user_message", + "assistant_message", + "tool_call", + "tool_result", + } + assert all(raw.host == "claude-code" for raw in stored) + assert all(raw.thread_id == f"cc:{_SESSION}" for raw in stored) + + again = import_session(agent_memory, path) + assert again.imported == 0 + assert again.skipped_existing == 4 + assert len(agent_memory.list_raw(session_id=f"cc:{_SESSION}", limit=100)) == 4 + + +def test_import_appends_only_new_turns(tmp_path: Path, agent_memory: Memory) -> None: + records = _turn_records() + path = _write(tmp_path, records) + import_session(agent_memory, path) + + records.append( + _record(type="user", uuid="u3", message={"role": "user", "content": "继续，加上 --dry-run"}) + ) + path = _write(tmp_path, records) + + stats = import_session(agent_memory, path) + assert stats.imported == 1 + assert stats.skipped_existing == 4 + + +def test_import_dry_run_writes_nothing(tmp_path: Path, agent_memory: Memory) -> None: + path = _write(tmp_path, _turn_records()) + + stats = import_session(agent_memory, path, dry_run=True) + assert stats.imported == 0 + assert stats.parsed == 4 + assert agent_memory.list_raw(session_id=f"cc:{_SESSION}", limit=100) == [] + + +def test_import_caps_long_tool_results(tmp_path: Path, agent_memory: Memory) -> None: + huge = "x" * 20_000 + records = [ + _record( + type="user", + uuid="u1", + message={ + "role": "user", + "content": [ + {"type": "tool_result", "tool_use_id": "t", "content": huge, "is_error": False} + ], + }, + ) + ] + import_session(agent_memory, _write(tmp_path, records)) + + stored = agent_memory.list_raw(session_id=f"cc:{_SESSION}", limit=10) + assert len(stored) == 1 + assert len(stored[0].content) == 4000 + + +def test_user_attribution_is_stored(tmp_path: Path, agent_memory: Memory) -> None: + import_session(agent_memory, _write(tmp_path, _turn_records()), user="alice") + stored = agent_memory.list_raw(session_id=f"cc:{_SESSION}", limit=10) + assert {raw.user for raw in stored} == {"alice"} + + +@pytest.mark.parametrize( + "tool_name", + [ + "memory_search", + "mcp__octop-memory-local__memory_search", + "mcp__octop-memory-prod__memory_search", + "mcp__octop-memory-oauth__memory_search", + "mcp__whatever-the-user-called-it__memory_recall", + "mcp_octop_memory_local_memory_search", + "mcp_octop_memory_prod_memory_get", + ], +) +def test_our_read_tools_are_recognized_under_any_mcp_name(tool_name: str) -> None: + """MCP 服务名可能是 local / prod / oauth 或用户自取的名字：只按工具名认。""" + from octop.infra.agents.claude_transcript import _is_own_read_tool + + assert _is_own_read_tool(tool_name) + # 写工具（memory_capture / memory_save）不是「读工具」，别把用户自己的调用误丢 + assert not _is_own_read_tool("mcp__octop-memory-prod__memory_capture") + assert not _is_own_read_tool("Bash") + + +def test_parse_drops_our_own_memory_read_results(tmp_path: Path) -> None: + """Re-importing our own recall output would close the echo loop.""" + records = [ + _record( + type="assistant", + uuid="a1", + message={ + "role": "assistant", + "content": [ + { + "type": "tool_use", + "id": "t1", + "name": "mcp__octop-memory-local__memory_search", + "input": {"query": "x"}, + }, + {"type": "tool_use", "id": "t2", "name": "Bash", "input": {"command": "ls"}}, + ], + }, + ), + _record( + type="user", + uuid="u1", + message={ + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "t1", + "content": '{"hits": [{"snippet": "旧记忆"}]}', + "is_error": False, + }, + { + "type": "tool_result", + "tool_use_id": "t2", + "content": "file list", + "is_error": False, + }, + ], + }, + ), + ] + + parsed = parse_session(_write(tmp_path, records)) + + assert [event.event_type for event in parsed.events] == [ + "tool_call", + "tool_call", + "tool_result", + ] + assert parsed.events[-1].content == "file list" + assert parsed.skipped_own_results == 1 + + +def test_import_content_drops_marked_events(agent_memory: Memory) -> None: + """A recalled block must not be written back into L0 as new evidence.""" + records = [ + _record(type="user", uuid="u1", message={"role": "user", "content": "正常提问"}), + _record( + type="user", + uuid="u2", + message={"role": "user", "content": "## Memory Recall\n- [atom] 旧记忆块"}, + ), + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + stats = import_content(agent_memory, text, skip_markers=("## Memory Recall",)) + + assert stats.parsed == 2 + assert stats.skipped_echo == 1 + assert stats.imported == 1 + stored = agent_memory.list_raw(session_id=stats.session_id, limit=10) + assert [raw.content for raw in stored] == ["正常提问"] + + +def test_user_messages_are_attributed_to_the_caller(agent_memory: Memory) -> None: + """transcript 形态也要让发送者进正文：提取器 prompt 看不到 raw 的 ``user`` 列。""" + records = [ + _record(type="user", uuid="u1", message={"role": "user", "content": "帮我加一个导入命令"}), + _record( + type="assistant", + uuid="a1", + message={"role": "assistant", "content": [{"type": "text", "text": "好的"}]}, + ), + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + stats = import_content(agent_memory, text, session_id="cc:s1", user="alice") + + by_type = {raw.event_type: raw for raw in agent_memory.list_raw(session_id="cc:s1", limit=10)} + assert by_type["user_message"].content == "alice说：帮我加一个导入命令" + assert by_type["assistant_message"].content == "好的" # 助手侧不加前缀 + assert by_type["user_message"].user == "alice" # raw 的 user 列照旧 + assert stats.imported == 2 + + # 重发（offset 丢失后的 mode=full）靠 src_key 去重，也不会把前缀叠成两层 + again = import_content(agent_memory, text, session_id="cc:s1", user="alice") + assert again.imported == 0 + + +def test_sidechain_user_records_are_not_attributed(agent_memory: Memory) -> None: + """子代理记录里的 user 消息是系统合成提示，不是调用者说过的话。""" + records = [ + _record(type="user", uuid="u1", message={"role": "user", "content": "真实提问"}), + _record( + type="user", + uuid="u2", + isSidechain=True, + message={"role": "user", "content": "子代理提示"}, + ), + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + import_content(agent_memory, text, session_id="cc:s2", user="alice") + + contents = {raw.content for raw in agent_memory.list_raw(session_id="cc:s2", limit=10)} + assert "alice说：真实提问" in contents + assert "子代理提示" in contents + assert "alice说：子代理提示" not in contents + + +# --------------------------------------------------------------------------- +# Kiro (``~/.kiro/sessions///messages.jsonl``) +# --------------------------------------------------------------------------- + +_KIRO_RECORDS = [ + { + "id": "r1", + "timestamp": _TS, + "payload": {"type": "user", "content": "Kiro 里问的问题", "source": "chat"}, + }, + {"id": "r2", "timestamp": _TS, "payload": {"type": "turn_start", "executionId": "e1"}}, + { + "id": "r3", + "timestamp": _TS, + "payload": { + "type": "ContextualHookInvoked", + "hookId": "h", + "output": "[octop memory search] 旧记忆", + }, + }, + { + "id": "r4", + "timestamp": _TS, + "payload": { + "type": "tool_call", + "toolCallId": "c1", + "toolName": "readFiles", + "args": {"files": ["a.ts"]}, + }, + }, + { + "id": "r5", + "timestamp": _TS, + "payload": { + "type": "tool_result", + "toolCallId": "c1", + "content": "file body", + "success": True, + }, + }, + { + "id": "r6", + "timestamp": _TS, + "payload": {"type": "assistant", "content": "scratchpad", "operationType": "Reasoning"}, + }, + { + "id": "r7", + "timestamp": _TS, + "payload": {"type": "assistant", "content": "这是回答", "operationType": "Say"}, + }, + {"id": "r8", "timestamp": _TS, "payload": {"type": "turn_end", "stopReason": "end_turn"}}, +] + + +def _kiro_text() -> str: + return "\n".join(json.dumps(r, ensure_ascii=False) for r in _KIRO_RECORDS) + + +def test_parse_kiro_maps_records() -> None: + parsed = parse_kiro_text(_kiro_text()) + + assert [e.event_type for e in parsed.events] == [ + "user_message", + "tool_call", + "tool_result", + "assistant_message", + ] + assert parsed.events[3].content == "这是回答" # Reasoning 是草稿，跳过 + assert parsed.events[1].payload["tool_name"] == "readFiles" + assert parsed.events[2].payload["success"] is True + # ContextualHookInvoked 是我们自己 hook 的调用记录（连 output 都在里面） + assert parsed.skipped_own_results == 1 + assert all(e.event_type != "tool_result" or "旧记忆" not in e.content for e in parsed.events) + + +def test_import_content_kiro_is_idempotent(agent_memory: Memory) -> None: + first = import_content( + agent_memory, _kiro_text(), session_id="kiro:s1", host="kiro", parser="kiro" + ) + assert first.session_id == "kiro:s1" + assert first.imported == 4 + assert first.by_type == { + "user_message": 1, + "tool_call": 1, + "tool_result": 1, + "assistant_message": 1, + } + + again = import_content( + agent_memory, _kiro_text(), session_id="kiro:s1", host="kiro", parser="kiro" + ) + assert again.imported == 0 + assert again.skipped_existing == 4 + + +def test_import_content_rejects_unknown_parser(agent_memory: Memory) -> None: + with pytest.raises(ValueError, match="unknown transcript parser"): + import_content(agent_memory, _kiro_text(), parser="no-such-host") + + +# --------------------------------------------------------------------------- +# DSH (``~/.dsh/sessions///session.jsonl.zstd``) +# --------------------------------------------------------------------------- + +_DSH_MS = 1788952324078 +_DSH_RECORDS = [ + {"type": "session", "version": 0, "id": "session-x", "createdAt": _DSH_MS, "cwd": "/tmp"}, + { + "type": "user/message", + "seq": 1, + "time": _DSH_MS, + "data": {"content": [{"type": "text", "text": "真实提问"}], "source": {"kind": "user"}}, + }, + { + "type": "user/message", + "seq": 2, + "time": _DSH_MS, + "data": { + "content": [ + {"type": "text", "text": "Current runtime context. This snapshot supersedes…"} + ], + "source": {"kind": "plugin"}, + }, + }, + {"type": "assistant/chunk", "seq": 3, "time": _DSH_MS, "data": {"text": "流式增量"}}, + { + "type": "assistant/message", + "seq": 4, + "time": _DSH_MS, + "data": { + "turn": 1, + "step": 1, + "message": { + "role": "assistant", + "content": [ + {"type": "reasoning", "text": "草稿"}, + {"type": "text", "text": "回答正文"}, + { + "type": "tool-call", + "id": "call_1", + "name": "bash", + "arguments": '{"command": "ls"}', + }, + ], + }, + }, + }, + { + "type": "tool/call", + "seq": 5, + "time": _DSH_MS, + "data": { + "turn": 1, + "step": 1, + "callId": "call_1", + "name": "bash", + "arguments": '{"command": "ls"}', + }, + }, + { + "type": "tool/result", + "seq": 6, + "time": _DSH_MS, + "data": { + "turn": 1, + "step": 1, + "message": { + "source": {"kind": "tool", "callId": "call_1"}, + "content": [ + { + "type": "tool-result", + "toolCallId": "call_1", + "content": [{"type": "text", "text": "file list"}], + "isError": False, + } + ], + }, + }, + }, + {"type": "turn/end", "seq": 7, "time": _DSH_MS, "data": {"turn": 1, "stopReason": "end_turn"}}, +] + + +def _dsh_text() -> str: + return "\n".join(json.dumps(r, ensure_ascii=False) for r in _DSH_RECORDS) + + +def test_parse_dsh_maps_records() -> None: + parsed = parse_dsh_text(_dsh_text()) + + assert parsed.session_id == "session-x" + assert [e.event_type for e in parsed.events] == [ + "user_message", + "assistant_message", + "tool_call", + "tool_result", + ] + # 流式增量与 Reasoning 草稿都不入库 + assert all("流式增量" not in e.content for e in parsed.events) + assert all("草稿" not in e.content for e in parsed.events) + # 同一个 call 既在 assistant/message 的 tool-call 块里、又是独立 tool/call 记录 —— 只留一条 + assert [e.content for e in parsed.events if e.event_type == "tool_call"] == [ + 'bash {"command": "ls"}' + ] + assert parsed.events[3].content == "file list" + assert parsed.events[3].payload["is_error"] is False + # plugin / skill-catalog / agent-instructions 注入是管道，不是用户说的话 + assert parsed.skipped_own_results == 1 + + +def test_import_content_dsh_uses_terminal_prefix(agent_memory: Memory) -> None: + stats = import_content(agent_memory, _dsh_text(), host="dsh", parser="dsh") + + assert stats.session_id == "dsh:session-x" + assert stats.imported == 4 + assert stats.by_type == { + "user_message": 1, + "assistant_message": 1, + "tool_call": 1, + "tool_result": 1, + } + + +def test_echo_rule_differs_by_role() -> None: + """回声判定按角色区分：助手侧看是否以标记开头，用户侧看标记是否出现。""" + from octop.infra.agents.claude_transcript import _is_echo + + markers = ("[octop memory search]", "[octop memory capture]") + # 助手侧：只是引用标记的回答必须保留 + assert not _is_echo( + "靠两个 hook：召回把 [octop memory search] 块注入上下文", "assistant_message", markers + ) + assert _is_echo("[octop memory search] related memories…", "assistant_message", markers) + # 用户侧：出现标记就是我们注入的上下文（CC 会加 "Stop hook feedback:" 前缀） + assert _is_echo( + "Stop hook feedback:\n[octop memory search] 请记录要点", "user_message", markers + ) + assert not _is_echo("正常提问", "user_message", markers) + + +def test_a_reply_that_quotes_our_marker_survives(agent_memory: Memory) -> None: + """模型回答只是**引用**召回标记时必须入库——否则解释接入方式的对话会被当回声丢掉。""" + records = [ + _record(type="user", uuid="u1", message={"role": "user", "content": "记忆接入怎么做的？"}), + _record( + type="assistant", + uuid="a1", + message={ + "role": "assistant", + "content": [ + { + "type": "text", + "text": "靠两个 hook：召回会把 [octop memory search] 块注入上下文。", + } + ], + }, + ), + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + stats = import_content(agent_memory, text, skip_markers=("[octop memory search]",)) + + assert stats.skipped_echo == 0 + assert stats.imported == 2 + stored = agent_memory.list_raw(session_id=stats.session_id, limit=10) + assert any("[octop memory search]" in raw.content for raw in stored) + + +def test_codex_rollout_maps_to_events() -> None: + """Codex rollout：event_msg 带对话，response_item 带工具与（重复的）消息副本。""" + from octop.infra.agents.claude_transcript import parse_codex_text + + records = [ + { + "timestamp": "2026-08-08T13:47:31.000Z", + "type": "session_meta", + "payload": {"id": "s-1", "cwd": "/tmp", "cli_version": "0.142.5"}, + }, + { + "timestamp": "2026-08-08T13:47:31.163Z", + "type": "event_msg", + "payload": {"type": "user_message", "message": "现在是什么模型\n"}, + }, + { + "timestamp": "2026-08-08T13:47:32.000Z", + "type": "response_item", + "payload": { + "type": "message", + "role": "developer", + "content": [{"type": "input_text", "text": ""}], + }, + }, + { + "timestamp": "2026-08-08T13:47:33.000Z", + "type": "response_item", + "payload": { + "type": "message", + "role": "assistant", + "id": "a1", + "content": [{"type": "output_text", "text": "我是 Codex。"}], + }, + }, + { + "timestamp": "2026-08-08T13:47:34.000Z", + "type": "response_item", + "payload": { + "type": "function_call", + "id": "t1", + "name": "shell", + "arguments": '{"command":"ls"}', + }, + }, + { + "timestamp": "2026-08-08T13:47:35.000Z", + "type": "response_item", + "payload": {"type": "function_call_output", "id": "t2", "output": "a.txt"}, + }, + { + "timestamp": "2026-08-08T13:47:36.000Z", + "type": "turn_context", + "payload": {"cwd": "/tmp", "model": "x"}, + }, + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + parsed = parse_codex_text(text) + + assert [event.event_type for event in parsed.events] == [ + "user_message", + "assistant_message", + "tool_call", + "tool_result", + ] + assert parsed.events[0].content == "现在是什么模型\n" + assert parsed.events[1].content == "我是 Codex。" + assert parsed.events[2].content.startswith("shell ") + assert parsed.events[3].content == "a.txt" + assert parsed.skipped_own_results == 1 # developer instructions 是宿主管线 + + +def test_codex_does_not_store_the_same_turn_twice() -> None: + """event_msg 与 response_item 会带同一段文本：只入库一次。""" + from octop.infra.agents.claude_transcript import parse_codex_text + + records = [ + { + "timestamp": "2026-08-08T13:47:31.163Z", + "type": "event_msg", + "payload": {"type": "user_message", "message": "同一个问题"}, + }, + { + "timestamp": "2026-08-08T13:47:31.900Z", + "type": "response_item", + "payload": { + "type": "message", + "role": "user", + "id": "u9", + "content": [{"type": "input_text", "text": "同一个问题"}], + }, + }, + ] + text = "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + parsed = parse_codex_text(text) + + assert [event.content for event in parsed.events] == ["同一个问题"] + assert parsed.skipped_own_results == 1 + + +def test_cursor_transcript_shapes_are_tolerated() -> None: + """Cursor 的 transcript_path 格式未公开：JSONL / 数组 / {messages:[]} 都要能读。""" + from octop.infra.agents.claude_transcript import parse_cursor_text + + pairs = [("user", "怎么接入？"), ("assistant", "两条 hook 就够了。")] + shapes = [ + "\n".join(json.dumps({"role": r, "content": c}, ensure_ascii=False) for r, c in pairs), + json.dumps([{"role": r, "text": c} for r, c in pairs], ensure_ascii=False), + json.dumps( + {"messages": [{"role": r, "content": [{"type": "text", "text": c}]} for r, c in pairs]}, + ensure_ascii=False, + ), + ] + + for text in shapes: + parsed = parse_cursor_text(text) + assert [event.event_type for event in parsed.events] == [ + "user_message", + "assistant_message", + ], text[:60] + assert parsed.events[0].content == "怎么接入？" + + +def test_cursor_agent_transcript_jsonl_keeps_text_and_drops_tool_blocks() -> None: + """stop 交给的真实文件：一行一个 {role, message:{content:[{type,text}]}}。""" + from octop.infra.agents.claude_transcript import parse_cursor_text + + lines = [ + { + "role": "user", + "message": {"content": [{"type": "text", "text": "怎么接入？"}]}, + }, + { + "role": "assistant", + "message": { + "content": [ + {"type": "text", "text": "两条 hook 就够了。"}, + {"type": "tool_use", "name": "read", "input": {"path": "a"}}, + ] + }, + }, + ] + text = "\n".join(json.dumps(line, ensure_ascii=False) for line in lines) + + parsed = parse_cursor_text(text) + + assert [event.event_type for event in parsed.events] == [ + "user_message", + "assistant_message", + ] + assert parsed.events[0].content == "怎么接入？" + assert parsed.events[1].content == "两条 hook 就够了。" + + +def test_cursor_skips_entries_without_a_conversational_role() -> None: + from octop.infra.agents.claude_transcript import parse_cursor_text + + text = json.dumps( + [ + {"role": "user", "content": "问题"}, + {"kind": "toolCall", "tool": "read", "args": {"path": "a"}}, + {"role": "assistant", "content": "回答"}, + ], + ensure_ascii=False, + ) + + parsed = parse_cursor_text(text) + + assert [event.event_type for event in parsed.events] == ["user_message", "assistant_message"] + assert parsed.skipped_own_results == 1 + + +def test_cursor_delta_uploads_do_not_collide_on_line_index_src_key() -> None: + """stop hook ships byte deltas; each chunk used to renumber from line0. + + Distinct user turns at the head of consecutive deltas must get distinct + ``src_key`` values, otherwise the later user message is dropped as + ``skipped_existing`` while later assistant lines still import. + """ + from octop.infra.agents.claude_transcript import parse_cursor_text + + def _line(role: str, text: str) -> str: + return json.dumps( + {"role": role, "message": {"content": [{"type": "text", "text": text}]}}, + ensure_ascii=False, + ) + + delta_a = "\n".join( + [ + _line("user", "上一轮用户问题"), + _line("assistant", "上一轮助手回答"), + ] + ) + delta_b = "\n".join( + [ + _line("user", "原始素材只有回复，没有用户消息"), + _line("assistant", "这是助手的长回复，超过十二个字符。"), + ] + ) + + keys_a = {event.payload["src_key"] for event in parse_cursor_text(delta_a).events} + parsed_b = parse_cursor_text(delta_b) + keys_b = [event.payload["src_key"] for event in parsed_b.events] + + assert parsed_b.events[0].event_type == "user_message" + assert parsed_b.events[0].content == "原始素材只有回复，没有用户消息" + assert keys_b[0] not in keys_a + assert keys_b[0].startswith("c") and keys_b[0].endswith(":0") + # Re-parsing the same delta stays idempotent. + assert parse_cursor_text(delta_b).events[0].payload["src_key"] == keys_b[0] + + +def test_cursor_prefers_explicit_record_id_for_src_key() -> None: + from octop.infra.agents.claude_transcript import parse_cursor_text + + text = json.dumps( + {"id": "msg_abc", "role": "user", "content": "有稳定 id"}, + ensure_ascii=False, + ) + parsed = parse_cursor_text(text) + assert parsed.events[0].payload["src_key"] == "msg_abc:0" diff --git a/tests/unit/agents/test_client_setup.py b/tests/unit/agents/test_client_setup.py new file mode 100644 index 000000000..d600c0e2c --- /dev/null +++ b/tests/unit/agents/test_client_setup.py @@ -0,0 +1,1146 @@ +"""Unit tests for the interactive local-terminal installer generator.""" + +from __future__ import annotations + +import json +import os +import shutil +import subprocess +import tomllib +from pathlib import Path + +import pytest + +from octop.infra.agents.client_setup import ( + HOOKS_DIRS, + TERMINALS, + render_setup_script, + render_uninstall_script, +) + +_URL = "http://127.0.0.1:8088/mcp/memory/" + +posix_only = pytest.mark.skipif(os.name != "posix", reason="假的 dsh 命令用 POSIX 可执行脚本") + + +def _script(**overrides) -> str: + kwargs = {"mcp_url": _URL} + kwargs.update(overrides) + return render_setup_script(**kwargs) + + +def test_installer_detects_the_local_terminals() -> None: + script = _script() + + assert '[ -e "$HOME/.claude.json" ] || [ -d "$HOME/.claude" ] || command -v claude' in script + assert '[ -d "$HOME/.kiro" ] || command -v kiro' in script + assert '[ -d "$HOME/.dsh" ] || command -v dsh' in script + assert 'echo " 1) Claude Code' in script + assert 'echo " 2) Kiro' in script + assert 'echo " 3) DSH' in script + + +def test_installer_asks_instead_of_demanding_parameters() -> None: + script = _script() + + # 交互走 /dev/tty：curl | sh 时 stdin 是脚本本身，直接 read 会把脚本吃掉 + assert "if (true /dev/null; then HAS_TTY=1; fi" in script + assert script.count("< /dev/tty") >= 3 # 探测 + 选择 + token + 专家 ID + assert "要配置哪些？" in script + assert "专家 ID: " in script + # 凭据提示按目标分流：本机实例与云端都问同一个实例共享 token + assert "实例共享 token（本机实例，走 Authorization: Bearer）" in script + assert "实例共享 token（云上实例）: " in script + assert "stty -echo" in script # 不回显 + # 没有 tty 时不盲读 stdin，而是明确要求环境变量 + assert 'TOKEN="${OCTOP_MEMORY_TOKEN:-}"' in script + assert "非交互运行时请设 OCTOP_MEMORY_TOKEN" in script + assert "非交互运行时请设 OCTOP_AGENT_ID" in script + # 只有一份凭据：不再有 OAuth 前置 / 个人令牌这套东西 + assert "OCTOP_ENTRY_TOKEN" not in script + assert "OCTOP_DISABLE_OAUTH" not in script + assert "个人令牌" not in script + # 会问用户标识（写进 X-Octop-User-Id） + assert "用户标识（建议填域账号，如 alice" in script + + +def test_terminal_pick_allows_multiselect() -> None: + """三个选项可多选：1,2 / 1 3 / 12 / a=全部，回车=全部检测到的。""" + script = _script() + + assert "可多选：1,2 / 1 3 / 12 / a=全部" in script + assert "''|a|A|all|ALL|全部) _chosen=\"$DETECTED\"" in script + assert "fold -w1" in script # 连写数字 12/13 按位拆开 + assert 'case ",$_chosen," in *",$_add,"*' in script # 去重 + assert "已忽略" in script # 未检测到/无法识别 + assert "没有选中任何已检测到的终端，请重选" in script # 选不中会重问 + assert 'while [ -z "$TERMINALS" ]' in script + + +def test_installer_embeds_no_credential() -> None: + """没内置默认值时不内嵌密钥：token 由运行的人输入（部署可选地内置一个，见 DEFAULT_TOKEN）。""" + script = _script() + + assert "DEFAULT_TOKEN=" not in script + assert "Bearer " in script # 只在运行时按输入拼头，不是写死的 token + # header 名可配、缺省 Authorization（不再按 token 前缀猜方言，也没有第二份凭据） + assert 'TOKEN_HEADER="${OCTOP_TOKEN_HEADER:-Authorization}"' in script + assert "OCTOP_ENTRY_TOKEN" not in script + + +def test_known_values_only_prefill_prompts() -> None: + script = _script(agent_id="RY9RQC", terminals="kiro") + + assert "DEFAULT_AGENT='RY9RQC'" in script + assert "DEFAULT_TERMINALS='kiro'" in script + + +def test_setup_script_shell_quotes_query_prefills() -> None: + script = _script(agent_id="x';id", terminals="cursor';id") + # Classic POSIX single-quote escape: ' → '"'"' + assert "DEFAULT_AGENT='x'\"'\"';id'" in script + assert "DEFAULT_TERMINALS='cursor'\"'\"';id'" in script + assert "DEFAULT_AGENT='x';id'" not in script + assert "DEFAULT_TERMINALS='cursor';id'" not in script + + +def test_each_terminal_gets_its_own_hooks_directory() -> None: + script = _script() + + assert "CC_DIR='~/.claude/hooks-events'" in script + assert "KIRO_DIR='~/.kiro/hooks'" in script + assert "DSH_DIR='~/.dsh/hooks'" in script + assert "CODEX_DIR='~/.codex/hooks'" in script + assert "CURSOR_DIR='~/.cursor/hooks'" in script + assert HOOKS_DIRS == { + "claude-code": "~/.claude/hooks-events", + "kiro": "~/.kiro/hooks", + "dsh": "~/.dsh/hooks", + "codex": "~/.codex/hooks", + "cursor": "~/.cursor/hooks", + } + + +def test_claude_code_uses_two_command_hooks() -> None: + """Claude Code：召回与上报都必须是 command hook。 + + - ``UserPromptSubmit`` 上 `mcp_tool` 会被静默跳过（实测 2.1.263）； + - ``Stop`` 用脚本，脚本自己后台脱离+延迟，等本轮落盘后再读 transcript + （``SessionEnd`` 不可用：那里没有 MCP 客户端上下文）。 + """ + script = _script() + + assert ( + 'search = {"type": "command", "command": hook_command(f"{hooks_dir}/claude-code/user-prompt-recall.mjs")}' + in script + ) + assert ( + 'stop = {"type": "command", "command": hook_command(f"{hooks_dir}/claude-code/stop-capture.mjs")}' + in script + ) + assert "该事件不支持 ``mcp_tool``" in script + assert "SessionEnd`` 不能用" in script + # 简写形态：事件直接挂 handler 数组，不套 matcher 组那层 + # （某些终端的 SDK 侧不保证 hooks 数组按顺序执行，少一层嵌套更稳） + assert 'return {"UserPromptSubmit": [search], "Stop": [stop]}' in script + # 收尾提示按事件名列出两端角色（Cursor 的事件名不同，故不能只写 UserPromptSubmit/Stop） + assert "用户提交时召回" in script and "memory_search" in script + assert "回合结束时上报" in script and "memory_capture" in script + assert "Cursor 的 beforeSubmitPrompt" in script + + +def test_claude_code_clears_the_legacy_session_end_hook() -> None: + """早期版本挂过 SessionEnd（mcp_tool 在那里不可用、会持续报错）：重装时要清掉。""" + script = _script() + + assert '_upsert_groups(st, "SessionEnd", [])' in script + assert 'st["hooks"].pop("SessionEnd", None)' in script + assert 'hooks["SessionEnd"]' not in script # 不再定义 + + +def test_previous_inline_hook_command_is_replaced_by_the_wrapper(tmp_path) -> None: + """老配置里那串内联 node 命令（含 ``.mjs`` 与转义字符）重装时换成包装脚本，不留重复。 + + 这也是升级路径：``_is_our_handler`` 同时认 ``.mjs`` / ``.sh`` 两种形态，所以老条目会被 + 替换掉而不是并列留下。 + """ + legacy = ( + "/usr/bin/node /home/u/.claude/hooks-events/claude-code/stop-capture.mjs " + '|| echo "octop-memory: hook 执行失败（node 路径不对或脚本报错）" >&2' + ) + home, _ = _run_claude_code_installer( + tmp_path, {}, {"hooks": {"Stop": [{"hooks": [{"type": "command", "command": legacy}]}]}} + ) + stops = json.loads((home / ".claude/settings.json").read_text(encoding="utf-8"))["hooks"][ + "Stop" + ] + assert len(stops) == 1 # 老命令被替换，没有并列 + assert stops[0]["type"] == "command" # 换成简写形态（事件直接挂 handler） + command = stops[0]["command"] + assert command.endswith("claude-code/stop-capture.sh") + assert ".mjs" not in command # 配置里不再有那串长命令 + assert (home / ".claude/hooks-events/claude-code/stop-capture.sh").exists() + + +def test_foreign_hook_groups_are_kept_next_to_ours(tmp_path) -> None: + """别人的 hook（matcher 组或裸 handler）都要原样留着，只动我们自己那两条。""" + settings = { + "hooks": { + "Stop": [ + {"matcher": "*", "hooks": [{"type": "command", "command": "git-ai checkpoint"}]}, + {"type": "command", "command": "/usr/local/bin/other-check"}, + ] + } + } + home, _ = _run_claude_code_installer(tmp_path, {}, settings) + stops = json.loads((home / ".claude/settings.json").read_text(encoding="utf-8"))["hooks"][ + "Stop" + ] + + assert stops[0] == { + "matcher": "*", + "hooks": [{"type": "command", "command": "git-ai checkpoint"}], + } + assert stops[1] == {"type": "command", "command": "/usr/local/bin/other-check"} + assert stops[2]["command"].endswith("claude-code/stop-capture.sh") # 我们的追加在最后 + + +def test_scripts_are_installed_for_every_host_and_mode() -> None: + """脚本直传不再挑 token 方言：谁能问的由 X-Octop-User-Id 表达，所以任何模式都装。 + + （早期版本用 ``SCRIPT_OK = LOCAL`` 之类的判据把共享 token 挡在脚本之外，结果 Kiro 上报 + 退化成模型自己总结、DSH 干脆不装 hooks。） + """ + script = _script() + + assert "SCRIPT_OK" not in script + assert "CAPTURE_PROMPT" not in script # 上报绝不走"让模型总结"的提示词 + # Claude Code 任何模式都要装脚本（召回 hook 是 command 脚本） + assert "def do_claude_code():\n # 两条 hook 都是 command 脚本" in script + assert '_install_hooks(CC_DIR, "claude-code")' in script + assert '_install_hooks(KIRO_DIR, "kiro")' in script + assert '_install_hooks(DSH_DIR, "dsh")' in script + assert '_install_hooks(CODEX_DIR, "codex")' in script + assert '_install_hooks(CURSOR_DIR, "cursor")' in script + + +def test_kiro_capture_is_always_a_command_hook() -> None: + """Kiro：召回用提示词（Kiro 只支持 runCommand/askAgent）；上报始终用脚本提交整段 jsonl。""" + script = _script() + + kiro = script[script.index("def do_kiro():") : script.index("def _dsh_bridge(")] + # 召回仍然是 agent 提示词（runCommand 的 stdout 不会被注入上下文） + assert '"action": {"type": "agent", "prompt": SEARCH_PROMPT}' in kiro + # 上报只有 command 一种形态，没有提示词兜底 + assert ( + '{"type": "command", "command": hook_command(f"{KIRO_DIR}/kiro/stop-capture.mjs")}' in kiro + ) + assert '"type": "agent", "prompt": CAPTURE_PROMPT' not in kiro + assert "提交整段 messages.jsonl" in kiro + assert "Kiro 上报 hook" in script + + +def test_prompts_carry_the_role_instructions() -> None: + script = _script() + + assert "先用 memory_search 检索" in script + # 上报没有提示词：模型"总结会话"不算捕获原始会话 + assert "用 memory_capture 记录" not in script + + +def test_generated_script_embeds_each_hosts_scripts() -> None: + """按宿主分别内嵌：共享传输模块 + 该宿主的入口脚本（含 detach/延迟逻辑）。""" + script = _script() + + assert "OCTOP_FILES_CLAUDE_CODE=" in script + assert "OCTOP_FILES_KIRO=" in script and "OCTOP_FILES_DSH=" in script + assert "OCTOP_FILES_CODEX=" in script and "OCTOP_FILES_CURSOR=" in script + assert "OCTOP_FILES_PRESET=" in script + assert "HOOK_FILES = {" in script and "OCTOP_FILES_" in script + assert "lib/octop-memory-client.mjs" in script + assert "claude-code/stop-capture.mjs" in script and "kiro/stop-capture.mjs" in script + assert "dsh/stop-capture.mjs" in script and "dsh/user-prompt-recall.mjs" in script + assert "dsh/memory-target.mjs" in script + assert "codex/user-prompt-recall.mjs" in script and "codex/stop-capture.mjs" in script + assert "cursor/user-prompt-recall.mjs" in script and "cursor/stop-capture.mjs" in script + # 旧版的单文件变量不再使用 + assert "OCTOP_STOP_B64" not in script and "OCTOP_RECALL_B64" not in script + + +def test_dsh_bridge_points_at_the_dsh_entry_scripts() -> None: + """DSH 的桥接只跑 command 钩子（mcp_tool / prompt 会被跳过），且必须指向 DSH 自己的脚本。 + + DSH 只装 ``dsh/`` 目录下的入口脚本；指向 ``claude-code/`` 等于两条钩子都指向不存在的文件。 + """ + script = _script() + + assert "def _dsh_bridge(hooks_dir):" in script + assert 'DSH_DIR / "octop-hooks.json"' in script + assert "{hooks_dir}/dsh/user-prompt-recall.mjs" in script + assert "{hooks_dir}/dsh/stop-capture.mjs" in script + bridge = script[script.index("def _dsh_bridge(") : script.index("def do_dsh():")] + assert "{hooks_dir}/claude-code/" not in bridge + + +def test_dsh_installs_the_preset_bundle() -> None: + """DSH ≥0.1.7 的 preset 由 bundle patch 声明：安装器生成 bundle 并装进 profile。""" + script = _script() + + assert "OCTOP_FILES_PRESET=" in script + assert "def _install_dsh_preset(bridge):" in script + assert "def _select_dsh_bundle(profile_dir, name):" in script + assert '"__OCTOP_HOOKS_JSON__"' in script # 模板占位符换成实际桥接文件 + assert '"dsh", "plugin", "--profile", profile, "add", str(bundle)' in script + assert '"pnpm", "add", str(bundle)' in script # 源码 checkout 常没有 dsh 命令 + assert "def _run_dsh_install(bundle, profile, profile_dir):" in script + assert ( + 'config.setdefault("dsh", {}).setdefault("profile", {}).setdefault("bundles", [])' in script + ) + assert "重启 DSH" in script + assert "没找到 DSH profile" in script + + +def test_loopback_targets_use_the_configured_header() -> None: + """本机实例的共享 token 缺省走 ``Authorization: Bearer``；header 名可配、不猜前缀。""" + script = _script() + + # 判据用 hooks 真正要打的地址 + assert 'URL_HOST="$(printf \'%s\' "$HOOK_URL"' in script + assert "127.0.0.1|localhost) IS_LOCAL=1" in script + assert "实例共享 token（本机实例，走 Authorization: Bearer）" in script + assert "实例共享 token（云上实例）: " in script + # 不再有 token 前缀方言判定或硬编码 header 名 + assert 'TOKEN_HEADER="${OCTOP_TOKEN_HEADER:-Authorization}"' in script + assert "客户端 MCP 条目与 hooks：" in script + + +def test_node_is_resolved_to_an_absolute_path() -> None: + """hook 由终端派生、常在非 login shell 里跑：node 必须解析成绝对路径。 + + 裸 node（nvm 场景）在那种 shell 里找不到，而 command hook 的非 0 退出是静默的。 + """ + script = _script() + + assert 'NODE_BIN="${OCTOP_NODE:-}"' in script + assert "command -v node" in script + assert "$HOME/.nvm/versions/node" in script # nvm 是最常见的坑 + assert "/opt/homebrew/bin/node" in script and "/usr/local/bin/node" in script + assert "找不到 node 的绝对路径" in script # 都找不到时明确告警 + assert 'export OCTOP_NODE="$NODE_BIN"' in script + + +def test_hook_commands_use_the_resolved_node_and_report_failures() -> None: + script = _script() + + assert 'NODE = os.environ.get("OCTOP_NODE") or "node"' in script + assert "def hook_command(script_path):" in script + assert 'hook_command(f"{hooks_dir}/claude-code/stop-capture.mjs")' in script + assert 'hook_command(f"{KIRO_DIR}/kiro/stop-capture.mjs")' in script + assert '|| echo "octop-memory: hook 执行失败' in script # 非 0 退出不再静默 + # 企业根在系统证书库里、不在 Node 自带库里：Node 22.15+ 才加 --use-system-ca + assert "OCTOP_USE_SYSTEM_CA" in script + assert "--use-system-ca" in script + assert 'NODE_OPTIONS="${NODE_OPTIONS:+$NODE_OPTIONS }--use-system-ca"' in script + + +def test_installer_names_the_log_and_state_location() -> None: + """日志与增量状态就落在终端自己的 hooks 目录（该目录不可写才退到临时目录）。""" + script = _script() + + assert "日志/增量状态：" in script + assert "{target}/octop-hook.log" in script + assert "{target}/octop-transcript-state.json" in script + assert "$TMPDIR/octop-hooks" in script + + +def test_mcp_entry_is_per_terminal_and_keeps_other_entries() -> None: + script = _script() + + assert 'home / ".claude.json"' in script + assert 'home / ".kiro" / "settings" / "mcp.json"' in script + assert "def _octop_servers(servers):" in script # 按 octop-memory* 认已有服务，不认具体名字 + assert "保持原样" in script + assert "if not any(" in script + + +def test_legacy_mcp_tool_hooks_are_recognized_by_tool_not_by_server_name() -> None: + """旧版 mcp_tool hook 的清理按工具名认：服务名可能是 local / prod / oauth 或改过名。""" + script = _script() + + assert '_OUR_MCP_TOOLS = ("memory_search", "memory_capture")' in script + assert 'handler.get("tool") in _OUR_MCP_TOOLS' in script + assert 'handler.get("server") == NAME' not in script + + +def test_endpoint_self_check_is_warn_only() -> None: + script = _script() + + assert "端点自检" in script + assert "安装仍继续" in script + # 自检与 hook 共用同一次探测；支持时先 export NODE_OPTIONS，再去 fetch + assert 'if "$NODE_BIN" --use-system-ca -e' in script + assert "export NODE_OPTIONS" in script + assert "AbortSignal.timeout(8000)" in script + + +def test_server_name_follows_the_endpoint() -> None: + assert "OCTOP_SERVER_NAME='octop-memory-local'" in _script() + remote = _script(mcp_url="https://octop.example.com/mcp/") + assert "OCTOP_SERVER_NAME='octop-memory'" in remote + assert "octop-memory-local" not in remote + + +def test_generator_knows_only_the_supported_terminals() -> None: + assert TERMINALS == ("claude-code", "kiro", "dsh", "codex", "cursor") + + +def test_codex_and_cursor_are_detected_and_offered() -> None: + """两个宿主都要进检测与菜单（此前只有脚本、没有安装入口）。""" + script = _script() + + assert '[ -d "$HOME/.codex" ] || command -v codex' in script + assert '[ -d "$HOME/.cursor" ] || command -v cursor' in script + assert 'echo " 4) Codex' in script and 'echo " 5) Cursor' in script + assert '"codex|Codex' in script and '"cursor|Cursor' in script + assert "请重选（1-5，或 a=全部）" in script + assert "4) _add=" in script and "5) _add=" in script + assert '"codex": do_codex' in script and '"cursor": do_cursor' in script + + +def test_terminal_selection_is_an_interactive_checklist() -> None: + """选终端是清单式：↑↓ 移动 · 空格 选择 · a 全选 · 回车 确认（参考常见安装器交互）。""" + script = _script() + + assert "OCTOP_SELECT" in script + assert "↑/↓ 移动 · 空格 选择 · a 全选 · 回车 确认" in script + assert 'if key == b"\\x1b":' in script # 方向键 ESC [ A/B + assert 'return {b"[A": "up", b"[B": "down"}' in script + assert 'if key == b" ":' in script # 空格切换 + assert "setraw(fd)" in script # raw 模式读单键 + assert 'box = "◉" if chosen[index] else "◯"' in script # 勾选状态可见 + # 一个都没选时回车不确认；无 tty / python3 不可用则回退数字输入 + assert "if not any(chosen):" in script + assert "可多选：1,2 / 1 3 / 12 / a=全部" in script + + +def test_checklist_helper_is_valid_python() -> None: + import ast + + from octop.infra.agents.client_setup import _SELECT_PY + + ast.parse(_SELECT_PY) # 片段直接内嵌进生成的脚本，必须能独立解析 + assert "sys.exit(main())" in _SELECT_PY + + +def test_generated_installer_actually_runs(tmp_path) -> None: + """把生成的安装脚本真跑一遍（沙箱 HOME）。 + + 脚本是 shell + 内嵌 python 拼出来的：语法正确不等于能跑，拼装错误（变量改名、 + 参数漏传）只有执行才会暴露。 + """ + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + home = tmp_path / "home" + for name in (".claude", ".kiro", ".dsh", ".codex", ".cursor"): + (home / name).mkdir(parents=True, exist_ok=True) + + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code,kiro,dsh,codex,cursor", + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + # 每个宿主只有自己的入口脚本 + 一份共享传输模块 + config.json + assert (home / ".claude/hooks-events/claude-code/stop-capture.mjs").exists() + assert (home / ".claude/hooks-events/claude-code/user-prompt-recall.mjs").exists() + assert (home / ".kiro/hooks/kiro/stop-capture.mjs").exists() + assert (home / ".dsh/hooks/dsh/stop-capture.mjs").exists() + assert (home / ".dsh/hooks/dsh/user-prompt-recall.mjs").exists() + assert (home / ".dsh/hooks/dsh/memory-target.mjs").exists() + assert (home / ".codex/hooks/codex/stop-capture.mjs").exists() + assert (home / ".codex/hooks/codex/user-prompt-recall.mjs").exists() + assert (home / ".cursor/hooks/cursor/stop-capture.mjs").exists() + assert (home / ".cursor/hooks/cursor/user-prompt-recall.mjs").exists() + for hooks_dir in ( + ".claude/hooks-events", + ".kiro/hooks", + ".dsh/hooks", + ".codex/hooks", + ".cursor/hooks", + ): + assert (home / hooks_dir / "lib/octop-memory-client.mjs").exists() + assert (home / hooks_dir / "config.json").exists() + # 宿主之间不串：CC 目录里不该出现 kiro/dsh 的入口 + assert not (home / ".claude/hooks-events/kiro").exists() + assert not (home / ".kiro/hooks/claude-code").exists() + # Codex / Cursor 的 hook 配置只引用包装脚本路径（长命令收在 .sh 里，配置里没有引号/转义坑） + assert ".codex/hooks/codex/stop-capture.sh" in (home / ".codex/config.toml").read_text( + encoding="utf-8" + ) + cursor_hooks = json.loads((home / ".cursor/hooks.json").read_text(encoding="utf-8")) + assert ( + "cursor/hooks/cursor/user-prompt-recall.sh" + in cursor_hooks["hooks"]["beforeSubmitPrompt"][0]["command"] + ) + assert "cursor/hooks/cursor/stop-capture.sh" in cursor_hooks["hooks"]["stop"][0]["command"] + # 每个宿主的包装脚本：可执行，内容是「绝对 node 路径 + .mjs + 失败兜底」 + for hooks_dir, host in ( + (".claude/hooks-events", "claude-code"), + (".kiro/hooks", "kiro"), + (".dsh/hooks", "dsh"), + (".codex/hooks", "codex"), + (".cursor/hooks", "cursor"), + ): + wrapper = home / hooks_dir / host / "stop-capture.sh" + text = wrapper.read_text(encoding="utf-8") + assert text.startswith("#!/bin/sh") + assert f"{host}/stop-capture.mjs" in text + assert "hook 执行失败" in text # 非 0 退出不再静默 + assert os.access(wrapper, os.X_OK) + if "node 使用系统证书库" in result.stdout: + assert "--use-system-ca" in text + # DSH 桥接指向 DSH 自己的包装脚本，且 preset bundle 已生成并把占位符换成了桥接文件 + bridge = home / ".dsh/hooks/octop-hooks.json" + bridge_text = bridge.read_text(encoding="utf-8") + assert "dsh/user-prompt-recall.sh" in bridge_text + assert "dsh/stop-capture.sh" in bridge_text + assert "claude-code/" not in bridge_text + bundle = home / ".dsh/bundles/dsh-memory-share" + assert (bundle / "package.json").exists() + patch = (bundle / "cordis.patch.yml").read_text(encoding="utf-8") + assert "__OCTOP_HOOKS_JSON__" not in patch + assert str(bridge) in patch + assert "octop-shared-memory" in patch + + +_CODEX_EXISTING = """model = "gpt-5" + +[[hooks.PreToolUse]] + +[[hooks.PreToolUse.hooks]] +command = "/usr/local/bin/git-ai checkpoint codex" +type = "command" + +[[hooks.Stop]] + +[[hooks.Stop.hooks]] +command = "/usr/local/bin/git-ai checkpoint codex" +type = "command" + +[hooks.state."CONFIG:stop:0:0"] +enabled = true +trusted_hash = "sha256:deadbeef" +""" + + +def _run_terminals(tmp_path, terminals: str, prepare=None, extra_env=None): + """按给定终端跑两遍安装器（第二遍验证幂等）。""" + home = tmp_path / "home" + home.mkdir(parents=True, exist_ok=True) + if prepare is not None: + prepare(home) + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + env = { + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": terminals, + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + **(extra_env or {}), + } + for _ in range(2): + result = subprocess.run( + ["sh", str(script)], env=env, capture_output=True, text=True, timeout=180 + ) + assert result.returncode == 0, result.stdout + result.stderr + return home + + +def test_user_id_is_written_and_sent_only_when_given(tmp_path) -> None: + """「谁问的」只能靠 X-Octop-User-Id 表达：给了就写，没给就不写。""" + home = _run_terminals( + tmp_path / "with-id", + "claude-code", + extra_env={"OCTOP_USER_ID": "alice"}, + ) + config = json.loads((home / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert config["userId"] == "alice" + + entry = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"][ + "octop-memory-local" + ] + assert entry["headers"]["X-Octop-User-Id"] == "alice" + + # 不传就保持匿名：配置里不写 userId，MCP 条目也不加头 + home2 = _run_terminals( + tmp_path / "anonymous", + "claude-code", + extra_env={"OCTOP_USER_ID": ""}, + ) + config2 = json.loads((home2 / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert "userId" not in config2 + entry2 = json.loads((home2 / ".claude.json").read_text(encoding="utf-8"))["mcpServers"][ + "octop-memory-local" + ] + assert "X-Octop-User-Id" not in entry2["headers"] + + +def test_entry_uses_authorization_bearer(tmp_path) -> None: + """条目缺省是 ``X-Octop-Agent-Id`` + ``Authorization: Bearer ``。""" + home = _run_terminals(tmp_path, "claude-code") + entry = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"][ + "octop-memory-local" + ] + assert entry["headers"]["Authorization"] == "Bearer fake-token" + assert "X-Octop-Memory-Token" not in entry["headers"] + + # header 名可配：非 Authorization 时原样写该 header（不加 Bearer 前缀） + home_custom = tmp_path / "custom-home" + (home_custom / ".claude").mkdir(parents=True) + custom_script = tmp_path / "setup-custom.sh" + custom_script.write_text( + _script(mcp_url="https://octop.example.com/mcp/memory/"), encoding="utf-8" + ) + result = subprocess.run( + ["sh", str(custom_script)], + env={ + **os.environ, + "HOME": str(home_custom), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "shared-token", + "OCTOP_TOKEN_HEADER": "X-Octop-Memory-Token", + "OCTOP_AGENT_ID": "AGT1", + }, + capture_output=True, + text=True, + timeout=180, + ) + assert result.returncode == 0, result.stdout + result.stderr + custom_entry = json.loads((home_custom / ".claude.json").read_text(encoding="utf-8"))[ + "mcpServers" + ]["octop-memory"] + assert custom_entry["headers"]["X-Octop-Memory-Token"] == "shared-token" + assert "Authorization" not in custom_entry["headers"] + + +def test_non_ascii_user_id_is_ignored(tmp_path) -> None: + """HTTP header 只能 ASCII（Node fetch 对中文直接抛错）→ 明确忽略并提示，别装完才崩。""" + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + "OCTOP_USER_ID": "用户甲", + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + assert "含非 ASCII" in result.stdout + config = json.loads((home / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert "userId" not in config + entry = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"][ + "octop-memory-local" + ] + assert "X-Octop-User-Id" not in entry["headers"] + + +def test_declared_public_url_becomes_the_entry_address(tmp_path) -> None: + """条目地址与 hooks 地址可以分开：条目走部署声明的公网地址，hooks 打实例挂载点。 + + 两份地址共用同一个**实例共享 token**（不再有个人令牌 / OAuth 模式）。 + """ + public = "https://octop-memory.example.com/mcp/memory/" + direct = "https://octop.example.com/mcp/memory/" + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + script = tmp_path / "setup.sh" + script.write_text(_script(mcp_url=public, hook_url=direct), encoding="utf-8") + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "shared-token", + "OCTOP_AGENT_ID": "AGT1", + "OCTOP_USER_ID": "alice", + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + servers = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"] + name, entry = next(iter(servers.items())) + assert name == "octop-memory" + assert entry["url"] == public + assert entry["headers"] == { + "X-Octop-Agent-Id": "AGT1", + "Authorization": "Bearer shared-token", + "X-Octop-User-Id": "alice", + } + config = json.loads((home / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert config["url"] == direct and config["token"] == "shared-token" + + +def test_entry_defaults_to_the_instance_mount(tmp_path) -> None: + """没有声明公网地址时，条目与 hooks 都落在实例自己的挂载点。""" + instance = "https://octop.example.com/mcp/memory/" + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + script = tmp_path / "setup.sh" + script.write_text(_script(mcp_url=instance), encoding="utf-8") + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "shared-token", + "OCTOP_AGENT_ID": "AGT1", + "OCTOP_USER_ID": "alice", + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + servers = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"] + entry = next(iter(servers.values())) + assert entry["url"] == instance + assert entry["headers"] == { + "X-Octop-Agent-Id": "AGT1", + "Authorization": "Bearer shared-token", + "X-Octop-User-Id": "alice", + } + + +def test_split_urls_keep_one_shared_token(tmp_path) -> None: + """条目与 hooks 地址不同也只有一个凭据：实例共享 token（不再有个人令牌）。 + + hooks 打实例（共享 token + ``X-Octop-User-Id``）；条目走部署声明的公网地址， + 用的是同一份共享 token——由 ingress 原样转发。 + """ + public = "https://octop-memory.example.com/mcp/memory/" + direct = "https://octop.example.com/mcp/memory/" + script = tmp_path / "setup.sh" + script.write_text(_script(mcp_url=public, hook_url=direct), encoding="utf-8") + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "shared-token", + "OCTOP_AGENT_ID": "AGT1", + "OCTOP_USER_ID": "alice", + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + config = json.loads((home / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert config["url"] == direct # hooks 直连实例 + assert config["token"] == "shared-token" and config["userId"] == "alice" + + entry = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"][ + "octop-memory" + ] + assert entry["url"] == public + assert entry["headers"] == { + "X-Octop-Agent-Id": "AGT1", + "Authorization": "Bearer shared-token", + "X-Octop-User-Id": "alice", + } + assert "实例共享 token" in result.stdout + + +def test_builtin_default_token_saves_the_prompt(tmp_path) -> None: + """部署内置的默认凭据（实例发布时的 OCTOP_MEMORY_MCP_TOKEN）→ 用户只给专家 ID 就能装。""" + script = tmp_path / "setup.sh" + script.write_text(_script(default_token="octopoctop"), encoding="utf-8") + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_AGENT_ID": "AGT1", # 没有 OCTOP_MEMORY_TOKEN / OCTOP_USER_ID + }, + capture_output=True, + text=True, + timeout=180, + ) + + assert result.returncode == 0, result.stdout + result.stderr + config = json.loads((home / ".claude/hooks-events/config.json").read_text(encoding="utf-8")) + assert config["token"] == "octopoctop" + assert "user" not in config # 没给 user id 就不写 + assert "用本实例部署时配置的默认凭据" in result.stdout + + +def test_without_a_builtin_token_the_installer_still_demands_one(tmp_path) -> None: + """没内置凭据 + 无 tty + 没给环境变量 → 明确报错，不盲读 stdin。""" + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + env = { + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_AGENT_ID": "AGT1", + } + env.pop("OCTOP_MEMORY_TOKEN", None) + result = subprocess.run( + ["sh", str(script)], env=env, capture_output=True, text=True, timeout=180 + ) + + assert result.returncode != 0 + assert "缺少 token" in result.stderr + assert "DEFAULT_TOKEN=" not in _script() # 没内置值就不该有赋值 + + +def test_codex_hooks_join_the_existing_config_without_clobbering_it(tmp_path) -> None: + """~/.codex/config.toml 已有别人的 hook：只追加我们的，别的字段与信任状态原样保留。""" + + def prepare(home): + (home / ".codex").mkdir(parents=True) + config = home / ".codex/config.toml" + (config).write_text(_CODEX_EXISTING.replace("CONFIG", str(config)), encoding="utf-8") + + home = _run_terminals(tmp_path, "codex", prepare) + cfg = tomllib.loads((home / ".codex/config.toml").read_text(encoding="utf-8")) + hooks = cfg["hooks"] + + def commands(event): + return [entry["command"] for group in hooks[event] for entry in group.get("hooks", [])] + + assert cfg["model"] == "gpt-5" # 其它字段不动 + assert len(commands("PreToolUse")) == 1 and "git-ai" in commands("PreToolUse")[0] + assert len(commands("Stop")) == 2 # 别人的 + 我们的（跑了两遍也不重复） + assert any("codex/stop-capture.sh" in command for command in commands("Stop")) + assert len(commands("UserPromptSubmit")) == 1 + assert any("codex/user-prompt-recall.sh" in command for command in commands("UserPromptSubmit")) + assert list(hooks["state"]) == [f"{home}/.codex/config.toml:stop:0:0"] # 信任状态保留 + + +def test_cursor_hooks_join_the_existing_json_without_clobbering_it(tmp_path) -> None: + """~/.cursor/hooks.json 其它事件（如 git-ai 的 postToolUse）原样保留。""" + + def prepare(home): + (home / ".cursor").mkdir(parents=True) + (home / ".cursor/hooks.json").write_text( + json.dumps( + {"hooks": {"postToolUse": [{"command": "git-ai cursor"}]}, "version": 1}, indent=2 + ) + + "\n", + encoding="utf-8", + ) + + home = _run_terminals(tmp_path, "cursor", prepare) + cfg = json.loads((home / ".cursor/hooks.json").read_text(encoding="utf-8")) + + assert cfg["version"] == 1 + assert cfg["hooks"]["postToolUse"] == [{"command": "git-ai cursor"}] + assert len(cfg["hooks"]["beforeSubmitPrompt"]) == 1 # 幂等：跑两遍仍只有一条 + assert ( + "cursor/hooks/cursor/user-prompt-recall.sh" + in cfg["hooks"]["beforeSubmitPrompt"][0]["command"] + ) + assert len(cfg["hooks"]["stop"]) == 1 + assert "cursor/hooks/cursor/stop-capture.sh" in cfg["hooks"]["stop"][0]["command"] + + +def test_uninstall_script_mentions_cursor_and_needs_no_token() -> None: + script = render_uninstall_script() + assert script.startswith("#!/bin/sh") + assert "octop memory 卸载" in script + assert "OCTOP_TERMINALS" in script + assert "Cursor" in script + assert "OCTOP_MEMORY_TOKEN" not in script + assert "undo_cursor" in script or "CURSOR_DIR" in script + + +def test_uninstall_script_shell_quotes_terminals_query() -> None: + """HTTP ``terminals`` must not break out of the shell assignment.""" + script = render_uninstall_script(terminals="cursor'; curl evil.example") + assert "DEFAULT_TERMINALS='cursor'\"'\"'; curl evil.example'" in script + assert "DEFAULT_TERMINALS='cursor'; curl evil.example'" not in script + + +def test_cursor_uninstall_removes_hooks_keeps_others(tmp_path) -> None: + """setup → uninstall：octop hooks/目录清掉，别人的 hook 与无关 MCP 保留。""" + + def prepare(home): + (home / ".cursor").mkdir(parents=True) + (home / ".cursor/hooks.json").write_text( + json.dumps( + {"hooks": {"postToolUse": [{"command": "git-ai cursor"}]}, "version": 1}, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + (home / ".cursor/mcp.json").write_text( + json.dumps( + { + "mcpServers": { + "octop-memory": {"url": "https://example/mcp"}, + "other": {"url": "https://other"}, + } + }, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + + home = _run_terminals(tmp_path, "cursor", prepare) + assert (home / ".cursor/hooks/cursor/stop-capture.mjs").exists() + + uninstall = tmp_path / "uninstall.sh" + uninstall.write_text(render_uninstall_script(), encoding="utf-8") + env = { + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "cursor", + } + result = subprocess.run( + ["sh", str(uninstall)], env=env, capture_output=True, text=True, timeout=60 + ) + assert result.returncode == 0, result.stdout + result.stderr + + cfg = json.loads((home / ".cursor/hooks.json").read_text(encoding="utf-8")) + assert cfg["hooks"]["postToolUse"] == [{"command": "git-ai cursor"}] + assert "beforeSubmitPrompt" not in cfg["hooks"] + assert "stop" not in cfg["hooks"] + assert not (home / ".cursor/hooks").exists() + + mcp = json.loads((home / ".cursor/mcp.json").read_text(encoding="utf-8")) + assert "octop-memory" not in mcp["mcpServers"] + assert mcp["mcpServers"]["other"]["url"] == "https://other" + + +def _run_claude_code_installer(tmp_path, claude_json: dict, settings: dict | None = None): + """只装 Claude Code：沙箱 HOME + 预置的 .claude.json / settings.json。""" + home = tmp_path / "home" + (home / ".claude").mkdir(parents=True) + (home / ".claude.json").write_text(json.dumps(claude_json, indent=2) + "\n", encoding="utf-8") + if settings is not None: + (home / ".claude/settings.json").write_text( + json.dumps(settings, indent=2) + "\n", encoding="utf-8" + ) + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + result = subprocess.run( + ["sh", str(script)], + env={ + **os.environ, + "HOME": str(home), + "OCTOP_TERMINALS": "claude-code", + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + }, + capture_output=True, + text=True, + timeout=180, + ) + assert result.returncode == 0, result.stdout + result.stderr + return home, result + + +def test_mcp_entry_is_added_when_no_octop_server_exists(tmp_path) -> None: + home, _ = _run_claude_code_installer( + tmp_path, {"mcpServers": {"other-mcp": {"type": "http", "url": "http://other/"}}} + ) + servers = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"] + assert "octop-memory-local" in servers # 全新安装才新建条目 + assert "other-mcp" in servers # 别人的条目原样保留 + + +def test_mcp_entry_reuses_an_existing_octop_entry_with_the_same_endpoint(tmp_path) -> None: + """已有同一个端点的 octop 服务（名字可能是 prod / oauth / 自定义）→ 复用，不新增条目。""" + home, result = _run_claude_code_installer( + tmp_path, + {"mcpServers": {"octop-memory-prod": {"type": "http", "url": _URL}}}, + ) + servers = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"] + assert list(servers) == ["octop-memory-prod"] + assert "复用" in result.stdout + + +def test_mcp_entry_does_not_add_a_duplicate_for_a_different_octop_endpoint(tmp_path) -> None: + """已有别的 octop 端点时保持原样：不按服务名硬塞第二个 octop 服务。""" + home, result = _run_claude_code_installer( + tmp_path, + { + "mcpServers": { + "octop-memory-oauth": {"type": "http", "url": "https://oauth.example/mcp/"} + } + }, + ) + servers = json.loads((home / ".claude.json").read_text(encoding="utf-8"))["mcpServers"] + assert list(servers) == ["octop-memory-oauth"] + assert "保持原样" in result.stdout + + +def test_legacy_mcp_tool_hook_under_another_server_name_is_replaced(tmp_path) -> None: + """旧版 mcp_tool hook 的服务名可能是 octop-memory-prod：第 5 步按工具名清掉它。""" + legacy = { + "hooks": { + "UserPromptSubmit": [ + { + "hooks": [ + { + "type": "mcp_tool", + "server": "octop-memory-prod", + "tool": "memory_search", + "input": {"query": "${prompt}"}, + } + ] + } + ] + } + } + home, _ = _run_claude_code_installer(tmp_path, {"mcpServers": {}}, settings=legacy) + entries = json.loads((home / ".claude/settings.json").read_text(encoding="utf-8"))["hooks"][ + "UserPromptSubmit" + ] + assert len(entries) == 1 # 旧 mcp_tool 那条被替换，只剩我们新写的 + assert entries[0]["type"] == "command" # 简写形态：事件直接挂 handler + assert not any(entry.get("type") == "mcp_tool" for entry in entries) + + +@posix_only +def test_dsh_bundle_is_auto_installed_when_the_dsh_cli_exists(tmp_path) -> None: + """有 dsh CLI 时优先用它装：pnpm 加依赖 + 把 bundle 写进 profile 的 dsh.profile.bundles。 + + DSH 只按 profile 的 bundle 名单加载 bundle，所以「装了依赖但没进名单」等于没装。 + """ + home = tmp_path / "home" + profile = home / ".dsh/profiles/web" + profile.mkdir(parents=True) + (profile / "package.json").write_text( + json.dumps( + {"name": "profile-web", "dsh": {"profile": {"bundles": ["@deepseek-ai/dsh-base"]}}}, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + fake_bin = tmp_path / "bin" + fake_bin.mkdir() + shim = fake_bin / "dsh" + shim.write_text('#!/bin/sh\nprintf \'%s\\n\' "$@" >> "$0.args"\n', encoding="utf-8") + shim.chmod(0o755) + + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + env = { + **os.environ, + "HOME": str(home), + "PATH": f"{fake_bin}{os.pathsep}{os.environ['PATH']}", + "OCTOP_TERMINALS": "dsh", + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + } + for _ in range(2): # 第二次验证幂等 + result = subprocess.run( + ["sh", str(script)], env=env, capture_output=True, text=True, timeout=180 + ) + assert result.returncode == 0, result.stdout + result.stderr + + bundle = home / ".dsh/bundles/dsh-memory-share" + calls = (fake_bin / "dsh.args").read_text(encoding="utf-8") + assert "plugin" in calls and "--profile" in calls and "web" in calls + assert str(bundle) in calls + assert not (fake_bin / "pnpm.args").exists() # 有 dsh 就不必自己调 pnpm + bundles = json.loads((profile / "package.json").read_text(encoding="utf-8"))["dsh"]["profile"][ + "bundles" + ] + assert bundles == ["@deepseek-ai/dsh-base", "octop-dsh-memory-share"] # 已存在的不重复追加 + + +@posix_only +def test_dsh_bundle_falls_back_to_pnpm_in_the_profile(tmp_path) -> None: + """源码 checkout 往往没有 dsh 命令：退回在 profile 目录直接 pnpm add，再写 bundle 名单。""" + home = tmp_path / "home" + profile = home / ".dsh/profiles/web" + profile.mkdir(parents=True) + (profile / "package.json").write_text( + json.dumps({"name": "profile-web", "dsh": {"profile": {"bundles": []}}}, indent=2) + "\n", + encoding="utf-8", + ) + fake_bin = tmp_path / "bin" + fake_bin.mkdir() + shim = fake_bin / "pnpm" + shim.write_text( + '#!/bin/sh\nprintf \'%s\\n\' "$PWD" >> "$0.cwd"\nprintf \'%s\\n\' "$@" >> "$0.args"\n', + encoding="utf-8", + ) + shim.chmod(0o755) + + script = tmp_path / "setup.sh" + script.write_text(_script(), encoding="utf-8") + # 保留 shell 自己要用的 /bin、/usr/bin 与解释器目录，但不含任何 dsh 命令 + path_dirs = [str(fake_bin), "/bin", "/usr/bin"] + path_dirs += [ + str(Path(which).parent) + for which in (shutil.which("python3"), shutil.which("node")) + if which + ] + path = os.pathsep.join(dict.fromkeys(path_dirs)) + assert shutil.which("dsh", path=path) is None, "测试环境里 dsh 必须在 PATH 之外" + env = { + **os.environ, + "HOME": str(home), + "PATH": path, + "OCTOP_TERMINALS": "dsh", + "OCTOP_MEMORY_TOKEN": "fake-token", + "OCTOP_AGENT_ID": "AGT1", + } + result = subprocess.run( + ["sh", str(script)], env=env, capture_output=True, text=True, timeout=180 + ) + + assert result.returncode == 0, result.stdout + result.stderr + bundle = home / ".dsh/bundles/dsh-memory-share" + assert (fake_bin / "pnpm.args").read_text(encoding="utf-8").splitlines() == [ + "add", + str(bundle), + ] + assert (fake_bin / "pnpm.cwd").read_text(encoding="utf-8").strip() == str(profile) + bundles = json.loads((profile / "package.json").read_text(encoding="utf-8"))["dsh"]["profile"][ + "bundles" + ] + assert bundles == ["octop-dsh-memory-share"] + assert "已装进 profile「web」" in result.stdout diff --git a/tests/unit/agents/test_hook_scripts.py b/tests/unit/agents/test_hook_scripts.py new file mode 100644 index 000000000..21007f2b7 --- /dev/null +++ b/tests/unit/agents/test_hook_scripts.py @@ -0,0 +1,118 @@ +"""Hook scripts are data files, but their layout and write locations are a contract.""" + +from __future__ import annotations + +from pathlib import Path + +_HOOKS = Path(__file__).parents[3] / "src" / "octop" / "infra" / "agents" / "hook_scripts" +_LIB = _HOOKS / "lib" / "octop-memory-client.mjs" +_HOSTS = { + "claude-code": ("stop-capture.mjs", "user-prompt-recall.mjs"), + "kiro": ("stop-capture.mjs",), + "dsh": ("stop-capture.mjs", "user-prompt-recall.mjs", "memory-target.mjs"), + "codex": ("stop-capture.mjs", "user-prompt-recall.mjs"), + "cursor": ("stop-capture.mjs", "user-prompt-recall.mjs"), +} + + +def test_every_host_has_its_own_entry_scripts() -> None: + """每个宿主一个入口脚本：一个宿主的改动不会碰到别人。""" + for host, names in _HOSTS.items(): + for name in names: + assert (_HOOKS / host / name).exists(), f"{host}/{name} 缺失" + + +def test_entries_do_not_detect_the_host_at_runtime() -> None: + """宿主差异体现在安装时选的脚本里，脚本内部不再做宿主嗅探。""" + for host, names in _HOSTS.items(): + if not (_HOOKS / host).exists(): + continue + for name in names: + path = _HOOKS / host / name + if not path.exists(): + continue + text = path.read_text(encoding="utf-8") + for sniff in ("findDshSessionFile", "kiroRoot", "dshRoot", "sourceOverride"): + assert sniff not in text, f"{host}/{name} 仍在运行时探测宿主（{sniff}）" + + +def test_transport_module_is_host_agnostic() -> None: + """共享模块只做传输（配置/鉴权/偏移/日志/压缩），不认识任何宿主。""" + text = _LIB.read_text(encoding="utf-8") + + assert "export async function capture" in text + assert "export async function callTool" in text + assert "decompressZstdContainer" in text + # 用户标识是显式传进来的（env / config.json），不是从 token 里解出来的 + assert "process.env.OCTOP_USER_ID || fromFile.userId" in text + assert '"X-Octop-User-Id"' in text + for host_word in ("claude-code", "kiro", "dsh", "codex", "cursor", "transcript_path"): + assert host_word not in text, f"共享模块不应出现宿主细节：{host_word}" + + +def test_every_entry_uses_the_shared_configured_guard() -> None: + """「有没有配好」只由共享模块的 isConfigured() 判，入口脚本别各写一套。""" + assert "export function isConfigured" in _LIB.read_text(encoding="utf-8") + for host, names in _HOSTS.items(): + for name in names: + text = (_HOOKS / host / name).read_text(encoding="utf-8") + if "not configured" not in text: + continue + assert "isConfigured(config)" in text, f"{host}/{name} 没用共享守卫" + assert "!config.token" not in text, f"{host}/{name} 自己拼了 token 判断" + + +def test_scratch_dir_prefers_the_hook_directory() -> None: + """日志/增量状态默认写 hooks 目录，不可写才退到临时目录。""" + text = _LIB.read_text(encoding="utf-8") + assert "for (const dir of [HOOK_DIR," in text + assert "${scratchDir}/octop-hook.log" in text + + +def test_falling_back_to_tmp_is_announced() -> None: + """DSH 的 workspace-write 沙盒里 $HOME 只读，回退必须可见而不是静默。""" + text = _LIB.read_text(encoding="utf-8") + assert "if (scratchDir !== HOOK_DIR)" in text + assert "不可写，日志与增量状态改写到" in text + + +def test_claude_code_capture_defers_to_a_detached_worker() -> None: + """Stop 时本轮还没落盘：后台脱离 + 轮询到「最后一条 user 之后有 assistant」。""" + text = (_HOOKS / "claude-code" / "stop-capture.mjs").read_text(encoding="utf-8") + + assert "OCTOP_CAPTURE_DELAY_MS" in text and "const IS_WORKER" in text + assert 'spawn(process.execPath, [fileURLToPath(import.meta.url), "--worker"]' in text + assert "detached: true" in text and "worker.unref()" in text + assert "OCTOP_HOOK_PAYLOAD_B64" in text + assert "turnLanded" in text + assert '"cc:"' in text or 'sessionPrefix: "cc:"' in text + + +def test_kiro_and_dsh_capture_are_self_contained() -> None: + """Kiro 只认 messages.jsonl；DSH 只认 session*.jsonl.zstd 且自带解压。""" + kiro = (_HOOKS / "kiro" / "stop-capture.mjs").read_text(encoding="utf-8") + dsh = (_HOOKS / "dsh" / "stop-capture.mjs").read_text(encoding="utf-8") + + assert "messages.jsonl" in kiro and '"kiro:"' in kiro + assert "/.kiro/sessions" in kiro + assert "decompressZstdContainer" in dsh and '"dsh:"' in dsh + assert "/.dsh/sessions" in dsh and "session.*\\.jsonl\\.zstd" in dsh + + +def test_dsh_recall_answers_in_the_bridge_dialect() -> None: + """DSH 桥只从 hookSpecificOutput.additionalContext 取上下文：纯 stdout 会被丢弃。""" + text = (_HOOKS / "dsh" / "user-prompt-recall.mjs").read_text(encoding="utf-8") + + assert "hookSpecificOutput" in text + assert 'hookEventName: "UserPromptSubmit"' in text + assert "additionalContext" in text + + +def test_dsh_capture_probes_the_endpoint_before_sending_turns() -> None: + """能分开角色就按轮上报（content + assistant），否则退回服务端解析的 transcript 模式。""" + text = (_HOOKS / "dsh" / "stop-capture.mjs").read_text(encoding="utf-8") + + assert "supportsAssistant" in text and "captureTurns" in text + assert "OCTOP_CAPTURE_MODE" in text and "OCTOP_CAPTURE_MAX_TURNS" in text + assert 'kind && kind !== "user"' in text # 插件注入/系统提醒不算用户提问 + assert 'source: "dsh"' in text diff --git a/tests/unit/agents/test_memory_mcp.py b/tests/unit/agents/test_memory_mcp.py new file mode 100644 index 000000000..0d417ea3a --- /dev/null +++ b/tests/unit/agents/test_memory_mcp.py @@ -0,0 +1,1772 @@ +"""Unit tests for the expert memory MCP server (infra/agents/memory_mcp).""" + +from __future__ import annotations + +import base64 +import gzip +import json +from datetime import UTC, datetime +from unittest import mock + +import pytest + +from octop.infra.agents import memory_mcp as mm + +_TRANSCRIPT = ( + json.dumps( + { + "type": "user", + "uuid": "u1", + "sessionId": "s1", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "先看看 CLI 结构"}, + } + ) + + "\n" + + json.dumps( + { + "type": "assistant", + "uuid": "a1", + "sessionId": "s1", + "timestamp": "2026-09-22T05:00:05Z", + "message": { + "role": "assistant", + "content": [ + {"type": "text", "text": "好的，先读文件"}, + { + "type": "tool_use", + "id": "t1", + "name": "Read", + "input": {"file_path": "x.py"}, + }, + ], + }, + } + ) +) + + +@pytest.fixture +def fake_memory(monkeypatch): + mem = mock.MagicMock() + mem.recall.return_value = [] + node = mock.MagicMock() + node.id = "node1" + node.content = "remember X" + mem.store.return_value = node + + def _add_raw(content, **_kwargs): + raw = mock.MagicMock(id="evt1") + raw.content = content + return raw + + mem.add_raw.side_effect = _add_raw + mem.deprecate_atom.return_value = True + monkeypatch.setattr(mm, "_open_memory", lambda server, agent_id: mem) + # Capture tests use a MagicMock server; do not let the real trigger schedule + # a fire-and-forget extract task against that mock (unawaited coroutine noise). + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + return mem + + +@pytest.fixture +def bind_agent(): + """Bind the request-scoped agent contextvar (main resolves the expert per request).""" + token = mm._current_agent_id.set("A1") + yield "A1" + mm._current_agent_id.reset(token) + + +@pytest.fixture +def bind_user(): + """Bind the ``X-Octop-User-Id`` caller contextvar (sender attribution).""" + token = mm._current_caller_user.set("alice") + yield "alice" + mm._current_caller_user.reset(token) + + +def _tools(mcp): + return mcp._tool_manager._tools + + +def test_build_resolves_agent_per_request(monkeypatch): + """Expert is bound per request (contextvar), not captured at build time.""" + captured = {} + + def fake_open(server, agent_id): + captured["agent_id"] = agent_id + mem = mock.MagicMock() + mem.store.return_value = mock.MagicMock(id="n1", content="x") + return mem + + monkeypatch.setattr(mm, "_open_memory", fake_open) + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = mm._current_agent_id.set("EXPERT42") + try: + _tools(mcp)["memory_save"].fn(content="x", source="s") + finally: + mm._current_agent_id.reset(token) + assert captured["agent_id"] == "EXPERT42" + + +def test_build_registers_nine_tools(fake_memory): + mcp = mm.build_memory_mcp(mock.MagicMock()) + assert set(_tools(mcp)) == { + "memory_search", + "memory_get", + "memory_save", + "memory_capture", + "memory_raws", + "memory_candidates", + "memory_extract", + "memory_promote", + "memory_reject", + } + + +def test_memory_save_goes_store(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_save"].fn(content="remember X", source="coding-agent") + kwargs = fake_memory.store.call_args.kwargs + assert kwargs["topic"] is None + assert kwargs["metadata"] == {"source": "coding-agent"} + assert result["source"] == "coding-agent" + + +def test_memory_capture_goes_add_raw(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="raw conversation", source="review-bot", session_id="review-1" + ) + kwargs = fake_memory.add_raw.call_args.kwargs + assert kwargs["event_type"] == "user_message" + assert kwargs["host"] == "mcp-external" + assert kwargs["session_id"] == "review-1" + assert kwargs["payload"] == {"source": "review-bot"} + assert result["recorded"] is True + assert "raw (L0)" in result["note"] + + +def test_memory_capture_is_idempotent(fake_memory, bind_agent): + """Re-capturing the same session + content + role reuses the existing L0 event.""" + existing = mock.MagicMock(id="evt-existing") + existing.content = "raw conversation" + existing.event_type = "user_message" + fake_memory.list_raw.return_value = [existing] + + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="raw conversation", source="review-bot", session_id="review-1" + ) + + fake_memory.add_raw.assert_not_called() + assert result["event_id"] == "evt-existing" + assert result["duplicate"] is True + assert "idempotent capture" in result["note"] + + +def test_memory_capture_implies_role_from_field(fake_memory, bind_agent, bind_user): + """content → user_message, assistant → assistant_message; no role parameter.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="", + assistant="助手结论：先不做 Replace", + source="coding-agent", + session_id="s1", + thread_id="thr-1", + ) + + kwargs = fake_memory.add_raw.call_args.kwargs + assert kwargs["event_type"] == "assistant_message" + assert kwargs["thread_id"] == "thr-1" + assert result["event_type"] == "assistant_message" + assert result["thread_id"] == "thr-1" + + +def test_memory_capture_attributes_only_the_user_side(fake_memory, bind_agent, bind_user): + """The assistant's own words must not be stamped as if the user had said them. + + The extractor prompt only sees event_id / event_type / content / timestamp / + session_id, so a sender reaches the atom text only through the prefix — which is + why the user side carries it while the assistant side (already marked by its + ``event_type``) must not. + """ + mcp = mm.build_memory_mcp(mock.MagicMock()) + + _tools(mcp)["memory_capture"].fn( + content="帮我改一下这个函数", + assistant="改好了，先保留旧签名", + source="dsh:cordis-memory", + ) + + stored = {c.kwargs["event_type"]: c.args[0] for c in fake_memory.add_raw.call_args_list} + assert stored["user_message"] == "alice说：帮我改一下这个函数" + assert stored["assistant_message"] == "改好了，先保留旧签名" + + +def test_memory_capture_writes_turn_in_one_call(fake_memory, bind_agent, bind_user): + """content + assistant write both halves of a turn, like MemoryService.capture_turn.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="接口先不要动", + assistant="好的，先保留旧签名", + source="coding-agent", + session_id="s1", + thread_id="thr-1", + ) + + assert [c.kwargs["event_type"] for c in fake_memory.add_raw.call_args_list] == [ + "user_message", + "assistant_message", + ] + assert [e["event_type"] for e in result["events"]] == ["user_message", "assistant_message"] + assert result["event_type"] == "user_message" + + +def test_memory_capture_drops_only_the_echoing_half(fake_memory, bind_agent, bind_user): + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="接口先不要动", + assistant="## Memory Recall\n- 旧记忆块", + source="hook", + ) + + assert [c.kwargs["event_type"] for c in fake_memory.add_raw.call_args_list] == ["user_message"] + assert [e["event_type"] for e in result["events"]] == ["user_message"] + + +def test_memory_capture_rejects_empty_turn(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + with pytest.raises(ValueError, match="both empty"): + _tools(mcp)["memory_capture"].fn(content=" ", source="review-bot") + fake_memory.add_raw.assert_not_called() + + +def test_memory_capture_same_content_different_role_not_deduped(fake_memory, bind_agent): + """Identical text in two roles is two distinct events, not a duplicate.""" + existing = mock.MagicMock(id="evt-existing") + existing.content = "接口先不要动" + existing.event_type = "assistant_message" + fake_memory.list_raw.return_value = [existing] + + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="接口先不要动", source="review-bot", session_id="s1" + ) + + assert fake_memory.add_raw.called + assert "duplicate" not in result + + +def test_memory_raws_queries_l0_with_query(fake_memory, bind_agent): + class _Evt: + id = "evt1" + timestamp = __import__("datetime").datetime(2026, 8, 19) + session_id = "review-1" + user = "u1" + event_type = "manual" + payload = {"source": "review-bot"} + content = "report panel banner hidden" + + fake_memory.search_raw.return_value = [_Evt()] + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_raws"].fn(query="report panel banner", limit=5) + fake_memory.search_raw.assert_called_once_with("report panel banner", limit=5) + assert result["count"] == 1 + assert result["events"][0]["event_id"] == "evt1" + assert result["events"][0]["source"] == "review-bot" + + +def test_memory_capture_transcript_writes_l0(fake_memory, bind_agent, bind_user, monkeypatch): + """A whole transcript uploads as L0 events carrying role + tool linkage.""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TRANSCRIPT, source="claude-code", session_id="cc:s1", tool_events="all" + ) + + assert result["session_id"] == "cc:s1" + assert result["parsed"] == 3 + assert result["imported"] == 3 + assert result["events_by_type"] == { + "user_message": 1, + "assistant_message": 1, + "tool_call": 1, + } + assert result["extract_scheduled"] is True + + raws = fake_memory.add_raw_batch.call_args.args[0] + assert {raw.event_type for raw in raws} == {"user_message", "assistant_message", "tool_call"} + assert all(raw.host == "claude-code" for raw in raws) + assert all(raw.user == "alice" for raw in raws) + assert all(raw.session_id == "cc:s1" for raw in raws) + + +def test_memory_capture_transcript_accepts_gzip_base64(fake_memory, bind_agent, monkeypatch): + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: False) + mcp = mm.build_memory_mcp(mock.MagicMock()) + blob = base64.b64encode(gzip.compress(_TRANSCRIPT.encode())).decode() + + result = _tools(mcp)["memory_capture"].fn(content=blob, source="claude-code", tool_events="all") + + assert result["imported"] == 3 + assert result["session_id"] == "cc:s1" + + +def test_memory_capture_transcript_dry_run_writes_nothing(fake_memory, bind_agent, monkeypatch): + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TRANSCRIPT, source="claude-code", dry_run=True + ) + + assert result["parsed"] == 3 + assert result["imported"] == 0 + assert result["extract_scheduled"] is False + fake_memory.add_raw_batch.assert_not_called() + + +def test_memory_capture_transcript_can_defer_extraction(fake_memory, bind_agent, monkeypatch): + """Backfill lands L0 first; promotion happens later in a paced sweep.""" + scheduled: list[str] = [] + monkeypatch.setattr( + mm, "_trigger_extract", lambda server, session_id: scheduled.append(session_id) or True + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TRANSCRIPT, source="claude-code", extract=False, tool_events="all" + ) + + assert result["imported"] == 3 + assert result["extract_scheduled"] is False + assert scheduled == [] + assert "paced sweep" in result["note"] + + +def test_memory_capture_terminal_source_with_prose_records_a_turn( + fake_memory, bind_agent, monkeypatch +): + """A terminal name used with prose must keep working (pre-transcript callers). + + ``source`` used to be free-form, so a caller that passes a sentence with + ``source="claude-code"`` would otherwise be parsed into zero events and dropped. + """ + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content="用户偏好：Windows 用 pwsh，不用 bash", source="claude-code" + ) + + assert result["recorded"] is True + assert result["events"][0]["event_type"] == "user_message" + assert "not a transcript" in result["note"] + assert "parsed" not in result # 走的是文本分支，不是 transcript 分支 + assert result["session_id"].startswith("ext:claude-code:") # 一轮对话的派生 session + fake_memory.add_raw.assert_called_once() + + +def test_memory_capture_transcript_drops_own_recall_results(fake_memory, bind_agent, monkeypatch): + """The transcript path must not re-ingest results of our own memory read tools.""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + records = [ + { + "type": "user", + "uuid": "u1", + "sessionId": "s9", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "上周改了什么"}, + }, + { + "type": "assistant", + "uuid": "a1", + "sessionId": "s9", + "timestamp": "2026-09-22T05:00:01Z", + "message": { + "role": "assistant", + "content": [ + { + "type": "tool_use", + "id": "t1", + "name": "mcp__octop-memory-local__memory_recall", + "input": {"query": "上周"}, + } + ], + }, + }, + { + "type": "user", + "uuid": "u2", + "sessionId": "s9", + "timestamp": "2026-09-22T05:00:02Z", + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "t1", + "content": "- [atom] (atom:high @ 2026-01-01) 旧记忆：部署在 8088", + "is_error": False, + } + ], + }, + }, + ] + + result = _tools(mcp)["memory_capture"].fn( + content="\n".join(json.dumps(record, ensure_ascii=False) for record in records), + source="claude-code", + session_id="cc:s9", + tool_events="all", + ) + + assert result["skipped_own_results"] == 1 + assert result["imported"] == 2 + assert "tool_result" not in result["events_by_type"] + raws = fake_memory.add_raw_batch.call_args.args[0] + assert all("旧记忆" not in raw.content for raw in raws) + + +def test_memory_capture_transcript_drops_marked_events(fake_memory, bind_agent, monkeypatch): + """A recalled block (marker present) is dropped even if no tool produced it.""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + records = [ + { + "type": "user", + "uuid": "u1", + "sessionId": "s8", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "正常提问"}, + }, + { + "type": "user", + "uuid": "u2", + "sessionId": "s8", + "timestamp": "2026-09-22T05:00:01Z", + "message": {"role": "user", "content": "## Memory Recall\n- [atom] 旧记忆块"}, + }, + ] + + result = _tools(mcp)["memory_capture"].fn( + content="\n".join(json.dumps(record, ensure_ascii=False) for record in records), + source="claude-code", + session_id="cc:s8", + ) + + assert result["parsed"] == 2 + assert result["skipped_echo"] == 1 + assert result["imported"] == 1 + + +_TOOL_TRANSCRIPT = "\n".join( + json.dumps(record) + for record in [ + { + "type": "user", + "uuid": "tu1", + "sessionId": "s5", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "跑一下测试"}, + }, + { + "type": "assistant", + "uuid": "ta1", + "sessionId": "s5", + "timestamp": "2026-09-22T05:00:01Z", + "message": { + "role": "assistant", + "content": [ + {"type": "text", "text": "我跑一下"}, + { + "type": "tool_use", + "id": "t1", + "name": "Bash", + "input": {"command": "pytest"}, + }, + ], + }, + }, + { + "type": "user", + "uuid": "tu2", + "sessionId": "s5", + "timestamp": "2026-09-22T05:00:02Z", + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "t1", + "content": "3 passed", + "is_error": False, + } + ], + }, + }, + ] +) + + +@pytest.mark.parametrize( + ("policy", "expected_by_type", "skipped_tools"), + [ + ( + "all", + {"user_message": 1, "assistant_message": 1, "tool_call": 1, "tool_result": 1}, + 0, + ), + ("calls", {"user_message": 1, "assistant_message": 1, "tool_call": 1}, 1), + ("none", {"user_message": 1, "assistant_message": 1}, 2), + ], +) +def test_memory_capture_tool_events_policy( + fake_memory, bind_agent, monkeypatch, policy, expected_by_type, skipped_tools +): + """``tool_events`` 默认只落 user/assistant（上传始终整份），可显式放开到 calls / all。""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", session_id="cc:s5", tool_events=policy + ) + + assert result["by_type" if "by_type" in result else "events_by_type"] == expected_by_type + assert result["skipped_tools"] == skipped_tools + assert result["tool_events"] == policy + assert result["imported"] == sum(expected_by_type.values()) + + +def test_memory_capture_transcript_drops_tools_by_default(fake_memory, bind_agent, monkeypatch): + """上传整份、落库只留人和助手：工具事件解析了但不进 L0（``parsed`` 仍计入）。""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", session_id="cc:s4" + ) + + assert result["tool_events"] == "none" + assert result["parsed"] == 4 # 整份 transcript 都解析了（含 call/result） + assert result["skipped_tools"] == 2 + assert result["events_by_type"] == {"user_message": 1, "assistant_message": 1} + assert result["imported"] == 2 + + +def test_no_caller_means_no_prefix(fake_memory, bind_agent, monkeypatch, tmp_path): + """headers 里没有「谁说的」就不加前缀——两种形态一致，不凭空造归属。""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + monkeypatch.setattr(mm, "_resolve_agent_scope", lambda server, agent_id: (tmp_path, {})) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + # 一轮形态：没有 X-Octop-User-Id + _tools(mcp)["memory_capture"].fn(content="接口先不要动", source="review-bot") + assert fake_memory.add_raw.call_args.args[0] == "接口先不要动" + + # transcript 形态：同样不加 + _tools(mcp)["memory_capture"].fn(content=_TRANSCRIPT, source="claude-code", session_id="cc:s7") + stored = fake_memory.add_raw_batch.call_args.args[0] + assert stored[0].content == "先看看 CLI 结构" + assert not stored[0].content.startswith("说：") + + +def test_memory_capture_archives_the_raw_session_as_markdown( + fake_memory, bind_agent, monkeypatch, tmp_path +): + """L0 只留人和助手，整份上传的会话另存 md 归档——工具过程在归档里可查。""" + from octop.infra.agents.workspace import host_system_dir + + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + monkeypatch.setattr(mm, "_resolve_agent_scope", lambda server, agent_id: (tmp_path, {})) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", session_id="cc:s5" + ) + + archive = host_system_dir(tmp_path, {}) / "memory-sessions" / "cc_s5.md" + assert result["events_by_type"] == {"user_message": 1, "assistant_message": 1} + assert result["session_archive"] == str(archive) + body = archive.read_text(encoding="utf-8") + assert "**tool_call" in body and "**tool_result" in body + assert "" in body + + # 整份重发（offset 丢失后的 mode=full）：不重复追加 + again = _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", session_id="cc:s5" + ) + assert again["session_archive"] is None + assert archive.read_text(encoding="utf-8") == body + + +def test_memory_capture_dry_run_writes_no_archive(fake_memory, bind_agent, monkeypatch, tmp_path): + """dry_run 只解析：不落 L0，也不写归档。""" + monkeypatch.setattr(mm, "_resolve_agent_scope", lambda server, agent_id: (tmp_path, {})) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", session_id="cc:s6", dry_run=True + ) + + assert result["session_archive"] is None + assert not (tmp_path / "memory-sessions").exists() + + +def test_memory_capture_rejects_unknown_tool_events(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + with pytest.raises(ValueError, match="unknown tool_events"): + _tools(mcp)["memory_capture"].fn( + content=_TOOL_TRANSCRIPT, source="claude-code", tool_events="whatever" + ) + + +def test_memory_capture_transcript_drops_host_plumbing(fake_memory, bind_agent, monkeypatch): + """Terminal-injected instructions and bundled skill docs are not conversation. + + Slash-command wrappers and local-command output stay: those record a user action + and its effect. + """ + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + records = [ + { + "type": "user", + "uuid": "u1", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "正常提问"}, + }, + { + "type": "user", + "uuid": "u2", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:01Z", + "message": { + "role": "user", + "content": "技能目录已更新", + }, + }, + { + "type": "user", + "uuid": "u3", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:02Z", + "message": { + "role": "user", + "content": "Base directory for this skill: /tmp/x\n# Building LLM apps", + }, + }, + { + "type": "user", + "uuid": "u4", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:03Z", + "message": { + "role": "user", + "content": "/modelmodel", + }, + }, + { + "type": "assistant", + "uuid": "a1", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:04Z", + "message": { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "t1", "name": "Read", "input": {"file_path": "x"}} + ], + }, + }, + { + "type": "user", + "uuid": "u5", + "sessionId": "s6", + "timestamp": "2026-09-22T05:00:05Z", + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "t1", + "content": "1→Base directory for this skill: /tmp/x", + "is_error": False, + } + ], + }, + }, + ] + + result = _tools(mcp)["memory_capture"].fn( + content="\n".join(json.dumps(record, ensure_ascii=False) for record in records), + source="claude-code", + session_id="cc:s6", + tool_events="all", + ) + + assert result["parsed"] == 6 + assert result["skipped_echo"] == 2 # system-reminder + skill 文档（都是消息事件） + assert result["imported"] == 4 # 真实提问 / 斜杠命令 / tool_call / tool_result + raws = fake_memory.add_raw_batch.call_args.args[0] + contents = [raw.content for raw in raws] + assert any("正常提问" in c for c in contents) + assert any("" in c for c in contents) + assert all("system-reminder" not in c for c in contents) + # 工具输出里出现同样字符串属于数据（读到的文件刚好含这句），不能当作注入丢弃 + assert any("Base directory for this skill" in c for c in contents) + + +def test_memory_capture_transcript_drops_own_hook_text(fake_memory, bind_agent, monkeypatch): + """Our own hook instruction is prompt plumbing, not something the user said.""" + monkeypatch.setattr(mm, "_trigger_extract", lambda server, session_id: True) + mcp = mm.build_memory_mcp(mock.MagicMock()) + records = [ + { + "type": "user", + "uuid": "u1", + "sessionId": "s7", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "AI预定是哪个项目?"}, + }, + { + "type": "user", + "uuid": "u2", + "sessionId": "s7", + "timestamp": "2026-09-22T05:00:01Z", + "message": { + "role": "user", + "content": "Stop hook feedback:\n[octop-memory hook] 本轮对话结束前，请把要点记录下来", + }, + }, + ] + + result = _tools(mcp)["memory_capture"].fn( + content="\n".join(json.dumps(record, ensure_ascii=False) for record in records), + source="claude-code", + session_id="cc:s7", + ) + + assert result["parsed"] == 2 + assert result["skipped_echo"] == 1 + assert result["imported"] == 1 + raws = fake_memory.add_raw_batch.call_args.args[0] + assert all("octop-memory hook" not in raw.content for raw in raws) + + +def test_memory_capture_transcript_rejects_broken_gzip_marker(fake_memory, bind_agent): + """``H4sI…`` says gzip+base64; if it does not decode, say so instead of storing junk.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + with pytest.raises(ValueError, match="gzip\\+base64"): + _tools(mcp)["memory_capture"].fn(content="H4sI!!!not-base64", source="claude-code") + + +def test_memory_capture_text_source_ignores_gzip_sniffing(fake_memory, bind_agent): + """A plain-text capture that happens to look base64-ish must stay verbatim.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + _tools(mcp)["memory_capture"].fn(content="H4sI 是一段普通笔记", source="review-bot") + assert fake_memory.add_raw.call_args.args[0] == "H4sI 是一段普通笔记" + + +def test_memory_save_can_supersede_an_atom(fake_memory, bind_agent): + """Correcting a fact = save the new one and name what it replaces.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_save"].fn( + content="new fact", source="review-bot", supersedes="atom1" + ) + fake_memory.deprecate_atom.assert_called_once_with( + "atom1", actor="user", note="superseded by memory_save" + ) + assert fake_memory.store.call_args.kwargs["metadata"] == { + "source": "review-bot", + "supersedes": "atom1", + } + assert result["deprecated"] is True + assert result["superseded_atom_id"] == "atom1" + + +def test_memory_save_without_supersedes_does_not_deprecate(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_save"].fn(content="plain fact", source="review-bot") + fake_memory.deprecate_atom.assert_not_called() + assert fake_memory.store.call_args.kwargs["metadata"] == {"source": "review-bot"} + assert "superseded_atom_id" not in result + + +def _asgi_scope(headers: list[tuple[bytes, bytes]] | None = None) -> dict: + return {"type": "http", "headers": headers or []} + + +@pytest.mark.asyncio +async def test_token_middleware_rejects_bad_token(): + inner_called = False + + async def _inner(scope, receive, send): + nonlocal inner_called + inner_called = True + + mw = mm._TokenAuthMiddleware(_inner, "secret") + sent = [] + scope = _asgi_scope([(b"authorization", b"Bearer wrong")]) + + async def _send(msg): + sent.append(msg) + + await mw(scope, lambda: {}, _send) + assert inner_called is False + assert sent[0]["status"] == 401 + + +@pytest.mark.asyncio +async def test_token_middleware_accepts_bearer(): + inner_called = False + + async def _inner(scope, receive, send): + nonlocal inner_called + inner_called = True + + mw = mm._TokenAuthMiddleware(_inner, "secret") + scope = _asgi_scope([(b"authorization", b"Bearer secret")]) + await mw(scope, lambda: {}, lambda msg: None) + assert inner_called is True + + +def test_mount_fail_closed_without_token(monkeypatch): + monkeypatch.delenv("OCTOP_MEMORY_MCP_TOKEN", raising=False) + app = mock.MagicMock() + assert mm.mount_memory_mcp(app, mock.MagicMock()) == [] + app.mount.assert_not_called() + + +def test_mount_unified_path_with_shared_app(monkeypatch): + """A single shared MCP app is mounted once at /mcp/memory.""" + from types import SimpleNamespace + + monkeypatch.setenv("OCTOP_MEMORY_MCP_TOKEN", "secret") + app = mock.MagicMock() + server = SimpleNamespace(services=SimpleNamespace(agent_repo=mock.MagicMock())) + managers = mm.mount_memory_mcp(app, server) + assert len(managers) == 1 + app.mount.assert_called_once() + assert app.mount.call_args.args[0] == "/mcp/memory" + + +@pytest.mark.asyncio +async def test_agent_router_routes_by_header(): + """_AgentRouter validates the agent, binds the contextvar, forwards to the shared app.""" + from types import SimpleNamespace + + seen = {} + + class _FakeApp: + async def __call__(self, scope, receive, send): + seen["agent"] = mm._current_agent_id.get() + + rows = { + "A1": SimpleNamespace(agent_id="A1", enabled=True), + "A2": SimpleNamespace(agent_id="A2", enabled=True), + } + server = SimpleNamespace( + services=SimpleNamespace(agent_repo=mock.MagicMock(get=lambda aid: rows.get(aid))) + ) + router = mm._AgentRouter(_FakeApp(), server) + scope = {"type": "http", "headers": [(b"x-octop-agent-id", b"A2")]} + await router(scope, lambda: {}, lambda msg: None) + assert seen["agent"] == "A2" + + +@pytest.mark.asyncio +async def test_agent_router_404_unknown_agent(): + """Unknown agent_id returns 404 before dispatching.""" + from types import SimpleNamespace + + server = SimpleNamespace( + services=SimpleNamespace(agent_repo=mock.MagicMock(get=lambda aid: None)) + ) + router = mm._AgentRouter(mock.MagicMock(), server) + scope = {"type": "http", "headers": [(b"x-octop-agent-id", b"NOPE")]} + sent = [] + + async def _send(msg): + sent.append(msg) + + await router(scope, lambda: {}, _send) + assert sent[0]["status"] == 404 + + +def test_trigger_extract_no_session_returns_false(): + """No session_id -> no extraction trigger.""" + assert mm._trigger_extract(mock.MagicMock(), None) is False + + +def test_trigger_extract_no_service_warns_after_start(monkeypatch, caplog): + """Stopped / memory-less agent: schedule wake+extract, warn if still no service.""" + import asyncio + import logging + + agent = mock.MagicMock() + agent._memory_runtime = mock.MagicMock(service=None) + registry = mock.MagicMock() + registry.get_agent.return_value = agent + + async def _start(_aid: str) -> None: + return None + + registry.start = _start + server = mock.MagicMock() + server.app_runtime.agent_registry = registry + created: list = [] + + def _capture(coro): + created.append(coro) + return mock.MagicMock() + + monkeypatch.setattr(asyncio, "create_task", _capture) + token = mm._current_agent_id.set("A1") + try: + with caplog.at_level(logging.WARNING, logger="octop.infra.agents.memory_mcp"): + assert mm._trigger_extract(server, "kiro-chat") is True + assert len(created) == 1 + asyncio.run(created[0]) + finally: + mm._current_agent_id.reset(token) + assert "MemoryService still unavailable" in caplog.text + + +def test_trigger_extract_starts_stopped_agent(monkeypatch): + """When get_agent fails (stopped), start the expert then extract.""" + import asyncio + + service = mock.MagicMock() + service.extract.return_value = {"candidates": 1} + agent = mock.MagicMock() + agent._memory_runtime = mock.MagicMock(service=service) + registry = mock.MagicMock() + registry.get_agent.side_effect = [ + RuntimeError("not running"), + agent, + ] + registry.start = mock.AsyncMock() + server = mock.MagicMock() + server.app_runtime.agent_registry = registry + created: list = [] + + def _capture(coro): + created.append(coro) + return mock.MagicMock() + + monkeypatch.setattr(asyncio, "create_task", _capture) + token = mm._current_agent_id.set("A1") + try: + assert mm._trigger_extract(server, "cursor:sess") is True + assert len(created) == 1 + asyncio.run(created[0]) + finally: + mm._current_agent_id.reset(token) + registry.start.assert_awaited_once_with("A1") + assert registry.get_agent.call_count == 2 + service.extract.assert_called_once_with( + "cursor:sess", incremental=True, promote=True, regen_pages=True + ) + + +def test_trigger_extract_schedules_service(monkeypatch): + """With a service, asynchronously schedule extract and return True.""" + import asyncio + + agent = mock.MagicMock() + service = mock.MagicMock() + service.extract.return_value = {"candidates": 0} + runtime = mock.MagicMock() + runtime.service = service + agent._memory_runtime = runtime + registry = mock.MagicMock(get_agent=lambda aid: agent) + server = mock.MagicMock() + server.app_runtime.agent_registry = registry + created: list = [] + + def _capture(coro): + created.append(coro) + return mock.MagicMock() + + monkeypatch.setattr(asyncio, "create_task", _capture) + + token = mm._current_agent_id.set("A1") + try: + assert mm._trigger_extract(server, "kiro-chat") is True + assert len(created) == 1 + asyncio.run(created[0]) + finally: + mm._current_agent_id.reset(token) + service.extract.assert_called() + assert service.extract.call_args.args[0] == "kiro-chat" + + +def _with_agent(agent_id: str): + """Bind the request-scoped agent contextvar for a single tool call.""" + return mm._current_agent_id.set(agent_id) + + +def test_memory_candidates_passes_status_string(fake_memory): + """``status`` is a typing.Literal of strings: pass the raw value, don't + instantiate it (previously crashed with "Cannot instantiate typing.Literal").""" + fake_memory.list_candidates.return_value = [] + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + result = _tools(mcp)["memory_candidates"].fn(status="pending", limit=5) + finally: + mm._current_agent_id.reset(token) + assert result["candidates"] == [] + assert fake_memory.list_candidates.call_args.kwargs["status"] == "pending" + + +def test_memory_candidates_not_a_status_raises(fake_memory): + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + with pytest.raises(ValueError): + _tools(mcp)["memory_candidates"].fn(status="no-such-status") + finally: + mm._current_agent_id.reset(token) + + +class _FakeBridge: + """Stand-in for ``octop_memory``'s ``Bridge``, recording dispatched RPCs.""" + + def __init__(self) -> None: + self.calls: list[tuple[str, dict]] = [] + self.results: dict[str, dict] = {} + self.errors: dict[str, str] = {} + + def handle(self, request): + method, params = request["method"], request["params"] + self.calls.append((method, params)) + key = params.get("candidate_id") + if key in self.errors: + return { + "jsonrpc": "2.0", + "id": request["id"], + "error": {"code": -32602, "message": self.errors[key]}, + } + return {"jsonrpc": "2.0", "id": request["id"], "result": self.results.get(key, {})} + + +@pytest.fixture +def fake_bridge(monkeypatch): + """Replace the dashboard bridge so review tools need no real Memory.""" + bridge = _FakeBridge() + monkeypatch.setattr(mm, "_dashboard_bridge", lambda memory: bridge) + return bridge + + +_PROMOTED = { + "promoted": 1, + "merged": 0, + "conflicts": 0, + "needs_review": 0, + "dropped": 0, + "llm_calls": 0, +} + + +def test_memory_promote_delegates_to_dashboard_rpc(fake_memory, fake_bridge): + """采纳走 dashboard 的 ``promote_candidate``：同参数、同返回、逐条聚合。""" + fake_bridge.results["c1"] = _PROMOTED + fake_bridge.results["c2"] = {**_PROMOTED, "promoted": 0, "merged": 1} + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + result = _tools(mcp)["memory_promote"].fn(candidate_ids=["c1", "c2"]) + finally: + mm._current_agent_id.reset(token) + assert fake_bridge.calls == [ + ("promote_candidate", {"candidate_id": "c1"}), + ("promote_candidate", {"candidate_id": "c2"}), + ] + assert result["promoted"] == 1 + assert result["merged"] == 1 + assert [c["candidate_id"] for c in result["candidates"]] == ["c1", "c2"] + assert result["errors"] == [] + + +def test_memory_promote_isolates_per_candidate_errors(fake_memory, fake_bridge): + """单条被 dashboard 拒绝（如非 pending）不影响同批其它候选。""" + fake_bridge.results["c1"] = _PROMOTED + fake_bridge.errors["c2"] = "candidate 'c2' not pending (current status='promoted')" + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + result = _tools(mcp)["memory_promote"].fn(candidate_ids=["c1", "c2"]) + finally: + mm._current_agent_id.reset(token) + assert result["promoted"] == 1 + assert [c["candidate_id"] for c in result["candidates"]] == ["c1"] + assert result["errors"] == [ + {"candidate_id": "c2", "error": "candidate 'c2' not pending (current status='promoted')"} + ] + + +def test_memory_reject_delegates_to_dashboard_rpc(fake_memory, fake_bridge): + """忽略走 dashboard 的 ``reject_candidate``；缺省actor与面板一致不传。""" + fake_bridge.results["c1"] = {"candidate_id": "c1", "status": "rejected"} + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + result = _tools(mcp)["memory_reject"].fn(candidate_id="c1", reason="dup") + finally: + mm._current_agent_id.reset(token) + assert fake_bridge.calls == [("reject_candidate", {"candidate_id": "c1", "reason": "dup"})] + assert result == {"ok": True, "candidate_id": "c1", "status": "rejected"} + + +def test_memory_reject_passes_actor_when_given(fake_memory, fake_bridge): + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + _tools(mcp)["memory_reject"].fn(candidate_id="c1", reason="dup", actor="auto") + finally: + mm._current_agent_id.reset(token) + assert fake_bridge.calls == [ + ("reject_candidate", {"candidate_id": "c1", "reason": "dup", "actor": "auto"}) + ] + + +def test_memory_reject_surfaces_bridge_error(fake_memory, fake_bridge): + fake_bridge.errors["c1"] = "candidate 'c1' cannot be rejected (current status='promoted')" + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + with pytest.raises(ValueError, match="cannot be rejected"): + _tools(mcp)["memory_reject"].fn(candidate_id="c1") + finally: + mm._current_agent_id.reset(token) + + +def test_memory_promote_end_to_end_through_real_bridge(tmp_path, monkeypatch): + """真实 harness-memory + Bridge：pending 候选经 MCP 采纳后落成原子。 + + 这是本次改动的核心契约——MCP 与记忆面板共用 dashboard_data，而不是各自 + 一份实现（旧实现直接调 ``Memory.promote_candidates``，对 conflict 候选无效）。 + """ + pytest.importorskip("octop_memory.adapters.bridge.handlers") + from octop_memory.core import Memory # noqa: PLC0415 + from octop_memory.types import Candidate, RawEvent # noqa: PLC0415 + + memory = Memory( + namespace="agent_a1", + backend="sqlite", + backend_config={"db_path": str(tmp_path / "memory.sqlite")}, + ) + memory.add_raw_batch( + [ + RawEvent( + id="raw-1", + host="octop", + session_id="s1", + thread_id="t1", + user="u", + timestamp=datetime.now(UTC), + event_type="user_message", + content="我喜欢喝美式咖啡", + ) + ] + ) + memory.add_candidate( + Candidate( + id="cand-1", + raw_event_ids=["raw-1"], + candidate_type="Preference", + status="pending", + title="美式咖啡偏好", + assertion="喜欢喝美式咖啡", + verbatim_quote="我喜欢喝美式咖啡", + quote_event_id="raw-1", + subject_name="user", + subject_entity_type="User", + target_entity_id=None, + confidence="high", + importance="high", + recommended_action="promote", + promotion_reason="seed", + extractor_version="test", + created_at=datetime.now(UTC), + ) + ) + monkeypatch.setattr(mm, "_open_memory", lambda server, agent_id: memory) + mcp = mm.build_memory_mcp(mock.MagicMock()) + token = _with_agent("A1") + try: + result = _tools(mcp)["memory_promote"].fn(candidate_ids=["cand-1"]) + again = _tools(mcp)["memory_promote"].fn(candidate_ids=["cand-1"]) + finally: + mm._current_agent_id.reset(token) + assert result["errors"] == [] + assert result["promoted"] == 1 + assert memory.get_candidate("cand-1").status == "promoted" + assert [a.assertion for a in memory.list_atoms()] == ["喜欢喝美式咖啡"] + # 重复采纳被 dashboard 的终态守卫挡下，且报错文本与面板一致（含当前状态）。 + assert again["promoted"] == 0 + assert len(again["errors"]) == 1 + assert "status=" in again["errors"][0]["error"] + + +def _stub_runtime(monkeypatch, *, hits=None, get_result=None, captured=None): + """Patch ``MemoryRuntime`` so search/get run against a fake runtime.""" + + class _FakeRuntime: + def __init__(self, memory): + if captured is not None: + captured["memory"] = memory + + def memory_search(self, params): + if captured is not None: + captured["search_params"] = params + return {"hits": list(hits or []), "total": len(hits or []), "empty_reason": None} + + def memory_get(self, params): + if captured is not None: + captured["get_params"] = params + return dict(get_result or {}) + + import octop_memory.application.runtime as _runtime + + monkeypatch.setattr(_runtime, "MemoryRuntime", _FakeRuntime) + + +def test_tool_responses_carry_the_family_marker(fake_memory, monkeypatch, bind_agent): + """One prefix (``[octop memory ``) lets any consumer filter all of our output.""" + _stub_runtime(monkeypatch, hits=[], get_result={"path": "atom/a1.md", "excerpt": ""}) + mcp = mm.build_memory_mcp(mock.MagicMock()) + tools = _tools(mcp) + + assert tools["memory_search"].fn(query="q")["marker"] == "[octop memory search]" + assert tools["memory_get"].fn(path="atom/a1.md")["marker"] == "[octop memory get]" + assert tools["memory_capture"].fn(content="x", source="s")["marker"] == "[octop memory capture]" + + +def test_memory_capture_drops_family_marker_content(fake_memory, bind_agent): + """A ``[octop memory …]`` payload is our own output, not something the user said.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn( + content="[octop memory search] related memories: - [atom] 旧内容", source="hook" + ) + assert result["recorded"] is False + assert result["skipped"] == "recall_echo" + fake_memory.add_raw.assert_not_called() + + +def test_memory_search_projects_hit_paths(fake_memory, monkeypatch, bind_agent): + """memory_search runs the shared pipeline and returns path-carrying hits.""" + captured = {} + _stub_runtime( + monkeypatch, + hits=[ + {"path": "atom/a1.md", "layer": "atom", "snippet": "s1", "source_id": "a1"}, + {"path": "raw/2026-08-19/r1.md", "layer": "raw", "snippet": "s2", "source_id": "r1"}, + ], + captured=captured, + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_search"].fn(query="billing", max_results=5) + assert captured["memory"] is fake_memory + assert captured["search_params"] == {"query": "billing", "maxResults": 5, "corpus": "memory"} + assert [hit["path"] for hit in result["hits"]] == ["atom/a1.md", "raw/2026-08-19/r1.md"] + assert result["total"] == 2 + assert result["corpus"] == "all" + + +def test_memory_search_atom_corpus_widens_then_filters(fake_memory, monkeypatch, bind_agent): + """``corpus=atom`` asks for a wider pool, then keeps only L2 atoms.""" + captured = {} + _stub_runtime( + monkeypatch, + hits=[ + {"path": "raw/2026-08-19/r1.md", "layer": "raw", "snippet": "s2", "source_id": "r1"}, + {"path": "atom/a1.md", "layer": "atom", "snippet": "s1", "source_id": "a1"}, + {"path": "atom/a2.md", "layer": "atom", "snippet": "s3", "source_id": "a2"}, + ], + captured=captured, + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_search"].fn(query="billing", max_results=2, corpus="atom") + assert captured["search_params"]["maxResults"] == 8 + assert [hit["path"] for hit in result["hits"]] == ["atom/a1.md", "atom/a2.md"] + assert result["corpus"] == "atom" + + +def test_memory_search_raw_corpus_uses_l0_fts(fake_memory, bind_agent): + """``corpus=raw`` bypasses the atom-first fallback and searches L0 directly.""" + from datetime import datetime + + event = mock.MagicMock() + event.id = "r1" + event.content = "nginx 需要 proxy /api/memory-mcp" + event.timestamp = datetime(2026, 8, 19, 12, 0) + fake_memory.search_raw.return_value = [event] + + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_search"].fn(query="proxy", max_results=3, corpus="raw") + fake_memory.search_raw.assert_called_once_with("proxy", limit=3) + assert result["hits"][0]["path"] == "raw/2026-08-19/r1.md" + assert result["hits"][0]["layer"] == "raw" + + +def test_memory_search_prefers_the_callers_own_hits( + fake_memory, monkeypatch, bind_agent, bind_user +): + """带了调用者就先把「这个人的」命中排前面：只调顺序，不过滤团队共享内容。""" + fake_memory.get_raw_events_by_ids.return_value = [mock.MagicMock(id="r1", user="alice")] + captured = {} + _stub_runtime( + monkeypatch, + hits=[ + {"path": "atom/a1.md", "layer": "atom", "snippet": "别人说的", "source_id": "a1"}, + { + "path": "raw/2026-08-19/r1.md", + "layer": "raw", + "snippet": "我说的", + "source_id": "r1", + }, + { + "path": "atom/a2.md", + "layer": "atom", + "snippet": "alice说：我的坑", + "source_id": "a2", + }, + ], + captured=captured, + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_search"].fn(query="q", max_results=2) + + assert captured["search_params"]["maxResults"] == 8 # 带 caller 先放大候选池 + assert [hit["path"] for hit in result["hits"]] == ["raw/2026-08-19/r1.md", "atom/a2.md"] + assert result["caller"] == "alice" + assert result["personal_hits"] == 2 + assert all(hit["personal"] for hit in result["hits"]) + assert result["hits"][0]["user"] == "alice" # raw 命中带上 raw 的 user 列 + + +def test_memory_search_without_caller_keeps_the_pipeline_order( + fake_memory, monkeypatch, bind_agent +): + """没有调用者：不放大候选池、不加 personal 字段、顺序原样。""" + captured = {} + _stub_runtime( + monkeypatch, + hits=[{"path": "atom/a1.md", "layer": "atom", "snippet": "x", "source_id": "a1"}], + captured=captured, + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_search"].fn(query="q", max_results=3) + + assert captured["search_params"]["maxResults"] == 3 + assert result["caller"] is None + assert result["personal_hits"] == 0 + assert "personal" not in result["hits"][0] + + +def test_memory_search_raw_corpus_prefers_the_callers_events(fake_memory, bind_agent, bind_user): + """``corpus=raw`` 同样按 raw 的 user 列把人自己的行排前面。""" + from datetime import datetime + + other, mine = mock.MagicMock(), mock.MagicMock() + for event, uid, event_id in ((other, "someone-else", "r1"), (mine, "alice", "r2")): + event.id, event.user, event.content = event_id, uid, "内容" + event.timestamp = datetime(2026, 8, 19, 12, 0) + fake_memory.search_raw.return_value = [other, mine] + + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_search"].fn(query="q", max_results=2, corpus="raw") + + fake_memory.search_raw.assert_called_once_with("q", limit=8) + assert [hit["source_id"] for hit in result["hits"]] == ["r2", "r1"] + assert [hit["personal"] for hit in result["hits"]] == [True, False] + assert [hit["user"] for hit in result["hits"]] == ["alice", "someone-else"] + + +def test_memory_search_rejects_unknown_corpus(fake_memory, bind_agent): + mcp = mm.build_memory_mcp(mock.MagicMock()) + with pytest.raises(ValueError): + _tools(mcp)["memory_search"].fn(query="billing", corpus="wiki") + + +def test_memory_get_returns_excerpt_as_content(fake_memory, monkeypatch, bind_agent): + """memory_get surfaces the excerpt under ``content`` and forwards paging.""" + captured = {} + _stub_runtime( + monkeypatch, + get_result={ + "path": "atom/a1.md", + "kind": "atom", + "excerpt": "# body", + "metadata": {"id": "a1"}, + "total_lines": 3, + "from_line": 1, + "to_line": 3, + "truncated": False, + }, + captured=captured, + ) + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_get"].fn(path="atom/a1.md", start=1, lines=10) + assert captured["get_params"] == {"path": "atom/a1.md", "from": 1, "lines": 10} + assert result["content"] == "# body" + assert result["kind"] == "atom" + assert result["total_lines"] == 3 + + +def test_memory_get_returns_hint_for_bad_path(fake_memory, monkeypatch, bind_agent): + """A stale/mistyped path degrades to an error payload instead of raising.""" + + class _Boom: + def __init__(self, memory): + pass + + def memory_get(self, params): + raise ValueError("path must be non-empty and unpadded: 'nope'") + + import octop_memory.application.runtime as _runtime + + monkeypatch.setattr(_runtime, "MemoryRuntime", _Boom) + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_get"].fn(path="nope") + assert "path must be non-empty" in result["error"] + assert "atom/.md" in result["hint"] + + +def test_memory_capture_prefixes_sender(fake_memory, bind_agent, bind_user): + """The header user id is folded into the captured text, not just the payload.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn(content="接口先不要动", source="review-bot") + assert fake_memory.add_raw.call_args.args[0] == "alice说：接口先不要动" + assert fake_memory.add_raw.call_args.kwargs["user"] == "alice" + assert result["content"] == "alice说：接口先不要动" + assert result["user"] == "alice" + + +def test_memory_capture_does_not_double_prefix(fake_memory, bind_agent, bind_user): + mcp = mm.build_memory_mcp(mock.MagicMock()) + _tools(mcp)["memory_capture"].fn(content="alice说：接口先不要动", source="review-bot") + assert fake_memory.add_raw.call_args.args[0] == "alice说：接口先不要动" + + +def test_memory_save_prefixes_sender(fake_memory, bind_agent, bind_user): + mcp = mm.build_memory_mcp(mock.MagicMock()) + _tools(mcp)["memory_save"].fn(content="部署约定：端点挂在 /mcp/memory", source="coding-agent") + assert fake_memory.store.call_args.args[0] == "alice说：部署约定：端点挂在 /mcp/memory" + assert fake_memory.store.call_args.kwargs["metadata"]["user"] == "alice" + + +def test_memory_save_supersede_prefixes_sender(fake_memory, bind_agent, bind_user): + mcp = mm.build_memory_mcp(mock.MagicMock()) + _tools(mcp)["memory_save"].fn(content="端点改到 /api/memory-mcp", source="s", supersedes="a1") + assert fake_memory.store.call_args.args[0] == "alice说：端点改到 /api/memory-mcp" + + +def test_write_without_caller_keeps_content(fake_memory, bind_agent): + """No ``X-Octop-User-Id`` -> no attribution prefix.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + _tools(mcp)["memory_save"].fn(content="no sender", source="coding-agent") + assert fake_memory.store.call_args.args[0] == "no sender" + + +@pytest.mark.parametrize( + "content", + [ + "[memory] Earlier in this workspace, related to your question:\n- [atom] x", + "结论见下：\n## Memory Recall\n- [atom] y\n[/memory]", + ], +) +def test_memory_capture_drops_recall_echo(fake_memory, bind_agent, content): + """Injected recall blocks are not captured back (mirrors skip_memory_echo).""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn(content=content, source="hook") + assert result["recorded"] is False + assert result["skipped"] == "recall_echo" + assert "recall_echo" in result["skipped"] + fake_memory.add_raw.assert_not_called() + + +def test_memory_capture_still_records_normal_content(fake_memory, bind_agent, bind_user): + """The echo guard must not block ordinary captures.""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + result = _tools(mcp)["memory_capture"].fn(content="接口先不要动", source="hook") + assert result["recorded"] is True + assert fake_memory.add_raw.call_args.args[0] == "alice说：接口先不要动" + + +def _installer_scope(query: str = "", method: str = "GET") -> dict: + return { + "type": "http", + "method": method, + "path": "/mcp/memory/setup", + "root_path": "/mcp/memory", + "query_string": query.encode(), + "headers": [(b"host", b"127.0.0.1:8088")], + } + + +async def _run_installer(scope: dict) -> tuple[list[dict], str]: + app = mm._InstallerApp(mock.MagicMock()) + sent: list[dict] = [] + + async def _send(msg): + sent.append(msg) + + await app(scope, lambda: {}, _send) + return sent, sent[1]["body"].decode() if len(sent) > 1 else "" + + +@pytest.mark.asyncio +async def test_installer_serves_the_script_without_authentication(monkeypatch): + """公共端点：只返回脚本，不要求鉴权；实例没配默认凭据时脚本里也没有凭据。""" + monkeypatch.delenv("OCTOP_MEMORY_MCP_TOKEN", raising=False) + sent, body = await _run_installer(_installer_scope("agent=AGT1&terminals=kiro")) + + assert sent[0]["status"] == 200 + assert (b"content-type", b"text/x-shellscript; charset=utf-8") in sent[0]["headers"] + assert "octop-stop-capture.mjs" in body + assert "DEFAULT_AGENT='AGT1'" in body + assert "DEFAULT_TERMINALS='kiro'" in body + assert "DEFAULT_TOKEN=" not in body + + +@pytest.mark.asyncio +async def test_installer_bakes_the_instance_token_as_the_default(monkeypatch): + """实例发布时配的 OCTOP_MEMORY_MCP_TOKEN 直接内置成默认凭据：用户只需专家 ID。""" + monkeypatch.setenv("OCTOP_MEMORY_MCP_TOKEN", "octopoctop") + _, body = await _run_installer(_installer_scope()) + + assert "DEFAULT_TOKEN='octopoctop'" in body + assert 'TOKEN="${OCTOP_MEMORY_TOKEN:-}"' in body # 现场变量仍可覆盖 + assert "OCTOP_ENTRY_TOKEN" not in body # 只有一份凭据 + + +@pytest.mark.asyncio +async def test_installer_defaults_to_the_host_it_was_fetched_from(): + _, body = await _run_installer(_installer_scope()) + + assert "URL='http://127.0.0.1:8088/mcp/memory/'" in body + + +@pytest.mark.asyncio +async def test_installer_url_query_wins_over_the_default(): + _, body = await _run_installer(_installer_scope("url=https://octop-memory.example.com/mcp")) + + assert "URL='https://octop-memory.example.com/mcp'" in body + assert "OCTOP_SERVER_NAME='octop-memory'" in body + + +@pytest.mark.asyncio +async def test_installer_public_url_override(monkeypatch): + """部署可通过环境变量指定"客户端该连哪个地址"（例如另一个 ingress 域名）。""" + monkeypatch.setenv("OCTOP_MEMORY_PUBLIC_MCP_URL", "https://oauth.example.com/mcp") + _, body = await _run_installer(_installer_scope()) + + assert "URL='https://oauth.example.com/mcp'" in body + + +@pytest.mark.asyncio +async def test_installer_rejects_non_get(): + sent, body = await _run_installer(_installer_scope(method="POST")) + + assert sent[0]["status"] == 405 + assert "GET only" in body + + +def _one_turn_transcript() -> str: + return json.dumps( + { + "type": "user", + "uuid": "tp1", + "sessionId": "s-tp", + "timestamp": "2026-09-22T05:00:00Z", + "message": {"role": "user", "content": "从文件读的一段会话"}, + } + ) + + +def test_memory_capture_reads_a_local_transcript_path( + fake_memory, bind_agent, tmp_path, monkeypatch +): + """mcp_tool hook 只传 ${transcript_path}：同机服务端把文件读进来。""" + monkeypatch.setattr(mm, "_TRANSCRIPT_PATH_ROOTS", (str(tmp_path),)) + session_file = tmp_path / "s-tp.jsonl" + session_file.write_text(_one_turn_transcript(), encoding="utf-8") + mcp = mm.build_memory_mcp(mock.MagicMock()) + + result = _tools(mcp)["memory_capture"].fn( + content="", source="claude-code", transcript_path=str(session_file), session_id="cc:s-tp" + ) + + assert result["parsed"] == 1 + assert result["imported"] == 1 + assert result["events_by_type"] == {"user_message": 1} + + +def test_memory_capture_refuses_paths_outside_the_session_stores(fake_memory, bind_agent, tmp_path): + """不给任意文件读取留后门：只允许终端会话目录。""" + mcp = mm.build_memory_mcp(mock.MagicMock()) + stray = tmp_path / "secrets.txt" + stray.write_text("nope", encoding="utf-8") + + with pytest.raises(ValueError, match="terminal session store"): + _tools(mcp)["memory_capture"].fn( + content="", source="claude-code", transcript_path=str(stray) + ) + + +def test_memory_capture_refuses_a_compressed_path(fake_memory, bind_agent, tmp_path, monkeypatch): + monkeypatch.setattr(mm, "_TRANSCRIPT_PATH_ROOTS", (str(tmp_path),)) + packed = tmp_path / "session.v3.jsonl.zstd" + packed.write_bytes(b"zstd-bytes") + mcp = mm.build_memory_mcp(mock.MagicMock()) + + with pytest.raises(ValueError, match=r"gzip\+base64"): + _tools(mcp)["memory_capture"].fn( + content="", source="claude-code", transcript_path=str(packed) + ) + + +@pytest.mark.asyncio +async def test_installer_defaults_to_the_deployment_config(monkeypatch, tmp_path): + """部署在 config.json 里声明一次 memory_public_mcp_url，同学就不用传 ?url=。""" + monkeypatch.delenv("OCTOP_MEMORY_PUBLIC_MCP_URL", raising=False) + monkeypatch.setenv("OCTOP_HOME", str(tmp_path)) + (tmp_path / "config.json").write_text( + json.dumps({"memory_public_mcp_url": "https://octop-memory.example.com/mcp"}), + encoding="utf-8", + ) + + _, body = await _run_installer(_installer_scope()) + + assert "URL='https://octop-memory.example.com/mcp'" in body + + +@pytest.mark.asyncio +async def test_installer_env_overrides_the_config_file(monkeypatch, tmp_path): + monkeypatch.setenv("OCTOP_HOME", str(tmp_path)) + (tmp_path / "config.json").write_text( + json.dumps({"memory_public_mcp_url": "https://from-config.example.com/mcp"}), + encoding="utf-8", + ) + monkeypatch.setenv("OCTOP_MEMORY_PUBLIC_MCP_URL", "https://from-env.example.com/mcp") + + _, body = await _run_installer(_installer_scope()) + + assert "URL='https://from-env.example.com/mcp'" in body + + +_EXTERNAL_HOST = "octop.example.com" + + +@pytest.mark.parametrize("host", [_EXTERNAL_HOST, "octop.internal.example"]) +@pytest.mark.parametrize("proto", ["", "http", "https"]) +@pytest.mark.asyncio +async def test_installer_defaults_to_this_instance_for_any_host(monkeypatch, host, proto): + """没有显式覆盖时，条目与 hooks 都指向本实例挂载点；非本机一律 https。""" + monkeypatch.delenv("OCTOP_MEMORY_PUBLIC_MCP_URL", raising=False) + scope = _installer_scope() + scope["headers"] = [(b"host", host.encode())] + if proto: + scope["headers"].append((b"x-forwarded-proto", proto.encode())) + + _, body = await _run_installer(scope) + + # 精确匹配整行：只写 "URL=" 会被 "HOOK_URL=" 子串误命中 + assert f"\nURL='https://{host}/mcp/memory/'\n" in body + assert f"\nHOOK_URL='https://{host}/mcp/memory/'\n" in body + + +@pytest.mark.asyncio +async def test_installer_splits_the_client_entry_from_the_hook_target(monkeypatch): + """条目地址可以被部署指到公网入口，hooks 仍拿本实例的直连挂载点（凭据同一份共享 token）。""" + monkeypatch.setenv("OCTOP_MEMORY_PUBLIC_MCP_URL", "https://octop-memory.example.com/mcp") + scope = _installer_scope() + scope["headers"] = [ + (b"host", _EXTERNAL_HOST.encode()), + (b"x-forwarded-proto", b"https"), + ] + + _, body = await _run_installer(scope) + + assert "URL='https://octop-memory.example.com/mcp'" in body + assert f"HOOK_URL='https://{_EXTERNAL_HOST}/mcp/memory/'" in body + + +@pytest.mark.asyncio +async def test_installer_hook_url_can_be_overridden(monkeypatch): + """直连挂载点不在本机时，?hook_url= 显式指定。""" + monkeypatch.delenv("OCTOP_MEMORY_PUBLIC_MCP_URL", raising=False) + scope = _installer_scope() + scope["query_string"] = b"hook_url=https%3A%2F%2Foctop.example.com%2Fmcp%2Fmemory%2F" + + _, body = await _run_installer(scope) + + assert "HOOK_URL='https://octop.example.com/mcp/memory/'" in body + + +@pytest.mark.parametrize("proto", ["", "http", "https"]) +@pytest.mark.asyncio +async def test_installer_keeps_loopback_on_this_instance(monkeypatch, proto): + """本机开发实例指向自己的挂载点：没有 TLS 前置，照请求的 scheme 走 http。""" + monkeypatch.delenv("OCTOP_MEMORY_PUBLIC_MCP_URL", raising=False) + scope = _installer_scope() + if proto: + scope["headers"].append((b"x-forwarded-proto", proto.encode())) + + _, body = await _run_installer(scope) + + scheme = "https" if proto == "https" else "http" + assert f"URL='{scheme}://127.0.0.1:8088/mcp/memory/'" in body + + +def test_mount_serves_setup_before_the_token_check(monkeypatch): + """GET /mcp/memory/setup 免鉴权（只发脚本）；同一挂载下的其它路径仍要 token。""" + from types import SimpleNamespace + + from starlette.applications import Starlette + from starlette.testclient import TestClient + + monkeypatch.setenv("OCTOP_MEMORY_MCP_TOKEN", "secret") + app = Starlette() + server = SimpleNamespace(services=SimpleNamespace(agent_repo=mock.MagicMock())) + mm.mount_memory_mcp(app, server) + client = TestClient(app) + + served = client.get("/mcp/memory/setup") + assert served.status_code == 200 + assert served.text.startswith("#!/bin/sh") + + refused = client.post("/mcp/memory/setup") + assert refused.status_code == 405 # 只支持 GET + + assert client.post("/mcp/memory/", json={}).status_code == 401 # 其它路径照旧鉴权 + + +def test_mount_serves_uninstall_before_the_token_check(monkeypatch): + """GET /mcp/memory/uninstall 同样免鉴权，只发卸载脚本。""" + from types import SimpleNamespace + + from starlette.applications import Starlette + from starlette.testclient import TestClient + + monkeypatch.setenv("OCTOP_MEMORY_MCP_TOKEN", "secret") + app = Starlette() + server = SimpleNamespace(services=SimpleNamespace(agent_repo=mock.MagicMock())) + mm.mount_memory_mcp(app, server) + client = TestClient(app) + + served = client.get("/mcp/memory/uninstall") + assert served.status_code == 200 + assert served.text.startswith("#!/bin/sh") + assert "卸载" in served.text + + assert client.post("/mcp/memory/uninstall").status_code == 405 diff --git a/tests/unit/agents/test_memory_recall_patch.py b/tests/unit/agents/test_memory_recall_patch.py new file mode 100644 index 000000000..87d51140e --- /dev/null +++ b/tests/unit/agents/test_memory_recall_patch.py @@ -0,0 +1,42 @@ +"""Unit tests for the octop_memory recall-quality patch (infra/agents/memory_recall_patch). + +The patch monkey-patches three symbols inside the upstream ``harness-memory`` +package at startup, so these tests only assert the wiring contract: the three +symbols are replaced, the originals are kept as the fallback, and a second call +is a no-op. The patched behaviour itself is covered by the memory eval corpus, +not here. +""" + +from __future__ import annotations + +import octop_memory.pipeline.recall.multi_source as multi_source +import octop_memory.pipeline.recall.router as router + +from octop.infra.agents import memory_recall_patch as patch + + +def test_apply_patches_the_three_recall_symbols(): + """router.route + the two multi_source helpers are replaced, originals kept.""" + patch.apply_memory_recall_patch() + + assert router.route is patch._patched_route + assert multi_source._per_token_atom_search is patch._patched_per_token_atom_search + assert multi_source._gather_atoms is patch._patched_gather_atoms + + # The upstream implementations stay reachable as the fallback path. + assert router._orig_route is not patch._patched_route + assert multi_source._orig_per_token_atom_search is not patch._patched_per_token_atom_search + assert multi_source._orig_gather_atoms is not patch._patched_gather_atoms + + +def test_apply_is_idempotent(): + """A second call must not re-wrap the originals (that would nest the patch).""" + patch.apply_memory_recall_patch() + original_route = router._orig_route + original_gather = multi_source._orig_gather_atoms + + patch.apply_memory_recall_patch() + + assert router._orig_route is original_route + assert multi_source._orig_gather_atoms is original_gather + assert router.route is patch._patched_route diff --git a/tests/unit/agents/test_memory_session_archive.py b/tests/unit/agents/test_memory_session_archive.py new file mode 100644 index 000000000..e30b8e385 --- /dev/null +++ b/tests/unit/agents/test_memory_session_archive.py @@ -0,0 +1,139 @@ +"""The raw-session markdown archive: L0 keeps messages, the archive keeps everything.""" + +from __future__ import annotations + +import json + +from octop.infra.agents.memory_session_archive import ( + _MAX_BODY_CHARS, + append_session, + archive_path, +) + + +def _jsonl(records: list[dict]) -> str: + return "\n".join(json.dumps(record, ensure_ascii=False) for record in records) + + +def _user(uuid: str, text: str = "跑一下测试", session: str = "s1") -> dict: + return { + "type": "user", + "uuid": uuid, + "sessionId": session, + "timestamp": "2026-09-29T02:00:00Z", + "message": {"role": "user", "content": text}, + } + + +def _assistant(uuid: str, blocks: list[dict], session: str = "s1") -> dict: + return { + "type": "assistant", + "uuid": uuid, + "sessionId": session, + "timestamp": "2026-09-29T02:00:05Z", + "message": {"role": "assistant", "content": blocks}, + } + + +def test_archive_renders_every_block_not_just_messages(tmp_path): + """L0 会丢 thinking / tool 事件，归档必须都留着。""" + text = _jsonl( + [ + _user("u1"), + _assistant( + "a1", + [ + {"type": "thinking", "thinking": "先想一下"}, + {"type": "text", "text": "我跑一下"}, + { + "type": "tool_use", + "id": "t1", + "name": "Bash", + "input": {"command": "pytest -q"}, + }, + { + "type": "tool_result", + "tool_use_id": "t1", + "content": "1 failed", + }, + ], + ), + ] + ) + + path = append_session( + tmp_path, session_id="cc:s1", text=text, host="claude-code", user="alice" + ) + + assert path == archive_path(tmp_path, "cc:s1") + assert path.name == "cc_s1.md" # 文件名净化 + body = path.read_text(encoding="utf-8") + assert "# claude-code session cc:s1" in body + assert "- user: alice" in body + assert "跑一下测试" in body + assert "*（thinking）*" in body and "先想一下" in body + assert "**tool_call · Bash**" in body and '"command": "pytest -q"' in body + assert "**tool_result" in body and "1 failed" in body + assert body.count("") == 1 + assert after_delta.count("") == 1 + assert after_delta.count("# claude-code session cc:s2") == 1 # 头只写一次 + + +def test_archive_keeps_long_tool_output_beyond_the_l0_cap(tmp_path): + """归档给人看：单条上限比 L0 的 4000 宽松得多。""" + body_text = "line\n" * 2000 # ~10KB，超过 L0 的 4000 + text = _jsonl( + [ + _user("u1"), + _assistant( + "a1", + [{"type": "tool_result", "tool_use_id": "t1", "content": body_text}], + ), + ] + ) + + path = append_session(tmp_path, session_id="cc:s3", text=text, host="claude-code") + body = path.read_text(encoding="utf-8") + + assert body.count("line") >= 1500 # 远多于 4000 字符 + assert len(body) <= _MAX_BODY_CHARS * 2 # 仍然有界 + + +def test_archive_skips_non_records_and_keeps_thinking_optional(tmp_path): + """非 user/assistant 记录（mode / cost-state / attachment）不进归档。""" + text = _jsonl( + [ + {"type": "mode", "mode": "acceptEdits"}, + _user("u1", text="问了"), + {"type": "cost-state", "total": 1}, + ] + ) + + path = append_session(tmp_path, session_id="cc:s4", text=text, host="claude-code") + body = path.read_text(encoding="utf-8") + + assert "问了" in body + assert "acceptEdits" not in body + assert body.count("