diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index adaf5e6b3..c654513ef 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -59,6 +59,12 @@ jobs: # runs the same typecheck / test / build / consumer-type-surface steps on # a real Windows runtime. The plugin-mount lane (real DSH + Playwright # Chromium) stays ubuntu-only and is deliberately NOT mirrored here. + # + # It also carries the search-engine gate: real fd/ripgrep binaries land on + # PATH (choco) before the suite runs, and the byte-level output-shape + # assertions run against them afterwards. Windows is where the engine's + # output format bites hardest (backslash separators, CR, GBK code pages) — + # see docs/ci-windows.md for the red-line list. ci-windows: runs-on: windows-latest timeout-minutes: 30 @@ -86,6 +92,12 @@ jobs: run: | git config --global user.name "dsh-better-sidebar-ci" git config --global user.email "ci@dsh.invalid" + # Real engine binaries for the byte-level assertions below. The probe + # inside src/search-engines.ts reads PATH, so the whole suite now runs + # with engines present; the specs inject their own probes and are + # unaffected either way (tests/search-engines.spec.ts). + - name: Install fd / ripgrep (choco) + run: choco install fd ripgrep -y --no-progress - name: Install dependencies run: pnpm install --frozen-lockfile - name: Typecheck @@ -118,6 +130,13 @@ jobs: run: pnpm build - name: Check consumer type surface run: pnpm check:consumer-types + # Byte-level assertions on the REAL engine binaries installed above: + # pinned (`--path-separator /`) output must carry no '\' and no CR, and + # a Unicode filename must survive the UTF-8 pipeline. An engine that is + # absent skips (so the script stays runnable on a dev machine); an + # assertion failure exits non-zero. + - name: Engine output-shape assertions (real fd/rg) + run: node scripts/win-engine-check.cjs --assert # Pack the plugin as an npm tarball, mount it into a REAL DSH instance # through the official `dsh plugin --profile web add` channel, and render diff --git a/AGENTS.md b/AGENTS.md index 41474b532..30690f0c2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,7 +19,7 @@ 「npm 打包 → 真实挂载 → 无头渲染」门禁(证明打包产物在真实 DSH 挂载后不 crash):`pnpm build && pnpm pack` 产 tarball → `scripts/e2e-mount.sh` 装进全新 scratch profile(`dsh plugin --profile web add `)并启动真实 `dsh web`(keyless,`--port 0`)→ `tests/e2e/mount.e2e.ts`(Playwright)断言 `[data-dsh-better-sidebar]` 挂载、无错误条/pageerror/console 错误,展开 DSH 原生右侧栏后经其 guide 页逐个打开插件 tab 类型(含终端懒加载 chunk),再经插件文件树(原生 `files` kind 接管)打开 seed 文件强制加载 editor chunk(`client-editor.js`),并跑 mermaid / README 预览与 sidechat 宿主路由烟测。 -本地:`pnpm build && pnpm pack && pnpm exec playwright install chromium && pnpm test:mount`。CI 钉 `@deepseek-ai/dsh@0.1.5-rc.2`(npm `next`;`latest` 仍是 rc.1;peer 下限 `^0.1.5-rc.1`)。该步骤用 `NODE_OPTIONS=--max-old-space-size=4096`:钉版自身的传递依赖是浮动 `^` 范围,上游分阶段发布预发布版时(rc.2 于 2026-09-10 的 14:43–14:57 逐包上线)npm 会组出混合 peer 图,3062 条 ERESOLVE 后 OOM(exit 134);另一个失败模式是对未发布兄弟包 ETARGET,靠「钉一个已完整发布的版本」修掉。**不要加 `--legacy-peer-deps`**:它跳过的正是全局安装必须提供的 peer,`@deepseek-ai/dsh-app-boot` 在 boot 时 require 的 `@deepseek-ai/cordis-plugin-group` 是 peer 而非 dependency,加了它 CLI 直接 `ERR_MODULE_NOT_FOUND`(实测过一次,见 rc.2 计划 C 节)。`ci-windows` 的 `Test` 跑 `pnpm test:windows`(`--maxWorkers=2`)而非 `pnpm test`——多个 spec 真起进程(`agent-pty` / `pty-deps` / `pty-helpers` / `install-powershell` / `smoke`),2 核 runner 上并行起 ConPTY / 冷启 `powershell.exe` 是超时与 worker 静默死亡的放大器;`vitest.config.ts` 的全局 `testTimeout: 15_000`(默认 5000 在 Windows 上对真起进程的用例太低,三个不同文件先后翻车)是配套的一半。e2e spec 命名 `*.e2e.ts` + vitest `exclude` 双保险;**改 `exclude` 必须保留默认排除项**(exclude 整体替换默认值)。 +本地:`pnpm build && pnpm pack && pnpm exec playwright install chromium && pnpm test:mount`。CI 钉 `@deepseek-ai/dsh@0.1.5-rc.2`(npm `next`;`latest` 仍是 rc.1;peer 下限 `^0.1.5-rc.1`)。该步骤用 `NODE_OPTIONS=--max-old-space-size=4096`:钉版自身的传递依赖是浮动 `^` 范围,上游分阶段发布预发布版时(rc.2 于 2026-09-10 的 14:43–14:57 逐包上线)npm 会组出混合 peer 图,3062 条 ERESOLVE 后 OOM(exit 134);另一个失败模式是对未发布兄弟包 ETARGET,靠「钉一个已完整发布的版本」修掉。**不要加 `--legacy-peer-deps`**:它跳过的正是全局安装必须提供的 peer,`@deepseek-ai/dsh-app-boot` 在 boot 时 require 的 `@deepseek-ai/cordis-plugin-group` 是 peer 而非 dependency,加了它 CLI 直接 `ERR_MODULE_NOT_FOUND`(实测过一次,见 rc.2 计划 C 节)。`ci-windows` 的 `Test` 跑 `pnpm test:windows`(`--maxWorkers=1`,单 fork,不再回收 worker)而非 `pnpm test`——多个 spec 真起进程(`agent-pty` / `pty-deps` / `pty-helpers` / `install-powershell` / `smoke`),2 核 runner 上并行起 ConPTY / 冷启 `powershell.exe` 是超时与 worker 静默死亡的放大器;`vitest.config.ts` 的全局 `testTimeout: 15_000`(默认 5000 在 Windows 上对真起进程的用例太低,三个不同文件先后翻车)是配套的一半。e2e spec 命名 `*.e2e.ts` + vitest `exclude` 双保险;**改 `exclude` 必须保留默认排除项**(exclude 整体替换默认值)。同一条 lane 还承载**搜索引擎门禁**:`choco install fd ripgrep` 把真二进制放进 PATH(整条套件因此在引擎在场的环境下跑),套件后对真引擎跑 `node scripts/win-engine-check.cjs --assert` 的字节级断言(钉死 `--path-separator /` 的输出不得含 `\`/CR,中文文件名必须命中;引擎缺失则跳过)。红线清单与标准动作见 [docs/ci-windows.md](docs/ci-windows.md)。 --- diff --git a/README.md b/README.md index 4db739eab..aa19cbe07 100644 --- a/README.md +++ b/README.md @@ -551,7 +551,7 @@ GitHub topic [`dsh-better-sidebar`](https://github.com/topics/dsh-better-sidebar **✨ 新功能** -- 📁 **文件窗口与资源管理器二合一**([#151](https://github.com/omdsh-dev/DSH-better-sidebar/pull/151)):新 `editorExplorer` 设置(编辑器卡齿轮)——文件 tab 增加路径输入框头部 + 可开关的右侧停靠文件树(每 tab 记忆展开/宽度,左缘拖拽调宽 160~480px,全局文件名搜索走 host `fs.search` 路由,预算封顶并跳过 `.git` / 符号链接目录);独立模式(默认)树点击 / 输入框 Enter **按路径新开**文件 tab,合并模式**原地切换**当前 tab;新会话默认 seed 空文件窗口(`Files`)替代 explorer tab,无路径窗口在独立模式为纯资源管理器、合并模式为带 chrome 的空文件窗口;树右键提供「在新 Tab 中打开」「在侧边打开」(split) +- 📁 **文件窗口与资源管理器二合一**([#151](https://github.com/omdsh-dev/DSH-better-sidebar/pull/151)):新 `editorExplorer` 设置(编辑器卡齿轮)——文件 tab 增加路径输入框头部 + 可开关的右侧停靠文件树(每 tab 记忆展开/宽度,左缘拖拽调宽 160~480px,全局文件名搜索走 host `fs.search` 路由(优先探测本机 fd / rg 原生引擎,DSH 自带 ripgrep 优先,缺失/失败时自动回退 JS 遍历,见 [#203](https://github.com/omdsh-dev/DSH-better-sidebar/issues/203);预算封顶并跳过 `.git` / `node_modules` 等噪声目录与符号链接目录));独立模式(默认)树点击 / 输入框 Enter **按路径新开**文件 tab,合并模式**原地切换**当前 tab;新会话默认 seed 空文件窗口(`Files`)替代 explorer tab,无路径窗口在独立模式为纯资源管理器、合并模式为带 chrome 的空文件窗口;树右键提供「在新 Tab 中打开」「在侧边打开」(split) - 🎛️ **声明式设置 select 行**([#151](https://github.com/omdsh-dev/DSH-better-sidebar/pull/151)):设置项新增 `type: 'select'`(`options` 支持 value/title/desc/icon,`multi` 多选存数组);带图标的选项渲染大图标选项卡、收起态同样显示图标;`editorExplorer` 改为图标化下拉(合并 / 独立);能力清单新增 `settingSelect` - 🔀 **与 dsh-web-ui 家族右侧面板互斥**([#181](https://github.com/omdsh-dev/DSH-better-sidebar/pull/181)):读取 `aionui-panel` 设置命名空间的提供方选择——当选择「使用 aionui-panel」时,整个 better-sidebar(右侧栏 / 底部面板 / 浮动入口 / 各类接管)不再挂载;选择 DSH-better-sidebar(或未安装 aionui)时正常。设置页保存后实时生效(settings-document 推送),无需刷新 diff --git a/docs/ci-windows.md b/docs/ci-windows.md new file mode 100644 index 000000000..ed5da1613 --- /dev/null +++ b/docs/ci-windows.md @@ -0,0 +1,70 @@ +# Windows CI 维护指南(ci-windows lane + 搜索引擎门禁) + +> 面向后续开发者:什么时候、为什么、怎么给 Windows 上的新功能加 CI 测试。 +> 本文的 lane 结构落在 `.github/workflows/ci.yml` 的 `ci-windows` job;搜索引擎门禁由其中的两个步骤承载。 + +## 1. 这条 lane 是什么 + +`ci-windows`(**windows-latest** runner,Git Bash shell)与 ubuntu 的 `ci` job 同等强度: + +1. `choco install fd ripgrep` —— 装**真引擎二进制**(`src/search-engines.ts` 的 probe 从 PATH 探测,与真实用户环境一致) +2. `pnpm install --frozen-lockfile` +3. `pnpm typecheck` +4. `pnpm lint` +5. `pnpm test:windows` —— **全量 vitest 套件**在真实 win32 Node 下跑(单 fork,理由见 ci.yml 内注释) +6. `pnpm build`(build script 已跨平台,**禁回退到 `rm -rf`**) +7. `pnpm check:consumer-types` +8. `node scripts/win-engine-check.cjs --assert` —— 用真 fd/rg 跑**字节级断言**:钉死的 `--path-separator /` 输出不得含 `\`/CR,中文文件名必须命中;任一失败 → 红灯 + +**Windows 全量是绿的**(2026-08-22 真机验证 773 passed / 3 skipped;2026-09-12 的 rc.2 基线为 127 文件 / 1356 用例)。全绿的达成靠两个平台修正,见 §3。这条 lane 不存在「Windows 例外测试不用跑」的说法——**新功能必须让全量保持绿**。 + +为什么曾有「只跑部分」的想法:最初 `pnpm test` 在 Windows 上有 4 个失败(3 个 smoke 的 CRLF 断言 + 1 个 pty-deps 的平台断言),都是测试自身不跨平台,不是产品 bug;修掉后全量直接绿,于是 lane 演进为全量门禁。**平台的坑应当修在测试里,而不是把测试从 Windows 门禁里排除**。 + +为什么引擎门禁并进 `ci-windows` 而不是单开一条 `search-windows`:单开要把「装依赖 + 跑全量」再付一遍(CI 分钟翻倍)却不增加覆盖;并进来则让**整个套件**都在真引擎在场的环境下跑,字节级断言也只是同一 runner 上多两个步骤。引擎相关 spec 是插桩式的(`tests/search-engines.spec.ts` 经 `setEngineHooks` 注入 probe),所以引擎在不在 PATH 上都不改变单测结论——ubuntu 的 `ci` job 因此继续覆盖「引擎缺失 → JS 遍历回退」这条路径。 + +## 2. 什么功能需要加 Windows 测试(红线清单) + +后续新功能**只要涉及以下任一项,就必须保证 `ci-windows` lane 全绿**(新增测试或修正平台断言): + +| 风险类别 | 具体坑(本仓库实测/已知) | 加测试的位置 | +|---|---|---| +| **子进程 spawn 外部命令** | PATH 解析、`.exe` 后缀、Windows 上 cmd/PowerShell 与 Git Bash 输出不同(rg#501:同一个 rg,`\` vs `/`) | spec 注入 runner + `--assert` 真二进制检查 | +| **路径字符串处理** | `\` vs `/` 分隔符、`.\` 前缀、CRLF 行尾、盘符/UNC、`MAX_PATH` 长度 | 把 Windows 形状的用例写进 spec(可给纯函数注入 `'\\'` 模拟,见 `normalizeEnginePaths` 测试) | +| **编码/Unicode 文件名** | UTF-8 管道 vs 系统代码页(GBK);中文/日文文件名 | `win-engine-check.cjs --assert` 的 Chinese 用例 | +| **依赖探测/环境布局** | 全局安装目录布局随 OS 不同(npm POSIX `lib/node_modules` vs Windows `%APPDATA%\npm\node_modules`) | `bundledRgCandidates` 的 win32 形状断言(已有);真实布局靠真机/CI 验证 | +| **文件系统语义** | symlink 权限(测试已有 `skipIf(!canSymlink)` 先例)、大小写不敏感、保留名(CON/NUL)、**git autocrlf 行尾**(smoke.spec.ts 先例:平台相关断言修进测试) | `skipIf` 平台门控或平台形状断言 | +| **原生模块/node-gyp** | node-pty 等需要 Windows 构建链(conpty);CI 的 `pnpm install` 已覆盖(含 approve-builds 配置) | 若新增原生依赖,确认 CI install 绿即可;修复命令的断言要按平台(pty-deps.spec.ts 先例) | + +## 3. 历史平台修正(保持警觉的样板) + +这些修正让 Windows 全量变绿,也是「平台坑修在测试里」的范例——**不要回退它们**(均已进 main): + +- `tests/smoke.spec.ts`:scratch repo 钉 `git config core.autocrlf false`(Windows 默认 autocrlf=true 会把 checkout 文件转 CRLF,精确 LF 断言会挂;测试测的是 git 驱动逻辑,不是行尾转换) +- `tests/pty-deps.spec.ts`:`depsStatus()` 按**真实平台**生成修复命令(win32 → `powershell -Repair`,POSIX → `bash --repair`),断言必须按平台写,fixture 建双安装脚本 +- `package.json` 的 `build`:`rm -rf lib` → `node -e "require('node:fs').rmSync('lib',{recursive:true,force:true})"`(Windows 无 `rm`) + +## 4. 怎么加(标准动作) + +```yaml +# ci.yml → ci-windows job → 在全量 Test 之后新增断言步骤: +- name: <你的断言说明> + run: pnpm vitest run tests/<你的spec>.spec.ts # 或 node scripts/<检查脚本> --assert +``` + +同时: + +1. **新 spec 必须放进全量**(vitest 默认收集 `tests/*.spec.ts`,新文件自动进 `pnpm test:windows`,无需改 ci.yml); +2. **必须考虑 Linux 行为**:同一 spec 在 ubuntu lane 也会跑,平台差异用显式注入/`skipIf` 表达(参考 `tests/fs-search.spec.ts` 的 `canSymlink` 模式与 `normalizeEnginePaths` 的 separator 注入); +3. **检查脚本**(如 `scripts/win-engine-check.cjs`)**必须可幂等**:自建 scratch 目录(勿硬编码路径)、引擎缺失时跳过而非失败(CI 有 fd/rg,本地无则跳过)、`--assert` 退出码非零才是失败; +4. 新脚本放 `scripts/`,命名 `win-*.cjs`,README/设计文档顺带记录。 + +## 5. 排查红灯速查 + +- **全量跑挂**:先本地(任意 OS)`pnpm vitest run`,看是否平台无关逻辑问题;再在 Windows 真机 `pnpm vitest run` 复现 +- **只有 Windows 挂**:八成是上面红线清单里的一项;用 `scripts/win-engine-check.cjs`(无 `--assert`)看引擎裸输出 +- **choco 装引擎失败**:runner 镜像偶发;重跑 job 即可(保底可改 winget) +- **真机复验**:见 `docs/plans/2026-08-22-windows-verification.md`(完整链路手册) + +## 6. 设计背景(为什么值得) + +issue #203:大目录下 JS 遍历又慢又截断;搜索模块因此引入 fd/rg 原生引擎。**Windows 恰好是格式坑最多的平台**(分隔符/CRLF/布局),此前完全无 CI 覆盖。这批真机验证(2026-08-22)先以独立 `search-windows` lane 落地(PR #335),main 拿到自己的 `ci-windows` lane(PR #520)后,引擎门禁改为并入该 lane(PR #303),避免两套 Windows 重复安装与重复跑套件。更多实测数据见设计文档 §5「Windows 真机验收」。 diff --git a/docs/external-plugin-guide.md b/docs/external-plugin-guide.md index 6537280b4..cae5d654d 100644 --- a/docs/external-plugin-guide.md +++ b/docs/external-plugin-guide.md @@ -1029,7 +1029,7 @@ better-sidebar 的内置 tab 和 viewer 就是参考实现("吃狗粮"), - **`src/client/SideCardSection.tsx`**:声明式设置页(注册表驱动清单 + 嵌套设置行 + 开关持久化) - **`src/client/api.ts`**:`/sidebar` API 的封装(复制其 fetch 模式到你的插件) - **`src/client/plugins-tabs.ts`** / **`plugins-viewers.ts`**:推荐插件目录(「添加插件」弹窗数据源;加一条数据即上架,`tests/plugin-list.spec.ts` 守护) -- **`src/client/FileTree.tsx`** / **`TreePanel.tsx`** / **`src/fs-search.ts`**:文件树 / 树面板 / host 文件名搜索(`fs.search`;`tests/fs-search.spec.ts`) +- **`src/client/FileTree.tsx`** / **`TreePanel.tsx`** / **`src/fs-search.ts`** / **`src/search-engines.ts`**:文件树 / 树面板 / host 文件名搜索(`fs.search`:优先探测本机 fd / rg 原生引擎(DSH 自带 ripgrep 优先),缺失/失败回退 JS walk;调试插桩 `DSH_SEARCH_DEBUG=1` 写 `$DSH_HOME/search-debug.log`(缺省 `~/.dsh`);测试 `tests/fs-search.spec.ts`、`tests/search-engines.spec.ts`) - **`src/client/markdown-html.ts`** / **`MarkdownHtml.tsx`** / **`md-toc.tsx`**:markdown 内嵌 HTML 管线与目录大纲(注意 `md-toc.tsx` 头注释的「子组件读父 ref 为 null」时序陷阱) - **`src/agent-opens.ts`** / **`/sidebar/ws/agent-opens`**:模型主动打开(`sidebar_open` 工具 + `agentOpenTools` 设置,默认关闭);文件夹窗口 = `meta.dir: true` 的 editor tab([设计文档](plans/2026-08-23-agent-open-tools-design.md)) - **`tests/service.spec.ts`** / **`tests/builtins.spec.ts`**:注册表生命周期 / 匹配算法 / dedupe / createTab / 启用态 gating;内置清单断言(7 tab + 6 viewer + 声明式元数据) diff --git a/docs/plans/2026-08-19-fs-search-engines-design.md b/docs/plans/2026-08-19-fs-search-engines-design.md new file mode 100644 index 000000000..a4c25e6fe --- /dev/null +++ b/docs/plans/2026-08-19-fs-search-engines-design.md @@ -0,0 +1,110 @@ +# fs.search 原生搜索引擎探测设计(fd / rg) + +**日期**:2026-08-19 +**状态**:已实施(feat/fs-search-engines,待 PR) +**作者**:opencode + 用户 +**关联**:issue #203(fs.search 文件名搜索优先嗅探并使用本机 fd/rg) + +## 1. 目标 + +编辑器侧栏全局文件名搜索(`fs.search` 路由 → `searchFiles`)在纯 JS 递归遍历下,大目录(十万级)单次搜索可达秒级。本设计在**零新依赖、路由和客户端零改动**的前提下: + +1. 探测本机已验证可用的原生引擎(fd → rg),按需调用,带宽返回 +2. 引擎不可用/运行失败时无缝回退纯 JS 遍历,行为与现状一致 +3. 引擎输出统一换算为朴素遍历的既有契约(根相对、`/` 分隔、大小写不敏感名字子串、排序、上限截断) + +## 2. 非目标(Out of Scope) + +- **懒索引 / mtime 校验缓存**:无引擎?环境的兜底加速,二期候选(见 §6「二期」) +- **应用层脏标记**:依赖 DSH jobs.output 事件流,延期 +- **自带引擎**(`@vscode/ripgrep` 作为依赖):包体积问题需维护者决策,二期候选 +- 修改 `fs.search` 路由签名、客户端 TreePanel 交互协议:不动 + +## 3. 设计 + +### 3.1 探测(进程内一次,懒,带缓存) + +- 探测顺序:`fd` → `rg` +- 候选位置 = PATH 展开 + 固定路径,并**优先 DSH 自带的 rg**: + - rg 第一候选:**DSH CLI 内置的 ripgrep**。因为 DSH 的安装布局因平台/包管理器而异,`bundledRgCandidates` 枚举多个候选根(每条经 `--version` 预检剔除):POSIX npm 全局(`/lib/node_modules`,由 `process.execPath` 推导——node 在 `/bin/`)、**Windows npm 全局(`%APPDATA%\npm\node_modules`,没有 `lib/` 层)**、Homebrew/pnpm 全局根、`~/.dsh/profiles/node_modules`(launchd/profile 布局,2026-08-22 本机实测 DSH 真实落在此处,旧的 execPath 单一定向反而 miss);之后 PATH `rg` + `/opt/homebrew/bin/rg`、`/usr/local/bin/rg`、`/usr/bin/rg` + - fd: PATH `fd` **与 `fdfind`**(Ubuntu 包装名)+ `/opt/homebrew/bin/fd`、`/usr/local/bin/fd`、`/usr/bin/fd`、`~/.cargo/bin/fd` + - ~~VS Code 捆绑 rg hack~~ **已弃用**:DSH 自身就捆绑 @vscode/ripgrep,不需要反向借用 VS Code 应用目录(issue #203 的核心洞察) +- 每个候选二进制跑 `--version`(500ms 超时)验证可执行,验证不过的路径不采用 +- 首次搜索时懒探测,结果 Promise 级缓存整个进程生命周期;`search-engines.ts` 导出 `setEngineHooks` / `resetEngines` 作为测试注入点 + +### 3.2 调用与语义对齐 + +| 引擎 | 命令 | 语义对齐 | +|---|---|---| +| fd | `fd --hidden --no-ignore --exclude .git --fixed-strings --ignore-case --path-separator / --max-results N+1 .`(cwd=root) | `--fixed-strings` 字面量子串匹配(对齐朴素语义,防 glob 注入);`-H -I` 不忽略隐藏/ignore 文件;"文件名+目录名" 都匹配;`--max-results` 天然限流;`.git` 目录显式排除(朴素遍历同样跳过) | +| rg | `rg --files --hidden --no-ignore --glob '!**/.git/**' --iglob '**' --iglob '**/**/**' --path-separator / .`(cwd=root) | `--iglob` 大小写不敏感(rg globset 不支持 `(?i)` 前缀);`--path-separator /` 把 Windows 上的 `\` 输出钉成 `/`(rg 在 cmd/PowerShell 下输出 `\`、Git Bash 下输出 `/`,见 rg#501,源头钉死);query 双 glob:无斜杠形态按 gitignore 语义**只匹配 basename**(真机 rg 15 验证),路径级形态 `**/*q*/**` 收纳「匹配目录下的文件」供 `deriveRgMatches` 倒推目录命中——与 fd/plain 的目录名匹配对齐(残留 lossy:rg 看不见空目录);`.git` 全程排除;无结果上限,流式读取到 N+1 杀进程;**exit 1 = 无匹配,属正常空结果**(rg 契约,streamLines 放行,不触发引擎禁用) | + +统一出口:子进程 stdout 流式逐行(全量结果用 readline + 超过 N+1 即 kill,不会 buffer 进内存),经 `normalizeEnginePaths` 换算(去 `./` 前缀、`/` 分隔),由 `searchFiles` 排序 + 截断。 + +### 3.3 降级链与失败策略 + +- 探测超时/非零 → 剔除该候选;无候选的引擎不进链 +- 运行期失败(非零退出 / 无法 spawn)→ **禁用该引擎(进程内)**,继续下一个引擎 +- **超时(15s)不禁用**:超时的语义是「这棵树太大」而非「二进制坏了」——broken 集按引擎记、不按搜索根记,若大目录搜一次就禁用,其余所有小目录的搜索也永远失去引擎加速。超时只降级本次(换下一个引擎 / plain walk),引擎保持可用 +- 全部不可用 → 纯 JS 遍历(`searchFilesPlain`,原逻辑原样) +- 调用方 signal abort → 杀子进程,跳过回退遍历直接返回空(请求已死,客户端不再消费) + +## 4. 文件变更 + +| 文件 | 类型 | 内容 | +|---|---|---| +| `src/search-engines.ts` | 新增 | 探测缓存 + 三引擎调用 + `normalizeEnginePaths`/`escapeGlob` + hooks | +| `src/search-debug.ts` | 新增 | 门控调试插桩:`DSH_SEARCH_DEBUG=1` 才写 `$DSH_HOME/search-debug.log`(+ console 镜像),默认零开销;写盘失败静默 | +| `src/fs-search.ts` | 改动 | 原逻辑改名 `searchFilesPlain`;新增 `searchFiles` dispatch(引擎优先,失败回退) | +| `tests/search-engines.spec.ts` | 新增 | 规范化/转义/探测缓存/失败禁用/abort 不禁用 | +| `tests/fs-search.spec.ts` | 改动 | 既有用例改测 `searchFilesPlain`;新增 dispatch 组(路由/截断/回退/无引擎/abort) | +| `docs/plans/2026-08-19-fs-search-engines-design.md` | 新增 | 本文档 | + +路由(`src/index.ts` 的 `fs.search`)与客户端:**零改动**。 + +## 5. 验证 + +- `pnpm typecheck` 通过;`pnpm test` 全量 637 passed / 5 skipped +- 真实机器集成(本机 macOS):PATH 不含 Homebrew 的容器里,`/opt/homebrew/bin/rg` 被固定路径探测命中,`searchFiles` 命中 `src/search-engines.ts`;DSH 内置 rg 由 `process.execPath` 前缀推导命中(enginetest profile 实测) +- **真实目录基准(本机 macOS,DSH 内置 rg 15.0,3 次取最优)**: + + | 目录(规模) | 查询 | plain(JS walk) | rg(--iglob) | + |---|---|---|---| + | `/opt/homebrew`(17 万文件) | `libruby` | 518ms,4 命中,`truncated=true` | 205ms,4 命中,完整 | + | `/opt/homebrew`(17 万文件) | 无匹配 | 513ms,0 命中,`truncated=true` | 198ms,0 命中,完整 | + | `/Users/y/workspace`(9 万文件) | `search-engines` | 870ms,4 命中,`truncated=true` | 184ms,4 命中,完整 | + | `/Users/y/workspace`(9 万文件) | 无匹配 | 872ms,0 命中,`truncated=true` | 191ms,0 命中,完整 | + | `/Applications`(24 万文件) | `README` | 453ms,85 命中,`truncated=true` | 23ms,200 命中,完整 | + | `/Applications`(24 万文件) | 无匹配 | 452ms,0 命中,`truncated=true` | 184ms,0 命中,完整 | + | `/Users/y/tools`(3.7 万文件) | `glob` | 217ms,102 命中,完整 | 56ms,99 命中,完整 | + + 结论:**真实嵌套目录下 rg 快 2.5~20 倍,且 plain 大目录一律预算截断(结果不完整,正是 issue #203 指出的问题)**;`tools "glob"` 的 99 vs 102 差异是 rg 只报文件、plain 把目录也算命中——后续以 query 双 glob + `deriveRgMatches` 倒推对齐(残留 lossy:空目录不可见);README 查询 rg 命中 200(结果上限)+ 完整遍历,而 plain 85 命中即因预算截断而漏掉其余。顺带修出两个 mock 测不出的 bug:`(?i)` 前缀无效(改 `--iglob`)与 rg exit 1 = 无匹配(放行,不触发引擎禁用)。极端场景(如 `~/Library/Containers`,数百万 iCloud 小文件)下 plain 与 rg 均需 >5min,两类实现都不可用,不作为基准 +- 本机无 fd,`fd` 候选探测自然剔除,链正常降级 +- **门控插桩验收(2026-08-21,本机 macOS,真机 dsh web + UI 实测)**: + - 默认静默:未设 `DSH_SEARCH_DEBUG` 时零 console、零磁盘写(测试与真机双重确认) + - 开启后:进程首搜 1 行 `engines probed: rg(<捆绑路径>)`(fd 缺失静默剔除,不报错);每搜索 1 行 `engine=rg bin=<捆绑 rg> root=~… query=… hits=… truncated=… ms`(root 以 `~` 缩写) + - `~/workspace`(9 万文件量级)实测 15~500ms:常见词毫秒级;罕见词(如 `.dsh` 仅 1 命中)全量遍历 ~478ms 且**结果完整不截断**——正是 issue #203 要的行为(`query="."` 则 hits=202 `truncated=true`,200 上限截断生效) + - `query="*"` → 0 命中(glob 元字符转义生效,字面量语义);真机复跑同款 rg 命令确认输出无任何 `.git/` 目录内部路径 + - **Windows 真机验收(2026-08-22,Windows 11 + npm 全局 DSH 0.1.1-rc.2 + scoop fd/rg)**: + - 输出格式字节级:`rg` 默认输出 `\` 分隔 + `.\` 前缀(commenter 担忧的坑实锤);`--path-separator /` 后全 `/`;fd 同效。CR 全 false / LF true(Rust 二进制管道输出 LF);中文文件名 UTF-8 无乱码 + - 探测命中:DSH 装于 `%APPDATA%\npm\node_modules`(npm 全局,无 `lib/` 层)→ `bundledRgCandidates` 的 `%APPDATA%` 候选 `existsSync` 命中,`probeEngines` 实测 `rg(捆绑 rg.exe)` 与 `fd(scoop shim)` 双双就位;PATH 滤掉 Scoop 后捆绑 rg 依旧命中(固定绝对路径兜底,与"无 PATH 环境"设计一致) + - 完整链路:`pnpm pack` tarball → `dsh plugin --profile web add`(首次 install 需先把 profile 模板的 `allowBuilds: node-pty` 占位写成 true)→ `dsh web --port ` → `POST /sidebar/api/fs.search` 返回 `ok:true`、200 匹配、`/` 分隔;`DSH_SEARCH_DEBUG` 日志见引擎行(web UI 搜索路径同款) + - **bench(D:\Project,>10 万元素,3 次最优)**:plain 一律 `truncated=true`(visited 100k 预算耗尽,**不含匹配预算**),`query="1234"` plain hits=5 vs 引擎 hits=40——"慢且结果不全"正是 issue #203;引擎快 3.7~34.7x(md 415→25ms、hollow 978→264ms、1234 1970→262ms、test 1008→29ms)。`scripts/win-bench.cjs` 可复跑 + - 顺手修复:fd `--max-results` 原钉在 `cap`(=maxMatches+1 哨兵)处,满集永不触发截断、多余返回 1 条;改为 `cap+1` 后 fd/rg 截断语义对齐,`fdArgv`/`rgArgv` 抽为导出纯函数由单测钉死 + +## 6. 二期候选(有意不做,记录在案) + +1. **懒索引 + 目录 mtime 校验**(`searchFilesPlain` 回退路径加速):文件名索引只需感知创建/删除/改名——必然改变父目录 mtime,故 mtime 校验即正确失效信号;TTL(如 5s)免校验补偿秒级粒度。有 fd/rg 时索引无意义甚至更慢(校验遍历 > 原生一次遍历),故**仅在无引擎路径生效** +2. **自带 `@vscode/ripgrep` 依赖**:消灭"用户没装/DSH 布局探不到"的概率问题,代价是 2-4MB 三平台二进制 + optionalDependencies 平台分片,需维护者拍板 +3. **应用层脏标记**:通过 DSH `jobs.output` 事件流解析 agent 写文件路径,即时失效 mtime 兜底的陈旧窗口(延迟/复杂度高) +4. rg 的目录名匹配缺失:`rg --files` 无目录输出,若反馈集中可考虑砍掉 rg 或换 `--no-ignore` 全量 + 客户端过滤 + +## 7. 已知取舍(诚实记录) + +- **truncated 语义**:引擎路径的截断顺序是引擎遍历序(非确定性),朴素路径是遍历序——截断结果本来就不保证全集,差异可接受 +- **DSH 内置 rg 路径由 `process.execPath` 推导**:依赖 DSH CLI 的全局 node_modules 布局(npm 风格 `lib/node_modules/@deepseek-ai/dsh/...`);用 `pnpm`/`bun` 全局安装等异构布局探不到,但 existSync + `--version` 预检会将其剔除,不影响正确性(还有 PATH/固定路径 rg 与 JS 兜底) +- **探测顺序决定引擎胜负**:rg 的 DSH 内置候选排在 PATH 之前——即使系统装了别的 rg,也优先用 DSH 自带的 15.x(行为一致、免环境依赖) +- **node_modules 不排除**:引擎与兜底 walk 同样沿用 no-ignore 语义(结果集一致,不引入两种模式的结果漂移);常见词查询的 200 名额可能被依赖树打满(实测 `query="test"` 202 条大半来自 node_modules)。加 `--exclude node_modules` 属产品决策(§6 候选),留待反馈 +- **rg glob 的花括号必须转义**:`{a,b}` 是 globset 交替组语法——查询含未闭合 `{` 会让 rg 解析 glob 失败(exit 2,引擎被误判损坏而进程内禁用),含成对 `{}` 则被静默当交替展开(`a{b}` 实搜 `ab`,rg 15.0 实测)。`escapeGlob` 对 `{}` 一并反斜杠转义后两类症状均消除 +- **`.git` 文件(worktree)三实现对齐**:git worktree 工作区根的 `.git` 是指针文件而非目录——plain walk 只匹配名字、fd `--exclude .git` 文件目录都排、rg 的 `!**/.git/**` 因模式要求 `.git` 后还有路径段而漏掉它(rg 15.0 实测会列出该文件)。现统一语义:凡名为 `.git` 的条目一律不算命中(rg 补 `!**/.git`,walk 去掉 isDirectory 条件) +- **超时不进入 broken 集**(见 §3.3):代价是同一个大根目录每次搜索都先付 15s 再落 plain walk,换来其余目录不受牵连 \ No newline at end of file diff --git a/docs/plans/2026-08-22-windows-verification.md b/docs/plans/2026-08-22-windows-verification.md new file mode 100644 index 000000000..42465b83c --- /dev/null +++ b/docs/plans/2026-08-22-windows-verification.md @@ -0,0 +1,109 @@ +# Windows 真机验证手册:fs.search 原生引擎(fd / rg) + +> 适用:PR #303(`feat/fs-search-engines`)。目标:在 Windows 真机验证引擎**探测、输出格式、降级链**三项,回应 review 对 Windows 格式问题的担忧。 +> 预计耗时:方案 A 约 10 分钟,方案 B(完整链路)视环境多 10~20 分钟。 + +## 背景:要验证什么 + +| 验证点 | 风险 | 单测能否覆盖 | +|---|---|---| +| DSH 捆绑 rg 探测路径命中(Windows npm 布局 `%APPDATA%\npm\node_modules`) | 推导公式若错则静默回退 JS 遍历 | 仅形状断言,真机命中需实测 | +| rg 输出分隔符:`--path-separator /` 已钉死,Windows 上不再出现 `\` | 低(flag 自 rg 0.8 起存在) | 单测注入模拟,真机确认 | +| fd 输出:`--path-separator /` + CRLF | 低 | 同上 | +| 引擎输出 `.\` 前缀 / CRLF 残留 | 已由 `normalizeEnginePaths` 剥离 | 单测已钉 | +| 运行失败 → 引擎禁用 → 回退 JS 遍历 | 中(Windows 上 spawn 行为差异) | 单测已钉,真机补一刀 | +| 中文/Unicode 文件名搜索 | 编码问题 Windows 特有 | **不能**,只能真机 | + +## 方案 A:模块级验证(不依赖 DSH,推荐先做) + +在 Windows 主力机上,任意目录: + +```powershell +# 1. 取代码(或直接在你已有的 clone 上 fetch) +git fetch origin feat/fs-search-engines +git checkout feat/fs-search-engines + +# 2. 装依赖(已有 pnpm 跳过)与引擎 +pnpm install +winget install sharkdp.fd # fd:winget 包名 sharkdp.fd +winget install BurntSushi.ripgrep.MSVC # rg:winget 包名(或 scoop install fd rg) + +# 3. 跑 search 两组单测(Windows 上会走真实 win32 分支) +pnpm vitest run tests/search-engines.spec.ts tests/fs-search.spec.ts +``` + +期望:28 个用例全绿。若 `bundledRgCandidates` 的 win32 用例失败,把输出贴给我。 + +### 4. 真引擎输出形状检查(关键,单测模拟不了的) + +```powershell +# 在旧版 cmd 中直接看引擎裸输出(fd 已验证 --path-separator;rg 是本次新加的) +cd $env:TEMP; mkdir wintest; cd wintest +New-Item -ItemType Directory src; New-Item src\a.ts; New-Item README.md +fd --hidden --no-ignore --exclude .git --fixed-strings --ignore-case --path-separator / --max-results 201 a . +rg --files --hidden --no-ignore --glob "!**/.git/**" --iglob "*a*" --path-separator / . +``` + +期望:全部输出 `/` 分隔、无 `\r`(若 PowerShell 里看有 `\n` 即可,重点是没有 `\` 和路径尾部 `\r`)。任意一行出现 `\` 或 `\r` 就是回归,贴给我。(**注意用旧版 cmd/PowerShell 跑**,Git Bash 下 rg 本来就输出 `/`,测不出问题。) + +### 5. Unicode 文件名(编码验证) + +```powershell +New-Item "中文文件名.ts"; New-Item "名前テスト.txt" +fd --hidden --no-ignore --fixed-strings --ignore-case --path-separator / 中文 . +rg --files --hidden --no-ignore --iglob "*中文*" --path-separator / . +``` + +期望:两行都命中 `中文文件名.ts`,且输出不是乱码。这是"编码坑"仅存的最大真机风险点。 + +## 方案 B:完整链路(DSH + 侧边栏 UI) + +前提:Windows 上有一份可跑的 DSH(用 `dsh --version` 确认;没有则 `npm i -g @deepseek-ai/dsh` 或参照 DSH 官方安装方式,装完再回来)。 + +```powershell +# 1. 确认 DSH 全局安装位置(决定捆绑 rg 探测根) +npm root -g # 期望 %APPDATA%\npm\node_modules 或等价 npm 全局根 + +# 2. 确认捆绑 rg 存在(新候选公式应命中) +Test-Path "$(npm root -g)\@deepseek-ai\dsh\node_modules\@vscode\ripgrep-win32-x64\bin\rg.exe" +# 期望 True。False = 探测布局假设不成立,贴给我。 + +# 3. 装插件到 profile(按仓库 README/AGENTS.md 的挂载流程),然后: +$env:DSH_SEARCH_DEBUG = "1" +dsh web + +# 4. 浏览器打开,在文件窗口搜一个常见词 + 一个中文词,各搜一次 +# 5. 看调试日志 +Get-Content "$env:USERPROFILE\.dsh\search-debug.log" -Tail 20 +``` + +期望: +- 首搜有一行 `engines probed: rg(<捆绑 rg.exe 全路径>), fd(...)`——证明探测命中(Windows 布局修复生效) +- 每搜一行 `engine=rg ... hits=N ... ms`,命中数合理、无报错 +- 中文词能搜到中文文件名(编码链路 OK) + +若日志显示 `engines probed: none` 或 `engine=plain`,探测有问题;若命中但结果乱,是编码问题。两种情况都贴日志给我。 + +## 验收后要做的 + +- 把上面三段期望的结果(或失败日志)发我,由我决定是否还差修复 +- 确认无误后回复 PR 评论,写明"Windows 真机已验证(装 fd + rg,模块级 + UI 链路)",附上本手册链接 +--- + +## 实施偏差记录(2026-09-12):CI 车道从 #335 并入本 PR + +原计划由独立 PR #335 新增 `search-windows` lane 固化这批验证。main 在 2026-09-03 拿到自己的 `ci-windows` lane(#520:windows-latest 上跑 `pnpm test:windows` 全量套件)之后,那条独立 lane 的形态已经过时,本次把它并入 #303: + +**保留并搬进 `ci-windows`**(`.github/workflows/ci.yml` 的两个步骤): +- `choco install fd ripgrep -y --no-progress` —— 真引擎二进制进 PATH(在 `pnpm install` 之前) +- `node scripts/win-engine-check.cjs --assert` —— 套件结束后跑字节级断言 + +**为什么并进而不是单开 lane**:单开要把「装依赖 + 跑全量」再付一遍(CI 分钟翻倍)却不增加覆盖;并进来则整条套件都在引擎在场的环境下跑(覆盖面严格更大),断言只多两个步骤。引擎相关 spec 是插桩式的(`tests/search-engines.spec.ts` 经 `setEngineHooks` 注入 probe,文件头注释即写明「CI 机器没有 fd/rg」),因此引擎在不在 PATH 上都不改变单测结论——ubuntu 的 `ci` lane 继续覆盖「引擎缺失 → JS 遍历回退」。 + +**随 #303 一并升级**:`scripts/win-engine-check.cjs` 从「硬编码 `C:\Users\y\wintest` 的裸输出 demo」升级为 #335 的 `--assert` 版——每次运行自建 scratch 目录、引擎缺失即跳过、钉死 `--path-separator /` 的输出不得含 `\`/CR、中文文件名必须命中、失败非零退出。新增 `docs/ci-windows.md`(红线清单 / 历史平台修正 / 标准动作)并在 AGENTS.md §2 埋入口。 + +**丢弃的部分**(原因见上,均已由 main 独立落地,无需随本 PR 携带): +- 独立的 `search-windows` job 与「全量测试」步骤 —— 由 `ci-windows` 承担 +- `package.json` build 跨平台(`rm -rf lib` → `node -e rmSync`)—— 已在 main +- `tests/smoke.spec.ts` 钉 `core.autocrlf=false`、`tests/pty-deps.spec.ts` 平台断言 —— 已在 main +- #335 对 README/AGENTS.md 的引擎功能描述 —— 本 PR 的 rebase 已把该描述锚定到 README 的 v0.12.3 条目与 `docs/external-plugin-guide.md` diff --git a/scripts/win-bench.cjs b/scripts/win-bench.cjs new file mode 100644 index 000000000..988673bf7 --- /dev/null +++ b/scripts/win-bench.cjs @@ -0,0 +1,42 @@ +// This is an ops script run directly with `node ` on a real host (not +// bundled by tsdown): CommonJS require() is intentional here. +/* eslint-disable @typescript-eslint/no-require-imports */ +// Windows engine benchmark: plain walk vs fd vs rg on the same root/query. +// Usage: node win-bench.cjs [query...] +// PATH is extended with Scoop shims + npm global (ssh sessions lack them). +const { performance } = require('node:perf_hooks') +const os = require('node:os') + +const home = os.homedir() +const extra = [ + `${home}\\Scoop\\shims`, + `${process.env.APPDATA ?? ''}\\npm`, +] +process.env.PATH = [...extra, process.env.PATH ?? ''].filter(Boolean).join(';') + +const root = process.argv[2] ?? 'D:\\Project' +const queries = process.argv.slice(3).length > 0 ? process.argv.slice(3) : ['md', 'hollow', '1234', 'test'] + +;(async () => { + const { searchFilesPlain, searchFiles } = await import('./src/fs-search.ts') + console.log(`root=${root} queries=[${queries.join(', ')}] runs=3 best-of`) + for (const q of queries) { + const row = {} + for (const [label, fn] of [ + ['plain', () => searchFilesPlain(root, q)], + ['fd+rg', () => searchFiles(root, q)], + ]) { + const times = [] + let hits = 0, truncated = false + for (let i = 0; i < 3; i += 1) { + const t0 = performance.now() + const r = await fn() + times.push(performance.now() - t0) + hits = r.matches.length + truncated = r.truncated + } + row[label] = `${Math.min(...times).toFixed(0)}ms hits=${hits}${truncated ? ' TRUNC' : ''}` + } + console.log(`query="${q}" plain: ${row['plain']} | engines: ${row['fd+rg']}`) + } +})().catch((err) => { console.error('BENCH FAILED:', err); process.exit(1) }) \ No newline at end of file diff --git a/scripts/win-e2e/e2e-helper.cjs b/scripts/win-e2e/e2e-helper.cjs new file mode 100644 index 000000000..162543994 --- /dev/null +++ b/scripts/win-e2e/e2e-helper.cjs @@ -0,0 +1,47 @@ +// This is an ops script run directly with `node ` on a real host (not +// bundled by tsdown): CommonJS require() is intentional here. +/* eslint-disable @typescript-eslint/no-require-imports */ +// Windows DSH e2e helper: check dsh web process/log state, then hit +// /sidebar/api/fs.search if a URL is available. +const { execSync, execFileSync } = require('node:child_process') +const fs = require('node:fs') + +const scratch = 'C:\\Users\\y\\dsh-e2e-scratch' + +function ps() { + try { + const out = execSync('powershell -NoProfile -Command "Get-CimInstance Win32_Process -Filter \\"Name=\'node.exe\'\\" | ForEach-Object { $_.ProcessId.ToString() + \\" | \\" + $_.CommandLine }"', { encoding: 'utf8', maxBuffer: 1 << 20 }) + console.log('--- node processes ---') + console.log(out) + } catch (err) { + console.log('ps failed:', String(err.message).slice(0, 300)) + } +} + +const args = process.argv.slice(2) +if (args.includes('--ps')) { + ps() +} else if (args.includes('--logs')) { + for (const name of ['web.log', 'web.err.log']) { + const p = `${scratch}\\${name}` + if (fs.existsSync(p)) { + const s = fs.statSync(p) + console.log(`--- ${name} (${s.size} bytes) ---`) + console.log(fs.readFileSync(p, 'utf8').split('\n').slice(-15).join('\n')) + } else { + console.log(`--- ${name}: MISSING ---`) + } + } +} else if (args.includes('--search')) { + const url = args[args.indexOf('--search') + 1] + const query = args[args.indexOf('--search') + 2] ?? 'src' + const payload = JSON.stringify({ sessionId: 'test', query }) + try { + const out = execFileSync('curl.exe', ['-s', '-X', 'POST', `${url}/sidebar/api/fs.search`, '-H', 'content-type: application/json', '-d', payload], { encoding: 'utf8', maxBuffer: 1 << 20 }) + console.log('fs.search response:', out.slice(0, 2000)) + } catch (err) { + console.log('search failed:', String(err.message).slice(0, 500)) + } +} else { + console.log('usage: --ps | --logs | --search [query]') +} \ No newline at end of file diff --git a/scripts/win-e2e/pnpm-workspace.yaml b/scripts/win-e2e/pnpm-workspace.yaml new file mode 100644 index 000000000..ff93acb91 --- /dev/null +++ b/scripts/win-e2e/pnpm-workspace.yaml @@ -0,0 +1,12 @@ +packages: + - . + +nodeLinker: hoisted +autoInstallPeers: false + +allowBuilds: + node-pty: true + protobufjs: true + +minimumReleaseAgeExclude: + - dsh-better-sidebar \ No newline at end of file diff --git a/scripts/win-e2e/profile-package.json b/scripts/win-e2e/profile-package.json new file mode 100644 index 000000000..99bc5674e --- /dev/null +++ b/scripts/win-e2e/profile-package.json @@ -0,0 +1,10 @@ +{ + "name": "dsh-profile-web", + "private": true, + "dependencies": {}, + "dsh": { + "profile": { + "bundles": ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-web-app"] + } + } +} \ No newline at end of file diff --git a/scripts/win-engine-check.cjs b/scripts/win-engine-check.cjs new file mode 100644 index 000000000..1058b8a40 --- /dev/null +++ b/scripts/win-engine-check.cjs @@ -0,0 +1,73 @@ +// This is an ops script run directly with `node ` on a real host (not +// bundled by tsdown): CommonJS require() is intentional here. +/* eslint-disable @typescript-eslint/no-require-imports */ +/** + * Windows(任意 OS)引擎输出形状检查/断言: + * - 无参:打印 rg/fd 在有无 `--path-separator /` 时的字节级输出(人工检查用) + * - --assert:CI 门禁模式——存在的引擎必须满足: + * * `--path-separator /` 输出不含 `\`(backslash)与 `\r`(CR) + * * 中文文件名可正常命中(UTF-8 编码链路) + * 任一断言失败退出码非零(红灯);引擎缺失则跳过(不算失败)。 + * 用法: node scripts/win-engine-check.cjs [--assert] + */ +const { execFileSync } = require('node:child_process') +const { mkdtempSync, rmSync, writeFileSync, mkdirSync } = require('node:fs') +const { tmpdir } = require('node:os') +const { join } = require('node:path') + +const assertMode = process.argv.includes('--assert') + +// Every run builds its own scratch tree: the CI runner and a real machine +// have no shared fixed path (the original hard-coded C:\Users\y\wintest). +const scratch = mkdtempSync(join(tmpdir(), 'dsh-engine-check-')) +mkdirSync(join(scratch, 'src')) +writeFileSync(join(scratch, 'README.md'), 'readme') +writeFileSync(join(scratch, 'src', 'a.ts'), 'code') +writeFileSync(join(scratch, '中文文件名.ts'), 'code') + +let failed = false + +function run(label, binary, args, opts = {}) { + try { + const out = execFileSync(binary, args, { cwd: scratch, encoding: 'buffer' }) + const hasBackslash = out.includes(92) + const hasCR = out.includes(13) + console.log(`=== ${label} (${binary}) ===`) + console.log('backslash:', hasBackslash, '| CR:', hasCR) + console.log('raw:', JSON.stringify(out.toString('utf8'))) + if (assertMode && opts.pinned !== false) { + // The default-shape runs (opts.pinned === false) are DEMO cases — + // un-pinned engines leak '\' by design; only the pinned + // (--path-separator /) and Chinese-name runs are asserted. + if (hasBackslash || hasCR) { + console.error(`ASSERT FAIL: ${label} leaked backslash/CR into output`) + failed = true + } + // Chinese filename must be hit through the UTF-8 pipeline. + if (opts.chinese && !out.toString('utf8').includes('中文文件名.ts')) { + console.error(`ASSERT FAIL: ${label} missed the Chinese filename`) + failed = true + } + } + } catch (err) { + console.log(`=== ${label} (${binary}) SKIPPED/FAILED ===`) + console.log(String(err.stderr ?? err.message).split('\n')[0]) + } +} + +// The default-shape runs are DEMO cases (un-pinned output leaks '\' by +// design — that is exactly the review concern on #303); only the pinned +// (--path-separator /) and Chinese-name runs assert. `--exclude .git` keeps +// a worktree's `.git` pointer file out of the listing the engine emits. +run('rg default (no path-separator)', 'rg', ['--files', '--hidden', '--no-ignore', '.'], { pinned: false }) +run('rg with --path-separator /', 'rg', ['--files', '--hidden', '--no-ignore', '--path-separator', '/', '.']) +run('fd default (no path-separator)', 'fd', ['--hidden', '--no-ignore', '--exclude', '.git', '--fixed-strings', '--ignore-case', 'a', '.'], { pinned: false }) +run('fd with --path-separator /', 'fd', ['--hidden', '--no-ignore', '--exclude', '.git', '--fixed-strings', '--ignore-case', '--path-separator', '/', 'a', '.']) + +const rgCn = ['--files', '--hidden', '--no-ignore', '--iglob', '*中文*', '--path-separator', '/', '.'] +const fdCn = ['--hidden', '--no-ignore', '--exclude', '.git', '--fixed-strings', '--ignore-case', '--path-separator', '/', '中文', '.'] +run('rg Chinese filename', 'rg', rgCn, { chinese: true }) +run('fd Chinese filename', 'fd', fdCn, { chinese: true }) + +rmSync(scratch, { recursive: true, force: true }) +if (failed) process.exit(1) diff --git a/scripts/win-probe-check.cjs b/scripts/win-probe-check.cjs new file mode 100644 index 000000000..90d650dab --- /dev/null +++ b/scripts/win-probe-check.cjs @@ -0,0 +1,17 @@ +// This is an ops script run directly with `node ` on a real host (not +// bundled by tsdown): CommonJS require() is intentional here. +/* eslint-disable @typescript-eslint/no-require-imports */ +// Windows real-machine probe verification: does bundledRgCandidates hit +// the DSH global install's ripgrep, and does probeEngines actually pick it? +const { existsSync } = require('node:fs') +const { homedir } = require('node:os') +;(async () => { + const m = await import('./src/search-engines.ts') + console.log('platform:', process.platform, 'arch:', process.arch) + console.log('APPDATA:', process.env.APPDATA ?? '(unset)') + const paths = m.bundledRgCandidates(process.platform, process.arch, process.execPath, process.env, homedir()) + for (const p of paths) console.log(existsSync(p) ? 'EXISTS ' : ' - ', p) + console.log('--- probeEngines (real spawn --version check) ---') + const probes = await m.probeEngines() + console.log('probed engines:', JSON.stringify(probes, null, 2)) +})().catch((err) => { console.error('FAILED:', err); process.exit(1) }) \ No newline at end of file diff --git a/src/fs-search.ts b/src/fs-search.ts index c78d98f07..eecb77ab4 100644 --- a/src/fs-search.ts +++ b/src/fs-search.ts @@ -11,9 +11,27 @@ * the flat list) and `maxVisited` (a runaway tree — a home directory root * — must not stall the host). Exceeding either stops early with * `truncated: true`. + * + * `searchFiles` (the dispatch the fs.search route calls) first tries the + * probed native engines (fd / rg — see search-engines.ts); when + * none are available or all failed at runtime it falls back to this walk + * (exported as `searchFilesPlain` for tests). */ import { opendir } from 'node:fs/promises' import { join, relative, sep } from 'node:path' +import { homedir } from 'node:os' +import { runEngine, usableEngines, SKIP_DIR_NAMES } from './search-engines.ts' +import { debugLog } from './search-debug.ts' + +/** Shorten an absolute search root for log lines: ~/ for the config home. */ +function relRoot(root: string): string { + const home = process.env.DSH_HOME !== undefined && process.env.DSH_HOME.trim() !== '' + ? process.env.DSH_HOME + : homedir() + if (root === home) return '~' + const boundary = home.endsWith(sep) ? home : home + sep + return root.startsWith(boundary) ? '~' + root.slice(home.length) : root +} /** One search: the relative paths of the matching entries (dirs included so * the client can hint where matches live) plus the truncation flag. */ @@ -37,28 +55,13 @@ const DEFAULT_MAX_VISITED = 100_000 * Directory names that are never useful filename-search results and would * burn the visit budget before the walk reaches project files. Compared * case-insensitively so `Node_Modules` / `.GIT` stay skipped on every - * platform. The directory itself is neither matched nor descended. + * platform. The name list lives in search-engines.ts (SKIP_DIR_NAMES) — + * the engine argvs exclude the same names so the fallback and the engines + * return identical result shapes. An entry with a skip name is neither + * matched nor descended — including a worktree-style `.git` FILE (parity + * with the engines' .git exclusion — see SKIP_DIR_NAMES in search-engines.ts). */ -const SEARCH_SKIP_DIRS = new Set([ - '.git', - 'node_modules', - '.pnpm-store', - '.yarn', - '.turbo', - '.turbopack', - '.next', - '.nuxt', - '.output', - '.cache', - '.parcel-cache', - 'coverage', - 'dist', - 'build', - 'out', - '.umi', - '.umi-production', - '.dumi', -]) +const SEARCH_SKIP_DIRS = new Set(SKIP_DIR_NAMES) /** * Search `root` recursively for entries whose name contains `query` @@ -70,7 +73,7 @@ const SEARCH_SKIP_DIRS = new Set([ * plus whether a budget cut the walk short. An unreadable level is skipped * (permission errors never fail the whole search). */ -export async function searchFiles(root: string, query: string, opts: FsSearchOptions = {}): Promise { +export async function searchFilesPlain(root: string, query: string, opts: FsSearchOptions = {}): Promise { const needle = query.trim().toLowerCase() if (needle === '') return { matches: [], truncated: false } const maxMatches = opts.maxMatches ?? DEFAULT_MAX_MATCHES @@ -90,8 +93,11 @@ export async function searchFiles(root: string, query: string, opts: FsSearchOpt truncated = true return } - // Dependency / VCS / build-output forests: never matched, never descended. - if (dirent.isDirectory() && SEARCH_SKIP_DIRS.has(dirent.name.toLowerCase())) continue + // Dependency / VCS / build-output forests: never matched, never + // descended. A worktree-style `.git` FILE (pointer to the real + // gitdir) is VCS noise too — the name check covers both shapes, + // parity with the engines' .git exclusion (SKIP_DIR_NAMES in search-engines.ts). + if (SEARCH_SKIP_DIRS.has(dirent.name.toLowerCase())) continue if (dirent.name.toLowerCase().includes(needle)) { matches.push(join(relative(root, dir), dirent.name)) if (matches.length >= maxMatches) { @@ -111,3 +117,45 @@ export async function searchFiles(root: string, query: string, opts: FsSearchOpt // '/' separators on every platform: the client joins onto the cwd itself. return { matches: matches.sort().map(path => path.split(sep).join('/')), truncated } } + +/** + * The fs.search dispatch: native engines first (when verified and healthy), + * the plain walk as fallback. Engine output matches the walk contract: + * root-relative, '/'-separated, sorted; the engine's cap (maxMatches + 1) + * decides `truncated`. A query that matches nothing up front short-circuits + * before any engine or walk runs. + * @param signal - aborts the engine child; an aborted search skips the + * fallback walk too (the client has already discarded the request). + */ +export async function searchFiles( + root: string, + query: string, + opts: FsSearchOptions = {}, + signal?: AbortSignal, +): Promise { + const needle = query.trim() + if (needle === '') return { matches: [], truncated: false } + const maxMatches = opts.maxMatches ?? DEFAULT_MAX_MATCHES + const t0 = performance.now() + for (const probe of await usableEngines()) { + try { + const { paths, truncated } = await runEngine(probe, root, needle, maxMatches, signal) + const matches = paths.sort() + const elapsed = (performance.now() - t0).toFixed(0) + debugLog(`[dsh-search] engine=${probe.engine} bin=${probe.binary} root=${relRoot(root)} query="${needle}" hits=${matches.length} truncated=${truncated} ${elapsed}ms`) + return { + matches: truncated ? matches.slice(0, maxMatches) : matches, + truncated, + } + } catch { + // runEngine disabled the engine; try the next one (an aborted signal + // rethrows untouched, but matching nothing is just as good — the + // request is dead either way). + if (signal?.aborted) return { matches: [], truncated: false } + } + } + const result = await searchFilesPlain(root, query, opts) + const elapsed = (performance.now() - t0).toFixed(0) + debugLog(`[dsh-search] engine=plain root=${relRoot(root)} query="${needle}" hits=${result.matches.length} truncated=${result.truncated} ${elapsed}ms`) + return result +} diff --git a/src/search-debug.ts b/src/search-debug.ts new file mode 100644 index 000000000..5018e9005 --- /dev/null +++ b/src/search-debug.ts @@ -0,0 +1,39 @@ +/** + * Gated debug instrumentation for the fs.search engine pipeline (host half). + * Off by default: with DSH_SEARCH_DEBUG unset nothing is logged, nothing + * touches the disk, and no console output is emitted. Setting + * `DSH_SEARCH_DEBUG=1` in the dsh web environment appends one line per + * probe/search/engine-failure to `/search-debug.log` and mirrors + * it to the host console. + * + * The log file lives under $DSH_HOME (the DSH config dir) when set, exactly + * like pty-deps.ts resolves it; otherwise it falls back to `~/.dsh`. A + * missing/unwritable log dir must never break a search — write errors are + * swallowed. + * + * The flag is read once at module load: toggling it requires restarting the + * host, the same as any other host-side change. + */ +import { appendFileSync } from 'node:fs' +import { join } from 'node:path' +import { homedir } from 'node:os' + +const SEARCH_DEBUG = process.env.DSH_SEARCH_DEBUG === '1' + +const LOG_FILE = join( + process.env.DSH_HOME !== undefined && process.env.DSH_HOME.trim() !== '' + ? process.env.DSH_HOME + : join(homedir(), '.dsh'), + 'search-debug.log', +) + +/** Append one debug line (gated; no-op unless DSH_SEARCH_DEBUG=1). */ +export function debugLog(msg: string): void { + if (!SEARCH_DEBUG) return + try { + appendFileSync(LOG_FILE, `[${new Date().toISOString()}] ${msg}\n`) + } catch { + // A missing/unwritable log path must never break a search. + } + console.log(msg) +} \ No newline at end of file diff --git a/src/search-engines.ts b/src/search-engines.ts new file mode 100644 index 000000000..5370e714c --- /dev/null +++ b/src/search-engines.ts @@ -0,0 +1,481 @@ +/** + * Optional native search engines (fd / ripgrep) for the editor's file-name + * search (fs.search). The probe runs lazily once per process: each candidate + * binary is verified with `--version` under a 500ms budget so a dead path is + * never used. Engine stdout streams are always capped (fd's --max-results + * included; rg is capped by killing the child) and every invocation honors + * the caller's AbortSignal. + * + * Engine output is normalized to the plain-JS walk contract: root-relative, + * '/'-separated path lines (encoded as a name-substring, case-insensitive + * match — fd is pinned to literal semantics with --fixed-strings, rg globs + * are escaped and forced case-insensitive). Directory entries are kept + * where the engine can report them (fd); rg --files only reports files, + * which is a documented lossy difference. + * + * A runtime failure disables that engine for the rest of the process (a + * broken binary should not slow every later search); the caller falls back + * to the plain walk. Hooks are swappable for tests (setEngineHooks). + */ +import { spawn, type ChildProcess } from 'node:child_process' +import { createInterface } from 'node:readline' +import { dirname, join, sep } from 'node:path' +import { homedir } from 'node:os' +import { debugLog } from './search-debug.ts' + +export type Engine = 'fd' | 'rg' + +/** A probed, verified engine binary. */ +export interface EngineProbe { + engine: Engine + binary: string +} + +/** The engine output contract: root-relative, '/'-separated, UNSORTED. */ +export interface EngineResult { + paths: string[] + /** true when the match cap cut the stream short. */ + truncated: boolean +} + +const PROBE_TIMEOUT_MS = 500 +const ENGINE_TIMEOUT_MS = 15_000 + +const PATH_SEPARATOR = process.platform === 'win32' ? ';' : ':' + +/** + * Where the DSH CLI's own dependency tree may live, as `@vscode/ripgrep` + * platform-package candidates (@vscode/ripgrep--/bin/rg). + * npm global installs on POSIX lay out as /lib/node_modules/ + * (execPath's node lives directly under /bin); the Windows npm + * global prefix has NO lib/ layer — packages land in %APPDATA%\npm\node_modules + * instead (nvm-windows keeps the same %APPDATA%\npm global). The + * launchd-spawned profile layout under ~/.dsh/profiles is covered last. + * Wrong guesses are cheap: every candidate goes through verify() + * (--version under a 500ms budget) and unusable ones are dropped. + */ +export function bundledRgCandidates( + platform: NodeJS.Platform, + arch: string, + execPath: string, + env: NodeJS.ProcessEnv, + home: string, +): string[] { + const binName = platform === 'win32' ? 'rg.exe' : 'rg' + const pkg = `@vscode/ripgrep-${platform}-${arch}` + const roots: string[] = [] + const dshDepRoot = (globalModules: string) => + join(globalModules, '@deepseek-ai/dsh/node_modules') + if (platform === 'win32') { + // Windows npm global: %APPDATA%\npm\node_modules (no lib/ layer). + if (env.APPDATA !== undefined && env.APPDATA !== '') { + roots.push(dshDepRoot(join(env.APPDATA, 'npm', 'node_modules'))) + } + // Portable / per-user node installs next to the executable. + roots.push(dshDepRoot(join(dirname(execPath), 'node_modules'))) + } else { + // POSIX npm global: /lib/node_modules — execPath's node is + // /bin/node, so the prefix is two dirnames up. + roots.push(dshDepRoot(join(dirname(dirname(execPath)), 'lib', 'node_modules'))) + // Homebrew and pnpm global layouts missed by the execPath derivation. + roots.push(dshDepRoot('/opt/homebrew/lib/node_modules'), dshDepRoot('/usr/local/lib/node_modules')) + roots.push(dshDepRoot(join(home, '.local', 'share', 'pnpm'))) + } + // The launchd-style profile layout (no user PATH, dsh deps under ~/.dsh). + roots.push(dshDepRoot(join(home, '.dsh', 'profiles', 'node_modules'))) + const seen = new Set() + const out: string[] = [] + for (const root of roots) { + const candidate = join(root, pkg, 'bin', binName) + if (!seen.has(candidate)) { + seen.add(candidate) + out.push(candidate) + } + } + return out +} + +/** Candidate binaries for one engine: PATH entries then fixed well-known + * locations. A PATH that misses Homebrew (seen in launchd-spawned + * processes) is covered by the fixed paths; rg additionally probes the + * ripgrep binary DeepSeek Harness itself ships (the @vscode/ripgrep + * optional platform package under the DSH CLI's global install — the + * harness's own agent search tool already runs on it). */ +function candidates(engine: Engine): string[] { + const env = process.env + const pathNames: Record = { + fd: ['fd', 'fdfind'], + rg: ['rg'], + } + const out: string[] = [] + // DSH's bundled rg — the CLI's own dependency tree holds the platform + // package (@vscode/ripgrep--). The harness's agent-side + // search tool already uses this binary — the sidebar search should too, + // so it wins over any system rg. + if (engine === 'rg') { + out.push(...bundledRgCandidates(process.platform, process.arch, process.execPath, env, homedir())) + } + for (const name of pathNames[engine]) { + if (env.PATH !== undefined && env.PATH !== '') { + for (const dir of env.PATH.split(PATH_SEPARATOR)) { + if (dir !== '') out.push(join(dir, name)) + } + } + } + if (engine === 'fd') { + out.push('/opt/homebrew/bin/fd', '/usr/local/bin/fd', '/usr/bin/fd', join(homedir(), '.cargo/bin/fd')) + } else if (engine === 'rg') { + out.push('/opt/homebrew/bin/rg', '/usr/local/bin/rg', '/usr/bin/rg') + } + return out +} + +/** Verify one binary actually runs (a broken install must not be used). */ +function verify(binary: string): Promise { + return new Promise((resolve) => { + let settled = false + const finish = (ok: boolean): void => { + if (settled) return + settled = true + clearTimeout(timer) + resolve(ok) + } + let child: ChildProcess + try { + child = spawn(binary, ['--version'], { stdio: 'ignore' }) + } catch { + finish(false) + return + } + const timer = setTimeout(() => { + child.kill() + finish(false) + }, PROBE_TIMEOUT_MS) + child.once('error', () => { finish(false) }) + child.once('exit', (code) => { finish(code === 0) }) + }) +} + +/** Escape glob metacharacters so a query matches literally inside -g. + * Braces are metacharacters too ({a,b} alternation): an unbalanced '{' + * makes rg fail to parse the glob (exit 2 → the engine looks broken), and + * a balanced one silently matches a DIFFERENT literal ('a{b}' searches + * 'ab'). Verified against rg 15: '\{' is a valid brace escape. */ +export function escapeGlob(query: string): string { + // The output is a GLOB string, not a regex: rg globs are gitignore-style, + // so '[' opens a character class and MUST be escaped there even though + // escaping it is unnecessary inside this regex's own character class. + // eslint-disable-next-line no-useless-escape + return query.replace(/[\[\]{}*?\\]/g, '\\$&') +} + +/** Directory names that are never useful filename-search results (VCS + * internals, dependency forests, package-manager stores, build caches). + * The plain walk in fs-search.ts builds its case-insensitive skip set + * from this list, and BOTH engine argvs exclude every name — with + * --no-ignore active an rg/fd run would otherwise re-enter node_modules + * and regress the budget-saving behavior of the walk. Excludes are + * matched case-insensitively (fd honors --ignore-case; rg uses --iglob), + * mirroring the walk's toLowerCase comparison. */ +export const SKIP_DIR_NAMES: readonly string[] = [ + '.git', + 'node_modules', + '.pnpm-store', + '.yarn', + '.turbo', + '.turbopack', + '.next', + '.nuxt', + '.output', + '.cache', + '.parcel-cache', + 'coverage', + 'dist', + 'build', + 'out', + '.umi', + '.umi-production', + '.dumi', +] + +/** A timeout means the tree is too big, not that the binary is broken — + * it must not disable the engine for every other search root. */ +export class EngineTimeoutError extends Error {} + +/** Stream a child's stdout line-by-line, capped at `max` lines (the child is + * killed past the cap so a huge result set never buffers into memory). */ +function streamLines( + child: ChildProcess, + max: number, + signal: AbortSignal | undefined, +): Promise<{ lines: string[]; truncated: boolean }> { + return new Promise((resolve, reject) => { + const lines: string[] = [] + let truncated = false + let closed = false + const finish = (error: unknown): void => { + if (closed) return + closed = true + clearTimeout(timer) + signal?.removeEventListener('abort', onAbort) + if (error !== undefined) { + child.kill() + reject(error) + } else { + resolve({ lines, truncated }) + } + } + const onAbort = (): void => { finish(new Error('search aborted')) } + // A signal that already fired never dispatches again — check upfront. + if (signal?.aborted) { + finish(new Error('search aborted')) + return + } + signal?.addEventListener('abort', onAbort, { once: true }) + const timer = setTimeout(() => { finish(new EngineTimeoutError('search engine timed out')) }, ENGINE_TIMEOUT_MS) + child.once('error', (error) => { finish(error) }) + child.once('exit', (code) => { + if (truncated) { + finish(undefined) + } else if (code !== 0 && code !== 1) { + // rg exits 1 when nothing matched — that is a successful empty run, + // not an error (fd exits 0 either way). + finish(new Error(`search engine exited with ${String(code)}`)) + } else { + finish(undefined) + } + }) + const stdout = child.stdout + if (stdout === null) { + finish(new Error('search engine stdout unavailable')) + return + } + const rl = createInterface({ input: stdout }) + rl.on('line', (line) => { + if (line === '' || truncated) return + lines.push(line) + if (lines.length > max) { + truncated = true + child.kill() + } + }) + }) +} + +/** Normalize one engine's stdout lines to the walk contract: root-relative, + * '/'-separated, no leading './'. fd/rg emit relative paths with the + * PLATFORM separator; the walk contract is '/'-separated on every platform. + * `separator` defaults to the platform separator — pass '\\' to model + * Windows engine output (rg on Windows emits '\' paths and '.\' prefixes; + * Windows file names can never contain '\' or '/', so the substitution is + * lossless there). A trailing '\r' is also stripped: Windows engines emit + * CRLF line endings, and while readline usually swallows the CR, a leftover + * one must never leak into a path (safe on every platform — a POSIX file + * whose name ends in CR is pathological). */ +export function normalizeEnginePaths(lines: readonly string[], separator: string = sep): string[] { + const out: string[] = [] + for (const line of lines) { + if (line === '') continue + const trimmed = line.endsWith('\r') ? line.slice(0, -1) : line + if (trimmed === '') continue + // A leading '.\' becomes './' after the separator substitution below; + // the './' prefix strip then covers both POSIX and Windows output. + const normalized = trimmed.split(separator).join('/') + const clean = normalized.startsWith('./') ? normalized.slice(2) : normalized + if (clean !== '' && clean !== '.') out.push(clean) + } + return out +} + +/** The fd argv: the query as a literal substring, '/' separators, every + * SKIP_DIR_NAMES excluded (a bare name excludes the entry at any depth; + * a worktree-style `.git` FILE is covered too — verified against real + * fd), and --max-results one ABOVE the stream sentinel (cap + 1) so a + * full result set overflows into the same truncation detected for rg + * (see runChild). */ +export function fdArgv(cap: number, query: string): string[] { + return [ + '--hidden', '--no-ignore', + ...SKIP_DIR_NAMES.flatMap(name => ['--exclude', name]), + '--fixed-strings', '--ignore-case', + '--path-separator', '/', '--max-results', String(cap + 1), query, '.', + ] +} + +/** The rg argv: --files listing filtered by a case-insensitive literal-name + * glob, '/' separators pinned (rg emits '\' in cmd/PowerShell on Windows, + * '/' in Git Bash — rg#501; the flag exists since rg 0.8, a build without + * it fails at spawn and is disabled at runtime). Every SKIP_DIR_NAME gets + * a glob pair: a '-anywhere' glob prunes the directory tree, and the second + * entry-only glob excludes an entry NAMED the skip word itself — a git + * worktree has a `.git` FILE (not a directory) at its root, and the + * directory-exclusion glob requires a path segment AFTER the word so it + * does not cover the pointer file (verified against real rg). fd's + * --exclude and the plain walk both skip it without the extra glob. */ +export function rgArgv(query: string): string[] { + const skipGlobs = SKIP_DIR_NAMES.flatMap(name => [ + '--iglob', `!**/${name}/**`, + '--iglob', `!**/${name}`, + ]) + const escaped = escapeGlob(query) + return [ + '--files', '--hidden', '--no-ignore', + ...skipGlobs, + // Two query globs: the slash-free form matches BASENAMES only (rg + // follows gitignore semantics — no '/' in the pattern means basename + // match, verified on real rg 15: 'util/helper.ts' never matches + // '*util*'), so a second path-level glob admits files UNDER a + // matching directory — deriveRgMatches turns those into the + // directory matches the walk/fd contract reports. + '--iglob', `*${escaped}*`, + '--iglob', `**/*${escaped}*/**`, + '--path-separator', '/', '.', + ] +} + +/** One child invocation per engine, emitting normalized relative paths. + * rg's query globs admit basename hits AND files under matching + * directories; deriveRgMatches converts the latter into the directory + * matches the walk contract reports and drops nothing else. + * + * Truncation symmetry: `streamLines` marks truncated when the stream + * exceeds `cap = maxMatches + 1` lines (the +1 is a sentinel proving + * "there is more"). rg has no --max-results so it naturally overflows + * into the sentinel; fd MUST NOT cap at `cap` (it would stop exactly at + * the sentinel and never be seen as truncated) — pin its --max-results + * one higher (maxMatches + 2) so a full result set trips the same + * sentinel and both engines report truncated identically, with the + * caller slicing back to maxMatches. */ +function runChild( + probe: EngineProbe, + root: string, + query: string, + maxMatches: number, + signal: AbortSignal | undefined, +): Promise { + const cap = maxMatches + 1 + let child: ChildProcess + if (probe.engine === 'fd') { + child = spawn(probe.binary, fdArgv(cap, query), { cwd: root, stdio: ['ignore', 'pipe', 'ignore'] }) + } else { + child = spawn(probe.binary, rgArgv(query), { cwd: root, stdio: ['ignore', 'pipe', 'ignore'] }) + } + return streamLines(child, cap, signal).then(({ lines, truncated }) => { + const paths = normalizeEnginePaths(lines) + if (probe.engine !== 'rg') return { paths, truncated } + const derived = deriveRgMatches(paths, query, maxMatches) + return { paths: derived.paths, truncated: truncated || derived.truncated } + }) +} + +/** rg reports FILES only (`rg --files` never emits a directory line), while + * the walk contract matches entry names — files AND directories. Every rg + * line is a file path whose glob already guarantees the query appears + * somewhere in it, so: a basename hit stays a file match, and a matching + * DIRECTORY segment is derived as a directory match (a directory named X + * holding at least one file surfaces exactly as fd/walk report it). + * EMPTY directories stay invisible to rg — no file path carries them — + * the one irreducible gap versus fd (documented lossy, see the design + * doc). Output is deduped and sorted; when the derived set exceeds the + * match budget it is capped at maxMatches + 1 and truncated is raised — + * the budget, not the engine, cut the result short (the caller slices + * back to maxMatches on truncated, same as the fd/rg stream sentinel). */ +export function deriveRgMatches( + paths: readonly string[], + query: string, + maxMatches: number, +): { paths: string[]; truncated: boolean } { + const needle = query.toLowerCase() + const out = new Set() + for (const path of paths) { + const segments = path.split('/') + const base = segments[segments.length - 1] + if (base !== undefined && base.toLowerCase().includes(needle)) out.add(path) + let prefix = '' + for (let index = 0; index < segments.length - 1; index += 1) { + const segment = segments[index] + if (segment === undefined) continue + prefix = prefix === '' ? segment : `${prefix}/${segment}` + if (segment.toLowerCase().includes(needle)) out.add(prefix) + } + } + const derived = [...out].sort() + return { paths: derived.slice(0, maxMatches + 1), truncated: derived.length > maxMatches } +} + +let prober = (): Promise => probeOnce() +let runner = runChild +/** Engines failed at runtime: skipped for the rest of this process. */ +const broken = new Set() +let probePromise: Promise | null = null + +/** Probe once per process (lazy): each engine's first working candidate. */ +async function probeOnce(): Promise { + const found: EngineProbe[] = [] + for (const engine of ['fd', 'rg'] as const) { + for (const binary of candidates(engine)) { + if (await verify(binary)) { + found.push({ engine, binary }) + break + } + } + } + const names = found.length > 0 ? found.map(p => `${p.engine}(${p.binary})`).join(', ') : 'none (plain-walk fallback)' + debugLog(`[dsh-search] engines probed: ${names}`) + return found +} + +/** The verified engines for this process (cached across searches). */ +export function probeEngines(): Promise { + probePromise ??= prober() + return probePromise +} + +/** Run one engine and get capped, normalized matches; a runtime failure + * disables that engine for the rest of the process (a broken binary must + * not slow every later search) — but a TIMEOUT does not: a timeout means + * THIS tree was too big, while the broken-set is per-engine, so disabling + * here would strip every other (small) root of the engine over one huge + * directory. An aborted signal rethrows untouched so the caller skips the + * fallback walk too. */ +export async function runEngine( + probe: EngineProbe, + root: string, + query: string, + maxMatches: number, + signal: AbortSignal | undefined, +): Promise { + let result: EngineResult + try { + result = await runner(probe, root, query, maxMatches, signal) + } catch (error) { + if (error instanceof EngineTimeoutError && !signal?.aborted) { + debugLog(`[dsh-search] engine ${probe.engine} timed out (tree too large), falling back`) + } else if (!signal?.aborted) { + broken.add(probe.engine) + debugLog(`[dsh-search] engine ${probe.engine} failed at runtime, disabled: ${error instanceof Error ? error.message : String(error)}`) + } + throw error + } + return result +} + +/** The engines a caller may actually try (verified, not broken). */ +export async function usableEngines(): Promise { + const probes = await probeEngines() + return probes.filter((probe) => !broken.has(probe.engine)) +} + +/** Test seam: replace the probe / child-runner implementations. */ +export function setEngineHooks(next: { prober?: typeof prober; runner?: typeof runner }): void { + if (next.prober !== undefined) prober = next.prober + if (next.runner !== undefined) runner = next.runner +} + +/** Reset probe cache, broken-set and hooks (test isolation). */ +export function resetEngines(): void { + probePromise = null + broken.clear() + prober = (): Promise => probeOnce() + runner = runChild +} \ No newline at end of file diff --git a/tests/fs-search.spec.ts b/tests/fs-search.spec.ts index ca10763f6..f45e87c7c 100644 --- a/tests/fs-search.spec.ts +++ b/tests/fs-search.spec.ts @@ -6,11 +6,13 @@ * never descended (cycle safety), and the maxMatches/maxVisited budgets * stop a runaway walk with `truncated: true`. */ -import { describe, expect, it } from 'vitest' +import { describe, afterEach, expect, it } from 'vitest' import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { searchFiles } from '../src/fs-search.ts' +import { searchFilesPlain, searchFiles } from '../src/fs-search.ts' +import type { EngineProbe } from '../src/search-engines.ts' +import { resetEngines, setEngineHooks } from '../src/search-engines.ts' /** * Symlink creation needs extra privileges on Windows; the symlink case skips @@ -48,10 +50,10 @@ describe('fs-search', () => { it('matches name substrings and reports root-relative /-separated paths', async () => { const dir = makeFixture() try { - const result = await searchFiles(dir, 'util') + const result = await searchFilesPlain(dir, 'util') expect(result).toEqual({ matches: ['src/util.ts'], truncated: false }) // A multi-level match list is sorted and relative (never absolute). - const md = await searchFiles(dir, '.md') + const md = await searchFilesPlain(dir, '.md') expect(md.truncated).toBe(false) expect(md.matches).toEqual(['README.md', 'docs/guide.md']) for (const match of md.matches) { @@ -66,10 +68,10 @@ describe('fs-search', () => { it('matches case-insensitively on the entry name', async () => { const dir = makeFixture() try { - expect((await searchFiles(dir, 'index.ts')).matches).toEqual(['src/Index.TS']) - expect((await searchFiles(dir, 'INDEX.TS')).matches).toEqual(['src/Index.TS']) + expect((await searchFilesPlain(dir, 'index.ts')).matches).toEqual(['src/Index.TS']) + expect((await searchFilesPlain(dir, 'INDEX.TS')).matches).toEqual(['src/Index.TS']) // Directory names match too (the client can hint where matches live). - expect((await searchFiles(dir, 'SRC')).matches).toEqual(['src']) + expect((await searchFilesPlain(dir, 'SRC')).matches).toEqual(['src']) } finally { rmSync(dir, { recursive: true, force: true }) } @@ -79,9 +81,9 @@ describe('fs-search', () => { const dir = makeFixture() try { // 'readme' would hit .git/objects/readme-pack if the walk entered .git. - expect((await searchFiles(dir, 'readme')).matches).toEqual(['README.md']) - expect((await searchFiles(dir, 'config')).matches).toEqual([]) - expect((await searchFiles(dir, '.git')).matches).toEqual([]) + expect((await searchFilesPlain(dir, 'readme')).matches).toEqual(['README.md']) + expect((await searchFilesPlain(dir, 'config')).matches).toEqual([]) + expect((await searchFilesPlain(dir, '.git')).matches).toEqual([]) } finally { rmSync(dir, { recursive: true, force: true }) } @@ -97,12 +99,28 @@ describe('fs-search', () => { writeFileSync(join(dir, 'web', 'app.ts'), 'src') // A match hidden behind node_modules / dist must not appear; project // files after those forests must still be reachable within budget. - expect((await searchFiles(dir, 'guide')).matches).toEqual(['docs/guide.md']) - expect((await searchFiles(dir, 'left-pad')).matches).toEqual([]) - expect((await searchFiles(dir, 'bundle')).matches).toEqual([]) - expect((await searchFiles(dir, 'app.ts')).matches).toEqual(['web/app.ts']) - expect((await searchFiles(dir, 'node_modules')).matches).toEqual([]) - expect((await searchFiles(dir, 'dist')).matches).toEqual([]) + expect((await searchFilesPlain(dir, 'guide')).matches).toEqual(['docs/guide.md']) + expect((await searchFilesPlain(dir, 'left-pad')).matches).toEqual([]) + expect((await searchFilesPlain(dir, 'bundle')).matches).toEqual([]) + expect((await searchFilesPlain(dir, 'app.ts')).matches).toEqual(['web/app.ts']) + expect((await searchFilesPlain(dir, 'node_modules')).matches).toEqual([]) + expect((await searchFilesPlain(dir, 'dist')).matches).toEqual([]) + } finally { + rmSync(dir, { recursive: true, force: true }) + } + }) + + // A git worktree carries a `.git` FILE (a pointer to the real gitdir), + // not a directory. It is VCS-internal noise exactly like the .git + // directory and must never surface as a match — parity with fd's + // --exclude .git and rg's '!**/.git' glob. + it('never matches a worktree-style .git file', async () => { + const dir = mkdtempSync(join(tmpdir(), 'dsh-sidebar-search-')) + try { + writeFileSync(join(dir, '.git'), 'gitdir: /elsewhere/.git/worktrees/wt') + writeFileSync(join(dir, 'util.ts'), 'code') + expect(await searchFilesPlain(dir, '.git')).toEqual({ matches: [], truncated: false }) + expect((await searchFilesPlain(dir, 'util')).matches).toEqual(['util.ts']) } finally { rmSync(dir, { recursive: true, force: true }) } @@ -111,8 +129,8 @@ describe('fs-search', () => { it('an empty (or whitespace) query matches nothing without walking', async () => { const dir = makeFixture() try { - expect(await searchFiles(dir, '')).toEqual({ matches: [], truncated: false }) - expect(await searchFiles(dir, ' ')).toEqual({ matches: [], truncated: false }) + expect(await searchFilesPlain(dir, '')).toEqual({ matches: [], truncated: false }) + expect(await searchFilesPlain(dir, ' ')).toEqual({ matches: [], truncated: false }) } finally { rmSync(dir, { recursive: true, force: true }) } @@ -125,7 +143,7 @@ describe('fs-search', () => { // src would duplicate its matches. Neither must be entered. symlinkSync(dir, join(dir, 'loop')) symlinkSync(join(dir, 'src'), join(dir, 'src-link')) - const result = await searchFiles(dir, 'util') + const result = await searchFilesPlain(dir, 'util') expect(result).toEqual({ matches: ['src/util.ts'], truncated: false }) } finally { rmSync(dir, { recursive: true, force: true }) @@ -138,7 +156,7 @@ describe('fs-search', () => { for (let index = 0; index < 5; index += 1) { writeFileSync(join(dir, `match-${index}.txt`), 'x') } - const result = await searchFiles(dir, 'match', { maxMatches: 2 }) + const result = await searchFilesPlain(dir, 'match', { maxMatches: 2 }) expect(result.truncated).toBe(true) expect(result.matches.length).toBe(2) } finally { @@ -153,7 +171,7 @@ describe('fs-search', () => { writeFileSync(join(dir, `file-${index}.txt`), 'x') } // The walk visits more entries than the budget allows and gives up. - const result = await searchFiles(dir, 'nomatch', { maxVisited: 3 }) + const result = await searchFilesPlain(dir, 'nomatch', { maxVisited: 3 }) expect(result.truncated).toBe(true) expect(result.matches).toEqual([]) } finally { @@ -165,7 +183,99 @@ describe('fs-search', () => { const dir = makeFixture() try { const missing = join(dir, 'does-not-exist') - expect(await searchFiles(missing, 'x')).toEqual({ matches: [], truncated: false }) + expect(await searchFilesPlain(missing, 'x')).toEqual({ matches: [], truncated: false }) + } finally { + rmSync(dir, { recursive: true, force: true }) + } + }) +}) + +/** The fs.search dispatch: engine-first with the plain walk as fallback. */ +describe('fs-search dispatch', () => { + const fakeFd: EngineProbe = { engine: 'fd', binary: '/fake/fd' } + + afterEach(() => { + resetEngines() + }) + + it('routes to the probed engine and normalizes its output', async () => { + setEngineHooks({ + prober: async () => [fakeFd], + runner: async (_probe, _root, query) => { + expect(query).toBe('util') + return { paths: ['src/util.ts', 'README.md'], truncated: false } + }, + }) + expect(await searchFiles('/workspace', 'util')).toEqual({ + matches: ['README.md', 'src/util.ts'], + truncated: false, + }) + }) + + it('an empty query never touches the engines', async () => { + let probed = false + setEngineHooks({ + prober: async () => { probed = true; return [fakeFd] }, + runner: async () => ({ paths: [], truncated: false }), + }) + expect(await searchFiles('/workspace', ' ')).toEqual({ matches: [], truncated: false }) + expect(probed).toBe(false) + }) + + it('caps engine output at maxMatches and reports truncated', async () => { + setEngineHooks({ + prober: async () => [fakeFd], + runner: async () => ({ paths: ['a', 'b', 'c'], truncated: true }), + }) + expect(await searchFiles('/workspace', 'x', { maxMatches: 2 })).toEqual({ + matches: ['a', 'b'], + truncated: true, + }) + }) + + it('falls back to the plain walk when the engine fails at runtime', async () => { + const dir = makeFixture() + try { + setEngineHooks({ + prober: async () => [fakeFd], + runner: async () => { throw new Error('engine exploded') }, + }) + expect(await searchFiles(dir, 'util')).toEqual({ matches: ['src/util.ts'], truncated: false }) + // The failed engine is disabled for the rest of the process. + let attempts = 0 + setEngineHooks({ + runner: async () => { attempts += 1; return { paths: [], truncated: false } }, + }) + expect(await searchFiles(dir, 'util')).toEqual({ matches: ['src/util.ts'], truncated: false }) + expect(attempts).toBe(0) + } finally { + rmSync(dir, { recursive: true, force: true }) + } + }) + + it('uses the plain walk when no engine is probed', async () => { + const dir = makeFixture() + try { + setEngineHooks({ prober: async () => [] }) + expect(await searchFiles(dir, 'util')).toEqual({ matches: ['src/util.ts'], truncated: false }) + } finally { + rmSync(dir, { recursive: true, force: true }) + } + }) + + it('an aborted search skips both the engine retry and the walk', async () => { + const dir = makeFixture() + const controller = new AbortController() + controller.abort() + try { + setEngineHooks({ + prober: async () => [fakeFd], + runner: async () => { throw new Error('search aborted') }, + }) + expect(await searchFiles(dir, 'util', {}, controller.signal)).toEqual({ + matches: [], + truncated: false, + }) } finally { rmSync(dir, { recursive: true, force: true }) } diff --git a/tests/search-engines.spec.ts b/tests/search-engines.spec.ts new file mode 100644 index 000000000..d7608b4bd --- /dev/null +++ b/tests/search-engines.spec.ts @@ -0,0 +1,327 @@ +/** + * search-engines: the native-engine probe and runner behind fs.search. + * The probe is process-cached (verified binaries only); a runtime failure + * disables one engine without disturbing the others. `normalizeEnginePaths` + * re-bases raw stdout lines onto the walk contract (root-relative, + * '/'-separated, no './' prefix). Child processes are exercised entirely + * through injected hooks — CI machines have no fd/rg, and the probe/runner + * contracts are what the dispatch depends on. + */ +import { afterEach, describe, expect, it } from 'vitest' +import { join } from 'node:path' +import { + bundledRgCandidates, + deriveRgMatches, + EngineTimeoutError, + escapeGlob, + fdArgv, + normalizeEnginePaths, + probeEngines, + resetEngines, + rgArgv, + runEngine, + setEngineHooks, + SKIP_DIR_NAMES, + usableEngines, +} from '../src/search-engines.ts' +import type { EngineProbe } from '../src/search-engines.ts' + +const fdProbe: EngineProbe = { engine: 'fd', binary: '/fake/fd' } +const rgProbe: EngineProbe = { engine: 'rg', binary: '/fake/rg' } + +describe('normalizeEnginePaths', () => { + it('keeps root-relative /-separated lines as-is (fd contract)', () => { + expect(normalizeEnginePaths(['src/util.ts', 'README.md'])).toEqual([ + 'src/util.ts', + 'README.md', + ]) + }) + + it('strips a leading ./ from engine output', () => { + expect(normalizeEnginePaths(['./src/a.ts', './b.ts'])).toEqual([ + 'src/a.ts', + 'b.ts', + ]) + }) + + it('drops empty lines and the bare root', () => { + expect(normalizeEnginePaths(['', '.', 'src/x.ts'])).toEqual(['src/x.ts']) + }) + + // Windows shape (review concern: fd/rg "weird formats" on Windows): + // rg emits '\'-separated paths with a '.\' prefix and CRLF line endings — + // all of it must still land on the '/'-separated walk contract. + it('normalizes Windows engine output: backslash separators + .\\ prefix (rg shape)', () => { + expect(normalizeEnginePaths(['src\\util.ts', '.\\README.md'], '\\')).toEqual([ + 'src/util.ts', + 'README.md', + ]) + }) + + it('strips a trailing CR from engine lines (Windows CRLF endings)', () => { + expect(normalizeEnginePaths(['src/util.ts\r', './b.ts\r'])).toEqual([ + 'src/util.ts', + 'b.ts', + ]) + // Same protection applies to the Windows shape. + expect(normalizeEnginePaths(['src\\util.ts\r', '.\\docs\\guide.md\r'], '\\')).toEqual([ + 'src/util.ts', + 'docs/guide.md', + ]) + }) +}) + +describe('deriveRgMatches', () => { + // rg --files never emits a directory line: a matching DIRECTORY segment + // must be derived from the file path so rg-only machines see the same + // results as fd / the plain walk (which both report directory names). + it('keeps basename hits and derives matching directory segments', () => { + expect(deriveRgMatches( + ['src/util.ts', 'util/helper.ts', 'web/dist/bundle.js'], + 'util', + 10, + )).toEqual({ paths: ['src/util.ts', 'util'], truncated: false }) + }) + + it('dedupes repeated matching segments across lines', () => { + expect(deriveRgMatches(['lib/a/x.ts', 'lib/a/y.ts'], 'lib', 10)).toEqual({ + paths: ['lib'], + truncated: false, + }) + }) + + it('matches case-insensitively (walk parity)', () => { + expect(deriveRgMatches(['SRC/Util.ts'], 'UTIL', 10)).toEqual({ + paths: ['SRC/Util.ts'], + truncated: false, + }) + }) + + it('caps the derived set at maxMatches + 1 and raises truncated', () => { + // 4 derived entries ('a' + 3 files) over a budget of 2: the budget, not + // the engine, cut the result short — same sentinel semantics as the + // fd/rg stream (cap = maxMatches + 1, caller slices back). + expect(deriveRgMatches(['a/a1', 'a/a2', 'a/a3'], 'a', 2)).toEqual({ + paths: ['a', 'a/a1', 'a/a2'], + truncated: true, + }) + }) + + it('derives both the directory and the basename hit from one line', () => { + // 'util/util.ts': basename hit keeps the file, the 'util' segment + // derives the directory — runChild ORs this with the stream's own + // truncation flag (a stream-truncated run with a small derived set + // stays true). + expect(deriveRgMatches(['util/util.ts'], 'util', 10)).toEqual({ + paths: ['util', 'util/util.ts'], + truncated: false, + }) + }) +}) + +describe('escapeGlob', () => { + it('escapes glob metacharacters for rg -g literal matching', () => { + expect(escapeGlob('a*b?c[d]')).toBe('a\\*b\\?c\\[d\\]') + }) + + // '{' is globset alternation syntax: unbalanced it breaks the glob parse + // (rg exits 2 → the engine looks broken and gets disabled process-wide), + // balanced 'a{b}' silently searches 'ab' instead of the literal. Both + // verified against real rg 15; '\{' is the accepted escape. + it('escapes braces so alternation syntax cannot hijack a literal query', () => { + expect(escapeGlob('a{b}')).toBe('a\\{b\\}') + expect(escapeGlob('{')).toBe('\\{') + expect(escapeGlob('util{bar')).toBe('util\\{bar') + }) +}) + +describe('engine argv symmetry', () => { + it('fd --max-results sits one ABOVE the sentinel (cap + 1) so full result sets trip truncation', () => { + // cap = maxMatches + 1 is the stream sentinel: the runner marks + // truncated when a line arrives past it. fd must not stop AT the + // sentinel (never seen as truncated) — it caps one line higher. + const argv = fdArgv(201, 'util') + expect(argv).toContain('--max-results') + expect(argv[argv.indexOf('--max-results') + 1]).toBe('202') + expect(argv).toContain('--fixed-strings') + expect(argv).toContain('--path-separator') + }) + + it('fd argv keeps the literal-fixed, hidden, no-ignore contract', () => { + const argv = fdArgv(10, 'a*b') + expect(argv.slice(0, 2)).toEqual(['--hidden', '--no-ignore']) + expect(argv).toContain('--fixed-strings') + expect(argv).toContain('--ignore-case') + expect(argv).toContain('--path-separator') + expect(argv).toContain('/') + expect(argv[argv.length - 2]).toBe('a*b') // literal, unescaped + expect(argv[argv.length - 1]).toBe('.') + }) + + // --no-ignore bypasses .gitignore: without explicit excludes the engines + // would re-enter node_modules etc. and regress the walk's budget savings. + // fd excludes each skip name at any depth (incl. a worktree .git FILE); + // rg needs the directory glob + entry-only glob pair per name. + it('fd and rg exclude every SKIP_DIR_NAMES entry (walk parity)', () => { + const fd = fdArgv(10, 'util') + const fdExcludes: (string | undefined)[] = [] + for (let index = 0; index < fd.length; index += 1) { + if (fd[index] === '--exclude') fdExcludes.push(fd[index + 1]) + } + expect(fdExcludes).toEqual([...SKIP_DIR_NAMES]) + + const rg = rgArgv('util') + const rgIglobs: (string | undefined)[] = [] + for (let index = 0; index < rg.length; index += 1) { + if (rg[index] === '--iglob') rgIglobs.push(rg[index + 1]) + } + for (const name of SKIP_DIR_NAMES) { + expect(rgIglobs).toContain(`!**/${name}/**`) + expect(rgIglobs).toContain(`!**/${name}`) + } + // The query iglobs are the last ones (skip globs precede them), still + // case-insensitive and escaped: basename form + path-level form. + expect(rgIglobs.slice(-2)).toEqual(['*util*', '**/*util*/**']) + }) + + it('rg argv escapes glob metacharacters and pins / separators', () => { + const argv = rgArgv('a*b') + expect(argv).toContain('--files') + expect(argv).toContain('--path-separator') + expect(argv).toContain('*a\\*b*') + expect(argv[argv.length - 1]).toBe('.') + }) + + // A git worktree has a `.git` FILE at its root: '!**/.git/**' needs a + // path segment AFTER .git, so the pointer file itself leaks through + // (verified against real rg). fd's --exclude .git covers both shapes; + // rg needs the second entry-only glob for parity. + it('rg argv excludes .git directories AND a bare worktree .git file', () => { + const argv = rgArgv('util') + expect(argv).toContain('--iglob') + expect(argv).toContain('!**/.git/**') + expect(argv).toContain('!**/.git') + }) +}) + +describe('bundledRgCandidates', () => { + it('derives the POSIX npm global layout from execPath (darwin)', () => { + const paths = bundledRgCandidates( + 'darwin', 'arm64', + '/opt/homebrew/bin/node', + {}, '/Users/me', + ) + expect(paths).toContain(join( + '/opt/homebrew/lib/node_modules', + '@deepseek-ai/dsh/node_modules', + '@vscode/ripgrep-darwin-arm64/bin/rg', + )) + }) + + // Windows npm has NO lib/ layer: the global prefix is %APPDATA%\npm, so + // the execPath derivation used on POSIX would resolve to a bogus + // C:\lib\node_modules\… path. The review concern (fd/rg "weird formats" + // on Windows) includes the probe itself — pin the %APPDATA%\npm shape. + it('derives the Windows npm global layout from %APPDATA% (win32)', () => { + const paths = bundledRgCandidates( + 'win32', 'x64', + 'C:\\Program Files\\nodejs\\node.exe', + { APPDATA: 'C:\\Users\\me\\AppData\\Roaming' }, 'C:\\Users\\me', + ) + // join() renders the platform separator, so this also validates the + // real backslash shape on Windows CI. + expect(paths).toContain(join( + 'C:\\Users\\me\\AppData\\Roaming', 'npm', 'node_modules', + '@deepseek-ai', 'dsh', 'node_modules', + '@vscode', 'ripgrep-win32-x64', 'bin', 'rg.exe', + )) + }) + + it('covers the launchd profile layout under ~/.dsh on every platform', () => { + const darwin = bundledRgCandidates('darwin', 'arm64', '/usr/local/bin/node', {}, '/Users/me') + expect(darwin).toContain(join( + '/Users/me/.dsh/profiles/node_modules', + '@deepseek-ai', 'dsh', 'node_modules', + '@vscode', 'ripgrep-darwin-arm64', 'bin', 'rg', + )) + // Windows: node:path is platform-bound, so a test on POSIX cannot pin + // the exact backslash shape — assert the layout structure (a real + // Windows CI/true-positive run pins the literal '\\' separators). + const win = bundledRgCandidates('win32', 'x64', 'C:\\node.exe', {}, 'C:\\Users\\me') + expect(win.some(path => path.includes('.dsh') && path.includes('@deepseek-ai'))).toBe(true) + }) + + it('dedupes identical candidates across derivations', () => { + const paths = bundledRgCandidates( + 'darwin', 'arm64', + '/usr/local/bin/node', + {}, '/Users/me', + ) + // /usr/local/bin/node → /usr/local/lib/node_modules AND the fixed + // /usr/local/lib/node_modules root — the same file must appear once. + const duplicates = paths.filter((path, index) => paths.indexOf(path) !== index) + expect(duplicates).toEqual([]) + }) + + it('drops APPDATA roots when the env var is absent (win32)', () => { + const paths = bundledRgCandidates('win32', 'x64', 'C:\\node.exe', {}, 'C:\\Users\\me') + expect(paths.some(path => path.includes('AppData'))).toBe(false) + // The per-executable node_modules root (portable installs) survives. + expect(paths.some(path => path.includes('node_modules') && path.includes('@deepseek-ai'))).toBe(true) + }) +}) + +describe('probe cache and broken-disable', () => { + afterEach(() => { + resetEngines() + }) + + it('caches the probe result across calls', async () => { + let calls = 0 + setEngineHooks({ prober: async () => { calls += 1; return [fdProbe] } }) + expect(await probeEngines()).toBe(await probeEngines()) + expect(calls).toBe(1) + }) + + it('usableEngines hides engines broken at runtime', async () => { + setEngineHooks({ prober: async () => [fdProbe, rgProbe] }) + await runEngine(fdProbe, '/w', 'x', 10, undefined).catch(() => { + /* the failing runner below */ + }) + setEngineHooks({ + runner: async () => { throw new Error('boom') }, + }) + await runEngine(fdProbe, '/w', 'x', 10, undefined).catch(() => { + /* expected failure */ + }) + const usable = await usableEngines() + expect(usable.map(probe => probe.engine)).toEqual(['rg']) + }) + + it('an aborted run does not disable the engine', async () => { + setEngineHooks({ prober: async () => [fdProbe] }) + setEngineHooks({ + runner: async () => { throw new Error('search aborted') }, + }) + const controller = new AbortController() + controller.abort() + await runEngine(fdProbe, '/w', 'x', 10, controller.signal).catch(() => { + /* expected failure */ + }) + expect((await usableEngines()).map(probe => probe.engine)).toEqual(['fd']) + }) + + // A timeout means THIS tree was too big — the broken-set is per-engine, + // so disabling here would strip every other root of the engine over one + // huge directory. The engine must stay usable after a timeout. + it('a timed-out run does not disable the engine', async () => { + setEngineHooks({ prober: async () => [fdProbe] }) + setEngineHooks({ + runner: async () => { throw new EngineTimeoutError('search engine timed out') }, + }) + await runEngine(fdProbe, '/huge-tree', 'x', 10, undefined).catch(() => { + /* expected failure */ + }) + expect((await usableEngines()).map(probe => probe.engine)).toEqual(['fd']) + }) +}) \ No newline at end of file