Skip to content

fix(store): 隔离 SQLite 与 TCVDB 的关键词查询语义#473

Open
Qiyuanqiii wants to merge 1 commit into
TencentCloud:mainfrom
Qiyuanqiii:fix/issue-160-backend-keyword-query-boundary
Open

fix(store): 隔离 SQLite 与 TCVDB 的关键词查询语义#473
Qiyuanqiii wants to merge 1 commit into
TencentCloud:mainfrom
Qiyuanqiii:fix/issue-160-backend-keyword-query-boundary

Conversation

@Qiyuanqiii

Copy link
Copy Markdown

Description | 描述

本 PR 修复关键词查询在 SQLite 与 TCVDB 后端之间的语义边界问题。

维护者已将 #178 定位为 #160 的标准窄范围修复,用于净化 SQLite FTS5 运算符。本 PR 不修改运算符净化逻辑,也不替代 #178,而是处理一个可以独立合并的后端抽象问题。

问题根因

当前调用方会先通过 buildFtsQuery() 将用户文本转换为 SQLite FTS5 MATCH 表达式,再传给 IMemoryStore.searchL1Fts() / searchL0Fts()

例如:

  • 原始文本:alpha beta
  • SQLite 表达式:"alpha" OR "beta"

SQLite 确实需要这种表达式,但 TCVDB 的实现会把同一个参数作为原始查询文本传给服务端 embedding、BM25 和混合检索。因此,TCVDB 实际收到的是包含引号和 OR 的 SQLite 语法,而不是原始自然语言,可能影响向量表示和关键词召回。

修改内容

  • 新增后端中立的 searchL1Keyword()searchL0Keyword() 接口。
  • 调用方始终传递原始用户文本,不再直接依赖 SQLite 的 buildFtsQuery()
  • SQLite 在存储实现内部生成 FTS5 MATCH 表达式。
  • TCVDB 保留原始自然语言,仅清理首尾空白后交给 embedding/BM25 混合检索。
  • 更新自动召回、L1 去重、记忆搜索和会话搜索调用链。
  • 保留原有 searchL1Fts() / searchL0Fts() 作为兼容入口。
  • 增加后端契约测试,无需连接真实 TCVDB 即可验证查询参数差异。

Related Issue | 关联 Issue

Related to #160

说明:#178 仍是 #160 的标准 FTS5 运算符净化修复;本 PR 解决独立的跨后端查询语义问题,因此不使用 Fixes #160

Change Type | 修改类型

  • Bug fix | Bug 修复
  • Code optimization | 代码优化
  • New feature | 新功能
  • Documentation update | 文档更新

Verification | 验证

  • npm.cmd test -- src/core/store/keyword-query.test.ts:5/5 通过
  • npm.cmd test:72/72 通过
  • npm.cmd run build:通过
  • git diff --check origin/main...HEAD:通过

新增测试明确验证:

  • SQLite L1/L0 在存储边界把 alpha beta 转换为 "alpha" OR "beta"
  • TCVDB L1/L0 收到原始的 alpha beta
  • 纯空白查询不会触发 TCVDB 检索。

Self-test Checklist | 自测清单

验证截图

image

@YOMXXX YOMXXX left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

I verified the PR locally on the branch with Node 24.15.0:

  • COREPACK_ENABLE_AUTO_PIN=0 pnpm vitest run src/core/store/keyword-query.test.ts — 1 file / 5 tests passed
  • COREPACK_ENABLE_AUTO_PIN=0 pnpm test — 5 files / 72 tests passed
  • COREPACK_ENABLE_AUTO_PIN=0 pnpm build — passed
  • git diff --check HEAD~1..HEAD — passed

This is not a duplicate of the #178 FTS5 operator-sanitization fix. It cleanly separates raw keyword text from backend-specific query syntax: SQLite builds FTS5 MATCH expressions inside the SQLite store, while TCVDB receives trimmed natural-language text for its hybrid search path. Existing low-level searchL*Fts() methods remain as compatibility entry points, and the new tests cover both L1/L0 SQLite and TCVDB boundaries.

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program!
We have successfully received your submission. The program is currently in full swing, and we will complete the Code Review for you as soon as possible. Please keep an eye on the status notifications for this PR so you can follow up promptly once the review feedback is provided.
Thanks again for your contribution and open-source spirit! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants