fix(store): 同步清理 TTL 过期记录的 FTS 索引#491
Open
Qiyuanqiii wants to merge 1 commit into
Open
Conversation
Collaborator
|
Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
关联 Issue
Related to #160
背景与定位
#160 的标准窄范围修复已经由 #178 负责,其他相关增量也有明确边界:
本 PR 不修改查询构造或 tokenizer,而是修复 SQLite 存储层另一条独立的一致性问题:TTL 已删除的记忆仍残留在 FTS 表中,并可继续被关键词搜索召回。
问题根因
deleteL1Expired()与deleteL0Expired()的事务当前只清理两类数据:l1_records/l0_conversations主表记录l1_vec/l0_vec向量记录l1_fts与l0_fts是独立的 FTS5 虚拟表,不会随着主表删除自动级联。关键词搜索又直接读取 FTS 表,因此 TTL 清理完成后,已过期内容仍可能作为“幽灵索引”返回。真实复现中,L1/L0 主表均从 5 条降至 4 条,但已删除的
l1-0与l0-0仍分别出现在 FTS 查询结果中。这不仅影响召回正确性,也违背 TTL 的数据保留语义。实现方案
兼容性
未启用 FTS5 时继续沿用原有 TTL 清理路径。启用 FTS5 时,只删除已经被同一 cutoff 判定为过期的索引行。
本 PR 保留:
updated_time与 L0recorded_at过期语义本 PR 不包含:
因此它是 SQLite TTL 事务的索引同步修复,不替代 #178 或 #484。
验证
1个文件、2项测试通过5个文件、69项测试通过git diff --check:通过定向测试覆盖
验证截图