Skip to content

fix(analytics): un-binary usage-ledger.js — escape the NUL separator - #27

Closed
oratis wants to merge 1 commit into
mainfrom
claude/fervent-spence-5f680f
Closed

fix(analytics): un-binary usage-ledger.js — escape the NUL separator#27
oratis wants to merge 1 commit into
mainfrom
claude/fervent-spence-5f680f

Conversation

@oratis

@oratis oratis commented Aug 9, 2026

Copy link
Copy Markdown
Owner

问题

server/usage-ledger.js 里有 2 个裸字节 0x00 直接写在源码文本中,而不是转义序列 \u0000。位置:summarize() 中复合键的注释,以及 cellKey 模板字符串。

分隔符的选择本身没问题 —— NUL 不可能与 YYYY-MM 月份或任何 agent_id 冲突。错的只是编码方式。

但因为 NUL 落在文件前 8000 字节内,git 把整个文件判成二进制,后果有两层

第一层:从 review 里消失

命令 修复前
git diff Bin 9679 -> 10779 bytes
git blame 完全失效
file data,而非 UTF-8 文本

这是 #17 带进来的,而它能活到 merge,恰恰是因为文件不可 diff。

第二层:文件无法合并(本 PR 过程中现场撞到)

本分支第一版基于 #21 时的 main,而 #23 在 main 上也改了这个文件。GitHub 直接报 CONFLICTING —— git 对二进制文件没有三方合并策略,任何并发修改都是死锁。已 rebase 到当前 main,冲突按「取 main 版本 + 重新施加同一处转义」解决。

改动

两处裸 0x00 替换为 \u0000 转义。运行时行为零变化 —— 模板字面量里的 \u0000 求值仍是 U+0000,getWorkspaceUsage / getPlatformUsage 的分组结果逐字节一致。

验证

检查 结果
文件 NUL 计数 0
file server/usage-ledger.js Unicode text, UTF-8 text(原 data
npm test 678/678 通过
usage-ledger.test.js 单跑 29/29 通过
全仓受跟踪文件扫描 仅此一个文件受影响
CI 全绿

除既有测试外,额外做了一次对抗性分组检查:agent_id 07-x- 分隔符下会与 2026-08-07 + x 撞键,改用 NUL 分隔后仍正确分桶(同月同 agent 合并、不同 agent 分离、总计正确)。

Reviewer 请注意

本 PR 的 diff 在 GitHub 上仍会显示为二进制。 这不是修复失败 —— git 只要 diff 任意一侧是二进制就走二进制路径,而旧 blob 确实含 NUL。把两侧分开检测就清楚了:

新内容(空 -> 文件):  1 file changed, 258 insertions(+)   ← 文本
旧内容(HEAD):        Bin 0 -> 9679 bytes                 ← 二进制

想看真实 diff 用 git diff --text,只有两行变化。合并之后该文件的 git diff / git blame 恢复正常。

配套的 #29 加了 .gitattributes,让这类 diff 即使跨 NUL blob 也强制按文本渲染(已实测对本 PR 的 blob 有效:Bin 9679 -> 9689 bytes4 ++--)。两个 PR 互相独立,但 #29 先合的话,本 PR 的 diff 就直接可读了。

🤖 Generated with Claude Code

`summarize()` 用 NUL 作 `month`/`agent_id` 复合键的分隔符,这个选择本身是对的
(NUL 不可能和 `YYYY-MM` 或任何 agent_id 冲突)。问题出在编码:两个 NUL 是以
**裸字节 0x00** 直接写进源文件的,而不是转义序列。

由于 NUL 落在文件前 8000 字节内,git 把整个文件判定为二进制,后果有两层:
- `git diff` 只显示 `Bin 9679 -> 10779 bytes`,`git blame` 完全失效,
  `file` 报 `data` —— 这个 bug 把自己从 review 里藏了起来(它是 #17 带进来的)
- 更糟的是**文件无法三方合并**:#23 在 main 上也改了这个文件,于是本分支
  第一版直接 CONFLICTING,git 对二进制文件没有合并策略可用

改动:
- 两处裸 0x00 换成 `\u0000` 转义 —— 注释里的复合键说明,以及 `cellKey` 模板字符串
- 运行时字符串完全不变:模板字面量里的 `\u0000` 求值仍是 U+0000,
  `getWorkspaceUsage` / `getPlatformUsage` 的分组逐字节一致

验证:
- 文件 NUL 计数 0;`file` 现在报 `Unicode text, UTF-8 text`
- 全仓受跟踪文件扫描,没有第二个文件有同样问题
- `npm test` 678/678 通过;`usage-ledger.test.js` 单独跑 29/29
- 对抗性分组检查:agent_id `07-x` 在 `-` 分隔符下会与 `2026-08-07`/`x` 撞键,
  用 NUL 分隔后仍正确分桶(同月同 agent 合并、不同 agent 分离、总计正确)

已 rebase 到当前 main(含 #23 对本文件的改动),冲突按「取 main 版本 + 重新施加
同一处转义」解决。

注意:本次 diff 对 HEAD 仍显示 `Bin`,因为 git 只要有一侧是二进制就走二进制路径,
而旧 blob 确实含 NUL。合并后该文件的 `git diff` / `git blame` 恢复正常。
#29 的 `.gitattributes` 会让这类 diff 即使跨 NUL blob 也强制按文本渲染。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oratis
oratis force-pushed the claude/fervent-spence-5f680f branch from dbd331e to a744855 Compare August 9, 2026 15:13
oratis added a commit that referenced this pull request Aug 9, 2026
…binary (#30)

The compound cache key in summarize() used a raw 0x00 byte written
directly into the source — a literal control character, not the escape
sequence. Git classifies a file with a NUL in its first 8000 bytes as
binary, so `git diff` reported "Bin 9679 -> 10779 bytes" instead of the
change, `git blame` stopped working, and `file` reported `data` rather
than UTF-8 text.

Worth noting how it got in: I merged it in #17, and the reason nobody
caught it is the bug itself — the file was undiffable, so review had
nothing to look at.

The separator choice is fine: NUL cannot collide with a YYYY-MM month or
any agent_id. Only the encoding was wrong. Both occurrences are now the
escape sequence, which evaluates to the same U+0000 character, so
getWorkspaceUsage / getPlatformUsage group byte-identically.

Verified: 0 NUL bytes remain, `file` reports UTF-8 text, the template
still evaluates to a string containing U+0000, and an end-to-end
aggregation still separates same-month agents correctly. npm test
678/678.

Found by the background session on PR #27, which was branched before #23
and would have reverted that PR's documentation on this file; applied
directly to main instead.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
oratis added a commit that referenced this pull request Aug 9, 2026
… review (#29)

#27 修了 `server/usage-ledger.js` 里两个裸 `0x00`(复合键分隔符本该写成
`\u0000` 转义)。真正的教训不在那两个字节,而在于**它是怎么活到 merge 的**:
NUL 一旦落进文件前 8000 字节,git 就把整个文件判成二进制,`git diff` 退化成
`Bin N -> M bytes`,`git blame` 完全失效 —— 这个 bug 把自己藏起来了。

它也不需要谁疏忽。任何一个会吞反斜杠转义的管线都足以产生它(这次修复过程中,
同样的转换在临时脚本和 commit message 草稿里各发生过一次)。所以值得做结构性
兜底,而不是只修实例。

改动:
- 新增 `.gitattributes`:给所有源码 / 文档扩展名加 `diff` 属性。git 文档明确
  说明该属性让路径「即使包含正常文本里绝不出现的字节值(如 NUL)也按文本处理」
- 同时加 `* text=auto` 做行尾规范(当前 283 个受跟踪文件 100% 已是 LF)
- 常见二进制资产(图片 / 字体 / pdf)显式标 `binary`,避免误规范化
- `docs/memory.md` §6「值得记住的失效模式」补第 4 条

验证:
- A/B 实测该属性确实生效,对 #27 那个含 NUL 的 blob:
    无 .gitattributes:  server/usage-ledger.js | Bin 9679 -> 9689 bytes
    有 .gitattributes:  server/usage-ledger.js | 4 ++--  (2 insertions, 2 deletions)
- `git add --renormalize .` 未产生任何暂存变更 —— `text=auto` 不动现有文件
- `npm test` 678/678 通过

如果本 PR 先于 #27 合入,#27 的 diff 在 GitHub 上就会直接显示为可读文本。

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@oratis

oratis commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #30, which applies the same two-byte fix on current main.

This branch was cut before #23 landed, so merging it as-is would have reverted that PR's documentation block on server/usage-ledger.js — the file-level diff looked clean only because git was still treating the file as binary.

The find itself was right and the write-up was the useful part: the reason nobody caught the raw NUL in #17 is the bug itself, since an undiffable file gives review nothing to look at. #29 turns that into a systemic guard (.gitattributes forces textual diffs), which is the half a one-file fix can't cover. Thanks — both merged.

@oratis oratis closed this Aug 9, 2026
oratis added a commit that referenced this pull request Aug 9, 2026
…nded (#31)

Two defects found while reviewing the #23#30 batch, both now on main.

**Wrong PR credited.** #29's .gitattributes header and memory.md §6 both
say #27 fixed the usage-ledger NUL bytes. #27 was closed as a duplicate —
#30 landed the identical fix (verified byte-for-byte identical trees).
Anyone following the reference lands on a closed PR with an empty diff.

**The count treadmill.** #26 corrected five hardcoded test counts from
234/377/656 to the then-accurate 678. #28 merged minutes later and made it
679, so a PR whose entire purpose was de-staling docs shipped a number that
was stale on arrival. Five copies of a figure that changes on every
test-bearing merge cannot stay right.

Counts are now stated as a magnitude pinned to a commit ("~680 as of
c7c7d5b") with an instruction to measure instead of cite, and the two
places that only needed "all green" no longer carry a number at all. The
prose that never had one ("Vitest files under client/src/{...}") aged fine
through this whole batch, which is the argument.

memory.md §5.2 records the pattern so the next doc pass doesn't reinstate
it. The remaining 234/656 mentions are deliberate — they are the history
being explained, not live facts.

Verified: 679/679 serialized on this branch; grep confirms no stale
hardcoded count or bare #27 reference survives.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant