Context
PR #75 fixed a bug in buildCompiledArticle (tools/compile.go:585-589) where the history summary truncation used byte-slicing (summary[:80]) instead of rune-slicing. When a multibyte UTF-8 character (e.g. em-dash U+2014, 3 bytes) spanned the byte-80 boundary, the slice cut the character mid-encoding, producing invalid UTF-8 (e.g. 0xE2 0x80 0x2E).
The fix converts to []rune before slicing:
if runes := []rune(summary); len(runes) > 80 {
summary = string(runes[:80]) + "..."
}
## What's needed
A regression test that:
- Constructs a Cluster with a learning whose Content contains a multibyte character positioned so it would span byte offset 80 under the old byte-slicing approach (e.g. an em-dash at rune position 79-80, or a string of 79 ASCII chars followed by an em-dash)
- Calls buildCompiledArticle and extracts the history table row
- Asserts the output is valid UTF-8 (utf8.ValidString)
- Asserts the summary ends with "..." and does not contain a corrupted byte sequence
- Ideally uses a table-driven approach covering multiple multibyte scenarios: em-dash (3-byte), CJK character (3-byte), emoji (4-byte), and a string that is exactly 80 runes (no truncation needed)
Convention pack reference
TC-006 MUST: Bug fixes MUST include a regression test that reproduces the original failure and verifies the fix. This was deferred from PR #75 to a follow-up issue.
Acceptance criteria
- Test exists in tools/compile_test.go (or a new tools/compile_truncation_test.go)
- Test fails if the fix is reverted to byte-slicing (summary[:80])
- Test passes with the current rune-slicing implementation
- go test -race -count=1 ./tools/... passes
- Valid UTF-8 output is asserted for all multibyte boundary cases
Context
PR #75 fixed a bug in
buildCompiledArticle(tools/compile.go:585-589) where the history summary truncation used byte-slicing (summary[:80]) instead of rune-slicing. When a multibyte UTF-8 character (e.g. em-dash U+2014, 3 bytes) spanned the byte-80 boundary, the slice cut the character mid-encoding, producing invalid UTF-8 (e.g.0xE2 0x80 0x2E).The fix converts to
[]runebefore slicing:## What's needed
A regression test that:
Convention pack reference
TC-006 MUST: Bug fixes MUST include a regression test that reproduces the original failure and verifies the fix. This was deferred from PR #75 to a follow-up issue.
Acceptance criteria