Skip to content

Optimize think tag parser with early-exit lastIndexOf and cursor indexing - #1219

Open
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:perf-streaming-think-tag-parser
Open

Optimize think tag parser with early-exit lastIndexOf and cursor indexing#1219
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:perf-streaming-think-tag-parser

Conversation

@nordicnode

Copy link
Copy Markdown

Optimize think tag parser with early-exit lastIndexOf and cursor indexing

Summary

• In cli/src/utils/think-tag-parser.ts, optimize streaming think-tag detection and segment parsing.
• Previously, getPartialTagLength was called on every streamed token chunk and sequentially tested 12 hardcoded prefix strings across PARTIAL_CLOSE_PREFIXES and PARTIAL_OPEN_PREFIXES with text.endsWith(prefix). Replaced this with an $O(1)$ check using text.lastIndexOf('<'). If < does not appear within the last 7 characters of the string (since the longest partial prefix is '</think', length 7), it returns 0 immediately without evaluating prefix arrays. If < is found in the trailing window, it verifies the prefix against the constant tags.
• Measured getPartialTagLength:

  • 20,000 evaluations on 5,000-char string: 10.12 ms down to 2.91 ms (3.47x faster).
  • 100,000 realistic stream chunk batches: 84.74 ms total execution time.
    • In parseThinkTags, replaced continuous remaining = remaining.slice(...) string allocations with an index cursor let cursor = 0 to scan tags directly on the source string without allocating intermediate substrings.
    • Removed unused PARTIAL_OPEN_PREFIXES and PARTIAL_CLOSE_PREFIXES arrays.
    • Verified 100% behavioral and edge-case parity against all 22 existing unit tests.

Test plan

[✓] bun test --config=/dev/null src/utils/__tests__/think-tag-parser.test.ts — 22 pass, 0 fail
[✓] bun run --cwd cli typecheck — 0 errors
[✓] PR hygiene check passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant