perf(highlight): stop re-parsing huge lines on every frame (fixes #2838)

[sinelaw]

Fixes #2838. Independent of the wrapped-line navigation PRs (#2784, #2813) — this touches only highlight_engine.rs and branches straight off master.

Problem

Viewing a 400+ KB single-line JSON pins a CPU core. Any input event — a keystroke, or just mouse motion over the window — triggers a re-render, and every render re-parsed the entire line.

Root cause

The highlight cache only advanced its commit point on newline-terminated lines. That rule guards streaming buffers, whose trailing partial line may still grow: resuming from a state captured mid-line loses the diff background (the dark-bar artefact inside + lines). A file that is one huge line with no trailing newline — a minified JSON one-liner — therefore never committed anything, so every frame re-parsed from scratch. Checkpoints are newline-aligned too, so there were none inside the line to resume from either.

Changes

• A parse that reaches the buffer end commits its trailing partial line. The streaming hazard cannot fire through it: both cache-consuming paths in highlight_viewport already require last_buffer_len == buffer.len() and no pending edit, so a grown buffer never resumes from that state. Checkpoints stay newline-aligned — find_parse_resume_point is not length-gated, so a mid-line checkpoint could still be chosen after growth.
• Lines longer than MAX_PARSE_BYTES have only the viewport-overlapping window parsed, from a clone of the parse state. Past that size the parse range is already a viewport window, so the EOF commit cannot apply and bounded-but-approximate beats unbounded. Below it the EOF commit parses such a line once with exact colours.

Measurements

On the issue's attached file at 80x24, per input event at 2 Hz so the editor is not saturated (at higher rates CPU saturates near 100% regardless and hides the difference):

	before	after
mouse / scroll	~480 ms/event	~150 ms/event
typing (edits)	~468 ms/event	~152 ms/event

Remaining cost is the wrap pipeline, not highlighting.

Testing

The guard being relaxed had no behavioural test, only one asserting the commit boundary. This PR adds test_streaming_partial_diff_line_keeps_bg_when_rest_arrives, which pins the rendered outcome: when the rest of a partially-streamed + line arrives, every byte of the completed line still carries the diff background. It passes both before and after the change, so it validates the guard's purpose rather than its mechanism. The old boundary assertion is replaced by its EOF-commit counterpart.

Full lib suite (3076), plus the syntax (145), highlight (191), and diff (114) e2e suites pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J3uiv3CFZEAFARmTkqxpaQ

---

Generated by Claude Code