perf(agent): skip session JSONL sanitize when file is unchanged - #3977
perf(agent): skip session JSONL sanitize when file is unchanged#3977arnohillen wants to merge 5 commits into
Conversation
Every reconnect re-runs sanitizeSessionJsonl, which reads and JSON-parses every line of the native Claude transcript. For long sessions that file reaches tens of MB, adding seconds to each resume. Remember the file stat of the last clean pass and skip the read when it matches. The SDK only appends, which changes size and mtime, so any real change forces a re-parse; staleness only ever causes extra work, not a skipped heal. Generated-By: PostHog Code Task-Id: 5d24ea17-aec0-4334-884e-c2639867e260
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
|
Reviews (1): Last reviewed commit: "perf(agent): skip session JSONL sanitize..." | Re-trigger Greptile |
Same swap as the conversation caches in #3976: lru-cache v11 is already a workspace dependency, so the hand-rolled capped Map goes away. Generated-By: PostHog Code Task-Id: 5d24ea17-aec0-4334-884e-c2639867e260
The healed path recorded the stat of the renamed file, taken after the rename, so bytes appended by a concurrent writer in that window would have been certified clean and skipped by every later pass. Record the tmp file's stat instead: rename preserves it, and any post-rename append already mismatches it, forcing a re-parse. Generated-By: PostHog Code Task-Id: 47cdc119-1c24-42ec-ac36-74c43cdd7f4e
Not run in CI (test globs only match *.test.*). Numbers land in the PR description; rerun with pnpm vitest bench in packages/agent. Generated-By: PostHog Code Task-Id: 47cdc119-1c24-42ec-ac36-74c43cdd7f4e
Picks up the vr-cli install fix from #4029 so the visual-regression check passes. Generated-By: PostHog Code Task-Id: 58678630-1e4b-4565-9de0-c74c91b9d995
|
Hey @arnohillen, we've migrated this repo into the PostHog/posthog monorepo, so this PR needs to be recreated there. Easiest path: check out the monorepo and run the Shout if you get stuck. |
|
@charlesvien ported to the monorepo as PostHog/posthog#76791, via the /porting-code-prs skill (original commits and authorship preserved with git am, lockfile change re-derived against the monorepo lockfile). Fine to close this one once the port merges. |
Problem
Part 2 of the task re-open latency work (see #3976). Every agent reconnect runs
sanitizeSessionJsonlover the native Claude transcript (~/.claude/projects/.../<sessionId>.jsonl): a full read plus aJSON.parseper line. These files grow to tens of MB on long sessions, so every re-open of a cold task pays seconds of parse work before the session can resume, even when the file was already verified clean moments earlier.Changes
lru-cache, already a workspace dependency), and return early when the current stat matches.How did you test this?
jsonl-hydrationsuite: 66 tests passing.Automatic notifications
Created with PostHog Code