[Performance] Session switch re-runs a full eager transcript backfill every time (no cross-session cache); cost scales with session length

[andrewreid]

[Performance] Session switch re-runs a full eager transcript backfill every time (no cross-session cache); cost scales with session length

Summary

Switching between sessions in the web/desktop UI — including switching back
to a session just viewed — takes ~5-10s. Each open rebuilds the transcript from
scratch via a sequential, capped eager backfill. There is no per-session
transcript cache, so the "and back again" case pays the full cost a second time.
The server pod stays idle throughout — it's a client-side sequential-round-trip
waterfall, not server load.

Mechanism

No cross-session cache. The chat store holds a single conversation. On
switch it resets blocks: [] and bumps historyGeneration
(web/src/store/chatStore.ts:1587,2529), so returning to a prior session
reloads from zero.

Eager backfill spine. The turn rail wants INITIAL_TURNS = 20 user-turn
ticks (web/src/pages/TurnRail.tsx:33,199) and fires
loadHistoryUntilUserMessages(20). That loop
(web/src/store/chatStore.ts:1887-1935) pages up to
MAX_EAGER_PAGES=10 × EAGER_PAGE_LIMIT=200, sequentially (each page
awaits the previous — cursor chain), counting only non-system user turns.

Why it maxes out on our sessions. Autonomous / tool-heavy sessions have huge
tool + reasoning + sub-agent item volume and sparse real user turns. 2000
items back rarely contains 20 user turns, so the loop hits the 10-page cap on
every open ⇒ 10 serial round-trips. Each page is keyset-paginated and fast
on its own (WHERE position > cursor … LIMIT n,
omnigent/stores/conversation_store/sqlalchemy_store.py:1630), but serial ⇒ a
multi-second floor that grows with how tool-heavy the session is.

This is the concrete mechanism behind the earlier "longer-history sessions get
slower to open" observation.

Evidence

A single isolated session open = ~63 requests over ~9s, of which the transcript
backfill is 14× /items (1×limit=20 first paint + 10×limit=200 eager
backfill + a few cursor pages). Server/pod CPU flat the whole time.

Secondary, panel-conditional: when the Files panel is mounted, its folder
tree (useWorkspaceAllFiles, PAGE_SIZE=1000,
web/src/shell/FilesPanel.tsx:305) lists ~38 directories per open, each a
runner-proxied round-trip (many 404). This adds to the open cost only when that
panel is open; the transcript backfill fires unconditionally.

Suggested fix directions

1. Cache transcript blocks per session across switches — biggest win;
   eliminates the "switch back" re-backfill entirely.
2. Relax the eager backfill for sparse-user sessions — the 10×200 cap scans
   up to 2000 items just to surface 20 rail ticks. Consider a smaller initial
   reveal with on-demand rail fill, or deriving early ticks without eagerly
   materializing every intervening item.
3. Files tree: list directories lazily on expand rather than probing many
   directories on open.

Impact

UX / annoyance, but scales with session length and so hits long autonomous
sessions hardest. No data-correctness effect.

Found via server-log + web-client + codebase correlation.