Brain summary poll parses every brain record body every 30s to render three counts

[atomantic]

Problem

GET /api/brain/summary (server/routes/brainSettings.js:76-79) calls brainStorage.getSummary() (server/services/brainStorage.js:1278-1310), which runs getAll() on seven entity stores (people, projects, ideas, admin, memories, links, buckets) plus getInboxLogCounts() (brainStorage.js:853-873, itself getAll('inbox')). getAll → loadRawMap (brainStorage.js:250-258) lists the type directory and loadOnes every record body (readFile + JSON.parse per data/brain/<type>/<id>/index.json). All of that is thrown away except .length, one status field, and isGitHubRepo.

Brain already has the cheap primitive for exactly this question: the per-record projection index (createRecordIndex / resolveRecordIndex, brainStorage.js:313-379) that #3508 (listLiveIds, :395) and #3509 (link summary index, :1047-1114) introduced so a store the process has already walked answers from memory with zero body reads. getSummary and getInboxLogCounts never got moved onto it.

Trigger

client/src/pages/Brain.jsx:43-51 — useAutoRefetch(fetchData, 30_000) fetches /brain/summary on every Brain page mount and every 30 s while the page is visible, on every tab (Inbox, Links, Daily Log, Graph, …). The result renders three numbers in the header (needsReview, links, projects, people) and a "last digest" time.

Impact

Per poll: 8 readdir + N lstat + N readFile + N JSON.parse, where N is every record (including tombstones) across the eight stores. A single ChatGPT import puts ~1,400 memories records on disk with up to 9,800 chars of content each (server/services/chatgptImport.js:27), so one poll parses on the order of 14 MB of JSON; at 120 polls/hour that is ~1.6 GB parsed per hour of having the Brain page open, to display counts that change a few times a day. The same walk runs concurrently with whatever the active tab is loading (Links limit: 5000, Inbox, Graph context), so the tab's own requests queue behind it on the same disk.

Fix

Add a summary projection index in server/services/brainStorage.js and answer both counters from it — no body reads in steady state.

1. Next to projectLinkSummary (:1047), add

   const projectSummary = (record) => (record && !isTombstone(record)
     ? { status: record.status, isGitHubRepo: record.isGitHubRepo }
     : null);
   const summaryIndex = createRecordIndex(projectSummary);

   createRecordIndex self-registers for invalidation (recordIndexes, :306), so every existing freshness signal (${type}:upserted / ${type}:deleted / record:changed, wired at :411-415) and invalidateAllCaches() (:1262) already cover it — nothing new to wire. A null projection = "not a user-visible record", exactly what getAll drops (missing, unparseable, tombstoned), so counts stay identical.
2. Rewrite getSummary() to Promise.all a resolveRecordIndex(summaryIndex, type, 0) per type, keep only non-null rows, and derive the same fields with the same predicates it uses today: counts.<type> = live rows (archived records are counted today via getAll — keep that), activeProjects = status === 'active', activeIdeas = !status || status === 'active', openAdmin = status === 'open', gitHubRepos = truthy isGitHubRepo, needsReview = inbox needs_review. loadMeta() stays as is.
3. Rewrite getInboxLogCounts() to tally status over the same index for 'inbox' (same seven keys; unknown statuses still only count toward total). Its other callers — GET /api/brain/inbox (server/routes/brainCapture.js:40), jobGates.brainReviewGate (server/services/jobGates.js:22) — need no change.

Steady-state cost becomes 8 readdir + N lstat (the listIds() membership check the #3508/#3509 indexes already accept) and zero reads/parses.

Rejected: caching the whole summary object with a TTL — a 30 s poll against a 2 s cache still walks every body 30× an hour, and a longer TTL makes the "needs review" badge lag captures; the projection index is invalidated per record and is the pattern the file already uses. Rejected: counting from listIds().length alone — can't distinguish tombstones or answer the status splits.

Files: server/services/brainStorage.js, server/services/brainStorage.test.js.

Tests (in brainStorage.test.js, beside describe('getLinksPage …', :438), using the same temp-dir store and the readJSONFile spy the #3508 test at :386 uses):

• getSummary over a seeded store (live + archived + tombstoned + one unparseable record per type, mixed statuses, one GitHub link) returns the same object as the current implementation.
• a second getSummary call reads zero record bodies (spy call count unchanged).
• update(type, id, { status: 'done' }) and remove(type, id) are reflected on the next call (per-id invalidation).
• getInboxLogCounts after remove('inbox', id) drops the tombstone from total and its status bucket.

Acceptance criteria

• GET /api/brain/summary returns a byte-identical shape for the same data before/after (fields, predicates, archived-included counts).
• A repeat getSummary() / getInboxLogCounts() call with no writes in between performs no readFile of any data/brain/<type>/<id>/index.json (asserted by test).
• Creating, updating (status flip), deleting, and a peer-applied applyRemoteRecord change are each reflected on the next call.
• server/services/brainStorage.test.js covers the four cases above; cd server && npm test green.