feat(sps): Live Notes with schedule and email event triggers#841
Closed
saxster wants to merge 536 commits into
Closed
feat(sps): Live Notes with schedule and email event triggers#841saxster wants to merge 536 commits into
saxster wants to merge 536 commits into
Conversation
…us/kill-switch
Phase 2: make Hermes' (already-built) inbound Telegram control safe, scoped, and
visible. Locked decision: read/info ONLY to start (no terminal/file/computer-use).
The security core — toolset scoping:
- Telegram's DEFAULT toolset (`hermes-telegram`) includes terminal + file +
cronjob + messaging, so an unscoped Telegram agent can mutate the machine
(gated only by the approval prompt). PR2 writes `platform_toolsets.telegram`
to a read/info-only list (web, x_search, browser, vision, memory,
session_search, clarify) — research + info, no machine mutation.
- New pure, tested helpers in tools.ts: upsertPlatformToolsets / readPlatformToolsets
(surgical config.yaml edits that preserve comments + other sections),
getTelegramScope ("read-info"|"broad"|"custom"), setTelegramReadInfoScope
(backs up config.yaml first, then restarts the gateway). 7 unit tests + a
dry-run against the real config confirmed cli/known_plugin_toolsets/approvals
stay intact and exactly the telegram block is added.
- approvals.mode is already `manual` (the dangerous-command gate is on).
Surfacing — the wizard gains a manage/status mode:
- On open, if Telegram is already connected it shows status: capability scope
(Read & info only ✓ / Broad ⚠ with a "Lock to read/info only" button), a
Re-pair action, a Two-Step-Verification reminder, and a "Disable remote
control" KILL SWITCH (disables the platform + restarts). Setup now applies the
read/info scope automatically (saveToken). Reachable via "Manage Telegram" in
the Scheduled modal + the wizard's Done step.
IPC: telegram-get-scope / telegram-set-read-info-scope (config.ts) + preload
parity. Verified: typecheck (both), eslint, build, verify:note-index, vitest 1563
passed/3 skipped (+ tools.test.ts), real-config dry-run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…yle) Skills were half-wired: SKILL.md content read IPC and the slash-command system both existed, but only skill NAMES reached the model (buildSelfAwarenessSystemMessage). The model knew a skill existed but never saw its instructions. This closes the loop in both chat surfaces. - Main: src/main/active-skills.ts — per-profile in-memory active set, load/unload/list + buildActiveSkillsSystemMessage (reads getSkillContent, empty-store fast path before profile resolution, defensive try/catch, soft-cap warning). Injected at the front of sendMessageViaApi (Hermes Chat) and after SYSTEM_PROMPT in buildSpsAssistantMessages (SPS assistant). - IPC: load-skill-to-chat / unload-skill-from-chat / list-active-skills, mirrored in both preload bridge + d.ts (parity test enforced). - Renderer: shared useChatSkills hook (slug resolution, IPC, state) + ActiveSkillChips. Hermes Chat gets dynamic /<skill> menu entries, /skill and /unload commands, and chips. SPS composer gets its own slash menu + chips. Built-in commands are never shadowed by a skill slug. Injects instructions only — it does NOT execute skill main.py (that stays the Python backend's job); the injected body still helps the agent delegate. Verification: typecheck (both) + eslint touched + vitest (1586 pass, +23 new) + verify:note-index + build all green. scripts/probe-skill-slash.mjs drives the real IPC round trip through a built Electron app (8/8: discovery, load, slug dedup, unload). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Shared types for the External Context Bridge (ExternalSource, conversation/ search/status/progress models, formatProvenance) and the single index-time redaction choke point: pattern-based secret matching (provider keys, GitHub/ AWS/Slack/Google tokens, JWTs, PEM blocks, key=value heuristic) plus an exact known-secrets layer mirroring StreamRedactor's >8-char guard. Redaction runs before any write to the index so a leaked secret never lands in messages or messages_fts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…grok Pure (vitest-safe, no electron/sqlite) tolerant parsers for the four local AI-tool transcript formats: - claude-code: per-session JSONL, cwd/gitBranch per line, text+thinking blocks - codex: handles BOTH the 2026 enveloped (session_meta + clean event_msg user_message/agent_message) and the 2025 bare-message generations - gemini: whole-file JSON (replace strategy); projectHash→real path via sha256 match against history/<slug>/.project_root - grok: append JSONL; project path from url-decoded dir, uuid conversation id Shared helpers (streaming JSONL reader with byte-offset cursors + prefix sniff, whole-file reader, text-block extraction that drops tool_use/ tool_result/thinking, 4000-char cap) and pure decideFileAction cursor logic (skip / tail-append / reparse-on-truncate-or-replace). Fixtures + adapter and scan-logic tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ExternalContextDb — a derived, rebuildable better-sqlite3 FTS5 index over external transcripts. applyFragments is the SINGLE writer and redacts every message at the choke point before INSERT, so secrets never reach messages or messages_fts. Schema: conversations / messages / messages_fts (porter) / files (incremental byte-offset cursors) + schema_version drop-and-rebuild. Reads: provenance-joined search (source/project filters), windowed getConversation, listProjects, per-source stats. Orchestrator: lazy machine-global singleton (HERMES_HOME/external-context.db), single-flight chunked scan with setImmediate yields and progress events, decideFileAction-driven skip/tail-append/reparse, vanished-cursor cleanup. verify:external-context (ELECTRON_RUN_AS_NODE) proves per-source counts, FTS + provenance, index-time redaction (seeded sk-ant key + known secret absent from messages AND messages_fts; FTS cannot retrieve them), incremental append delta, truncation reparse, and rebuild. Full suite: 1618 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Main-process IPC surface (get-config/set-source/status/scan/rebuild/search/ get-conversation/list-projects) plus app-level per-source toggles in desktop-store (default all OFF). Enabling a source backfills it immediately; disabling purges it. Known secrets (api-server key, remote bearer, env values >8 chars) are gathered per scan and handed to the redacting writer. App-start (+10s) and 15-min interval background scans, progress forwarded to the renderer. Registered in setupIPC; DB + timers torn down on before-quit. Preload bridge mirrored in index.ts and index.d.ts — preload-api-surface parity test green (135). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ExternalSessionsModal (ResearchModal-pattern): full-text search over the redacted external index with source/project filters and provenance lines; a read-only conversation viewer that opens with an explicit UNTRUSTED banner and renders every message as escaped plain text (never markdown, never auto-injected); and a Sources settings view with per-source on/off toggles, availability + counts, live scan progress, and Scan/Rebuild. Opens disabled sources first when nothing is enabled. Mounted in Overlays; reachable from a CommandPalette action. ui slice gains externalSessionsOpen. Note: the Save-to-KB store action (commit/undo) lands with commit 6; commit 5 keeps read paths as direct hermesAPI calls, matching ResearchModal precedent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Distill an external AI-tool session into ONE durable wiki page (decision brief: ## Decisions / ## Constraints / ## Open questions / ## Sources where the sole citation is a provenance line, no URLs). buildExternalSessionFile- Messages fences the transcript with the buildGroundingMessage untrusted wording so the synthesis pass treats it as data, never commands. spsExternalSaveToKb mirrors spsFileResearch (remote guard, schema, related pages, retry-once, parseChangeset). IPC assembles provenance + the already- redacted transcript (24k head+tail cap) from the index. Renderer slice commits via the same ingest path (commitChangeset → wiki log → select + one-click undo) and a Save-to-KB button in the viewer. +2 ingest tests asserting the untrusted fence; full suite 1620 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
external-context-server.ts → resources/external-context-mcp.cjs (esbuild,
better-sqlite3 external). Self-contained read-only SQL — does NOT import the
main db.ts — opens the machine-global index read-only via
HERMES_EXTERNAL_CONTEXT_DB. Tools: list_external_sources,
search_external_context{query,source?,project?,limit≤50},
read_external_conversation{conversationId,around?,limit≤100}. EVERY response
opens with an untrusted banner and wraps excerpts in <external_transcripts>
fences with per-hit provenance; 2000-char per-message cap.
ensureExternalContextMcpRegistered (twin of ensureResearchMcpRegistered) +
externalContextMcpServerPath; 'Expose to Hermes agent' button in the modal.
verify:external-context now also drives a real stdio JSON-RPC roundtrip
asserting the untrusted banner + fence and that the redacted key never
surfaces. Full suite 1620 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
external-context-smoke.mjs drives the BUILT app through hermetic seeded source roots (HERMES_EC_*_ROOT) — opens the modal from the palette, enables Claude Code (real backfill), searches, and opens the untrusted viewer. The smoke caught a redaction leak: conversation titles are derived from the first (secret-bearing) message and were stored UNREDACTED, surfacing the raw key in the viewer header, search hits and Save-to-KB. Fixed at the choke point — mergeConversation now redacts the title too; verify:external-context asserts titles never leak the sk-ant key. CLAUDE.md documents the bridge + its two structural invariants (index-time redaction, untrusted fencing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…han N days) Bounds backfill cost (the Codex tree alone is ~1.4 GB). A single pure seam: decideFileAction(file, record, olderThanMs?) skips files whose mtime predates the cutoff, threaded from scanExternalSources. App-level config get/setExternalContextMaxAgeDays in desktop-store (mirrors the source-toggle pattern), an external-context-set-max-age IPC channel that rebuilds+rescans so tightening the window also DROPS now-excluded sessions, ExternalIndexStatus gains maxAgeDays, preload bridge + d.ts (parity green), and an All time / 1y / 90 / 30 / 7-day dropdown in the modal Sources view. +3 scan-logic tests; full suite 1623 green; verify:external-context green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ExternalContextDb.listConversationsSince(sinceMs, {source?, project?, limit?})
— conversations whose last_at >= the cutoff, newest first, with the same
source/project scoping as search(). This is the query the weekly digest run is
built on. Extracted convMetaFromRow to share the row→meta mapping with
getConversationMeta. verify:external-context asserts the window + source
scoping against the fixtures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Model the weekly digest as a new schedule KIND reusing the Scheduled-Research
machinery. shared/scheduledResearch: kind?('research'|'digest', absent⇒research
for back-compat) + scope? + a pure periodStart(cadence,now) helper; digest
topic is optional in validateScheduleInput. sps-ingest gains
buildExternalDigestMergeMessages — smart-merge the period's sessions into a
living page (## Highlights / ## Decisions / ## Sources provenance-only-no-URLs /
## Updates), digest source fenced untrusted. mergeBriefAndQueue is generalized
to take a pluggable merge-messages builder (research default unchanged) so
research + digest share one pending/hash/notify path. +ingest tests; full
suite 181 files green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runDigest queries the period's external sessions from the redacted index (periodStart→listConversationsSince, scope-aware), assembles a provenance- labelled, per-message-capped source, and smart-merges it into the living digest page via the generalized mergeBriefAndQueue (passing the digest merge-builder) → review-first pending. runSchedule dispatches by kind; tick() and triggerScheduleNow route through it. createSchedule stamps kind/scope, defaults the digest topic/page, and SKIPS the paired gateway cron for digests (local data → app-open isDue fallback only). +periodStart unit tests; full suite 1627 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ExternalSessionsModal Sources view gains a '+ Weekly digest' button (srCreate kind:digest → opens the Scheduled modal) below the new date-filter dropdown. ScheduledModal renders digest schedules with a 'Digest' chip and an external- sessions/source label instead of a topic, and always shows 'app-open only' (digests have no paired cron). srCreate already takes the shared ScheduleInput (now carrying kind/scope) — no preload change. Typecheck/lint/build green; full suite 1627; UI smoke 4/4 with both new controls rendering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/digest-dogfood.mjs — drives the full digest journey against a throwaway HERMES_HOME + local stub gateway (only the LLM is mocked): enable source → +Weekly digest → Run now → pending → Apply → page, plus empty-period and gateway-failure modes. Findings in docs/external-digest-dogfood-2026-06-10.md: feature works end-to-end (0 Critical/High); 1 Medium (research-flavored copy leaks into the digest UX) + 3 Low. Report only — no feature code changed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modal served both research + digests but was titled 'Scheduled research' and a digest's empty run flashed the research-only 'No new info this run'. Retitle to 'Scheduled'; surface the run's own summary on no-change so a digest shows 'No external sessions this period' (and dedupe shows 'No meaningful change') instead of the generic line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…arch (dogfood fathah#3) Add 'digest' to WikiLogOp (+ the two preload unions); ScheduledModal.onApply looks up the originating schedule's kind and logs 'digest' for a digest pending instead of the hardcoded 'research'. +formatWikiLogLine test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…athah#4) The digest was hardcoded weekly/all-tools. The External Sessions Sources view now offers a cadence (Daily/Weekly/Monthly) + optional source dropdown next to a '+ Digest' button; createDigest builds the scope and encodes it into the topic so distinct-scope digests get distinct pages (and surfaces the 'already exists' error). Also hardens scripts/digest-dogfood.mjs (Sources-chip select + backfill wait + run-outcome toast capture) — re-dogfooded all four fixes end-to-end (title, 'No external sessions this period' toast, 'digest' wiki-log op, cadence/scope controls). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(P1.1) The old health poll self-cancelled the instant the gateway first reported healthy, so a *hang* after startup (process alive, /health unresponsive) was never re-detected — chat would silently stall with no recovery short of an app restart. Replace it with a permanent 30s supervisor (local mode only, while a gateway is started): - Pure decision state machine in hermes/gateway-supervisor.ts (vitest-tested, 9 cases): N consecutive failures -> kill + restart with exponential backoff (bounded attempts) -> a persistent visible "down" state. Never restarts underneath an open interactive stream (activeChatAborts.size>0). - Effects live in gateway-process.ts; index.ts injects the renderer broadcaster and the stream-open provider (no Electron dep leaks into the supervisor). - New push event gateway-health-changed + pull gateway-health-status; preload bridge + d.ts parity; useGatewayHealth hook; Gateway screen surfaces recovering/down. Shared status vocab in src/shared/gateway.ts. Stream-stall reality-check: req.setTimeout(120000) already aborts a stalled stream into a chat-error (Node request timeout is a socket-inactivity timer), so that half of the item was already satisfied — left as-is. Gate: typecheck x2, eslint touched, vitest 1637 pass (+9), verify:note-index, build — all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…P1.2)
The routine lock was a bare existsSync on /tmp/hermes-routine-<id>.lock that
wrote the PID but never read it: a job that crashed mid-run orphaned the file
and was then skipped FOREVER — the "system silently does nothing" PKM-killer.
- Pure decision logic in scheduler-lock.ts (vitest, 9 cases): acquire / steal
(dead-pid | stale) / blocked, with injectable liveness. Tolerant record parse
(legacy bare-PID lock => startedAt 0 => immediately stealable, so upgrades
never wedge a job behind a pre-upgrade lock).
- scheduler.ts: lock moves to <HERMES_HOME>/locks/<id>.lock as JSON {pid,startedAt};
acquisition steals when process.kill(pid,0) shows the owner dead or the lock
outlives JOB_TIMEOUT_MS (15m). A reap timer SIGKILLs a wedged child, releases
the lock and resolves false. Per-job skip telemetry persisted to
scheduler-skips.json, cleared on healthy acquisition, exposed via new
get-scheduler-skips IPC (Scheduled-modal surfacing lands in P2.2).
- launchd-autonomy.test.ts updated for the new semantics (the old tests asserted
the /tmp + bare-PID implementation 1.2 replaces) + a new dead-lock-steal
integration test.
Gate: typecheck x2, eslint touched, vitest 1647 pass, verify:note-index, build
— all green. (Lone vitest "error" is pre-existing jsdom getContext noise from
GraphView, untouched here.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SSH-remote API key cached in gateway-process.ts had a setter but no clear, so a key fetched for one SSH host lingered in memory forever and would be sent to a different host after the user switched connections. - Export clearSshRemoteApiKey() — clears the key and invalidates the readiness cache (apiServerAvailable=null). - Call it on every connection-mode change (set-connection-config when mode differs), on every new SSH target (set-ssh-config), and on tunnel teardown (stop-ssh-tunnel). - vitest-safe unit proves getRemoteAuthHeader stops sending the bearer after clear (mocks ../config to ssh mode). Gate: typecheck x2, eslint touched, vitest 1649 pass, verify:note-index, build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sps-index-rebuild handler returned the new status but emitted nothing, so the
renderer's search / graph / backlink hooks kept showing stale results until some
unrelated action triggered a refetch.
- Handler now broadcasts sps-index-rebuilt {profile, status} after rebuild.
- Preload onSpsIndexRebuilt listener + d.ts parity.
- useNoteIndex: a shared useIndexRebuildVersion() signal wired into the deps of
useVaultQuery / useVaultGraph / useVaultBacklinks / useVaultSearch, so all four
refetch the moment the index is rebuilt — no reload needed.
Scope note: the mirror-write-failure COUNT half of 1.7 is deferred to 2.6, which
builds the Storage settings surface that displays it (per the plan).
Gate: typecheck x2, eslint touched, vitest 1649 pass, verify:note-index, build.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No structured logging existed; long-lived background paths used scattered console.* and the gateway stderr log grew unbounded across restarts. - New src/main/log.ts: dependency-free JSON-lines logger -> <HERMES_HOME>/logs/ desktop.log, size-rotated (5MB, keep 3). Pure formatLogLine / shouldRotate split out electron-free for vitest (5 cases). Console mirror in dev only. - rotateGatewayStderrIfLarge() rotates gateway-stderr.log at gateway start when it passes 10MB (keep 2). - Adopted in the gateway supervisor (health transitions + auto-restart) and the scheduler (lock steal + wedged-job reap) — the paths 1.1/1.2 added. The IPC envelope (1.3) will adopt log.error next. Gate: typecheck x2, eslint touched, vitest 1654 pass (clean run; the transient "1 failed" is the pre-existing GraphView getContext jsdom flake), verify:note-index, build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-existing errors across 17 files (none from the P1 stability work), cleared with proper fixes — not blanket disables or any→unknown blind casts: - Mechanical/durable: prefer-const (ForceSimulation), unused vars (ObsidianEditor caught-err, equity-smoke arg, vault-search import), explicit return types (control-server), require()→dynamic import (grounding, keeps the circular-dep laziness) / →vi.importActual (semantic-index.test). - semantic-index.ts: replaced 8 `any` with proper typed response interfaces (SemanticSearch/Rag/Graph/IndexResult) — a real improvement; consumers (sps-agent, the test) now type-check without casts. - Chat/* screens (slated for removal in 2.3): typed the council `any`s against the existing CouncilTurnMessage/ChatBubbleMessage types (added optional messageId, which the UI already read via `|| key` fallback). useChatActions: file-level disable of react-hooks/preserve-manual-memoization with justification — fixing the dep arrays would change recreation behaviour in soon-deleted code. - GraphView drag-pin: scoped disable of react-hooks/immutability — SimNode objects are ref-held mutable physics state (ForceSimulation mutates them every frame), so in-place mutation is intentional, not React state. eslint errors 63→0 (137 prettier *warnings* left untouched — a 136-file --fix sweep is out of scope for an error-cleanup). Gate: typecheck x2, eslint 0 errors, vitest 1654 pass, verify:note-index, build — all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist the owner-approved transformation plan to its documented home so any future session can execute Phases 2-5 from the approved source, not a summary. Includes all per-item inventory/files/accept criteria, the non-negotiable constraints, the explicitly-NOT-doing list, the dependency graph, and a live status section. Live progress is tracked in project memory, which points here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wrap every ipcMain.handle in src/main/ipc/safe-handle.ts's safeHandle:
on throw it emits one structured, redacted log line
(log.error("ipc", { channel, message, stack })) and rethrows a clean,
serializable Error carrying only the redacted message. Return shapes are
unchanged — successful results pass through untouched, so renderer hooks
keep their existing contracts (Electron already serializes thrown errors
down to their message across the IPC boundary, so collapsing to
new Error(message) changes nothing observable except that secrets are now
scrubbed).
- safeHandle's handler type is derived as Parameters<typeof
ipcMain.handle>[1], making it a true drop-in: all 235 typed handlers
assign with no `any` and no parameter-variance surprises.
- describeIpcError extracted as pure logic (electron-free) → 4 vitest
cases proving channel/message shaping + secret redaction in both the
message and the stack, reusing the canonical pattern-based
redactExternalText (sk-ant / GitHub PAT / AWS / JWT / PEM / key=value).
- registerDualHandler now routes through safeHandle, so all dual-mode
(local + SSH) handlers gain the envelope in one place.
- index.ts uncaughtException / unhandledRejection handlers upgraded from
console.error to structured + redacted log.error("main", ...).
- ipc-handlers.test.ts source-scanner regex taught about safeHandle so
the preload↔main channel-parity contract still holds.
Reality-check corrections vs the plan: all 235 handlers were already in
src/main/ipc/*.ts (none left in index.ts), and redactor.ts exposes only
StreamRedactor (known-secret stream) — the canonical free-text scrubber
is external-context/redact.ts's redactExternalText, which this reuses.
Accept: zero raw `ipcMain.handle(` outside safe-handle.ts; gate green
(typecheck×2, eslint touched 0/0, vitest 1658 pass, verify:note-index,
build).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rd (P1.5) Prerequisite gate for all Phase 3+ background writers (non-negotiable fathah#5): workspace.json is whole-blob last-write-wins, so a writer saving a payload derived from a stale revision would silently drop pages a concurrent writer committed in between. New src/main/sps-write-queue.ts (pure, electron-free, fs injected) closes that hole: - WorkspaceWriteQueue serializes every save per profile behind a promise chain — overlapping saves can't interleave their read-modify-write — and stamps a monotonic __rev. A save whose declared baseRev is older than the on-disk rev is reload-merged via mergeWorkspaces (additive union: incoming wins for the pages it carries, current fills every gap, current-only pages stay reachable in the nav tree) instead of blind-overwritten. "No page is ever lost" is the explicit guarantee. - Rolling backups: first save per session + every 50th, pruned to the newest 5 (selectBackupsToPrune), reusing spsBackupWorkspace. - 14 vitest cases prove interleaved-stale-base → no page lost, the generation guard, serialization of overlapping enqueues, failure without rev-advance (EACCES), and backup-on-first-save. Surfacing (accept criteria): - spsSave now returns SpsSaveResult { ok, error?, bytes, rev, merged, oversize? } instead of a bare boolean; the sps-save IPC + preload (bridge + index.d.ts) carry an optional baseRev and the richer result. - Renderer persistence.ts tracks the on-disk rev across saves and echoes it back as baseRev; saveWorkspace is now async and returns the outcome. - A failed save raises a persistent shell badge (SaveStatus.tsx, pinned bottom-left, distinct from the transient toast) plus a one-shot warn toast on the ok→failed transition; recovery clears it. A >25 MB blob fires a one-time "consider vault migration" advisory (reportSaveResult). The null reset path (clearWorkspace) bypasses the queue and drops the cached queue so the next save re-reads the on-disk revision fresh. Vault-mode saves keep their existing path (blob queue is blob-mode only). Gate green: typecheck×2, eslint touched 0/0, vitest 1672 pass, verify:note-index, build; sps-smoke 01-home/02-palette pass (02b/03 are the pre-existing collapsed-nav fresh-seed failures, not regressions). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…u is parity (P2.1) Reality-check (non-negotiable #1) collapsed this item from "port + delete" to just "delete": SPS You (screens/SpsAgent/you/YouSurface.tsx) is already a strict SUPERSET of the admin Personalization screen. Both call the identical IPC — readMemory / readFocus / writeFocus / writeUserProfile / writeMemory / getDailyContextHookStatus / setDailyContextHookEnabled — and You renders every one of Personalization's four controls (focus.md, USER.md, MEMORY.md, the daily-context hook toggle, byte-identical UI) plus a structured Rules layer and the Telos auditor on top. The plan's anticipated "missing pieces" (hook enable/disable, allowlist UI) were wrong: the hook toggle already lives in You, and neither screen ever managed the shell-hooks allowlist. So the accept criterion ("every capability reachable in You before delete") was already met. Removed (renderer-only — ALL personalization IPC stays, You depends on it): - screens/Personalization/Personalization.tsx (the screen). parts.tsx KEPT — You imports EditorSection/hookStatusText/HookStatus/MemoryFile from it. - Layout.tsx: import, the "personalization" nav item (+ now-unused UserCog icon), and the render pane. - openSettings.ts: "personalization" dropped from the AdminView union and the KNOWN_VIEWS deep-link allowlist. - navigation.ts: the navigation.personalization label key. Grepped "personalization" to zero outside the kept parts.tsx + descriptive comments. Admin overlay drops from N tabs to N-1; the Agents nav group keeps agents/skills/capabilityReview. Gate green: typecheck×2, eslint touched 0/0, vitest 1672 pass, build. verify-admin-overlay a1/a2 are a pre-existing cold-start visibility flake (identical on origin/main; a5-skills — same nav group — passes, proving the grouped nav renders). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…te admin Schedules (P2.2) The main-process scheduler fires headless (digests, scheduled research, agent cron jobs) regardless of UI — so deleting the admin Schedules screen without a replacement would leave the owner unable to SEE or STOP background work. Ported that oversight into the SPS Scheduled modal first. Port (ScheduledModal.tsx — new "Agent tasks" section): - Lists every cron job via the existing listCronJobs IPC with schedule, next/last-run, and last status. - Per-job pause/resume, run-now (triggerCronJob), and delete — every running job is now visible + stoppable from SPS (the accept criterion). - Surfaces the P1.2 skip telemetry (getSchedulerSkips): a job that's been skipped shows "⚠ skipped N× · <reason>" so a wedged/dead-lock job is visible, not silently dead. Delete (admin Schedules screen): - screens/Schedules/Schedules.tsx + Layout import / nav item / Timer icon / render pane + the "schedules" member of the AdminView union and KNOWN_VIEWS deep-link allowlist. Grepped "schedules" view refs to zero. Kept (per plan — IPC/modules outlive their UI): ALL cron IPC (list/create/pause/resume/remove/trigger) and src/main/cronjobs.ts — research schedules create cron jobs internally via createCronJob, and IPC outlives a deleted UI by a release. The SPS Scheduled modal is reached from the sidebar (Sidebar.tsx). Capability delta (deliberate, per the plan's oversight-only scope): the raw cron-job CREATION wizard is not ported — SPS already creates research/digest schedules, and arbitrary cron jobs remain an agent/CLI concern. The i18n schedules.* namespace is left orphaned (harmless) rather than pruned across 9 locales in this focused PR. Gate green: typecheck×2, eslint touched 0/0, vitest 1672 pass, build; sps-smoke 01-home/02-palette pass (02b/02c/03 pre-existing collapsed-nav fresh-seed fails, not regressions). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Data-loss / reliability fixes on the email monitor: - Track IMAP UIDVALIDITY per account/folder; re-baseline the cursor when the server reassigns UIDs instead of stalling that folder forever (HIGH-4). - Drain the OLDEST unseen messages first (slice(0, pollLimit)) so a backlog larger than pollLimit is caught up on later runs rather than skipped past the cursor (HIGH-3). - Isolate each folder poll in its own try/catch so a renamed/deleted folder no longer starves every folder listed after it in the run (MED-7). Triage correctness (src/shared/email-monitor.ts): - Check importance keywords BEFORE the bulk-mail gate so a genuine incident alert carrying Auto-Submitted headers is captured, not dropped — and the "without an important match" reason is now truthful (MED-2). - Match keywords on whole-word boundaries so "site" no longer fires on "website" and "incident" no longer fires on "coincidentally" (MED-3). Adds regression tests for both triage fixes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
fetchRssArticles and discoverSubstackFeed defaulted their fetcher to the raw global fetch, so a user-added feed URL (add/sync feed, or the sps-rss-discover- substack IPC) was fetched with no IP-pinning, no private-range block, and no per-redirect re-validation — a bypass of the load-bearing SSRF invariant (CLAUDE.md), including the DNS-rebinding shape the string-literal pre-check misses. Default both to the guarded publicFetch, and reject non-http(s) / explicit-local feed URLs at the DB-insert boundary as defense in depth. Existing tests inject their own fetcher, so behavior for callers is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
drainCronBriefs advanced lastDrainedAt to the newest file unconditionally after the loop, so a transient failure (gateway hiccup, parse error, vault read error) during a headless cron drain permanently lost that period's research brief — the file stayed on disk but the cursor had passed it and it was never re-read. Advance the cursor only across the contiguous prefix of successfully-handled briefs; a read/merge failure now holds the cursor so the brief is retried on the next run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
…failure - The 3AM Dream Cycle guard compared a UTC date (toISOString) against a local getHours() gate, double-firing the "once daily" cycle east of UTC+3 (e.g. IST) when the UTC day rolled over after 03:00 local. Key off a local calendar date. - A lockfile write failure was logged but the job ran anyway, silently dropping the cross-process/crash-recovery single-runner guarantee. Skip the run, record a skip so it's visible, and return false instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
Self-healing overwrote a skill/script with an LLM-generated patch and kept no copy of the original (the audit log records only the NEW content), so a wrong patch — or several across cron cycles — had no revert path back to the last human-written version. Write a timestamped .selfheal-bak-<ts> before patching and refuse to patch if that backup can't be written; record the backup path in the audit detail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
audit:ci was red on main: the allowlist was reset empty on 2026-07-04 while 12 advisories exist. All 12 are transitive via @excalidraw/excalidraw (whiteboard) and @wesbos/code-icons (icon font) — lodash-es/nanoid inside the diagram parser and esbuild/vite build tooling, none remotely reachable in a packaged desktop app. Document each as a reviewed residual so the required check job passes; the proper fix (major bump of both parents, needs UI regression testing) is a tracked follow-on, not part of this correctness pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
ingestCommitPage parsed the full page meta out of the proposal markdown via pageFromMarkdown but kept only `blocks`, silently discarding tags/aliases/source/ingestedAt/cover/custom-properties. OKF import carefully translates all of that into frontmatter, so every imported page collapsed to bare title + body. Merge the parsed meta into both the create and update paths (caller title stays the display title of record; makePageWithId's icon/source fallbacks preserved when frontmatter omits them). Body-only proposals are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
… auth The email engine was multi-account-shaped but inert: enabled was hardcoded false, nothing polled it, and every account fell back to the shared EMAIL_PASSWORD env key (account fathah#2+ silently read account #1's password). Slice 1 of the "useful email agent" build (HIGH-5): - Scheduler now polls enabled IMAP accounts on the tick, throttled to ~5m and skipped entirely when no account is active (emailMonitorHasActiveAccount). - configFromEnv respects an explicit EMAIL_MONITOR_ENABLED=true opt-in so the env-derived account can actually run (GUI toggle comes in Slice 2). - normalizeEmailMonitorConfig assigns each keyless account a distinct passwordEnvKey (first stays EMAIL_PASSWORD for back-compat; additional accounts get EMAIL_PASSWORD_<ID>), fixing the multi-account auth collision. +7 unit tests (per-account key defaulting + active-account predicate). App-password/IMAP path per product decision; OAuth deferred. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
The email IPC (get/save-config, status, run-now, feedback) was already wired end-to-end; only the renderer account manager was missing, so the GUI couldn't even switch an account on. Adds a full account editor to the Sources tab: add/remove accounts, edit connection fields (label, address, IMAP host/port, password env-var), and an enable toggle. Edits stage locally (emailDirty) and persist as a whole config through the existing spsEmailMonitorSaveConfig on "Save changes" — the explicit-Save model per the product decision. New rows carry an empty id so normalize derives the id from the address and a distinct per-account password key; the sender-rule feedback block is guarded to saved accounts. +3 InboxSurface tests (enable-toggle+save, add+edit+save, remove+save); existing email test updated for the editable label field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQcUN3ukQMkagfWoY59pYX
Layer a gateway LLM classification over the deterministic keyword rules in shared/email-monitor.ts, mirroring task-triage.ts's gatewayChat pattern. - New src/main/email-triage.ts: the rules stay a cheap offline pre-filter; the gateway is only consulted for BORDERLINE mail — the "no strong signal" verdict the rules capture under the neutral "archive" label. Every decisive verdict (block/ignore/allow/important/bulk) is authoritative and never hits the gateway, keeping cost ~0 for the common case. - Produces real triageLabel + triageReason + confidence (extends, does not replace, the rule fields the capture card already carries). - Wires the previously-dead captureThreshold: it governs ONLY the uncertain lane — a borderline capture below the account bar is dropped to a skip, while an explicit allow/importance rule is never thresholded away. - Degradable: gateway down / timeout / garbage response falls back to the rule verdict; triageEmailCandidate never throws, so a poll is never hard-failed by classification. - Email body is fenced as untrusted data in the prompt (classify, don't obey) to blunt prompt injection from adversarial senders. handleParsedEmailMessage now awaits triageEmailCandidate and threads `profile` through the poll chain so the gateway URL/auth resolves per profile. Tests: email-triage.test.ts (borderline routing, threshold, parse coercion, decisive-no-gateway), email-triage-network.test.ts (gateway-down degrade), plus email-monitor.test.ts integration (borderline → LLM capture, threshold drop). Full gate green: both typechecks, lint, vitest (33 new pass; only the 3 known-baseline fails remain), electron-vite build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012VszUhmUc9efEb47rK6UHC
…Slice 4) Surface the triage the engine already computes and close the feedback loop: - Capture cards now show a triage label chip (mapped onto the existing SPS chip variants, confidence as tooltip) and the triage reason as a muted sub-line. - Per-capture "triage is wrong" flag button opens the four feedback actions; account + sender resolve from new emailAccountId/emailFrom capture frontmatter (label-lookup fallback for pre-Slice-4 captures). A sender-scoped raise-priority allowlists the sender (per-sender priority tiers documented as the upgrade path). - Newsletter digest lane: per-account digestBulk toggle (default off, staged via the existing Save-changes flow) captures bulk mail with digest: true instead of skipping it; digest verdicts are decisive rule outcomes and never consult the LLM. The inbox folds digest rows into one collapsible "Newsletters (N)" card; normal rows keep the virtualized list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
…gration
- electron-security: the updater warning is log.warn("updater", {...}) now,
not the old console literal — assert the structured call.
- obsidian-config: the hoisted electron mock factory hit a TDZ on
safeStorageMock (module-load import of installer/paths); hoist the mock +
its state with vi.hoisted and give the mock an app.getPath stub.
- remote-mode-url-and-spawn: cronjobs logs via log.error, whose dev mirror
is console.error("[cronjobs]", payload) — assert that shape.
Test-only; the full vitest suite is now fully green (2714 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
The missing recovery path. A snapshot copies the three authoritative artifacts — workspace.json, vault/**/*.md, and _manifest.json — into <profileHome>/sps-agent/backups/<epochMs>/; the derived .note-index.db is never included (WAL-mode, rebuildable). Restore replaces the artifacts, drops the in-memory write queue (so a late autosave can't clobber the restored blob), rebuilds the note index, and the Settings panel reloads the renderer so the store rehydrates. - src/main/sps-backups.ts: path-parameterized snapshot/restore cores (vitest-testable) + profile wrappers with keep-10 retention and an epoch-id traversal guard. Restore takes a safety snapshot of the current state first and refuses to proceed without it (self-healing doctrine: a still-broken state beats an unrecoverable one). Only markdown is touched — external-vault assets, .obsidian, and dot-files are ignored. - IPC sps-list-backups / sps-create-backup / sps-restore-backup + preload parity (bridges/sps.ts + sps.types.ts). - Auto-snapshot before destructive ops: toggleStorageMode (refuses on explicit snapshot failure), resetWorkspace, and bulk commitChangeset. - "Workspace Backups" section in Settings (dataPrivacy tab): list, create-now, restore with confirm. - Tests: snapshot/restore round-trip, backups-dir recursion guard, retention pruning; verify:note-index gains a restore→rebuild→searchable proof. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
…en one (MED-8) The blob autosave subscriber mirrored only s.page, so background pages written by ingestCommitPage / makePageWithId for arbitrary ids (OKF import, wiki-ingest, land reports) never got a vault/<id>.md until manually opened — invisible to search(), links(), and backlinks(). Replace the single-page mirror with a diff-mirror: a module-level mirroredDocs/mirroredMeta map records what the markdown mirror last saw; each blob-mode save exports exactly the pages whose docs[id]/meta[id] reference changed (Zustand immutability makes reference inequality reliable) and deletes mirror files for ids gone from docs (trash keeps its docs, so this only fires on permanent deletes/resets). Seeded from the hydrate-time mirrorAllPages pass so the first post-hydrate edit diffs correctly. Also updates the ingestCommitPage comment that previously (incorrectly) claimed autosave mirrored committed pages. Tests: jsdom store tests prove a background page is exported without being opened, unchanged pages are not re-exported, and removed pages get their mirror file deleted. Golden serializer tests unchanged; full gate green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
…D-10) Durable diagnostics with nothing leaving the machine (owner decision: local-only, no telemetry). - src/main/log.ts grows an errors-only JSONL sink at <HERMES_HOME>/logs/hermes-errors.jsonl: every log.error/log.warn record also lands there, size-capped (2 MB, keep 2) via the existing rotation, and scrubbed of obvious secret shapes (bearer tokens, sk-/ghp-style keys, password/token/apiKey JSON fields — including escaped-quote nested JSON) before hitting disk. The whole sink is best-effort: logging can never crash a caller. - Uncaught failures are captured in src/main/index.ts: process uncaughtException/unhandledRejection and Electron render-process-gone/child-process-gone all route into the sink. - "Diagnostics" section in Settings (troubleshooting tab): open logs folder, view/copy the last errors, clear log. IPC system-open-logs / system-read-error-log / system-clear-error-log + preload parity. Tests: sink append (error/warn in, info out), secret scrubbing on-disk, tail limit, clear; preload parity green; full gate green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
…4 note) - A1: remove the write-only EMAIL_SMTP_SERVER gateway field (no consumer) — field object, platform-fields entry, and its two i18n keys; the email platform description now says "Receive via IMAP". - A2: drop the never-produced "document-folder" SpsImportSource union member (only markdown-folder and okf-bundle are ever produced or handled). - A5 (security): the control server's three token comparisons (Bearer control token, ?token=, calendar ?feedToken=) now go through a constant-time compare — sha256 both sides then crypto.timingSafeEqual, so no length-mismatch early return leaks timing. Helper lives in src/main/security.ts with unit tests. Localhost-bound, so low severity, but a real fix. - A3: consolidate frontmatter parsing onto shared/sps-frontmatter — the 4 CRLF-intolerant bare-\n regexes (equity basket/report/calibration contracts, deck-studio) now use splitSpsFrontmatter, and sps-ingest/vault-health import parseYamlFrontmatterMarkdown instead of local copies. The editor JSON-scalar vs index-YAML value grammar split is documented as intentional (deferred substrate decision), not changed. - A4: control-server's ~13-endpoint multiplexing documented as a deliberate deferral in a header note — structural observation, not a bug. Golden serializer + equity/deck contract tests green; verify:note-index green; full gate green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
New repo-shipped skill (.claude/skills/hermes-dev-workflow/) teaching the working method for this codebase: problem decomposition, a verification ladder keyed by change type, git/worktree discipline, output structure, and an incident-derived pitfall catalog (PITFALLS.md). CLAUDE.md rebalanced: keeps facts + one-line invariants, gains a load-the-skill-first mandate; expanded symptom/fix detail relocated to PITFALLS.md. Dropped the stale sps-agent-prototype/ reference (directory no longer exists). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017GnYkUbq8cDgFMuhuVYtAU
…verrides (MED-9) The plan assumed a major bump of @excalidraw/excalidraw and @wesbos/code-icons would clear the allowlisted advisories, but 0.18.1 and 1.2.4 are the LATEST published versions of both — npm's only "fix" was a downgrade (0.17.6 / 1.1.5). The advisories are all in transitive subtrees with patched versions available, so they are remediated with package.json overrides instead, no parent downgrades and no API migration: - lodash-es -> ^4.18.1 (high code-injection + prototype-pollution CVEs, reached via excalidraw -> mermaid-to-excalidraw -> langium/chevrotain) - nanoid@4.0.2 -> ^5.1.16 under mermaid-to-excalidraw (predictable-id CVE; excalidraw's own nanoid 3.3.12 was already patched by an earlier override) - vite -> ^7.3.6 under @wesbos/code-icons (which ships vite 4 as a PROD dependency), dropping the vulnerable vite 4.5 + esbuild 0.18 subtree — incl. two high advisories. Stale nested lock entries purged so the override reifies. npm audit: 12 -> 0. security/npm-audit-allowlist.json is now empty by design (notes updated); audit:ci green against the empty baseline. Regression guards, written before touching deps: a code-icons API test for the WorktreePanel icon source (new) + the existing excalidraw golden markdown round-trip — both green pre- and post-override. Full gate green (typechecks, vitest 2735, verify:note-index, build, 29/29 UI smoke). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1L9UNugWU18aHuR2bAQfD
Add self-updating SPS pages (registry + pending→ingestCommitPage path for blob-mode safety). Owners set an objective and optional cron/window/email match rules; timed ticks and post-capture email fan-out rewrite the page body via Hermes gateway. UI: Live… control on DocHeader + LiveNotePanel.
Contributor
|
Too many files changed for review. ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sps-agent/live-notes.json).live-notes/pending/then rendereringestCommitPage(same pattern as scheduled research — no raw vaultwriteFilefor page bodies).How to try
Test plan
npm run typecheck(node + web)npx vitest runfor liveNotes / live-notes / pendingDrain / preload-api-surface (211 tests)Non-goals (v1)
live:YAML frontmatter, gardener, entity extraction, app-closed email/apply, tool-using agent loop