feat: v0.6.0 — experimental compaction markers, user memories, unified notes, plugin message bus, doctor CLI, TUI improvements, cache-bust fix

Author: ualtinok

ARCHITECTURE.md

@@ -124,6 +124,21 @@
 - Location: `src/features/magic-context/dreamer/queue.ts`, `src/features/magic-context/dreamer/lease.ts`, `src/features/magic-context/dreamer/storage-dream-state.ts`
 - Pattern: SQLite-backed queue plus cooperative lease lock.
 
+**User memory pipeline:**
+- Purpose: Extract user behavioral observations from historian output, collect candidates, and promote recurring patterns to stable user memories.
+- Location: `src/features/magic-context/user-memory/storage-user-memory.ts`, `src/features/magic-context/user-memory/review-user-memories.ts`
+- Pattern: Historian extracts candidates, dreamer reviews and promotes, system prompt injects stable memories.
+
+**Plugin message bus:**
+- Purpose: Enable asynchronous communication between the TUI plugin and server plugin via SQLite.
+- Location: `src/features/magic-context/plugin-messages.ts`
+- Pattern: SQLite-backed message queue with direction, type, and payload columns; consumed atomically.
+
+**Compaction markers:**
+- Purpose: Inject OpenCode-compatible compaction boundaries into the message table so `filterCompacted` stops at historian's last compartment boundary.
+- Location: `src/hooks/magic-context/compaction-marker.ts`, `src/hooks/magic-context/compaction-marker-manager.ts`
+- Pattern: Write summary/compaction rows into OpenCode's DB after historian publishes; filter them out from raw reads.
+
 **Agent prompt pack:**
 - Purpose: Keep hidden-agent identities and prompt text isolated from runtime wiring.
 - Location: `src/agents/dreamer.ts`, `src/agents/historian.ts`, `src/agents/sidekick.ts`, `src/agents/magic-context-prompt.ts`

CONFIGURATION.md

@@ -266,6 +266,29 @@ It is useful when starting a new session. It's better to choose a fast and cheap
 
 ---
 
+## Experimental Features
+
+### `experimental_compaction_markers`
+
+| Key | Type | Default |
+|-----|------|---------|
+| `experimental.compaction_markers` | `boolean` | `false` |
+
+When enabled, after historian publishes compartments a compaction boundary is injected into OpenCode's message/part tables. This causes `filterCompacted` to stop at the boundary, dramatically reducing the number of messages processed per transform pass.
+
+### `experimental_user_memories`
+
+| Key | Type | Default |
+|-----|------|---------|
+| `experimental.user_memories.enabled` | `boolean` | `false` |
+| `experimental.user_memories.promotion_threshold` | `number` | `3` |
+
+When enabled, historian extracts behavioral observations about the user alongside compartments. These are stored as candidates and reviewed by dreamer during scheduled runs. Recurring patterns that appear across multiple historian runs are promoted to stable user memories, injected into all sessions via `<user-profile>` in the system prompt.
+
+**Requires dreamer to be enabled.** Without dreamer, candidates accumulate but are never promoted to stable memories. The `doctor` command warns about this misconfiguration.
+
+- `promotion_threshold`: minimum number of semantically similar candidate observations before dreamer considers promoting to a stable memory (2–20, default 3).
+
 ## Commands
 
 | Command | Description |
@@ -323,6 +346,14 @@ It is useful when starting a new session. It's better to choose a fast and cheap
     "model": "github-copilot/gpt-5.4",
     "fallback_models": ["anthropic/claude-sonnet-4-6"],
     "timeout_ms": 30000
+  },
+
+  "experimental": {
+    "compaction_markers": false,
+    "user_memories": {
+      "enabled": false,
+      "promotion_threshold": 3
+    }
   }
 }
 ```

README.md

@@ -39,6 +39,12 @@ Your agent should never stop working to manage its own context. Magic Context is
 
 **5.** TUI sidebar with live context breakdown, token usage, historian status, and memory counts — right inside the terminal.
 
+### 🧪 Experimental
+
+**Compaction Markers** — after historian compresses older messages, Magic Context injects a boundary into OpenCode's message table. OpenCode's built-in message loader stops at the boundary, so the transform processes only the live tail instead of the full session history. Dramatically reduces per-turn overhead on long sessions. Enable with `experimental.compaction_markers: true`.
+
+**User Memories** — historian extracts behavioral observations about you alongside its normal compartment output (communication style, expertise level, review focus, working patterns). Recurring observations are promoted by the dreamer to stable user memories that appear in all sessions via `<user-profile>`. Enable with `experimental.user_memories.enabled: true` (requires dreamer).
+
 ---
 
 ## Get Started
@@ -253,11 +259,20 @@ An optional background agent that maintains memory quality overnight:
 - **Improve** — rewrite verbose memories into terse operational form
 - **Maintain docs** — update ARCHITECTURE.md and STRUCTURE.md from codebase changes
 - **Evaluate smart notes** — check pending smart note conditions and surface ready notes
+- **Review user memories** (experimental) — promote recurring user behavior observations to stable memories
 
 The dreamer runs during a configurable schedule window and creates ephemeral OpenCode child sessions for each task. Since it runs during idle time (typically overnight), it works well with local models — even slower ones like `ollama/mlx-qwen3.5-27b-claude-4.6-opus-reasoning-distilled` are fine since there's no user waiting.
 
 When dreamer is enabled, ARCHITECTURE.md and STRUCTURE.md are automatically injected into the agent's system prompt (configurable via `inject_docs`). Content is cached per-session and refreshed on cache-busting passes.
 
+### User Memories (Experimental)
+
+When `experimental.user_memories.enabled` is set, historian extracts behavioral observations about the user alongside its normal compartment output — things like communication style, expertise level, review focus, and working patterns. These go into a candidate pool.
+
+During dreamer runs, a dedicated review pass checks candidates for recurring patterns across sessions. Observations that appear at least `promotion_threshold` times (default 3) are promoted to stable user memories and injected into all sessions via `<user-profile>` in the system prompt.
+
+Stable user memories are visible and manageable in the dashboard's User Memories page. Requires dreamer to be enabled for the promotion step.
+
 ### TUI Sidebar
 
 When running in OpenCode's terminal UI, Magic Context shows a live sidebar panel with:
@@ -322,9 +337,8 @@ All durable states live in a local SQLite database. If the database can't be ope
 | `source_contents` | Raw content snapshots for persisted reductions |
 | `compartments` | Historian-produced structured history blocks |
 | `session_facts` | Categorized durable facts from historian runs |
-| `session_notes` | Session-scoped `ctx_note` content |
+| `notes` | Unified session notes and project-scoped smart notes |
 | `session_meta` | Per-session state — usage, nudge flags, anchors |
-| `smart_notes` | Project-scoped smart notes with surface conditions |
 | `memories` | Cross-session persistent memories |
 | `memory_embeddings` | Embedding vectors for semantic search |
 | `dream_state` | Dreamer lease locking and task progress |
@@ -335,6 +349,8 @@ All durable states live in a local SQLite database. If the database can't be ope
 | `message_history_index` | Tracks last indexed ordinal per session for incremental FTS population |
 | `recomp_compartments` | Staging for `/ctx-recomp` partial progress |
 | `recomp_facts` | Staging for `/ctx-recomp` partial progress |
+| `user_memory_candidates` | User behavior observations from historian (experimental) |
+| `user_memories` | Promoted stable user memories injected into all sessions (experimental) |
 
 ---
 

STRUCTURE.md

@@ -51,7 +51,7 @@
 **`src/features/`:**
 - Purpose: Group reusable subsystem logic by feature.
 - Contains: Magic-context services, dreamer runtime, sidekick support, storage, scheduler, tagger, and built-in commands.
-- Key files: `src/features/magic-context/storage-db.ts`, `src/features/magic-context/storage-meta-persisted.ts`, `src/features/magic-context/dreamer/runner.ts`, `src/features/magic-context/memory/storage-memory.ts`, `src/features/builtin-commands/commands.ts`
+- Key files: `src/features/magic-context/storage-db.ts`, `src/features/magic-context/storage-meta-persisted.ts`, `src/features/magic-context/dreamer/runner.ts`, `src/features/magic-context/memory/storage-memory.ts`, `src/features/magic-context/user-memory/storage-user-memory.ts`, `src/features/builtin-commands/commands.ts`
 
 **`src/tools/`:**
 - Purpose: Define the agent-facing tool surface.

packages/dashboard/src-tauri/src/commands.rs

@@ -106,12 +106,22 @@ pub fn get_session_facts(
 pub fn get_session_notes(
     state: State<'_, AppState>,
     session_id: String,
-) -> Result<Vec<db::SessionNote>, String> {
+) -> Result<Vec<db::Note>, String> {
     let path = state.get_db_path()?;
     let conn = db::open_readonly(&path).map_err(|e| e.to_string())?;
     db::get_session_notes(&conn, &session_id).map_err(|e| e.to_string())
 }
 
+#[tauri::command]
+pub fn get_smart_notes(
+    state: State<'_, AppState>,
+    project_path: String,
+) -> Result<Vec<db::Note>, String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readonly(&path).map_err(|e| e.to_string())?;
+    db::get_smart_notes(&conn, &project_path).map_err(|e| e.to_string())
+}
+
 #[tauri::command]
 pub fn get_session_meta(
     state: State<'_, AppState>,
@@ -349,6 +359,57 @@ pub async fn test_embedding_endpoint(
     }
 }
 
+// ── User Memory commands ────────────────────────────────────
+
+#[tauri::command]
+pub fn get_user_memories(
+    state: State<'_, AppState>,
+    status: Option<String>,
+) -> Result<Vec<db::UserMemory>, String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readonly(&path).map_err(|e| e.to_string())?;
+    db::get_user_memories(&conn, status.as_deref()).map_err(|e| e.to_string())
+}
+
+#[tauri::command]
+pub fn get_user_memory_candidates(
+    state: State<'_, AppState>,
+) -> Result<Vec<db::UserMemoryCandidate>, String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readonly(&path).map_err(|e| e.to_string())?;
+    db::get_user_memory_candidates(&conn).map_err(|e| e.to_string())
+}
+
+#[tauri::command]
+pub fn dismiss_user_memory(
+    state: State<'_, AppState>,
+    id: i64,
+) -> Result<(), String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readwrite(&path).map_err(|e| e.to_string())?;
+    db::dismiss_user_memory(&conn, id).map_err(|e| e.to_string())
+}
+
+#[tauri::command]
+pub fn delete_user_memory(
+    state: State<'_, AppState>,
+    id: i64,
+) -> Result<(), String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readwrite(&path).map_err(|e| e.to_string())?;
+    db::delete_user_memory(&conn, id).map_err(|e| e.to_string())
+}
+
+#[tauri::command]
+pub fn delete_user_memory_candidate(
+    state: State<'_, AppState>,
+    id: i64,
+) -> Result<(), String> {
+    let path = state.get_db_path()?;
+    let conn = db::open_readwrite(&path).map_err(|e| e.to_string())?;
+    db::delete_user_memory_candidate(&conn, id).map_err(|e| e.to_string())
+}
+
 // ── Health commands ─────────────────────────────────────────
 
 #[tauri::command]