feat(core): add LivePack mutable overlay

LIVE_KBS_MVP.md

@@ -0,0 +1,38 @@
+# Live KBs MVP
+
+## Summary
+
+- Add `LivePack` as a deterministic mutable overlay on top of an immutable mounted base pack.
+- Keep the existing flattened postings index, Cortex, and `query()` pipeline unchanged in phase 1.
+- Persist with full-snapshot `serialize()` only; no mutable postings map, patch-pack format, or mutation log yet.
+
+## Implementation
+
+- Add a new `packages/core/src/live.ts` module with `createLivePack()` and `LivePack`.
+- Make live mutations require stable doc ids; the initial `docs` array follows the same rule.
+- `addDocument()` inserts or replaces, `updateDocument()` merges partial fields onto the last known full doc and shadows any base copy, and `removeDocument()` tombstones the id while deleting any overlay copy.
+- Keep internal state as a read-only `base`, an overlay `Map` of docs, a `Set` of tombstones, a rebuilt `delta` pack, and only the deterministic build knobs that can be recomputed from docs alone.
+- Rebuild only the overlay pack after each mutation with the existing `buildPack()` and `mountPack()` primitives.
+- Query base and delta separately with the existing lexical/graph options, collapse duplicate ids with delta winning, and sort final hits by `score desc`, `source/docId asc`, then `blockId asc`.
+- Return the same `Hit[]` shape as `query()` so downstream code can swap in live packs without adapter changes.
+- Keep live querying lexical/graph-only in v1; semantic build options are rejected for live packs until we add a mutation-time embedding story.
+- Make `serialize()` materialize the merged live state in stable id order and return a normal `.knolo` snapshot.
+- Export `LivePack`, `createLivePack()`, and `LivePackOptions` from `packages/core/src/index.ts`.
+- Update the root README and `packages/core/README.md` to document `LivePack`, clarify that Cortex remains a separate append-only memory layer, and keep `knolo dev` as the watch/rebuild workflow instead of adding `build --watch` now.
+- Leave `packages/core/src/indexer.ts`, `packages/core/src/query.ts`, and the CLI command behavior unchanged for this phase.
+
+## Test Plan
+
+- Add a `node:test` suite under `packages/core/test` for the live pack lifecycle.
+- Cover adding a doc, updating a base doc, removing a base doc, re-adding a removed id, query merging, and `serialize()` round-tripping through `mountPack()`.
+- Verify repeated `serialize()` calls on the same live state are byte-identical.
+- Verify deterministic merge order for equal-score hits and delta-over-base precedence.
+- Verify missing ids and updates to unknown ids fail fast.
+
+## Assumptions
+
+- Phase 1 is lexical/graph-only; semantic live updates are deferred to a later `consolidate()` or embedding-aware follow-up.
+- Anonymous blocks in an existing base pack stay queryable but are read-only for live mutations.
+- Rebuilding the overlay pack on every mutation is the intended MVP tradeoff for incremental evidence and docs-scale writes.
+- Patch packs, mutation logs, source-watcher APIs, and `build --watch` stay out of scope for this phase.
+- No Rust/ICP changes are needed because the persisted output remains a standard `.knolo` pack.

README.md

@@ -4,7 +4,7 @@ Knolo is a **local-first knowledge base engine** built around deterministic retr
 
 It provides:
 
-* `@knolo/core` — pack format + deterministic retrieval engine
+* `@knolo/core` — pack format + deterministic retrieval engine, LivePack overlay, and Cortex memory layer
 * `@knolo/cli` — build workflows for `.knolo` artifacts
 * `create-knolo-app` — instant Next.js starter with playground
 * `@knolo/langchain` — LangChain-style retriever adapter
@@ -64,6 +64,8 @@ npm run knolo:build
 npm run dev
 ```
 
+For pack workflows, `knolo dev` is the watch/rebuild loop for configured sources. We are keeping that workflow instead of adding a separate `build --watch` command in this phase.
+
 Open:
 
 ```
@@ -89,16 +91,42 @@ Knolo is:
 
 You build `.knolo` packs once.
 You mount them anywhere — Node, web, React Native, offline.
+When you need a deterministic mutable overlay on top of a mounted pack, use `LivePack`.
 
 Retrieval is lexical-first and deterministic by default.
 
 Hybrid semantic reranking is optional and **never replaces lexical grounding**.
 
+# 🧪 Live KBs MVP
+
+`LivePack` is the phase-1 mutable overlay for mounted packs. The base pack stays immutable, live docs are keyed by stable ids, and `serialize()` returns a standard `.knolo` snapshot.
+
+For the rollout plan, implementation notes, and test matrix, see [`LIVE_KBS_MVP.md`](./LIVE_KBS_MVP.md).
+
+```ts
+import { createLivePack, mountPack } from '@knolo/core';
+
+const base = await mountPack({ src: './dist/knowledge.knolo' });
+const live = await createLivePack(base, [
+  { id: 'notes.alpha', text: 'alpha note', namespace: 'notes' },
+]);
+
+await live.updateDocument({ id: 'notes.alpha', text: 'alpha note v2' });
+await live.removeDocument('notes.alpha');
+await live.addDocument({ id: 'notes.alpha', text: 'alpha note restored' });
+
+const snapshot = await live.serialize();
+const rebuilt = await mountPack({ src: snapshot });
+```
+
+`LivePack` stays lexical/graph-only in v1. Cortex remains a separate append-only memory layer.
+
 ---
 
 # 🧠 Knolo Cortex
 
 Knolo Cortex adds a local-first overlay memory layer on top of `@knolo/core`.
+It is separate from `LivePack`: Cortex is append-only memory, while LivePack is a mutable document overlay for mounted packs.
 
 It gives you:
 

packages/core/README.md

@@ -28,8 +28,10 @@ It is:
 
 * A versioned binary pack format
 * A deterministic lexical retrieval engine
+* A deterministic `LivePack` overlay for mounted packs
 * An optional semantic rerank layer
 * A portable knowledge runtime
+* A separate append-only Cortex memory layer
 
 You build once.
 You mount anywhere — Node, browser, React Native, serverless, offline.
@@ -147,6 +149,51 @@ Properties:
 * Namespace-aware
 * Evaluation-friendly scoring
 
+For iterative pack builds, use `knolo dev` as the watch/rebuild workflow. We are keeping that flow instead of introducing `build --watch` in this phase.
+
+---
+
+## 4️⃣ LivePack Overlay
+
+`LivePack` is a deterministic mutable overlay on top of a mounted base pack.
+
+It is phase-1 lexical/graph-only. Stable doc ids are required for the initial `docs` array and for every live mutation, and semantic live updates are rejected until the embedding story exists.
+
+Construction accepts `LivePackOptions` for graph settings such as `maxEdgesPerDoc`, but semantic live options stay disabled in v1.
+
+It is designed for document-style live updates:
+
+* `addDocument()` inserts or replaces a live doc by stable id
+* `updateDocument()` merges partial fields onto the last known full doc and shadows any base copy
+* `removeDocument()` tombstones a doc id and hides the base copy
+* `query()` returns the same `Hit[]` shape as `query(pack, ...)`
+* `serialize()` materializes the merged live state as a normal `.knolo` snapshot
+* repeated `serialize()` calls on the same state are byte-identical
+
+Live querying in v1 stays lexical/graph-only.
+Semantic build or query options are rejected until live embeddings are added.
+
+```ts
+import { createLivePack, mountPack, query } from '@knolo/core';
+
+const base = await mountPack({ src: './dist/knowledge.knolo' });
+const live = await createLivePack(base, [
+  { id: 'notes.alpha', text: 'alpha note', namespace: 'notes' },
+]);
+
+await live.addDocument({ id: 'notes.beta', text: 'beta note' });
+await live.updateDocument({ id: 'notes.alpha', text: 'alpha note v2' });
+await live.removeDocument('notes.beta');
+await live.addDocument({ id: 'notes.beta', text: 'beta note restored' });
+
+const hits = live.query('alpha note', { topK: 5 });
+const snapshot = await live.serialize();
+const rebuilt = await mountPack({ src: snapshot });
+const roundTripHits = query(rebuilt, 'beta note', { topK: 5 });
+```
+
+For the phase-1 rollout notes and test matrix, see [`../../LIVE_KBS_MVP.md`](../../LIVE_KBS_MVP.md).
+
 ---
 
 # 🔀 Optional: Hybrid Semantic Rerank

packages/core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@knolo/core",
-  "version": "3.2.2",
+  "version": "3.2.4",
   "type": "module",
   "description": "Local-first knowledge packs for small LLMs.",
   "keywords": [
@@ -35,8 +35,8 @@
     "prepublishOnly": "npm run build",
     "smoke": "node scripts/smoke.mjs",
     "test": "npm run build && node scripts/check-runtime-no-node.mjs && node --test test/*.test.mjs && node scripts/test.mjs",
-    "format": "prettier --write src/agent.ts src/pack.ts src/pack.runtime.ts src/pack.node.ts src/node.ts src/builder.ts src/index.ts scripts/test.mjs scripts/check-runtime-no-node.mjs ../../README.md README.md",
-    "format:check": "prettier --check src/agent.ts src/pack.ts src/pack.runtime.ts src/pack.node.ts src/node.ts src/builder.ts src/index.ts scripts/test.mjs scripts/check-runtime-no-node.mjs ../../README.md README.md",
+    "format": "prettier --write src/agent.ts src/pack.ts src/pack.runtime.ts src/pack.node.ts src/node.ts src/builder.ts src/live.ts src/index.ts scripts/test.mjs scripts/check-runtime-no-node.mjs ../../README.md README.md",
+    "format:check": "prettier --check src/agent.ts src/pack.ts src/pack.runtime.ts src/pack.node.ts src/node.ts src/builder.ts src/live.ts src/index.ts scripts/test.mjs scripts/check-runtime-no-node.mjs ../../README.md README.md",
     "check:runtime-no-node": "node scripts/check-runtime-no-node.mjs"
   },
   "devDependencies": {

packages/core/src/index.ts

@@ -8,6 +8,7 @@ export {
 } from './query.js';
 export { makeContextPatch } from './patch.js';
 export { buildPack } from './builder.js';
+export { LivePack, createLivePack } from './live.js';
 export {
   quantizeEmbeddingInt8L2Norm,
   encodeScaleF16,
@@ -49,6 +50,7 @@ export { expandQueryWithGraph } from './graph/query_expand.js';
 export * from './memory/index.js';
 export type { MountOptions, PackMeta, Pack } from './pack.runtime.js';
 export type { QueryOptions, Hit } from './query.js';
+export type { LivePackOptions } from './live.js';
 export type { EmbeddingProvider, SemanticSidecar, SemanticQueryOptions, RetrievalEvidence } from './semantic/types.js';
 export type { ContextPatch } from './patch.js';
 export type { BuildInputDoc, BuildPackOptions } from './builder.js';