diff --git a/.gitignore b/.gitignore index 919d772b..f4937c68 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,9 @@ __pycache__/ *.pyo *.pyd +# Throwaway dev/analysis scripts (not part of the product) +scripts/tmp_*.py + # Migration build output scripts/export-tencent-vdb/dist/ scripts/migrate-sqlite-to-tcvdb/dist/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..dd5bf893 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,104 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +`@tencentdb-agent-memory/memory-tencentdb` — a memory plugin for the **OpenClaw** agent runtime (and **Hermes** via an HTTP gateway). It does two largely independent jobs: + +1. **Layered long-term memory** (`src/core/`): a semantic pyramid that distills conversations into a user profile. `L0 Conversation` (raw dialogue) → `L1 Atom` (atomic facts) → `L2 Scene` (scene blocks) → `L3 Persona` (user profile). Lower layers are evidence in a database; upper layers are human-readable Markdown. +2. **Symbolic short-term memory / context offload** (`src/offload/`): offloads verbose tool logs to external `refs/*.md` files and keeps only a compact Mermaid "canvas" (with `node_id`s) in context, drilling back down via `node_id` when needed. + +Both jobs are wired into host lifecycle hooks. The two subsystems share a data directory but have separate config, hooks, and code paths — when editing, know which one you're in. + +## Commands + +```bash +npm run build # full build: tsdown bundles index.ts → dist/ + tsc builds the bin/ scripts +npm run build:plugin # just the plugin bundle (tsdown) +npm test # vitest run (unit tests: src/**/*.test.ts) +npm run test:watch # vitest watch +npm run test:coverage # vitest with v8 coverage +npx vitest run path/to/file.test.ts # single test file +npx vitest run -t "substring of test name" # single test by name +npx vitest --config vitest.e2e.config.ts run # e2e tests (**/*.e2e.test.ts), excluded from default run +``` + +**No build is required for local development.** Node ≥ 22.16 strips TypeScript types natively and OpenClaw loads `.ts` source directly. Build only for publishing (`prepack` runs it). To dev against a real OpenClaw: `openclaw plugins install --link .`, then `openclaw gateway restart` after changes. + +`tsdown` treats every declared dependency + `openclaw` + `node:*` as external (see `tsdown.config.ts`) — the bundle is intentionally thin. + +## Architecture + +### Host-neutral core + adapters +The central abstraction is `TdaiCore` (`src/core/tdai-core.ts`) — a host-neutral facade exposing `handleBeforeRecall`, `handleTurnCommitted`, search, and pipeline management. It depends only on the `HostAdapter` / `LLMRunner` interfaces in `src/core/types.ts`, never on a specific runtime. Two adapters implement those interfaces: + +- **`src/adapters/openclaw/`** — in-process. `index.ts` (the plugin entry) is a thin shell: it registers tools/hooks via `api.registerTool` / `api.on(...)`, translates OpenClaw events into `TdaiCore` calls, and manages prompt/recall caches keyed by session. +- **`src/adapters/standalone/`** — used by `src/gateway/server.ts`, a native-`http` server (no Express) that exposes `TdaiCore` over HTTP (`/recall`, `/capture`, `/search/*`, `/session/end`, `/seed`) for the Hermes sidecar. + +When changing memory behavior, put logic in `TdaiCore` / `src/core/`, not in `index.ts` or the gateway — both hosts must get it. + +### Long-term memory pipeline (`src/core/`) +- `conversation/l0-recorder.ts` — appends raw turns to local JSONL (L0). +- `record/` — L1: `l1-extractor.ts` (LLM extracts atomic facts), `l1-dedup.ts` (smart dedup), `l1-writer.ts` / `l1-reader.ts`. +- `scene/` — L2: `scene-extractor.ts` aggregates L1 atoms into Markdown scene blocks; scene index/navigation/formatting helpers alongside. +- `persona/` — L3: `persona-generator.ts` synthesizes the user profile; `persona-trigger.ts` decides when (every N new memories). +- `hooks/auto-recall.ts` + `hooks/auto-capture.ts` — the recall (before prompt) and capture (after agent end) entry points called by both adapters. +- `prompts/` — all LLM prompt templates for L1/L2/L3 and dedup. +- `store/` — pluggable storage via `store/factory.ts`. Two backends behind `IMemoryStore` (`store/types.ts`): `sqlite.ts` (default: SQLite + `sqlite-vec` + FTS5, local) and `tcvdb.ts` (Tencent Cloud VectorDB, server-side embedding + hybrid search). `embedding.ts` + `bm25-local.ts` / `bm25-client.ts` handle vector + lexical retrieval. +- `tools/` — `memory-search.ts` and `conversation-search.ts` are the agent-callable search tools. +- `report/reporter.ts` — metric/health reporting. +- `profile/profile-sync.ts` — keeps L2/L3 Markdown in sync locally. + +### Context offload (`src/offload/`) +Registered separately via `registerOffload(api, offloadConfig)` from `index.ts`, **only when `offload.enabled`**. It registers the OpenClaw `contextEngine` slot and its own hooks (all via `api.on`): +- `hooks/after-tool-call.ts` — captures tool output, writes full text to `refs/*.md`. +- `hooks/before-prompt-build.ts` / `hooks/llm-input-l3.ts` — inject the Mermaid history canvas, compress non-current tool-use blocks, emergency-compress on token overflow. +- `hooks/before-agent-start.ts` — task transition / judgment handling. +- `pipelines/l2-mermaid.ts` — builds the Mermaid canvas and backfills `node_id`s. +- `local-llm/` — optional local LLM (`node-llama-cpp`) path for offload extraction; `backend-client.ts` is the remote alternative. +- `storage.ts` is the on-disk format (`refs/*.md`, `*.mmd`, jsonl); token accounting lives in `context-token-tracker.ts` / `*token*` files. + +### Scripts & bins (`scripts/`, `bin/`) +Standalone migration/inspection tools, each with its own `tsconfig.json` and a built `.mjs` in `bin/`: `migrate-sqlite-to-tcvdb`, `export-tencent-vdb`, `read-local-memory`. Built by `npm run build:scripts`. `scripts/memory-tencentdb-ctl.sh` and `setup-offload.sh` are operator helpers; `openclaw-after-tool-call-messages.patch.sh` patches the OpenClaw install so after-tool-call messages can be offloaded (run once per OpenClaw install, re-run after upgrades). + +## Configuration + +Plugin config is parsed by `parseConfig` in `src/config.ts` into flat groups: `capture` (L0), `extraction` (L1), `persona` (L2/L3), `pipeline`, `recall`, `embedding`, `offload`, and a `tcvdb` block. Zero-config (`{}`) is valid — every field has a default and the SQLite backend is the default. LLM `model` fields use `"provider/model"` and fall back to OpenClaw's default model when omitted. + +The **gateway** (Hermes path) is configured separately in `src/gateway/config.ts`, primarily from env vars: `TDAI_LLM_API_KEY` / `TDAI_LLM_BASE_URL` / `TDAI_LLM_MODEL`, `MEMORY_TENCENTDB_ROOT` / `TDAI_DATA_DIR` for the data dir, and `MEMORY_TENCENTDB_GATEWAY_HOST` / `_PORT`. + +## Active work: multi-tenant retrofit (`custom-multitenant` branch) + +This fork is being adapted so **one Gateway/sidecar process can safely serve multiple end-user accounts** (tenants), for the AI4ALL WeChat companion project. Design doc + upstream issue live in `docs/tdai_multitenant_design.md` and `docs/tdai_multitenant_issue.md`. + +> **Path caveat:** the design doc is written from the *AI4ALL* (weixin_bot) repo's perspective. Paths like `app/turn_service.py`, `scripts/seed_tdai_memory.py`, `docs/tech_design/...` belong to **that** repo, not this one. Only `src/...` paths refer to this TDAI repo. Our scope is the **TDAI side** (`§8.4` / `§8.5` / phase P0.5 of the design, and the standalone issue). + +**The core problem:** the standalone/SQLite store is single-tenant per `dataDir`. `session_key` only isolates L0 (raw dialogue) and pipeline/session state — **L1/L2/L3 recall and search are dataDir-global**, so one sidecar serving multiple accounts would recall across accounts, violating the hard isolation invariant. + +Verified change points (file:line confirmed against current tree, 2026-06): + +| # | Gap | Where | Note | +|---|---|---|---| +| 1 | L1 search has no session filter | `searchL1Vector`/`searchL1Fts`/`searchL1Hybrid?` (`core/store/types.ts`), SQL in `core/store/sqlite.ts` (`l1_vec`/`l1_fts MATCH`) | `L1Record` already carries `session_key` (`store/types.ts:65`) — **no schema change**; add `sessionKey` param + push filter into SQL/vector, thread through `executeMemorySearch → searchMemories → performAutoRecall` | +| 2 | L0 search has no session filter | `searchL0Vector`/`searchL0Fts` (`store/types.ts`); conversation search is **post-filter** (`core/tools/conversation-search.ts:224`) | push down, don't post-filter (post-filter topK can be all other tenants → 0 results after filter) | +| 3 | `/search/memories` has no `session_key` | `MemorySearchRequest` (`gateway/types.ts`), handler `server.ts:~424` doesn't pass it; `MemorySearchParams` (`core/types.ts:229`) lacks it | add field + thread to #1 | +| 4 | `/search/conversations` `session_key` is optional | `gateway/types.ts`, `ConversationSearchParams` has `sessionKey?` | make required in multi-tenant mode | +| 5 | L2/L3 are dataDir-root files | persona reads `pluginDataDir/persona.md` (`auto-recall.ts:148`, writes `persona-generator.ts:185`); scene `readSceneIndex(pluginDataDir)` (`auto-recall.ts:162`) | per-account subdir; change read+write sides together | +| 6 | `/recall` drops `prependContext` (**P0, smallest, do first**) | `server.ts:386` returns only `appendSystemContext` | `RecallResult.prependContext` **already exists** (`core/types.ts:201`) and `handleBeforeRecall` already takes `sessionKey` — just add `prepend_context` to `RecallResponse` and pass it through | +| 7 | reindex/count cross-session | `getAllL1Texts`/`rebuildFtsIndex`/`countL1` (`sqlite.ts`) | structural approach isolates for free; filter approach must decide per-session reindex | + +Background pipeline concerns (as important as isolation): TDAI runs background timers per session (L1 idle, L2 schedule — `utils/pipeline-manager.ts:181`) + a global L3 runner (`pipeline-manager.ts:956`). Only a global `extraction.enabled` switch exists, no per-tenant. L3 persona is an unlocked file read-modify-write (`persona-generator.ts:185`) — needs **atomic write (temp+rename)**; same-account L3 is already serialized by the in-process `SerialQueue` (concurrency=1) only under the single-dataDir-single-process contract. + +Two implementation routes (decision pending — see the design doc §8.4): +- **Structural (AI4ALL's recommendation):** Gateway keeps `Map`, one dataDir per account (`baseDir/{account}`), lazy + LRU. Isolation is physical (covers #1/#2/#5/#7 for free); still needs #3/#6 interface fields. Cost: N cores = N timer sets + N `SerialQueue`s → background LLM extraction fan-out ×N, so it **requires a cross-core global concurrency cap**. +- **Filter:** single core + shared store, `session_key` pushed into every query. Memory-light, single `SerialQueue` caps concurrency naturally, but "miss one WHERE = cross-tenant leak." + +Either way: #6, #5 (L2/L3 per-account), atomic L3 write, and a `session_key`-scoped **namespace wipe** API (for account hard-delete) are all required. + +## Conventions + +- Commits: Conventional Commits with a scope from `{store, hooks, persona, scene, record, conversation, gateway, hermes, offload, llm, embedding}`, e.g. `fix(embedding): ...`. **DCO is enforced** — every commit needs `Signed-off-by` (`git commit -s`). +- Import order: Node builtins → third-party → internal. +- Tests live next to source as `*.test.ts`; e2e as `*.e2e.test.ts` (run only via the e2e config). `vitest` uses the `forks` pool with a 120s timeout. +- The `src/conversation`, `src/record`, etc. paths in CONTRIBUTING.md are stale — the actual layout nests these under `src/core/`. diff --git a/docs/tdai_gateway_integration.md b/docs/tdai_gateway_integration.md new file mode 100644 index 00000000..6fa3f881 --- /dev/null +++ b/docs/tdai_gateway_integration.md @@ -0,0 +1,359 @@ +# TDAI Memory Gateway — Integration Guide + +> Audience: application teams (AI4ALL `weixin_bot`, and any similar chat/agent +> product) who want to bolt a **layered long-term memory** onto their bot by +> talking HTTP to one TDAI Gateway process. You do **not** need to embed the +> plugin, run OpenClaw, or understand the internals — you POST turns in and GET +> memory context back. + +--- + +## 1. What you get + +TDAI distils a user's conversation history into a **semantic pyramid** and hands +back a compact, ready-to-inject context block at recall time: + +``` +L0 raw dialogue turns (everything the user said) +L1 atomic facts ("studies guitar", "evening practice 30min") +L2 scene blocks (Markdown) (themed summaries of related atoms) +L3 persona profile (Markdown) (one human-readable user portrait) +``` + +You feed it turns (`/capture`); a background pipeline grows L1→L2→L3 with an LLM. +Before each model call you ask for context (`/recall`) and prepend/append what +you get to your own prompt. That's the whole contract. + +**One process, many users — safely.** A single Gateway can serve thousands of +end-user accounts with **hard isolation**: account A's memory can never surface +in account B's recall or search. Isolation is *structural* — each account gets +its own data directory, SQLite file, and pyramid — not a `WHERE session_key=…` +filter that one missed query could leak past. + +--- + +## 2. Architecture model + +``` + your app (weixin_bot turn_service) + │ HTTP (JSON, Bearer auth) + ▼ + ┌─────────────────────────────────────────┐ + │ TDAI Gateway (node, native http :8420) │ + │ │ + │ CoreRegistry Map │ + │ ├─ ai4all:alice → TdaiCore → baseDir/alice./ (own SQLite, persona.md…) + │ ├─ ai4all:bob → TdaiCore → baseDir/bob./ + │ └─ … (lazy create, LRU evict) │ + │ │ + │ one shared extraction semaphore │ ← caps total background LLM fan-out + └─────────────────────────────────────────┘ + │ + ▼ per-account dirs on disk (the isolation boundary) +``` + +- **`session_key` is the tenant identity.** You choose the scheme; the convention + is `"{namespace}:{account_id}"`, e.g. `ai4all:alice`, `psydt:User2993`. It is + business-supplied and **never trusted as a path** — the Gateway derives a + collision-free, traversal-safe directory name from it (slug + sha256 prefix). +- **Lazy + LRU.** A core is created on first request for an account and can be + evicted when idle (bounded by `maxResidentCores`) — its pending L1/L2/L3 work + is flushed before teardown, never dropped. +- **Bounded background cost.** N active accounts would otherwise fan out to ~3N + concurrent extraction LLM calls. One shared semaphore caps the global total. + +--- + +## 3. Deploy & configure + +Run it directly (no build needed; Node ≥ 22.16 strips TS types): + +```bash +node --env-file=.env --import tsx src/gateway/server.ts +``` + +### Configuration sources & precedence + +`env var` **>** `tdai-gateway.yaml` (CWD or `/`) **>** built-in default. + +| Concern | env | yaml | default | +|---|---|---|---| +| Bind port / host | `TDAI_GATEWAY_PORT` / `_HOST` | `server.port`/`.host` | `8420` / `127.0.0.1` | +| **Multi-tenant** | `TDAI_MULTI_TENANT` | `data.multiTenant` | `false` | +| Data root | `TDAI_DATA_DIR` (or `MEMORY_TENCENTDB_ROOT`) | `data.baseDir` | `~/.memory-tencentdb/memory-tdai` | +| API key (auth) | `TDAI_GATEWAY_API_KEY` | `server.apiKey` | unset → **auth off** | +| CORS allow-list | `TDAI_CORS_ORIGINS` | `server.corsOrigins` | `[]` → no CORS headers | +| Max resident cores (LRU) | `TDAI_MAX_RESIDENT_CORES` | `data.maxResidentCores` | `0` = unlimited | +| Idle core TTL (ms) | `TDAI_CORE_IDLE_TTL_MS` | `data.coreIdleTtlMs` | `0` = disabled | +| Global extraction cap | `TDAI_MAX_CONCURRENT_EXTRACTIONS` | `data.maxConcurrentExtractions` | multi-tenant `4`, single `∞` | +| LLM (chat/extraction) | `TDAI_LLM_API_KEY` / `_BASE_URL` / `_MODEL` | `llm.*` | OpenAI defaults | + +> **Embedding is yaml-only.** There are **no** `TDAI_EMBEDDING_*` env vars. Vector +> recall is configured under `memory.embedding.*` in `tdai-gateway.yaml`. The +> `TDAI_LLM_*` vars configure *chat/extraction only* — setting them does **not** +> turn on vector retrieval. See `tdai-gateway.yaml` in the repo root for a working +> DashScope/Bailian (`text-embedding-v3`, 1024-dim) example, and set +> `memory.recall.strategy: hybrid` for vector + BM25 fusion. + +### Storage backend & embedding — the supported stack + +These are two **independent** layers; this deployment fixes both: + +| Layer | Job | What we use | +|---|---|---| +| **Embedding** | text → vector (semantic encoding) | **Alibaba DashScope** `text-embedding-v3` (1024-dim), via `memory.embedding.*` | +| **Vector store** | store vectors + nearest-neighbour & keyword search | **local SQLite** (`sqlite-vec` for vectors + FTS5/BM25 for keyword), one DB file per account | + +This is the **only supported multi-tenant stack.** The code also contains a +**Tencent Cloud VectorDB (`tcvdb`) backend**, but it is a fully separate design — +tcvdb does its *own* server-side embedding (it would replace DashScope, not work +alongside it) and keeps all data in shared cloud collections. Because shared +collections defeat the per-account physical isolation this Gateway relies on, +**`multiTenant=true` + `storeBackend=tcvdb` is rejected at startup by design.** + +> If you only ever run DashScope + SQLite (the default — `storeBackend` is +> `sqlite` unless you set it), the tcvdb path is never exercised and the +> "tcvdb breaks multi-tenant isolation" issue **does not apply to you**. It is a +> guard against an unsupported combination, not a bug to fix in this deployment. +> Switching to tcvdb would mean re-embedding all history with Tencent's model and +> doing dedicated isolation work first — only worth it if local SQLite outgrows +> your scale/ops needs. + +Minimal multi-tenant `.env`: + +```bash +TDAI_MULTI_TENANT=true +TDAI_DATA_DIR=/var/lib/tdai +TDAI_GATEWAY_API_KEY= +TDAI_LLM_API_KEY=sk-... +TDAI_LLM_BASE_URL=https://api.deepseek.com/v1 +TDAI_LLM_MODEL=deepseek-chat +DASHSCOPE_API_KEY=sk-... # referenced by ${DASHSCOPE_API_KEY} in the yaml +``` + +--- + +## 4. HTTP API reference + +All bodies are JSON. When `TDAI_GATEWAY_API_KEY` is set, send +`Authorization: Bearer ` on every route except `GET /health`. In +**multi-tenant mode `session_key` is required** on every routed endpoint (omit → +`400`). + +### `GET /health` +Liveness; never requires auth. Multi-tenant response reports routing state, not +store internals: +```json +{ "status":"ok","version":"0.1.0","uptime":1234, + "multi_tenant":true,"active_cores":12, + "extraction":{"limit":4,"active":1,"waiting":0}, + "resident":{"count":12,"limit":200,"pinned":3}, + "embedding":{"configured":true,"provider":"dashscope","model":"text-embedding-v3", + "dimensions":1024,"recallStrategy":"hybrid"} } +``` +- `resident.pinned` — cores currently serving a request (held by a lease). If + `count > limit` with `pinned` close behind, the LRU cap is being held past its + bound because every core is busy — transient and safe. +- `embedding` — **configuration intent**, not a live probe: `configured:true` + means embedding is enabled with a real provider and no config error. It does + **not** prove the embedding endpoint is reachable (health stays a cheap + liveness check). For the live signal that vectors actually fired, read the + `strategy` field from `/search/memories` (§6). +> In multi-tenant mode `stores.vectorStore`/`embeddingService` stay `false` by +> design (cores are lazy/per-account — there is no single store to probe); the +> `embedding` block is the embedding signal health can give there. + +### `POST /recall` — get memory context for a turn +Call this *before* your LLM call. +```jsonc +// request +{ "query": "what am I practicing lately?", "session_key": "ai4all:alice" } +// response +{ "context": "…\n…", // append to system prompt + "prepend_context": "- studies guitar; practices 30min nightly", // prepend to user turn + "strategy": "hybrid", "memory_count": 3 } +``` +- `context` — stable system-prompt material (persona, scene navigation). +- `prepend_context` — query-relevant L1 memories for *this* turn. **Inject both.** + (A client that uses only `context` gets the persona but never the + query-specific memories.) + +### `POST /capture` — record a completed turn +Call this *after* the assistant replies. L0 is written synchronously; L1→L3 grow +in the background. +```jsonc +// request +{ "session_key":"ai4all:alice", + "user_content":"I started learning guitar.", + "assistant_content":"That's great — daily practice is key.", + "session_id":"optional-conversation-id" } +// response +{ "l0_recorded": 2, "scheduler_notified": true } +``` + +### `POST /search/memories` — L1 fact search +```jsonc +{ "query":"guitar", "session_key":"ai4all:alice", "limit":5 } +// → { "results":"…formatted…", "total":4, "strategy":"hybrid" } +``` +`strategy` tells you which retrieval path fired: `hybrid` (vector+BM25), +`embedding` (vector only), `fts` (keyword only), `none`. Use it to verify vector +recall is actually live (§6). + +### `POST /search/conversations` — L0 raw-dialogue search +```jsonc +{ "query":"guitar", "session_key":"ai4all:alice", "limit":5 } +// → { "results":"…", "total":2 } (no strategy field) +``` + +### `POST /session/end` — flush a session +Flushes buffered pipeline work for a session. No-op if the account has no +resident core (never spins one up just to tear it down). +```jsonc +{ "session_key":"ai4all:alice" } // → { "flushed": true } +``` + +### `POST /namespace/wipe` — account hard-delete *(multi-tenant only)* +Tears down the account's core and removes its entire dataDir from disk. +Idempotent; refuses any path outside `baseDir`. Backs your "delete my data" flow. +```jsonc +{ "session_key":"ai4all:alice" } // → { "wiped": true } +``` + +### `POST /seed` — bulk import historical dialogue +Validates + runs the L0→L1 pipeline over batches. +> **Caveat (multi-tenant):** `/seed` writes to a snapshot dir `baseDir/seed-/`, +> **not** the per-account dirs that recall/search read. To pre-load *per-account* +> memory in multi-tenant mode, drive `executeSeed` per account into +> `baseDir/safeAccountDir(session_key)` — see `scripts/import-psydt.ts` for the +> reference pattern. Treat the HTTP `/seed` route as single-tenant / snapshot use. + +--- + +## 5. Integration lifecycle (the turn flow) + +This is the whole loop your `turn_service` runs per user message: + +```python +# pseudo-code — one user turn +ctx = POST("/recall", {"session_key": sk, "query": user_text}) + +system_prompt = base_system + "\n" + ctx["context"] +user_prompt = (ctx["prepend_context"] + "\n\n" if ctx["prepend_context"] else "") + user_text + +reply = your_llm(system_prompt, user_prompt) + +# fire-and-forget; do not block the user's reply on this +POST("/capture", { + "session_key": sk, "session_id": conversation_id, + "user_content": user_text, "assistant_content": reply, +}) +return reply +``` + +- **`/recall` is on the hot path** — keep its timeout tight; it returns whatever + memory exists (degrades gracefully to keyword-only if embedding is down). +- **`/capture` is off the hot path** — fire it asynchronously after you've replied. + L1→L3 synthesis happens later in the background. +- Call **`/session/end`** when a conversation goes idle (or on a timer) to flush + the last turn's pipeline work promptly. +- Call **`/namespace/wipe`** from your account-deletion / GDPR path. + +--- + +## 6. Verifying vector recall is live + +Hybrid search returns hits regardless of which path fired, and `/health` only +reports embedding *config intent* (`embedding.configured`), not whether a vector +query actually succeeded — so test deliberately: + +1. Capture a turn with a distinctive fact (e.g. "我喜欢户外登山"). +2. Wait for L1 to form (seconds, with a real LLM). +3. `/search/memories` with a **paraphrase that shares no keywords** (e.g. + "徒步运动"). If you get a hit with `strategy: hybrid` or `embedding`, vector + recall works. If paraphrases only ever return `fts`/`none`, embedding isn't + contributing — check the `memory.embedding.*` yaml block and the DashScope key. + +### One-click smoke test + +`scripts/smoke-recall.mjs` automates exactly the steps above and **asserts** the +result — run it against a Gateway started with your real `.env` (DashScope key, +`TDAI_LLM_*`) as the first thing you do after deploy: + +```bash +node scripts/smoke-recall.mjs +# against a non-default address / with auth: +TDAI_GATEWAY_URL=http://127.0.0.1:8420 \ +TDAI_GATEWAY_API_KEY= node scripts/smoke-recall.mjs --timeout 120 +``` + +It captures a distinctive fact for a throwaway account, polls a no-keyword +paraphrase until L1 forms, and passes **only** when `strategy` is `hybrid` or +`embedding`. Exit codes: `0` PASS, `1` FAIL (vectors not contributing — the +message distinguishes "L1 never formed" from "formed but embedding silent"), `2` +setup error (gateway unreachable / bad config). It fails fast (~1s) if +`/health` reports `embedding.configured:false`, and cleans up via +`/namespace/wipe` in multi-tenant mode (`--keep` to retain the probe data). No +build, no deps — needs only Node ≥ 18. Override the probe text for non-Chinese +deployments via `SMOKE_FACT_USER` / `SMOKE_KEYWORD` / `SMOKE_PARAPHRASE`. + +The bundled **dev-console** (`scripts/dev-console`, `:8421`) drives all of this +through a UI and shows the `strategy` tag and the live pyramid per account. + +--- + +## 7. Operational knobs + +| Knob | Effect | Guidance | +|---|---|---| +| `maxResidentCores` | warm cores kept in RAM (LRU evicts idle ones) | A *count* bound. In-flight requests are pinned and never evicted, so a too-low value causes a transient over-limit, not a mid-request teardown. Still size it above your peak active-account count. | +| `coreIdleTtlMs` | reclaim cores idle longer than this | A *time* bound, complementing the count bound. Good for long-tail traffic (many accounts, each briefly active) — frees memory during quiet periods instead of waiting for an LRU push. Pinned cores are spared. | +| `maxConcurrentExtractions` | global cap on background L1/L2/L3 LLM calls | Raise for more throughput, lower to protect your LLM quota. Watch `/health.extraction.waiting`. | +| `strategy: hybrid` | vector + BM25 fusion | Falls back to keyword automatically if embedding is unavailable. | + +Cold-start cost: the first request for an (evicted or new) account opens SQLite +and warms the store — a few hundred ms. Size `maxResidentCores` so your active +set stays warm; pre-warm with a cheap `/recall` if first-turn latency matters. + +--- + +## 8. Security posture + +- **Auth is opt-in and off by default.** With no `TDAI_GATEWAY_API_KEY`, every + route except `/health` is open. The Gateway logs a loud WARN at startup if it + is bound to a non-loopback host without a key. **Always set a key before + exposing the port.** Token check is constant-time. +- **Bind localhost** unless you've set auth + a CORS allow-list. CORS defaults to + sending no headers (browsers block cross-origin); `["*"]` is dev-only. +- The dev-console (`:8421`) has **no auth of its own** — it injects the Bearer + server-side and must stay bound to localhost. Never expose it. + +--- + +## 9. Invariants & limitations (read before you scale) + +1. **One process per dataDir.** Structural isolation makes accounts independent, + but a single account's dataDir must be owned by exactly one process. Do not + run two Gateways over the same `baseDir`, and don't run the import script + against an account whose core is live. Atomic writes guard against torn files + *within* one process, not against two writers. +2. **`session_key` is the whole security boundary.** If your app reuses one + `session_key` across two real users, their memory merges. Make it stable and + unique per end-user. +3. **`maxResidentCores` is a memory bound, not a request guard.** It must be set + above your peak active-account count (see §7). +4. **`/seed` does not populate per-account dirs in multi-tenant mode** (§4). +5. **No cross-account aggregation API.** By design — there is no "search all + users" endpoint, because that would defeat isolation. + +--- + +## 10. Quickstart checklist + +- [ ] `.env` with `TDAI_MULTI_TENANT=true`, `TDAI_DATA_DIR`, `TDAI_GATEWAY_API_KEY`, `TDAI_LLM_*`. +- [ ] `tdai-gateway.yaml` with `memory.embedding.*` + `memory.recall.strategy: hybrid`. +- [ ] Start the Gateway; confirm `GET /health` → `multi_tenant:true`. +- [ ] Wire `turn_service`: `/recall` before the LLM, async `/capture` after. +- [ ] Pick a `session_key` scheme (`{namespace}:{account_id}`). +- [ ] Verify vector recall: run `node scripts/smoke-recall.mjs` → `✅ PASS` (§6). +- [ ] Wire `/namespace/wipe` into account deletion. diff --git a/docs/tdai_multitenant_design.md b/docs/tdai_multitenant_design.md new file mode 100644 index 00000000..be87a488 --- /dev/null +++ b/docs/tdai_multitenant_design.md @@ -0,0 +1,920 @@ +# TDAI 多账号记忆接入落地设计 + +> 状态：设计文档，待 PoC。 +> 最后更新：2026-06-26。 +> 适用范围：AI4ALL 微信个人 AI 陪伴项目。 +> 上游参考： +> - `/Users/suchong/workspace/TencentDB-Agent-Memory` +> - `docs/plans/记忆机制_tdai化对齐.md` +> - `docs/tech_design/thick_node_postgres_refactor.md` +> - `docs/tech_design/identity_model_and_wechat_binding.md` + +## 1. 结论 + +Gemini 原方案给出的方向有价值：TDAI 的 L0/L1/L2/L3 分层、query-time recall、异步提纯和多租户隔离，确实适合 AI4ALL。 +但它对现状有几个关键误判，不能直接作为实施方案： + +1. TDAI 当前不是 Python SDK，也没有可直接内嵌到 FastAPI 的 `AsyncMemoryClient`。 +2. TDAI 当前可落地接入面是 Node.js Gateway / OpenClaw 插件能力，HTTP API 包括 `/recall`、`/capture`、`/search/memories`、`/search/conversations`、`/session/end`、`/seed`。 +3. TDAI 当前 store backend 是 `sqlite` 或 `tcvdb`，不是 PostgreSQL / pgvector；如果要统一落到 AI4ALL PostgreSQL，需要新增 TDAI store backend，不能在 AI4ALL 侧单方面配置出来。 +4. AI4ALL 的业务隔离边界是 `ai4all_account_id`，代码和 DB 里历史字段叫 `account_id`；OpenClaw 原始 `session_key` / `channel_account_id` 不是记忆隔离主键。 +5. AI4ALL 已有 `messages`、`account_profile_files`、`memory_writer.py`、`dreaming.py`、`PromptBuilder` 和 PostgreSQL 垫片，接入必须沿这些边界做最小改动，而不是替换整条 turn 链路。 + +因此推荐采用两阶段策略： + +- **近期可落地**：把 TDAI 作为本机 sidecar，通过 HTTP 接入 AI4ALL turn 热路径。TDAI 自己维护本地 SQLite 或 TCVDB 记忆库；AI4ALL 继续以 PostgreSQL/SQLite 抽象作为业务真相。 +- **后续可选**：当 sidecar 效果验证后，再评估是否给 TDAI 增加 `postgres` store backend，把 L0/L1/L2/L3 统一收敛到中心 PG。 + +> **关键修正（2026-06，blocker）：`session_key` 不足以隔离多账号。** 经核实 TDAI 当前 standalone/sqlite store 是「单租户 per dataDir」：`session_key` 只隔离了 L0（原始对话）和 pipeline/session 状态，**L1/L2/L3 召回是 dataDir 全局的**——L1 搜索接口（`store/types.ts:269-270` 的 `searchL1Fts`/`searchL1Vector`）签名里没有 session 过滤；persona/scene 是 dataDir 根级文件（`auto-recall.ts:148/162`）。因此「一个 sidecar 靠 `session_key` 服务多账号」会跨账号召回，**违反 AI4ALL「按账号隔离」红线**。已确认方向：在 P1 灰度前对 TDAI 做**多租户改造（路 B，§8.4）**，并补 `/recall` response 丢失的 `prependContext`（§5.3、§8.4）。这两项是 P1 前置，不是可选优化。 + +### 1.1 部署现状（2026-06，影响接入形态） + +- 数据库为 **中心化 PostgreSQL**。线上两节点：一个节点跑 PostgreSQL，另一个节点经内网连接调用。PG 是全局业务真相，`messages` 等表集中存储。 +- 微信接入点的特殊性导致 **用户与节点强绑定**：每个用户的入口只能落到某一台机器节点，该用户的业务主逻辑也全部在这台节点上运行。用户不会在节点间漂移（除非显式迁移）。 + +这两点不推翻 sidecar 方案，反而强化它： + +1. **用户强绑定节点 → 本地 SQLite 不再是妥协，而是天然契合**。同一用户所有轮次都在同一节点处理，本地记忆不会跨节点碎片化（§4.2 旧约束被微信绑定语义自动满足）。 +2. **中心 PG 已持有全量 `messages` → TDAI 记忆可从中心重建**。节点故障/迁移的恢复路径是「新节点起 sidecar + 从中心 PG re-seed」，而非搬运本地 SQLite。源真相在中心、派生记忆在本地且可重建，是稳的混合形态。 +3. **中心 PG 让未来 `postgres` store backend 成为真实可选项（P4）**，但要改 TDAI TypeScript store、依赖 pgvector，并给热路径 recall 增加一次跨节点网络往返；近期不做。 + +### 1.2 接入点完整度（轴 A）≠ provider 抽象（轴 B） + +「完整实现 OpenClaw-like 接入」要区分两个正交的轴： + +- **轴 A — 接入点完整度**：是否把 OpenClaw 暴露的所有 host 点都接上（recall / capture / 模型主动 search 工具 / session-end / shutdown flush）。**目标做满**，详见 §2.3。 +- **轴 B — provider 抽象层**：是否建抽象 `MemoryProvider` 基类 + 多实现（TdaiHttp / NativePg / Noop）热插拔。**推迟到 P3/P4**：在出现第二个实现之前用单一实现去定接口，几乎必然返工；当前 `app/tdai_client.py` 的模块边界已提供「调用不散落、可降级、可灰度」的全部收益。 + +第一版接口面 = `tdai_client.py` 的几个具体函数，不套抽象基类。 + +## 2. 当前系统映射 + +### 2.1 AI4ALL 现有热路径 + +当前用户入站链路是： + +```text +OpenClaw bridge + -> /openclaw/turn + -> app/turn_service.py + -> build_turn_llm_input() + -> PromptBuilder.assemble() + -> generate_reply_with_tools() + -> insert_message / billing / moderation + -> memory_writer.write_memory() + -> dreaming scheduler 或 admin run-once +``` + +现有记忆材料： + +| 层次 | AI4ALL 当前实现 | 说明 | +|---|---|---| +| L0 原始会话 | `messages` 表 | 每轮用户/助手可见文本，按 `account_id` 隔离 | +| L0 daily notes | `app/memory_writer.py` -> `account_profile_files.filename = memory/YYYY-MM-DD.md` | turn 后异步追加，已入 DB-backed profile storage | +| session continuity | `sessions.carryover_summary` | session 轮转后承接上一段对话 | +| 长期上下文 | `SOUL.md` / `IDENTITY.md` / `USER.md` / `MEMORY.md` | 通过 `read_agent_context()` 进入 `Project Context` | +| 整理审计 | `dreaming_runs` / `dreaming_memory_items` / `memory_events` | Dreaming 输出、自动应用和 diff 审计 | + +### 2.2 TDAI 当前能力 + +TDAI 仓库关键入口： + +| 文件 | 作用 | +|---|---| +| `src/core/tdai-core.ts` | Host-neutral facade，提供 recall/capture/search/session end | +| `src/gateway/server.ts` | HTTP Gateway，暴露 `/recall`、`/capture`、`/search/*`、`/session/end`、`/seed` | +| `src/gateway/types.ts` | Gateway 请求/响应类型 | +| `src/config.ts` | TDAI 配置，包含 capture/extraction/persona/pipeline/recall/embedding/storeBackend | +| `src/core/store/factory.ts` | store backend 选择：`sqlite` 或 `tcvdb` | +| `src/core/hooks/auto-recall.ts` | query-time L1 recall + L3 persona + L2 scene navigation 注入 | +| `src/core/hooks/auto-capture.ts` | turn 后 L0 capture + pipeline notify | + +Gateway API 形态： + +```http +GET /health +POST /recall +POST /capture +POST /search/memories +POST /search/conversations +POST /session/end +POST /seed +``` + +其中 `/recall` 返回： + +```json +{ + "context": "......", + "strategy": "hybrid", + "memory_count": 3 +} +``` + +`/capture` 输入： + +```json +{ + "user_content": "用户本轮文本", + "assistant_content": "助手回复", + "session_key": "ai4all:aid_806382741", + "session_id": "123", + "messages": [ + {"role": "user", "content": "..."}, + {"role": "assistant", "content": "..."} + ] +} +``` + +注意：Gateway type 里有 `user_id`，但当前 `src/gateway/server.ts` 并未把它传入 `TdaiCore` 的 recall/capture 主逻辑。当前 TDAI 的隔离和 pipeline state 实际依赖 `session_key`。 + +### 2.3 OpenClaw host 点完整映射（轴 A 的目标范围） + +TDAI 的 `TdaiCore` 是 host-neutral facade，OpenClaw 插件与 Gateway 是它的两个 host adapter；**Gateway 就是 standalone host**，HTTP 端点逐一对应 facade 方法（核对自 TDAI `index.ts` 的 `api.on(...)` 与 `src/core/tdai-core.ts`）。OpenClaw 插件注册的 5 个 host 点映射到 AI4ALL： + +| OpenClaw host 点 | TdaiCore facade | Gateway 端点 | AI4ALL 落点 | 阶段 | +|---|---|---|---|---| +| `before_prompt_build`（被动召回 L1+L3+L2 注入 prompt） | `handleBeforeRecall(userText, sessionKey)` | `POST /recall` | `build_turn_llm_input()` → `extra_blocks` 的 `tdai_recall_memories` + `tdai_recall_persona` | P1 | +| `before_message_write`（剥离注入标签防污染历史） | — | — | **N/A**：recall 作为独立 system block 注入，从不写回 `messages.content`，架构天然满足 | — | +| `agent_end`（捕获 user+assistant 轮，推进 L0→L1→L2→L3） | `handleTurnCommitted(turn)` | `POST /capture` | after-turn 复用 `write_memory` 的 background_loop 派发 | P1 | +| `registerTool`：`tdai_memory_search` / `tdai_conversation_search`（模型主动召回，每轮合计上限 3 次） | `searchMemories` / `searchConversations` | `POST /search/memories`、`/search/conversations` | 加进 `generate_reply_with_tools` 的 tool registry（§5.6） | P1 | +| `gateway_stop`（flush pipeline、关 store） | `handleSessionEnd` + `destroy()` | `POST /session/end` | `session_lifecycle` 轮转 + FastAPI shutdown（§5.5） | P2 | + +要达到「完整 OpenClaw-equivalent」，除已规划的被动 recall + capture 外，还需补两块（旧版本漏列）： + +1. **模型主动 search 工具**（§5.6）：让模型在生成中途自行查记忆，覆盖被动召回未命中的「我上次说的那个事」类追问。Gateway `/search/*` 现成，只需加 tool 壳并复制「每轮合计 3 次」硬上限。 +2. **session-end / shutdown flush**（§5.5）：pipeline 是进程内串行队列，session 轮转与进程退出要给它 flush 机会，否则刚 capture 的 L0 可能未提纯就丢失。 + +`before_message_write` 我们不需要实现——独立 block 注入天然不污染历史。 + +## 3. 关键设计决策 + +### 3.1 接入形态：sidecar 优先，不做 Python 内嵌 + +推荐形态： + +```text +AI4ALL FastAPI 进程 + | + | HTTP localhost + Bearer token + v +TDAI Gateway Node 进程 + | + +-- 本地 SQLite/sqlite-vec 或 Tencent Cloud VectorDB + +-- TDAI pipeline: L0 -> L1 -> L2 -> L3 +``` + +理由： + +- 与 TDAI 真实实现一致，PoC 成本最低。 +- 不需要在 AI4ALL 里引入 Node/TypeScript 运行时代码。 +- 故障边界清晰：TDAI 失败时 AI4ALL 可降级为现有 memory/dreaming。 +- 保留后续替换 store backend 的空间。 + +不建议近期做： + +- 不建议直接改 TDAI 为 Python SDK。 +- 不建议一开始就把 TDAI 全部迁到 AI4ALL PostgreSQL。 +- 不建议让 TDAI OpenClaw 插件直接介入当前微信 bot 的 OpenClaw runtime；AI4ALL 已经自己接管了 prompt、tool、计费、审核和记忆链路，插件式 hook 容易和现有业务链路重叠。 + +注意「完整接入」≠ 现在建 provider 协议：接入点做满（轴 A，§2.3）与抽象接口（轴 B）是两件事，第一版接口面就是 `app/tdai_client.py` 的几个函数（见 §1.2）。 + +### 3.2 多账号隔离：TDAI session_key 必须绑定 AI4ALL account_id + +AI4ALL 隔离主键： + +```text +业务主键：ai4all_account_id +代码/DB 兼容字段：account_id +通道身份：channel_account_id +OpenClaw 会话键：session_key +``` + +TDAI 当前实际按 `session_key` 分 session/pipeline/checkpoint/recall。因此 AI4ALL 调 TDAI 时必须构造稳定 key： + +```python +tdai_session_key = f"ai4all:{account_id}" +tdai_session_id = str(session["id"]) +``` + +不要使用： + +- 不要用 OpenClaw payload 原始 `session_key` 作为 TDAI session key。 +- 不要用 `channel_account_id` 作为 TDAI session key。 +- 不要按 AI4ALL session id 单独隔离长期记忆，否则 L3 persona 和 L1 memories 会按会话碎片化，无法跨 session 生效。 + +`session_id` 可以传 AI4ALL 当前 `sessions.id`，用于 TDAI L0/L1 溯源；但长期隔离仍以 `tdai_session_key = ai4all:{account_id}` 为准。 + +### 3.3 Prompt 防污染 + +TDAI recall 返回的上下文只能进入本轮 prompt，不能写回： + +- `messages.content` +- `memory/YYYY-MM-DD.md` +- 用户原始消息 +- assistant 可见回复之外的材料 + +AI4ALL 当前 `build_turn_llm_input()` 已经先读 history，再构造 system prompt；只要把 TDAI recall block 放进 `PromptBuilder` 的动态 block，且不改落库的 `text/reply`，即可保持存储态干净。 + +建议新增两个 block name（对应 `/recall` 的 prepend/append 两段，§5.3）： + +```text +tdai_recall_memories # prepend_context：当前 query 相关 L1 记忆 +tdai_recall_persona # append_context：persona/scene 背景 +``` + +注入内容建议包一层 AI4ALL 自己的说明，避免让 TDAI 原始标签成为回答风格主导： + +```text +【系统召回记忆】 +以下材料来自长期记忆召回，只作为理解用户的参考，不是用户本轮原话。 +如果与用户本轮说法冲突，以用户本轮为准，可轻量确认。 + + +``` + +### 3.4 与现有 dreaming 的关系 + +近期不替换 `dreaming.py`。两套系统并行： + +| 能力 | 近期权威 | +|---|---| +| AI4ALL 业务上下文文件 | 现有 `account_profile_files` + `dreaming.py` | +| TDAI recall/capture 效果评估 | TDAI sidecar | +| billing/moderation/proactive | AI4ALL 现有链路 | +| 用户可见回复 | AI4ALL LLM 链路 | + +并行期的目标不是一次性迁移，而是回答两个问题： + +1. TDAI 的 L1 recall / L3 persona 是否明显改善陪伴感和长期一致性？ +2. TDAI pipeline 的成本、延迟、稳定性是否适合微信陪伴业务？ + +如果答案稳定为是，再决定是否减少 `MEMORY.md` 的 prompt 权重，或把 `dreaming.py` 改为生成 TDAI seed/records。 + +## 4. 目标架构 + +### 4.1 单机 PoC 架构 + +```text +┌───────────────────────────────┐ +│ AI4ALL FastAPI │ +│ - turn_service.py │ +│ - PromptBuilder │ +│ - messages / billing / audit │ +│ - existing dreaming │ +└───────────────┬───────────────┘ + │ localhost HTTP + │ Authorization: Bearer +┌───────────────▼───────────────┐ +│ TDAI Gateway │ +│ - /recall before LLM │ +│ - /capture after reply │ +│ - /seed for backfill │ +│ - pipeline L0/L1/L2/L3 │ +└───────────────┬───────────────┘ + │ +┌───────────────▼───────────────┐ +│ TDAI store │ +│ Phase 0: local SQLite │ +│ Phase 1 option: TCVDB │ +│ Future: PostgreSQL backend │ +└───────────────────────────────┘ +``` + +### 4.2 线上拓扑与多节点部署 + +线上现状（2026-06）：两节点，中心化 PostgreSQL。 + +```text +节点 A + PostgreSQL（中心业务真相：账号/绑定/计费/审核/messages） + + AI4ALL app + TDAI Gateway + OpenClaw + TDAI 本地 store + +节点 B（经内网连节点 A 的 PG） + AI4ALL app + TDAI Gateway + OpenClaw + TDAI 本地 store +``` + +> 已确认（2026-06）：两节点都跑 AI4ALL app，节点 A 同时是 PG 节点。因此**每个节点各起一个 TDAI sidecar**，服务本节点的多个账号。 +> +> ⚠️ **多账号共享一个 sidecar 必须先完成 TDAI 多租户改造（路 B，§8.4）**：当前 standalone store 的 L1/L2/L3 是 dataDir 全局的，未改造前一个 sidecar 服务多账号会跨账号召回。改造后该 sidecar 按账号命名空间隔离 L0/L1/L2/L3，对外仍只暴露一个 Gateway。 + +关键：**微信接入点把用户强绑定到某一台节点**，该用户所有轮次都在同一节点处理。因此： + +- TDAI sidecar **与处理该用户的 app 实例同节点**，localhost HTTP 调用；该用户的 L0/L1/L2/L3 都落在这台节点的本地 store，并**按账号命名空间隔离**（§8.4）。 +- 用户不跨节点漂移 → 本地 SQLite 记忆**不会碎片化**。§4.1 的「账号稳定归属同一节点」约束被微信绑定语义自动满足，不再是额外灰度门槛。 + +故障与迁移（DR，第一期）： + +- 第一期采用**硬绑定 + 故障暂停**：某节点宕机时，其上用户暂停服务，不做跨节点热备/重绑。 +- 中心 PG 持有全量 `messages`，TDAI 本地记忆是**可重建的派生数据**：节点恢复后 sidecar 重启，本地 SQLite 完好则续用；若本地 store 丢失，从中心 PG re-seed 重建（`scripts/seed_tdai_memory.py`），无需搬运 SQLite 文件。 +- 后续若上热备/重绑，再评估共享 store（TCVDB 或 PostgreSQL backend，§9 P4）；那时才需处理「用户在节点间漂移」的记忆一致性。 + +## 5. AI4ALL 代码接入点 + +### 5.1 新增 TDAI client 模块 + +建议新增： + +```text +app/tdai_client.py +tests/test_tdai_client.py +``` + +职责： + +- 读取配置。 +- 封装 `httpx` 调用。 +- 统一超时、鉴权、错误降级和日志。 +- 提供同步接口给当前同步 turn 热路径使用。 + +建议接口： + +```python +def tdai_session_key(account_id: str) -> str: + """Return stable TDAI session key scoped to one AI4ALL account.""" + return f"ai4all:{account_id}" + + +def recall( + *, + account_id: str, + query: str, + timeout_seconds: float | None = None, +) -> dict: + """Recall TDAI memory for one account. Return empty dict on disabled/failure.""" + + +def capture_turn( + *, + account_id: str, + session_id: int, + user_content: str, + assistant_content: str, + messages: list[dict] | None = None, +) -> dict: + """Capture one visible user/assistant turn into TDAI. Best-effort.""" +``` + +### 5.2 配置项 + +新增到 `app/config.py` 和 `.env.example`： + +```python +tdai_enabled: bool = False +tdai_gateway_url: str = "http://127.0.0.1:8420" +tdai_gateway_api_key: str = "" +tdai_recall_enabled: bool = True +tdai_capture_enabled: bool = True +tdai_recall_timeout_seconds: float = 1.5 +tdai_capture_timeout_seconds: float = 2.0 +tdai_recall_max_chars: int = 2500 +tdai_account_allowlist: str = "" +``` + +默认关闭，按账号 allowlist 灰度。 + +### 5.3 recall 注入点 + +位置：`app/turn_service.py` 的 `build_turn_llm_input()`。 + +⚠️ **依赖 TDAI `/recall` 补丁（§8.4 #6，P0 前置）**：TDAI recall 把最核心的「当前 query 相关 L1 记忆」放在 `prependContext`，把 persona/scene/tools 放在 `appendSystemContext`；但当前 Gateway `/recall` **只返回 `appendSystemContext`，丢弃 `prependContext`**（`server.ts:385`），且 `memory_count` 仍报 L1 条数 > 0。未打补丁就接 `/recall`，会拿到 persona/scene 却拿不到 query 相关记忆，PoC 必然误判召回效果。补丁后 response 同时返回两段。 + +建议流程： + +```text +1. 解析 account_id、session、history、agent_context。 +2. 如果 tdai_enabled 且账号命中灰度： + - 调 /recall，query = 当前用户文本。 + - 超时或失败返回空，不阻塞主对话。 + - 对 prepend/append 两段分别做最大字符数截断。 +3. 注入为 PromptBuilder.extra_blocks 中两个 volatile block： + - tdai_recall_memories（prepend_context：当前 query 相关 L1 记忆，每轮变化） + - tdai_recall_persona（append_context：persona/scene，变化慢） +4. metadata 记录 tdai_recall_enabled / status / mem_chars / persona_chars / memory_count / latency_ms。 +``` + +`trim_priority` 建议： + +```python +ContextBlock( + name="tdai_recall_memories", # prepend_context：当前 query 相关 L1 记忆 + text=formatted_tdai_memories, + section="volatile", + char_limit=settings.tdai_recall_max_chars, + trim_priority=25, +) +ContextBlock( + name="tdai_recall_persona", # append_context：persona/scene，相对稳定 + text=formatted_tdai_persona, + section="volatile", + char_limit=settings.tdai_recall_max_chars, + trim_priority=35, +) +``` + +优先级说明： + +- `tdai_recall_memories` 比 `carryover_summary` 稍高或接近，因为它是当前 query 相关材料；persona 块稍低，作为背景。 +- 低于 onboarding/context 基础身份，避免新用户流程被记忆块干扰。 + +### 5.4 capture 调用点 + +位置：主 LLM 回复成功、assistant message 落库后。 + +当前 `turn_service.py` 已调用 `write_memory()`。TDAI capture 可以与 `write_memory()` 同级，best-effort 异步执行： + +```text +1. 用户消息和助手回复都已确定。 +2. 入站内容未被同步审核拦截。 +3. 回复不是 no_reply / error fallback，或按配置允许记录 fallback。 +4. 调 /capture： + - session_key = ai4all:{account_id} + - session_id = 当前 AI4ALL session id + - user_content = 原始用户可见文本 + - assistant_content = 最终发送给用户的文本 +``` + +不要把 system prompt、tool result、TDAI recall context 写入 capture 的 messages，除非后续明确要做短期任务 offload。第一阶段只捕获用户和助手可见文本。 + +### 5.5 session end / seed + +AI4ALL session 生命周期由 `session_lifecycle.py` 控制。**session-end / shutdown flush 属 P2 范围**：pipeline 是进程内串行队列，需在 session 轮转、Dreaming run 后、以及 FastAPI shutdown 时给它 flush 机会，避免刚 capture 的 L0 未提纯丢失。调用： + +```http +POST /session/end +{"session_key": "ai4all:"} +``` + +`/seed` 用于历史回灌。建议先做脚本，不放进主链路： + +```text +scripts/seed_tdai_memory.py +``` + +输入： + +- `--account-id aid_...` +- `--since YYYY-MM-DD` +- `--limit-sessions N` +- `--dry-run` + +数据来源： + +- `messages` 表，按 `account_id` 和 session 分组。 +- 每组映射到 TDAI seed Format A/B。 + +### 5.6 模型主动 search 工具 + +对应 OpenClaw 的 `registerTool`（§2.3），把两个工具加进 `generate_reply_with_tools` 的 tool registry，让模型在生成中途主动召回： + +| 工具 | Gateway 端点 | 用途 | +|---|---|---| +| `tdai_memory_search` | `POST /search/memories` | 召回结构化长期记忆（偏好/事件/指令） | +| `tdai_conversation_search` | `POST /search/conversations` | 召回历史原始消息片段 | + +要求： + +- 两个工具调用都必须带 `session_key = ai4all:{account_id}`，复用 `tdai_client.tdai_session_key()` 唯一构造点。 +- ⚠️ **`/search/memories` 当前无 `session_key` 字段**（`gateway/types.ts:66`），`tdai_memory_search` 直接接会**全库跨账号搜 L1**。必须先完成 §8.4 #3（给 `MemorySearchRequest` 加 `session_key` 并下推过滤）才能开放此工具。`/search/conversations` 已带 `session_key`，我们的工具会强制传入，可先行；但注意它当前是**检索后过滤**（`conversation-search.ts:224`），多租户下召回质量需靠 §8.4 #2 的下推改善。 +- 复制 TDAI 的硬上限：两个工具**每轮合计最多 3 次**，超出直接拒绝，避免模型刷召回拖慢回复。 +- 工程量提示：tool registry 是 schema/handler 单一事实源 + 启动期双向校验（`app/tools/registry.py`），接入需改 `definitions.py`/`registry.py`、新增 handler、给 `TurnContext` 加 per-turn 计数、并按 allowlist/runtime flag gating，不是塞两个 schema 即可。 +- 失败降级：工具返回空结果而非抛错，不打断本轮生成。 +- 与被动 recall（§5.3）并存：被动注入覆盖高频上下文，主动 search 覆盖被动未命中的精确追问。 + +## 6. TDAI Gateway 部署配置 + +### 6.1 本地 Gateway + +Gateway 默认端口是 `8420`，建议绑定 loopback 并开启 Bearer token： + +```bash +export TDAI_GATEWAY_HOST=127.0.0.1 +export TDAI_GATEWAY_PORT=8420 +export TDAI_GATEWAY_API_KEY=change-me +export TDAI_DATA_DIR=/var/lib/ai4all/tdai +export TDAI_LLM_BASE_URL=https://api.deepseek.com/v1 +export TDAI_LLM_API_KEY=... +export TDAI_LLM_MODEL=deepseek-v4-flash +``` + +必须要求： + +- 非 loopback 暴露时必须设置 `TDAI_GATEWAY_API_KEY`。 +- AI4ALL 侧只配置同一个 token，不写死密钥。 +- TDAI data dir 纳入备份策略。 + +### 6.2 TDAI memory 配置建议 + +PoC 阶段建议： + +```yaml +server: + host: 127.0.0.1 + port: 8420 + apiKey: ${TDAI_GATEWAY_API_KEY} + +data: + baseDir: /var/lib/ai4all/tdai + +llm: + baseUrl: ${TDAI_LLM_BASE_URL} + apiKey: ${TDAI_LLM_API_KEY} + model: ${TDAI_LLM_MODEL} + timeoutMs: 120000 + +memory: + timezone: Asia/Shanghai + storeBackend: sqlite + capture: + enabled: true + l0l1RetentionDays: 0 + extraction: + enabled: true + enableDedup: true + maxMemoriesPerSession: 20 + pipeline: + everyNConversations: 5 + enableWarmup: true + l1IdleTimeoutSeconds: 600 + l2DelayAfterL1Seconds: 10 + l2MinIntervalSeconds: 900 + l2MaxIntervalSeconds: 3600 + sessionActiveWindowHours: 24 + recall: + enabled: true + maxResults: 5 + maxTotalRecallChars: 2000 + scoreThreshold: 0.3 + strategy: hybrid + timeoutMs: 1200 + embedding: + enabled: true + provider: dashscope # 非 local/none → 走 OpenAI 兼容远程路径 + baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1 + apiKey: ${TDAI_EMBEDDING_API_KEY} # 百炼 DashScope key，与 DeepSeek key 是两把不同的 key + model: text-embedding-v3 + dimensions: 1024 # 可降到 768/512 省存储 + sendDimensions: true # v3 支持指定维度；若兼容模式返回 400 改为 false + recallTimeoutMs: 1500 # recall 路径短超时，别拖慢出话 +``` + +说明： + +- **embedding 选型：阿里云百炼 `text-embedding-v3`（远程）**。决定性因素是轻量服务器内存：节点 A 已压着 PG + app + sidecar，<3G 可用，本地 `embeddinggemma-300m`（`node-llama-cpp` 进程内加载）常驻 ~0.5–1G，叠加后有真实 OOM 风险；远程方案在本机内存占用 ≈0，仅一次 HTTP。成本 0.0005 元/1k tokens（免费额度后），陪伴量级可忽略。 +- **LLM 与 embedding 是两把独立的 key**：TDAI 的 extraction(L1)/persona(L3) 复用 AI4ALL 的 DeepSeek `api_key` + `base_url`；embedding 单独用百炼 DashScope key。DeepSeek 无 embedding 端点，二者本就在 config 中分属不同字段，不冲突。 +- **故障可退化（已验证源码 `auto-recall.ts:searchHybrid`）**：hybrid 下 keyword(FTS) 与 embedding 并行、各自独立 try/catch。DashScope 超时/宕机时 embedding 腿返回空，FTS 腿照常出结果，本轮自动降级为关键词召回，不会整轮空；启动期 embedding 配置缺失则策略整体降级为 `keyword`。因此远程 embedding 是「锦上添花、坏了能退」的依赖，不是单点。 +- ⚠️ **`@node-rs/jieba` 必须装上**：它服务 hybrid 里的 FTS 腿（中文分词），与是否用远程 embedding 无关；装不上则中文 FTS 退化为单字切分，召回明显变差。 +- ⚠️ **隐私**：记忆文本会出到阿里云做 embedding；TDAI 写库前 `sanitize` 已剥离密码/验证码/证件号等敏感内容，但仍需业务侧确认可接受第三方云。 +- 上线前一次性验证：百炼兼容模式 `/embeddings` 是否接受请求体 `dimensions` 字段（接受→`sendDimensions:true`，报 400→false）。 +- 建议留一行 embedding token 计数日志/上限：embed 发生在 capture（后台、可批）+ recall（每轮），量级虽小但便于观测成本。 +- TDAI pipeline 内部 L1/L2/L3 队列当前是进程内串行队列；单 sidecar 下已经天然限流。 + +## 7. 数据与一致性策略 + +### 7.1 近期：双系统并行 + +AI4ALL PostgreSQL/SQLite 仍是业务真相；TDAI store 是记忆增强缓存。 + +| 数据 | 权威系统 | +|---|---| +| 账号、绑定、计费、审核、消息 | AI4ALL DB | +| SOUL/IDENTITY/USER/MEMORY | AI4ALL `account_profile_files` | +| TDAI L0/L1/L2/L3 | TDAI store | +| TDAI recall 注入 | 本轮 prompt-only | + +一致性要求： + +- TDAI capture 失败不回滚主 turn。 +- TDAI recall 失败不影响回复。 +- AI4ALL wipe/unbind 需要后续补 TDAI 删除能力；PoC 阶段至少在运行手册中要求删除对应 data dir 或 TCVDB 记录。 + +### 7.2 后续：统一 PostgreSQL 的前提 + +如果要达成 Gemini 原方案里的“所有记忆落 PG”，需要在 TDAI 仓库新增 store backend： + +```text +src/core/store/postgres.ts +``` + +它必须实现 `IMemoryStore`： + +- `upsertL0` / `searchL0Vector` / `searchL0Fts` / `queryL0ForL1` +- `upsertL1` / `searchL1Vector` / `searchL1Fts` / `queryL1` +- `pullProfiles` / `syncProfiles` / `deleteProfiles` +- `getCapabilities` + +AI4ALL 侧不能只靠 `database_url` 替 TDAI 完成这件事，因为 TDAI 进程是 Node.js，当前 store abstraction 与 AI4ALL `app/db/_backend.py` 没有关系。 + +## 8. 安全、隔离与降级 + +### 8.1 隔离红线 + +所有 AI4ALL -> TDAI 调用必须满足： + +```text +tdai_session_key == "ai4all:" + account_id +``` + +⚠️ **这是必要条件，但当前 TDAI 下不充分**：`session_key` 只隔离 L0 和 pipeline 状态，L1/L2/L3 召回仍是 dataDir 全局（§1 结论修正）。真正守住隔离红线还需 §8.4 的 TDAI 多租户改造；在改造完成前，一个 sidecar 只能服务单账号。 + +日志中必须记录： + +- `account_id` +- `tdai_session_key` hash 或原文的安全截断 +- recall/capture status +- latency +- chars / memory_count + +禁止： + +- 禁止跨账号复用 recall 结果缓存。 +- 禁止不带 `account_id` 查询 AI4ALL messages 后 seed。 +- 禁止把 OpenClaw `channel_account_id` 当业务账号。 + +### 8.2 敏感信息 + +AI4ALL 已有 moderation 和 dreaming 敏感过滤。TDAI capture 前仍应保守： + +- 同步审核拦截的入站内容不 capture。 +- 图片理解失败 fallback 不 capture 图片内容猜测。 +- 密码、验证码、token、证件、银行卡、精确地址和联系方式不应进入长期记忆。 + +TDAI 自身 extraction prompt 也有过滤，但 AI4ALL 不能只依赖下游。 + +### 8.3 失败降级 + +| 场景 | 行为 | +|---|---| +| `/health` 失败 | 标记 TDAI unavailable；主对话继续 | +| `/recall` 超时 | 不注入 TDAI block；metadata 记 `timeout` | +| `/capture` 失败 | 记录 warning；不影响用户回复 | +| Gateway 重启 | AI4ALL 自动重试下一轮；不补发本轮 capture，除非后续做 outbox | +| TDAI store 损坏 | 删除/恢复 TDAI data dir 后从 AI4ALL messages seed 重建 | + +第一阶段不建议为 capture 建复杂 outbox；等 PoC 证明有效后再补。 + +### 8.4 TDAI 多租户改造（路 B，P1 前置） + +经源码核实，TDAI standalone/sqlite store 是「单租户 per dataDir」：`session_key` 只隔离 L0 和 pipeline 状态，L1/L2/L3 召回是 dataDir 全局。要让一个 sidecar 安全服务多账号，需对 **TDAI 仓库**做以下改造（这是 TDAI 侧的开发量，AI4ALL 侧只消费结果）： + +| # | 改造项 | 证据/落点 | 说明 | +|---|---|---|---| +| 1 | **L1 搜索按 session 过滤** | `searchL1Fts`/`searchL1Vector`/`searchL1Hybrid`（`store/types.ts:269-271`）签名无 session；SQL `sqlite.ts:635-641`（`l1_vec MATCH`）/`796-805`（`l1_fts MATCH`）无 WHERE；`auto-recall.ts:125`、`tdai-core.ts:290` 调用未传 | L1Record 已存 `session_key`（`store/types.ts:65`），**schema 无需改**；给上述接口加 `sessionKey` 参数、把过滤**下推**进 SQL/向量检索（`WHERE session_key = ?`），并沿 `executeMemorySearch → searchMemories → performAutoRecall` 全链路透传 | +| 2 | **L0 搜索按 session 过滤** | `searchL0Vector`/`searchL0Fts`（`store/types.ts:295-296`）签名无 session；conversation search 是**检索后再 filter**（`tools/conversation-search.ts:224-228`），`/search/conversations` 的 `session_key` 还是可选（`gateway/types.ts:83-87`） | 同 #1 下推；**post-filter ≠ pushdown**——先全库取 topK 再 filter，多租户下 topK 可能全是别账号、filter 完剩 0，召回质量塌。必须下推 + 多租户模式 `session_key` 必填 | +| 3 | **`/search/memories` 加 `session_key`** | `MemorySearchRequest`（`gateway/types.ts:66`）无字段；handler `server.ts:431` 未传 | 加字段（强校验）+ 透传到 #1 的过滤；`tdai_memory_search` 工具必带（§5.6） | +| 4 | **L2/L3 per-account** | persona 读 `pluginDataDir/persona.md`（`auto-recall.ts:148`）、scene 读 `readSceneIndex(pluginDataDir)`（`auto-recall.ts:162`），均 dataDir 根级 | 改为按 `session_key` 派生 per-account 子目录；写入侧（persona `persona-generator.ts:185` / scene writer）与读取侧同步改 | +| 5 | **后台 pipeline 多租户限流 + L3 原子写** | 每 session 一组 timer（L1 idle/L2 schedule，`pipeline-manager.ts:181-182,422`）+ 全局 L3 runner（`pipeline-manager.ts:956-1000`）；L3 是裸文件 RMW 无锁（`persona-generator.ts:185`）；开关只有全局 `extraction.enabled`（`config.ts:488`），无 per-tenant 开关，也无 per-session 手动触发 L2/L3 的 API（只有 `flushSession`=L1） | 结构式下后台 pipeline 会 ×N（见下成本修正）→ 需**跨 core 全局并发上限**；本地 L3/L2 改**原子写（temp+rename）**；明确「单 dataDir 单进程」契约。详见 §8.5 | +| 6 | **`/recall` response 补 `prependContext`** | 仅返回 `appendSystemContext`（`server.ts:385`），丢弃 L1 query-time 记忆（`auto-recall.ts:186-204`） | `RecallResponse` 加 `prepend_context` 字段返回 `result.prependContext`；**P0 前置**（§5.3） | + +实现路线（两种满足方式，推荐结构式）： + +- **结构式（推荐）**：Gateway 维护 `Map`，每账号一份独立 dataDir（`baseDir/{account}`），按需 lazy 实例化 + LRU 淘汰空闲账号。隔离是**结构性**的（各账号物理分库分文件），不依赖「每条 SQL 都记得加 WHERE」，天然覆盖 #1/#2/#4/#7（reindex/计数类），最契合「按账号隔离是核心不变量」。仍需做 #3/#6 的接口字段。 + - ⚠️ **成本修正（2026-06 核查）**：先前「单 core ≈ sqlite 句柄 + 小状态，可控」**低估了**。每账号一个 core = **一套后台 timer + 一个独立 `SerialQueue`**，N 个活跃账号可**并发 N 路 LLM 提纯**（各 core 的串行队列互不相干），造成 DeepSeek 并发/成本尖峰 + N 套常驻 timer 内存。结构式必须在 TDAI 侧配一个**跨 core 的全局并发上限/共享工作队列**（改造项 #5），否则活跃账号一多后台 LLM 打满。 +- **过滤式**：单 TdaiCore + 共享 store，靠 #1/#2 的 `session_key` 过滤。内存省、后台 pipeline 被单一 `SerialQueue` 天然限到 concurrency=1，但「漏一个 WHERE 即跨账号泄漏」，与红线相悖，需配套强测试 + Gateway 层强制 `session_key` 放行。 + +无论哪种，#6（recall 补 `prependContext`）都必须做。最终内部实现由 TDAI 侧定，AI4ALL 只要求对外契约：`/recall`、`/search/*` 按 `session_key` 严格隔离且返回 query-time L1。 + +> 改造清单已抽成独立 issue 供 TDAI 团队推进：`docs/tech_design/tdai_multitenant_issue.md`。 + +wipe/unbind：账号硬删除路径 `unbind_and_wipe_account()` 需联动 TDAI namespace 清除——结构式下 = 删除该账号 dataDir（并卸载其 TdaiCore）；过滤式下 = 按 `session_key` 删 L1/L2/L3。**进入真实灰度前必须接通**，不能只靠手册删 data dir。 + +### 8.5 三条工程红线（结合 TDAI 源码核查） + +外部评审提出三条工程红线，逐条核查后**全部成立**，但有三处机制判断需按 TDAI 当前实现纠正。 + +#### 红线 1：接管 pipeline 控制权（不放任原生后台） + +- **成立**：TDAI 有后台常驻调度，非纯被动——每 session 一组 timer（L1 idle `l1IdleTimeoutSeconds` 默认 600s、L2 schedule `l2MaxIntervalSeconds` 默认 3600s，`pipeline-manager.ts:181-182,422`）、全局 L3 runner（`pipeline-manager.ts:956-1000`）、offload 5s poll + 24h reclaim（`offload/index.ts:946,1281`）。 +- **纠正点**：开关粒度是**全局** `extraction.enabled`（`config.ts:488`、`tdai-core.ts:151`），关掉则整条 L0→L1→L2→L3 都不跑，**无 per-tenant 开关**；手动触发只暴露 `flushSession(sessionKey)`（=L1 flush，`/session/end`）与 `/capture`（按阈值/idle 触发 L1），**没有按 session 手动触发 L2/L3 的公开 API**。所以「自己调 API 触发该用户的 L1/L3 提纯」当前**只能做到 L1**。 +- **对我们的结论**：结构式路线会把后台 pipeline ×N（§8.4 成本修正），是这条红线在多租户下的具象。优先方案 = **保留后台、但在 TDAI 侧加跨 core 全局并发上限**（改造项 #5）；若要完全接管算力（关 `extraction.enabled` + AI4ALL 按「聊满 N 句」自驱），前提是 TDAI 补 per-session L2/L3 触发 API，工程量更大，第一期不做。 + +#### 红线 2：暴力审计 SQL 与向量过滤 + +- **成立且范围更大**：除 L1 `searchL1Vector/Fts` 无过滤外，**L0 `searchL0Vector/Fts` 同样无过滤**（`store/types.ts:295-296`）；conversation search 是**检索后再 filter**（`tools/conversation-search.ts:224-228`）；`getAllL1Texts`/`rebuildFtsIndex`/`countL1` 等 reindex/统计类全跨 session。 +- **纠正点**：**TDAI store 是 SQLite/tcvdb，不是 PG**，「开 PostgreSQL 慢查询日志审计」用错了对象。审计应落在：(a) 代码层逐条核对 `store/sqlite.ts` 的 SELECT/向量检索是否带 session；(b) SQLite trace / query log；(c) Gateway 层强制每个 search 请求必带 `session_key` 再放行（防御纵深）。 +- **子要点**：**post-filter ≠ pushdown**。现状先全库取 topK 再 filter，多租户下 topK 可能全来自其他账号、filter 完剩 0，召回质量塌。必须把 session 过滤**下推进 SQL/向量检索**（§8.4 #1/#2），不是检索后过滤。 + +#### 红线 3：L3 画像读写一致性 + +- **成立且更脆**：L3 persona 是**磁盘文件 `persona.md`** 的裸 read-modify-write（`persona-generator.ts:185`，`fs.writeFile`，**无原子 rename、无锁、无 version**）；version 乐观锁只在 tcvdb 远程 sync（`tcvdb.ts:1117`），不保护本地文件。L2 scene 同样是无锁 `scene_blocks/*.md`。 +- **纠正点**：「用 PostgreSQL 乐观锁 / version 字段」**不适用**——L3 既不在 PG 也不在任何 SQL 表，是文件。正解 = (a) 原子写（temp + rename）；(b) 同账号 L3 串行。 +- **对我们的结论**：单进程内 TDAI 已用 `SerialQueue`（concurrency=1）全局串行 L1/L2/L3，故**单 sidecar 单进程下同账号 L3 不会自竞争**。竞争只在 (i) 同一 dataDir 被两个 sidecar 进程打开，或 (ii) P4 多实例扩容时复活。§4.2「用户硬绑定单节点 + 每节点单 sidecar」拓扑**天然规避** (i)(ii)。→ 当前拓扑**风险低**，但落一条硬约束：**禁止两个 sidecar 进程共享同一 dataDir**；P4 若上多实例/共享存储，必须补原子写或分布式锁。结构式额外好处：每账号独立 core+SerialQueue，跨账号 L3 本就串行隔离。 + +## 9. 分阶段实施 + +### P0：Sidecar 连通性 PoC + +目标：不改主链路，确认 Gateway 可运行；**单账号**形态（多租户改造前，一个 sidecar 只服务测试账号）。 + +任务： + +1. 本机启动 TDAI Gateway；装 `@node-rs/jieba`，配百炼 `TDAI_EMBEDDING_API_KEY`、`embedding.provider=dashscope`、`strategy=hybrid`（§6.2）。 +2. 一次性验证：百炼兼容模式 `/embeddings` 是否接受 `dimensions` 字段 → 定 `sendDimensions`。 +3. **TDAI 侧打 `/recall` 补丁（§8.4 #6）**：`RecallResponse` 补 `prepend_context`，否则 recall 拿不到 query-time L1，PoC 误判。 +4. 写最小 `app/tdai_client.py`。 +5. 加 health/recall/capture dry-run 测试。 +6. 用固定 `aid_806382741` 手工 capture 几轮，再 recall。 + +验收： + +- `GET /health` 返回 ok/degraded。 +- `/capture` 返回 `l0_recorded > 0`。 +- 多轮后 `/recall` 同时返回 `prepend_context`（L1 query 相关）和 persona/scene；hybrid 下 embedding 命中 > 0。 +- 断开百炼（错 key）时 recall 自动退回 FTS、不报错整轮空。 +- AI4ALL 不启动 TDAI 时测试仍绿。 + +### P0.5：TDAI 多租户改造（路 B，P1 前置，TDAI 侧） + +目标：让一个 sidecar 能按账号安全隔离 L0/L1/L2/L3，解除「一个 sidecar 只能服务单账号」的约束。详见 §8.4。 + +任务（TDAI 仓库）： + +1. 实现隔离（推荐结构式：per-account dataDir + `Map` lazy/LRU；或过滤式：L0/L1 搜索下推 `session_key` 过滤 + L2/L3 per-account 路径）。 +2. `/search/memories` 加 `session_key` 字段并下推过滤（§8.4 #3）；`/search/conversations` 多租户模式 `session_key` 必填。 +3. 提供 namespace wipe 能力（按 `session_key` 或删账号 dataDir），供 AI4ALL `unbind_and_wipe_account()` 联动。 +4. 结构式下加**跨 core 全局并发上限**，限制同时进行的 LLM 提纯路数（§8.4 #5、§8.5 红线 1）。 +5. 本地 L3/L2 写改**原子写（temp+rename）**；文档化「单 dataDir 单进程」契约（§8.5 红线 3）。 + +验收： + +- 两个账号交叉 capture 后，A 的 `/recall`、`/search/memories`、`/search/conversations` 不返回 B 的 L0/L1/persona/scene。 +- wipe 账号 A 后，A 的记忆全清，B 不受影响。 +- 多账号同时活跃时后台并发 LLM 提纯有上限、不随账号数线性膨胀。 + +### P1：主对话灰度接入 + +目标：按账号 allowlist 在真实 turn 中启用 recall/capture（**依赖 P0.5 完成**，一个 sidecar 服务多个灰度账号）。 + +任务： + +1. `build_turn_llm_input()` 调 recall 并注入 `tdai_recall_memories` + `tdai_recall_persona` 两个 block（§5.3）。 +2. 回复成功后 best-effort 调 capture。 +3. 在 tool registry 注册 `tdai_memory_search` / `tdai_conversation_search`，带 `ai4all:{account_id}`，每轮合计上限 3 次（§5.6）；`tdai_memory_search` 依赖 §8.4 #3。 +4. 接通 `unbind_and_wipe_account()` → TDAI namespace wipe（§8.4）。 +5. debug trace metadata 加 TDAI 字段。 +6. 聚焦测试覆盖： + - recall 成功注入 prompt（memories + persona 两块）。 + - recall 失败不影响 prompt。 + - capture 使用 `ai4all:{account_id}`。 + - search 工具命中正确账号，且每轮 3 次上限生效。 + - **跨账号隔离**：A 灰度账号的 recall/search 不返回 B 的记忆（依赖 P0.5）。 + - TDAI context 不写入 messages/daily notes。 + +验收： + +- allowlist 账号可看到 TDAI recall 生效。 +- 非 allowlist 账号行为逐字稳定。 +- 多账号共享同一 sidecar 下召回严格不跨账号。 +- TDAI Gateway 停止时主对话正常回复。 + +### P2：历史 seed 与质量评估 + +目标：让 TDAI 具备足够历史材料，并评估效果。 + +任务： + +1. 新增 `scripts/seed_tdai_memory.py`。 +2. 支持单账号 dry-run、limit、按日期过滤。 +3. 输出 seed 行数、session 数、失败数。 +4. 接 session-end / FastAPI shutdown flush（§5.5），确保 pipeline 退出前提纯。 +5. 建立人工评估样例： + - 用户偏好召回。 + - 过去事件追问。 + - 用户纠正旧记忆后是否以当前说法为准。 + +验收： + +- 主要测试账号 seed 成功。 +- 召回内容不跨账号。 +- 错误/敏感记忆不过度注入。 + +### P3：与 Dreaming 的职责重划 + +目标：决定长期方向。 + +可选路径： + +1. 保留 Dreaming 写 `USER.md/MEMORY.md`，TDAI 只做动态 recall。 +2. Dreaming 继续做审计，但减少自动写入整文件。 +3. TDAI 效果足够后，AI4ALL 自建 `account_memory_records/account_user_persona`，参考 `docs/plans/记忆机制_tdai化对齐.md`，逐步替代 sidecar。 +4. 给 TDAI 增加 PostgreSQL backend，保留 TDAI pipeline。 + +### P4：共享存储与多节点 + +触发条件： + +- TDAI 已进入核心体验。 +- 多节点账号迁移频繁。 +- 本地 sidecar SQLite 备份/迁移成本不可接受。 + +可选方案： + +| 方案 | 优点 | 风险 | +|---|---|---| +| TCVDB backend | TDAI 已支持，向量能力完整 | 新增云服务和凭证运维 | +| 新增 TDAI PostgreSQL backend | 统一中心 PG，符合 AI4ALL 厚节点方向 | 需要改 TDAI TypeScript store，工作量较大 | +| AI4ALL 自建 L1/L3 | 完全贴合现有 Python/PG 架构 | 放弃一部分 TDAI 现成 pipeline | + +> 中心 PG 已存在，PostgreSQL backend 在存储统一上最契合 AI4ALL 厚节点方向；但硬前置是 (a) 改 TDAI TypeScript store 实现 `IMemoryStore`（已知开发量）、(b) 中心 PG 装 `pgvector`（需实测，§13），并需接受 (c) 非 PG 节点（如节点 B）的 sidecar recall 每次跨内网连节点 A 的 PG——这与本地 SQLite recall 的低延迟相比是真实取舍。 + +## 10. 测试方案 + +### 10.1 文档阶段 + +本次只改文档，不运行代码测试。自审项： + +- 方案不把不存在的 Python SDK 作为实施依赖。 +- 方案不把 TDAI 当前不支持的 PostgreSQL backend 写成已可配置能力。 +- 所有隔离描述都使用 AI4ALL `account_id`。 +- 接入点与当前 `turn_service.py`、`PromptBuilder`、`memory_writer.py` 对齐。 + +### 10.2 实现阶段建议测试 + +聚焦测试： + +```bash +.venv/bin/pytest tests/test_prompt_builder.py -v +.venv/bin/pytest tests/test_turn_service.py -v +``` + +新增测试建议： + +```text +tests/test_tdai_client.py +tests/test_turn_tdai_memory.py +``` + +手工验证： + +```bash +.venv/bin/uvicorn app.main:app --reload --port 8180 +.venv/bin/python scripts/send_mock_turn.py --url http://127.0.0.1:8180 --text "hello" +.venv/bin/python scripts/check_prompt.py --url http://127.0.0.1:8180 --account aid_806382741 +``` + +预期： + +- TDAI disabled 时 prompt 不含 `tdai_recall_memories` / `tdai_recall_persona`。 +- TDAI enabled 且 recall 有内容时 prompt metadata 记录两块 chars。 +- messages 表只保存用户和助手可见文本。 + +## 11. 风险清单 + +| 风险 | 影响 | 缓解 | +|---|---|---| +| **多账号共享 sidecar 跨账号召回（blocker）** | L1/L2/L3 dataDir 全局，`session_key` 隔离不到，违反隔离红线 | P1 前置完成 §8.4 多租户改造（推荐结构式 per-account 物理隔离）；改造前一个 sidecar 只服务单账号；P1 测试含跨账号隔离用例 | +| **`/recall` 丢 `prependContext`** | 拿不到 query-time L1，PoC 误判召回无效 | P0 前置打 §8.4 #6 补丁，response 返回两段 | +| **结构式后台 pipeline ×N（成本）** | 活跃账号多时并发 N 路 LLM 提纯，DeepSeek 并发/成本尖峰 + N 套 timer 内存 | TDAI 侧加跨 core 全局并发上限/共享队列（§8.4 #5、§8.5 红线 1）；LRU 淘汰空闲 core | +| **L3 persona 文件并发覆盖** | 多进程/多实例下同账号 persona 丢更新 | 单 dataDir 单进程硬约束（§4.2 拓扑天然满足）；TDAI 侧改原子写；P4 多实例再上分布式锁（§8.5 红线 3） | +| **conversation/L0 搜索 post-filter 召回塌陷** | topK 全来自他账号、过滤后剩 0，召回质量差 | 把 session 过滤下推进检索而非检索后 filter（§8.4 #2、§8.5 红线 2） | +| TDAI Gateway API 仍在演进 | 接入代码可能跟随调整 | client 层封装，主链路只依赖内部接口 | +| sidecar SQLite 节点故障导致记忆丢失 | 该节点用户长期记忆暂失 | 微信绑定保证用户不漂移、本地记忆不分裂；中心 PG `messages` 可 re-seed 重建；仅频繁自由迁移时再升级 TCVDB/PG | +| 远程 embedding（百炼）超时/宕机 | 该轮丢失语义召回 | hybrid 并行、embedding 腿失败返空，自动退回 FTS 关键词召回（`auto-recall.ts:searchHybrid` 已验证）；`recallTimeoutMs` 短超时 | +| `@node-rs/jieba` 装不上 | 中文 FTS 退化为单字切分，召回变差 | 生产机预装并校验；纳入部署检查 | +| recall 延迟增加首 token 时间 | 微信回复变慢 | 1.2-1.5s 硬超时，失败跳过 | +| recall 内容污染长期记忆 | 错误记忆被二次写回 | prompt-only 注入，不写 messages/daily notes | +| TDAI extraction 产生敏感记忆 | 隐私风险 | capture 前过滤 + TDAI prompt + 后续 admin 观测 | +| 双记忆系统冲突 | prompt 里新旧记忆不一致 | 明确用户本轮优先；debug trace 记录来源 | +| 本地 data dir 未备份 | sidecar 记忆丢失 | 纳入备份；可从 AI4ALL messages seed 重建 | + +## 12. 最小实现摘要 + +第一版只需要这些代码改动： + +1. 新增 `app/tdai_client.py`。 +2. `app/config.py` / `.env.example` 新增 TDAI 配置。 +3. `app/turn_service.py`： + - `build_turn_llm_input()` 中调用 recall，并通过 `PromptBuilder.extra_blocks` 注入。 + - 回复成功后 best-effort capture。 +4. 新增 `tests/test_tdai_client.py` 和聚焦 turn 测试。 +5. 可选新增 `scripts/seed_tdai_memory.py`。 + +第一版不改： + +- 不改 `dreaming.py`。 +- 不改 DB schema。 +- 不改 `messages` 存储格式。 +- 不改 OpenClaw bridge。 +- 不引入 TDAI PostgreSQL backend。 + +## 13. 关键确认（2026-06）与遗留项 + +| # | 问题 | 结论 | +|---|---|---| +| 1 | 节点角色 | **已确认**：两节点都跑 AI4ALL app，节点 A 兼 PG。每节点各起一个 TDAI sidecar（§4.2）。 | +| 2 | pgvector 可用性 | **大概率可装，需实测**。仅在「托管 PG 扩展白名单不含 `vector` / 无 `CREATE EXTENSION` 权限 / PG 版本过低（需 11+，HNSW 需 pgvector 0.5+）/ 自建机无法装扩展二进制」时不可装。验证：`SELECT * FROM pg_available_extensions WHERE name='vector';`，或试 `CREATE EXTENSION vector;`。仅 P4 走 PG backend 时硬依赖；第一期 sidecar SQLite 不需要。 | +| 3 | TDAI 后台 LLM / embedding | **均已确认**。LLM 复用 DeepSeek `api_key`（覆盖 extraction/persona）；embedding 用阿里云百炼 `text-embedding-v3` 远程，`strategy=hybrid`（理由：轻量服务器 <3G 内存，本地模型有 OOM 风险；远程本机零内存、成本可忽略；hybrid 故障可退 FTS）。两把 key 独立（§6.2）。 | +| 4 | 故障策略 | **已确认（第一期）**：硬绑定 + 故障暂停，不做热备/重绑；DR 靠节点恢复后续用本地 SQLite 或从中心 PG re-seed（§4.2）。后续再评估热备。 | +| 5 | 多账号隔离模型 | **已确认（2026-06）**：走**路 B——改 TDAI 支持多租户**（§8.4），作为 P1 前置（P0.5）。`session_key` 仅隔离 L0/pipeline，L1/L2/L3 需 TDAI 侧改造；实现路线推荐结构式（per-account dataDir 物理隔离），过滤式为备选。改造前一个 sidecar 只服务单账号。 | +| 6 | `/recall` prependContext 补丁 | **已确认**：纳入 **P0 前置**。当前 `/recall` 只返回 `appendSystemContext`、丢弃 L1 query-time 记忆（`server.ts:385`）；P0 即在 TDAI 侧补 `prepend_context`，否则 PoC 误判（§5.3、§8.4 #6）。 | + +遗留待定（不阻塞 P0，但阻塞 P1）： + +- 路 B 实现路线终选（结构式 per-account dataDir vs 过滤式 session 下推）——由 TDAI 侧定，AI4ALL 只约束对外契约（§8.4）。 + +遗留待定（不阻塞 P0–P2）： + +- 百炼兼容模式 `/embeddings` 是否接受请求体 `dimensions` 字段（接受→`sendDimensions:true`，报 400→false）；上线前一次性验证。 +- `@node-rs/jieba` native 包在生产机的安装/兼容性（影响 hybrid 的 FTS 腿，装不上中文 FTS 退化为单字切分）。 +- 记忆文本出第三方云（百炼）的隐私签字。 +- pgvector 实测结果（决定 P4 是否走 PostgreSQL backend）。 diff --git a/docs/tdai_multitenant_issue.md b/docs/tdai_multitenant_issue.md new file mode 100644 index 00000000..6a15ad4a --- /dev/null +++ b/docs/tdai_multitenant_issue.md @@ -0,0 +1,65 @@ +# [TDAI] 多租户隔离改造 — 让一个 Gateway/Core 安全服务多账号 + +> 提给：TDAI（TencentDB-Agent-Memory）维护方 +> 提出方：AI4ALL 微信个人 AI 陪伴项目 +> 关联设计：AI4ALL `docs/tech_design/tdai_multitenant_design.md` §8.4 / §8.5 / P0.5 +> 证据基线：核对自 TDAI 仓库当前 `src/`（文件:行号见下） + +## 背景 + +AI4ALL 以 sidecar 形态接 TDAI：一个节点起一个 Gateway，服务该节点上的**多个**最终用户账号，账号隔离主键是 `session_key = "ai4all:{account_id}"`。 + +经源码核实，TDAI 当前 standalone/sqlite store 是「**单租户 per dataDir**」：`session_key` 只隔离 L0（原始对话写入）与 pipeline/session 状态，**L1/L2/L3 召回与搜索是 dataDir 全局的**。因此「一个 Gateway 靠 `session_key` 服务多账号」会跨账号召回，违反我们「按账号隔离」的硬不变量。 + +本 issue 列出让一个 Gateway 安全服务多账号所需的 TDAI 侧改造、验收标准，以及一个待 TDAI 侧拍板的实现路线选择。 + +## 一、隔离缺口清单（按需修复） + +| # | 缺口 | 证据（file:line） | 现状 | 需要 | +|---|---|---|---|---| +| 1 | **L1 搜索无 session 过滤** | `core/store/types.ts:269-270` `searchL1Vector(queryEmbedding, topK?, queryText?)` / `searchL1Fts(ftsQuery, limit?)`；SQL 见 `core/store/sqlite.ts:635-641`（`l1_vec ... WHERE embedding MATCH ?`）、`sqlite.ts:796-805`（`l1_fts MATCH ?`） | 签名与 SQL 都无 session 条件，返回全库 L1 | 加 `sessionKey` 参数；SQL 把 session 过滤**下推**（`WHERE session_key = ?` / 向量预过滤），并沿 `executeMemorySearch → searchMemories → performAutoRecall` 全链路透传 | +| 2 | **L0 搜索无 session 过滤** | `types.ts:295-296` `searchL0Vector` / `searchL0Fts` 签名无 session | conversation search 走的是**检索后过滤**：`core/tools/conversation-search.ts:224-228` `if (sessionFilter) results = results.filter(r => r.session_key === sessionFilter)` | 同 #1：把过滤**下推**进检索，而非取完 topK 再 filter（否则 topK 可能全来自其他账号，过滤后召回质量塌陷） | +| 3 | **`/search/memories` 请求无 `session_key`** | `gateway/types.ts:66-71` `MemorySearchRequest { query; limit?; type?; scene? }`；handler `gateway/server.ts:424-436` 未传 session | 该端点无法按账号约束，任意账号可搜全库 L1 | `MemorySearchRequest` 加 `session_key`（必填/强校验），handler 透传到 #1 的过滤 | +| 4 | **`/search/conversations` 的 `session_key` 是可选** | `gateway/types.ts:83-87` `session_key?`；`server.ts:447-458` 仅在提供时透传 | 缺省即跨账号 | 多租户模式下强制必填（缺省拒绝） | +| 5 | **L2/L3 是 dataDir 根级文件** | persona 读 `pluginDataDir/persona.md`（`core/hooks/auto-recall.ts:148-152`、写 `core/persona/persona-generator.ts:185`）；scene 读 `readSceneIndex(pluginDataDir)`（`auto-recall.ts:162-164`、写 `core/scene/*` → `scene_blocks/*.md`） | 多账号共用同一 dataDir 时直接互相覆盖/泄漏 | 按 `session_key` 派生 per-account 子目录；读写两侧同步改 | +| 6 | **`/recall` response 丢 `prependContext`** | `gateway/server.ts:385` 仅返回 `appendSystemContext`；query-time L1 在 `auto-recall.ts:186-204` 的 `prependContext` 被丢弃，但 `memory_count` 仍报 L1 条数 | 接入方拿到 persona/scene 却拿不到「当前 query 相关 L1 记忆」 | `RecallResponse` 加 `prepend_context` 返回 `result.prependContext`（AI4ALL 侧 P0 前置，最优先） | +| 7 | **reindex / 统计类全库跨 session** | `getAllL1Texts`/`getAllL0Texts`（`sqlite.ts:1767-1797`）、`rebuildFtsIndex`（`sqlite.ts:2207-2269`）、`countL1`/`countL0`（`sqlite.ts:1338/1745`） | 重建索引/计数跨所有账号 | 结构式天然隔离；过滤式需评估是否要 per-session 重建 | + +> #6 对 AI4ALL 最优先：未打补丁则 PoC 拿不到 query-time L1，必然误判召回效果。其余按多账号灰度前完成。 + +## 二、后台 pipeline 在多租户下的隐患（与隔离同等重要） + +TDAI 有后台常驻调度，非「纯被动」： + +- 每 session 一组 timer：L1 idle（`l1IdleTimeoutSeconds`，默认 600s）、L2 schedule（`l2MaxIntervalSeconds`，默认 3600s）——`utils/pipeline-manager.ts:181-182,422`。 +- 全局 L3 runner：`pipeline-manager.ts:956-1000`，`SerialQueue` concurrency=1，一次跑一轮但顺序触及所有账号 persona。 +- offload（若 `offload.enabled`）：5s poll + 24h reclaim——`offload/index.ts:946,1281`。 +- 开关粒度：只有全局 `extraction.enabled`（`config.ts:488`、`tdai-core.ts:151`），**无 per-tenant 开关**。 +- 手动触发：只暴露 `flushSession(sessionKey)`（=L1 flush，经 `POST /session/end`，`tdai-core.ts:359`）和 `/capture`（按阈值/idle 触发 L1）。**无按 session 手动触发 L2/L3 的公开 API。** + +多租户影响与诉求： + +1. **结构式路线（每账号一 core）会把后台 pipeline 乘以 N**：N 套 timer + N 个独立 SerialQueue → 可并发 N 路 LLM 提纯，造成后端 LLM 并发/成本尖峰。**诉求**：提供一个**跨 core/跨 session 的全局并发上限或共享工作队列**，把同时进行的 L1/L2/L3 提纯限到可控并发。 +2. 若接入方想完全接管算力（关掉后台、自行按「聊满 N 句」触发），**诉求**：提供**按 `session_key` 手动触发 L2/L3 的公开 API**（当前只有 L1 flush）。 +3. L3 一致性：persona 是磁盘文件 `persona.md` 的裸 read-modify-write（`persona-generator.ts:185`，无原子 rename / 无锁 / 无 version；version 乐观锁只在 tcvdb 远程 `tcvdb.ts:1117`，不保护本地文件）。**诉求**：本地 L3/L2 写改为**原子写（temp + rename）**，并明确「同一 dataDir 不可被两个进程同时打开」的契约；结构式下每账号独立 SerialQueue 已串行化同账号 L3。 + +## 三、实现路线（请 TDAI 侧拍板） + +满足上述隔离有两条路，AI4ALL 只约束**对外契约**（`/recall`、`/search/*` 按 `session_key` 严格隔离且返回 query-time L1），内部实现由 TDAI 定： + +- **结构式（AI4ALL 推荐）**：Gateway 维护 `Map`，每账号独立 dataDir（`baseDir/{account}`），lazy 实例化 + LRU 淘汰空闲账号。隔离是**结构性**的（物理分库分文件），天然覆盖 #1/#2/#5/#7，仍需做 #3/#4/#6 的接口字段。代价是 **N 个 core 常驻 + 后台 pipeline ×N**，必须配第二节第 1 条的全局并发上限。 +- **过滤式**：单 core + 共享 store，靠 #1/#2 的 `session_key` 下推过滤 + #5 的 per-account 路径。内存省、后台 pipeline 天然被单一 SerialQueue 限到 concurrency=1，但「漏一个 WHERE 即跨账号泄漏」，需配套强测试与 Gateway 层强制校验。 + +## 四、验收标准 + +1. 两账号交叉 capture 后，A 的 `/recall`、`/search/memories`、`/search/conversations` **不返回** B 的 L1/L0/persona/scene。 +2. `/recall` 同时返回 `prepend_context`（query-time L1）与 `appendSystemContext`（persona/scene）。 +3. 提供按 `session_key`（或删账号 dataDir）的 namespace wipe 能力，供接入方账号硬删除联动。 +4. 后台 pipeline 在多账号活跃时，并发 LLM 提纯有全局上限，不随账号数线性膨胀。 +5. 本地 L3/L2 写为原子写；明确单-dataDir-单-进程契约。 +6. 默认/示例配置文档化：多租户模式下 `/search/*` 的 `session_key` 为必填。 + +## 五、AI4ALL 侧已对齐的前置假设（供 TDAI 侧确认无冲突） + +- 部署：用户硬绑定单节点，每节点单 sidecar 单进程；故障暂停、不热备（第一期）。因此「单-dataDir-单-进程」契约对我们成立，第二节第 3 条的多实例竞争在第一期不触发。 +- 后端：TDAI store 用 SQLite + 远程 embedding（阿里云百炼 `text-embedding-v3`，OpenAI 兼容），`strategy=hybrid`；非 PG/pgvector。审计向量/SQL 请以 SQLite 为对象（SQLite trace / 代码层核对），不要按 PG 慢查询日志思路。 diff --git a/package.json b/package.json index 09abe4d8..0e0d9806 100644 --- a/package.json +++ b/package.json @@ -26,6 +26,8 @@ "export-tencent-vdb": "node ./bin/export-tencent-vdb.mjs", "build:read-local-memory": "tsc --project scripts/read-local-memory/tsconfig.json", "read-local-memory": "node ./bin/read-local-memory.mjs", + "dev-console": "node --env-file-if-exists=.env --import tsx scripts/dev-console/server.ts", + "import-psydt": "node --env-file-if-exists=.env --import tsx scripts/import-psydt.ts", "test": "vitest run", "test:watch": "vitest", "test:coverage": "vitest run --coverage", diff --git a/scripts/dev-console/README.md b/scripts/dev-console/README.md new file mode 100644 index 00000000..458b0ec8 --- /dev/null +++ b/scripts/dev-console/README.md @@ -0,0 +1,95 @@ +# TDAI dev-console + +A **build-free local console** for exercising the multi-tenant Gateway end to end +and watching the memory pyramid (L0→L1→L2→L3) build per account. Dev tool only — +not bundled, not published, no auth of its own. **Bind to localhost.** + +``` +browser (single-page UI, same-origin) + │ + ▼ +dev-console server (:8421) ──► TDAI Gateway (:8420, the 8 HTTP routes) + │ + └─► reads each account's dataDir read-only ──► the pyramid view +``` + +- **Proxies** the Gateway: the browser only talks to the console (same origin), + so the Gateway needs **no CORS** config and its API key never reaches the + client — the console injects `Authorization: Bearer` server-side. +- **Inspects** the on-disk pyramid the HTTP API can't fully show (L2 scenes, + L1 atoms, persona/raw text), via a read-only SQLite handle + file reads. +- **Zero duplicate config**: it loads the *same* `loadGatewayConfig()` the + Gateway uses, so baseDir / multiTenant / apiKey / port automatically agree. + +## Run + +**1. Start the Gateway.** Multi-tenant, and — to see L1→L3 actually grow — with a +real LLM: + +```bash +# connectivity only (no LLM): L0 + routing/isolation work; L1+ stay empty +TDAI_MULTI_TENANT=true \ +TDAI_DATA_DIR=/tmp/tdai-demo \ +node --import tsx src/gateway/server.ts + +# full effect (real LLM): the pyramid grows L1 → L2 → L3 +TDAI_MULTI_TENANT=true \ +TDAI_DATA_DIR=/tmp/tdai-demo \ +TDAI_LLM_API_KEY=sk-... \ +TDAI_LLM_BASE_URL=https://api.openai.com/v1 \ +TDAI_LLM_MODEL=gpt-4o \ +node --import tsx src/gateway/server.ts +``` + +**2. Start the console** (separate terminal, same env so it resolves the same +baseDir / port / apiKey): + +```bash +TDAI_MULTI_TENANT=true TDAI_DATA_DIR=/tmp/tdai-demo npm run dev-console +``` + +**3. Open** http://127.0.0.1:8421 + +## Env + +| var | default | purpose | +|---|---|---| +| `TDAI_MULTI_TENANT` | `false` | must match the Gateway | +| `TDAI_DATA_DIR` | gateway default | base data dir; must match the Gateway | +| `TDAI_GATEWAY_API_KEY` | unset | injected as Bearer if the Gateway requires it | +| `GATEWAY_URL` | `http://:` from config | override if the Gateway is elsewhere | +| `DEV_CONSOLE_PORT` | `8421` | console listen port | +| `DEV_CONSOLE_HOST` | `127.0.0.1` | console bind host | + +`TDAI_GATEWAY_PORT` / `TDAI_GATEWAY_HOST` (read by `loadGatewayConfig`) set both +the Gateway's bind and the console's default proxy target. + +## Walkthrough + +**Isolation (no LLM needed)** +1. With `ai4all:alice` in the session box, **Send turn** → the L0 bar ticks up. +2. Type `ai4all:bob`, **Inspect** → all bars 0. Alice's data is invisible to Bob. +3. **Search** (L0 conversations) for a word from Alice's turn: hits under + `ai4all:alice`, `total=0` under `ai4all:bob`. +4. **Ops → /health**: `active_cores`, `resident{count,limit}`, + `extraction{limit,active,waiting}`. +5. **Ops → /namespace/wipe** on Alice → her dataDir is gone; Bob untouched. + +**Overall effect (real LLM)** +6. Feed `ai4all:alice` a few informative turns. Toggle **auto 3s**. +7. Watch the pyramid fill: L0 immediately → L1 atoms in seconds → L2 scenes → + L3 persona. +8. **Recall** → `context` carries Alice's `` (highlighted) and + never Bob's. + +## Files + +| file | role | +|---|---| +| `server.ts` | console HTTP server — serves the UI, proxies `/api/gw/*`, `GET /api/inspect` | +| `inspector.ts` | read-only disk readers for L0/L1/L2/L3 (sqlite `query_only` + fs) | +| `public/index.html` | single-page vanilla UI (no build step) | + +Reuses `safeAccountDir` (`src/gateway/core-registry.ts`), `loadGatewayConfig` +(`src/gateway/config.ts`), and the read-only SQLite pattern from +`scripts/read-local-memory/`. diff --git a/scripts/dev-console/inspector.ts b/scripts/dev-console/inspector.ts new file mode 100644 index 00000000..c43a4755 --- /dev/null +++ b/scripts/dev-console/inspector.ts @@ -0,0 +1,383 @@ +/** + * dev-console / inspector — read-only disk readers for the memory pyramid. + * + * Reads a single account's on-disk dataDir and reports the L0/L1/L2/L3 layers + * so the console UI can visualise the pyramid building up after each turn. + * + * This NEVER writes. The SQLite handle is opened with `PRAGMA query_only = ON` + * (the proven pattern from `scripts/read-local-memory/read-local-memory.ts`), + * so it is safe to read `vectors.db` while the Gateway holds it open for writes + * — SQLite allows concurrent readers. Every layer is wrapped in try/catch and + * degrades to empty so a brand-new account (missing db / files) reads as zeros + * rather than erroring. + * + * Layout (verified against source), all relative to the account dataDir: + * L0 conversations/YYYY-MM-DD.jsonl + vectors.db table `l0_conversations` + * L1 vectors.db table `l1_records` + * L2 scene_blocks/*.md + .metadata/scene_index.json + * L3 persona.md + */ + +import { createRequire } from "node:module"; +import type { DatabaseSync } from "node:sqlite"; +import * as fs from "node:fs"; +import * as path from "node:path"; +import { safeAccountDir } from "../../src/gateway/core-registry.js"; + +const require = createRequire(import.meta.url); +function requireNodeSqlite(): typeof import("node:sqlite") { + return require("node:sqlite") as typeof import("node:sqlite"); +} + +const SQLITE_DB_NAME = "vectors.db"; +const SCENE_DIR = "scene_blocks"; +const SCENE_INDEX_REL = path.join(".metadata", "scene_index.json"); +const PERSONA_FILE = "persona.md"; +const CONVERSATIONS_DIR = "conversations"; + +const RECENT_LIMIT = 20; + +export interface L1Atom { + content: string; + type: string; + priority: number; + scene_name: string; + updated_time: string; +} + +export interface SceneMeta { + filename: string; + summary: string; + heat: number; + updated: string; +} + +export interface L0Turn { + role: string; + content: string; + recorded_at: string; +} + +export interface AccountSummary { + /** The true session_key (recovered from on-disk data — safeAccountDir is lossy). */ + sessionKey: string; + /** Friendly label: the part after the last ":" (e.g. "User2993"), else the full key. */ + display: string; + /** Namespace prefix before the first ":" (e.g. "psydt"), or "" when unprefixed. */ + namespace: string; + /** On-disk directory name under baseDir. */ + dir: string; + counts: { l0: number; l1: number; l2: number; l3: number }; +} + +export interface InspectResult { + sessionKey: string; + accountDir: string; + /** Whether the account's dataDir exists on disk at all. */ + exists: boolean; + counts: { l0: number; l1: number; l2: number; l3: number }; + l1ByType: { type: string; count: number }[]; + l1Recent: L1Atom[]; + scenes: SceneMeta[]; + persona: { text: string; chars: number } | null; + l0Recent: L0Turn[]; + /** Non-fatal problems encountered while reading (shown as hints in the UI). */ + notes: string[]; +} + +/** + * Map a session_key to its on-disk dataDir. Single-tenant: the shared baseDir. + * Multi-tenant: `baseDir/{safeAccountDir(key)}` — the SAME function the Gateway + * uses, so the console always points at the exact directory the Gateway writes. + */ +export function resolveAccountDir( + sessionKey: string, + baseDir: string, + multiTenant: boolean, +): string { + if (!multiTenant) return baseDir; + return path.join(baseDir, safeAccountDir(sessionKey)); +} + +function openReadonly(dbPath: string): DatabaseSync | null { + if (!fs.existsSync(dbPath)) return null; + const { DatabaseSync: DbSync } = requireNodeSqlite(); + const db = new DbSync(dbPath, { open: false }); + db.open(); + db.exec("PRAGMA query_only = ON"); + return db; +} + +function num(v: unknown): number { + return typeof v === "bigint" ? Number(v) : typeof v === "number" ? v : 0; +} + +/** Read L0 + L1 from vectors.db. Returns zeros if the db is absent. */ +function readSqliteLayers( + dataDir: string, + notes: string[], +): Pick & { + l0Count: number; + l1Count: number; +} { + const empty = { l1ByType: [], l1Recent: [], l0Recent: [], l0Count: 0, l1Count: 0 }; + let db: DatabaseSync | null = null; + try { + db = openReadonly(path.join(dataDir, SQLITE_DB_NAME)); + if (!db) return empty; + + const l0Count = num((db.prepare("SELECT COUNT(*) AS c FROM l0_conversations").get() as any)?.c); + const l1Count = num((db.prepare("SELECT COUNT(*) AS c FROM l1_records").get() as any)?.c); + + const l1ByType = (db + .prepare("SELECT type, COUNT(*) AS c FROM l1_records GROUP BY type ORDER BY c DESC") + .all() as any[]).map((r) => ({ type: String(r.type ?? ""), count: num(r.c) })); + + const l1Recent = (db + .prepare( + "SELECT content, type, priority, scene_name, updated_time FROM l1_records " + + "ORDER BY updated_time DESC LIMIT ?", + ) + .all(RECENT_LIMIT) as any[]).map((r) => ({ + content: String(r.content ?? ""), + type: String(r.type ?? ""), + priority: num(r.priority), + scene_name: String(r.scene_name ?? ""), + updated_time: String(r.updated_time ?? ""), + })); + + const l0Recent = (db + .prepare( + "SELECT role, message_text, recorded_at FROM l0_conversations " + + "ORDER BY timestamp DESC LIMIT ?", + ) + .all(RECENT_LIMIT) as any[]).map((r) => ({ + role: String(r.role ?? ""), + content: String(r.message_text ?? ""), + recorded_at: String(r.recorded_at ?? ""), + })); + + return { l1ByType, l1Recent, l0Recent, l0Count, l1Count }; + } catch (err) { + notes.push(`sqlite read failed: ${String((err as Error)?.message ?? err)}`); + return empty; + } finally { + try { + db?.close(); + } catch { + /* ignore */ + } + } +} + +/** Count L0 turns from the newest JSONL shard — fallback when there is no db. */ +function countL0FromJsonl(dataDir: string): { count: number; recent: L0Turn[] } { + try { + const dir = path.join(dataDir, CONVERSATIONS_DIR); + const files = fs + .readdirSync(dir) + .filter((f) => f.endsWith(".jsonl")) + .sort(); + let count = 0; + let lastLines: string[] = []; + for (const f of files) { + const lines = fs + .readFileSync(path.join(dir, f), "utf8") + .split("\n") + .filter((l) => l.trim().length > 0); + count += lines.length; + lastLines = lines; + } + const recent: L0Turn[] = lastLines + .slice(-RECENT_LIMIT) + .reverse() + .map((l) => { + try { + const o = JSON.parse(l); + return { role: String(o.role ?? ""), content: String(o.content ?? ""), recorded_at: String(o.recordedAt ?? "") }; + } catch { + return { role: "", content: l.slice(0, 200), recorded_at: "" }; + } + }); + return { count, recent }; + } catch { + return { count: 0, recent: [] }; + } +} + +/** Read L2 scenes from the index (preferred) or by scanning scene_blocks/. */ +function readScenes(dataDir: string, notes: string[]): SceneMeta[] { + try { + const indexPath = path.join(dataDir, SCENE_INDEX_REL); + if (fs.existsSync(indexPath)) { + const raw = JSON.parse(fs.readFileSync(indexPath, "utf8")); + if (Array.isArray(raw)) { + return raw.map((e: any) => ({ + filename: String(e.filename ?? ""), + summary: String(e.summary ?? ""), + heat: num(e.heat), + updated: String(e.updated ?? ""), + })); + } + } + // No index yet — list the raw scene block files so the count still shows. + const blocksDir = path.join(dataDir, SCENE_DIR); + if (fs.existsSync(blocksDir)) { + return fs + .readdirSync(blocksDir) + .filter((f) => f.endsWith(".md")) + .map((f) => ({ filename: f, summary: "(no index entry)", heat: 0, updated: "" })); + } + } catch (err) { + notes.push(`scene read failed: ${String((err as Error)?.message ?? err)}`); + } + return []; +} + +function readPersona(dataDir: string, notes: string[]): { text: string; chars: number } | null { + try { + const p = path.join(dataDir, PERSONA_FILE); + if (!fs.existsSync(p)) return null; + const text = fs.readFileSync(p, "utf8"); + if (text.trim().length === 0) return null; + return { text, chars: text.length }; + } catch (err) { + notes.push(`persona read failed: ${String((err as Error)?.message ?? err)}`); + return null; + } +} + +/** + * Inspect one account's on-disk memory pyramid. Pure read-only; safe to call + * repeatedly (e.g. an auto-refresh loop) while the Gateway is live. + */ +export function inspectAccount( + sessionKey: string, + baseDir: string, + multiTenant: boolean, +): InspectResult { + const notes: string[] = []; + const accountDir = resolveAccountDir(sessionKey, baseDir, multiTenant); + const exists = fs.existsSync(accountDir); + + const sql = readSqliteLayers(accountDir, notes); + + // L0: prefer the sqlite index; fall back to JSONL when the db is missing. + let l0Count = sql.l0Count; + let l0Recent = sql.l0Recent; + if (l0Count === 0 && l0Recent.length === 0) { + const jsonl = countL0FromJsonl(accountDir); + l0Count = jsonl.count; + l0Recent = jsonl.recent; + } + + const scenes = readScenes(accountDir, notes); + const persona = readPersona(accountDir, notes); + + return { + sessionKey, + accountDir, + exists, + counts: { + l0: l0Count, + l1: sql.l1Count, + l2: scenes.length, + l3: persona ? 1 : 0, + }, + l1ByType: sql.l1ByType, + l1Recent: sql.l1Recent, + scenes, + persona, + l0Recent, + notes, + }; +} + +/** + * Recover the true session_key for an account dir. `safeAccountDir` is a lossy + * one-way hash (`psydt:User2993` → `psydt_User2993.`), so the dir name + * alone can't be turned back into a key. Both the SQLite `l0_conversations` + * table and the L0 JSONL store the original `session_key` per row — read one. + */ +function recoverSessionKey(accountDir: string): string | null { + // Prefer sqlite (single row, indexed). + let db: DatabaseSync | null = null; + try { + db = openReadonly(path.join(accountDir, SQLITE_DB_NAME)); + if (db) { + const row = db.prepare("SELECT session_key FROM l0_conversations LIMIT 1").get() as any; + const key = row?.session_key ? String(row.session_key) : ""; + if (key) return key; + } + } catch { + /* fall through to JSONL */ + } finally { + try { db?.close(); } catch { /* ignore */ } + } + + // Fallback: first JSONL line carries `sessionKey`. + try { + const dir = path.join(accountDir, CONVERSATIONS_DIR); + const files = fs.readdirSync(dir).filter((f) => f.endsWith(".jsonl")).sort(); + for (const f of files) { + const first = fs.readFileSync(path.join(dir, f), "utf8").split("\n").find((l) => l.trim()); + if (first) { + const key = String(JSON.parse(first).sessionKey ?? ""); + if (key) return key; + } + } + } catch { + /* no recoverable key */ + } + return null; +} + +/** + * List every account with on-disk memory under `baseDir`, recovering each one's + * true session_key so the console can offer a real-account picker (the dir name + * is a lossy hash and cannot be reversed). Single-tenant has no per-account + * dirs, so it returns an empty list (the UI falls back to the free-text key). + * + * `seed-*` snapshot dirs (written by the single-dir /seed pipeline) and hidden + * dirs are skipped. Read-only and cheap — safe to call on an auto-refresh. + */ +export function listAccounts(baseDir: string, multiTenant: boolean): AccountSummary[] { + if (!multiTenant) return []; + + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(baseDir, { withFileTypes: true }); + } catch { + return []; + } + + const out: AccountSummary[] = []; + const seen = new Set(); + for (const ent of entries) { + if (!ent.isDirectory()) continue; + if (ent.name.startsWith(".") || ent.name.startsWith("seed-")) continue; + + const accountDir = path.join(baseDir, ent.name); + const sessionKey = recoverSessionKey(accountDir); + if (!sessionKey || seen.has(sessionKey)) continue; + seen.add(sessionKey); + + const info = inspectAccount(sessionKey, baseDir, multiTenant); + const colon = sessionKey.indexOf(":"); + out.push({ + sessionKey, + display: colon >= 0 ? sessionKey.slice(colon + 1) : sessionKey, + namespace: colon >= 0 ? sessionKey.slice(0, colon) : "", + dir: ent.name, + counts: info.counts, + }); + } + + // Group by namespace, then by display — stable, readable ordering in the picker. + out.sort((a, b) => + a.namespace === b.namespace + ? a.display.localeCompare(b.display, undefined, { numeric: true }) + : a.namespace.localeCompare(b.namespace), + ); + return out; +} diff --git a/scripts/dev-console/public/index.html b/scripts/dev-console/public/index.html new file mode 100644 index 00000000..b9046681 --- /dev/null +++ b/scripts/dev-console/public/index.html @@ -0,0 +1,355 @@ + + + + + +TDAI dev-console + + + +
+

TDAI dev-console

+ mode … + + + + + + + + +
+ +
+ +
+
+

① Capture — feed a turn (/capture)

+ + + + +
+ + +
+

L0 records immediately. L1→L3 grow asynchronously and only when the Gateway has a real LLM — Inspect again after a few seconds.

+
+ +
+

② Recall — what gets injected (/recall)

+ + +
+ + +
+
+
+ +
+

③ Search

+
+ + +
+
+ + + +
+
+
+ +
+

④ Ops

+
+ + + +
+
+
+ /seed — paste seed JSON + +
+
+
+
+ + +
+
+

Memory pyramid

+

Press Inspect.

+
+
+
+ + + + diff --git a/scripts/dev-console/server.ts b/scripts/dev-console/server.ts new file mode 100644 index 00000000..d982fbf8 --- /dev/null +++ b/scripts/dev-console/server.ts @@ -0,0 +1,183 @@ +/** + * dev-console / server — a build-free local console for exercising the TDAI + * Gateway end to end and watching the memory pyramid build per account. + * + * browser (single-page UI, same-origin) + * │ + * ▼ + * this console server (default :8421) + * ├─ proxies /api/gw/* → real Gateway (injects Bearer server-side) + * └─ GET /api/inspect → reads account dataDir read-only (the pyramid) + * + * Proxying means the browser only ever talks to this server (same origin), so + * the Gateway needs NO CORS config and its API key never reaches the client. + * + * Config is loaded with the SAME `loadGatewayConfig()` the Gateway uses, so the + * console resolves the identical baseDir / multiTenant / apiKey / port — point + * both at the same env and they automatically agree. + * + * Run: npm run dev-console (or: node --import tsx scripts/dev-console/server.ts) + * Dev tool only — not bundled, not published, no auth of its own. Bind localhost. + */ + +import http from "node:http"; +import fs from "node:fs"; +import { fileURLToPath } from "node:url"; +import { getEnv } from "../../src/utils/env.js"; +import { loadGatewayConfig } from "../../src/gateway/config.js"; +import { inspectAccount, listAccounts } from "./inspector.js"; + +const INDEX_HTML = fileURLToPath(new URL("./public/index.html", import.meta.url)); + +// Gateway routes the console is allowed to proxy to (closed allow-list). +const GW_POST_ROUTES = new Set([ + "/recall", + "/capture", + "/search/memories", + "/search/conversations", + "/session/end", + "/namespace/wipe", + "/seed", +]); + +const cfg = loadGatewayConfig(); +const baseDir = cfg.data.baseDir; +const multiTenant = cfg.data.multiTenant; +const apiKey = cfg.server.apiKey; + +// Where the real Gateway lives. Default to the loopback form of the Gateway's +// own configured host:port; override with GATEWAY_URL for a remote/odd setup. +const gwHost = cfg.server.host === "0.0.0.0" ? "127.0.0.1" : cfg.server.host; +const gatewayUrl = (getEnv("GATEWAY_URL") ?? `http://${gwHost}:${cfg.server.port}`).replace(/\/$/, ""); + +const consolePort = Number(getEnv("DEV_CONSOLE_PORT") ?? "8421"); +const consoleHost = getEnv("DEV_CONSOLE_HOST") ?? "127.0.0.1"; + +function sendJson(res: http.ServerResponse, status: number, body: unknown): void { + const payload = JSON.stringify(body); + res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" }); + res.end(payload); +} + +async function readBody(req: http.IncomingMessage): Promise { + const chunks: Buffer[] = []; + for await (const chunk of req) chunks.push(chunk as Buffer); + return Buffer.concat(chunks).toString("utf8"); +} + +function authHeaders(): Record { + return apiKey ? { Authorization: `Bearer ${apiKey}` } : {}; +} + +/** Forward a console request to the Gateway, returning its status + raw text. */ +async function proxyToGateway( + method: "GET" | "POST", + gwPath: string, + body?: string, +): Promise<{ status: number; text: string; contentType: string }> { + try { + const res = await fetch(`${gatewayUrl}${gwPath}`, { + method, + headers: { + ...(method === "POST" ? { "Content-Type": "application/json" } : {}), + ...authHeaders(), + }, + body: method === "POST" ? (body ?? "{}") : undefined, + }); + const text = await res.text(); + return { status: res.status, text, contentType: res.headers.get("content-type") ?? "application/json" }; + } catch (err) { + return { + status: 502, + text: JSON.stringify({ + error: `Cannot reach Gateway at ${gatewayUrl}${gwPath}: ${String((err as Error)?.message ?? err)}`, + hint: "Is the Gateway running? Set GATEWAY_URL if it is elsewhere.", + }), + contentType: "application/json", + }; + } +} + +const server = http.createServer(async (req, res) => { + try { + const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`); + const pathname = url.pathname; + const method = req.method ?? "GET"; + + // ── UI ──────────────────────────────────────────────────────────────── + if (method === "GET" && (pathname === "/" || pathname === "/index.html")) { + const html = fs.readFileSync(INDEX_HTML, "utf8"); + res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }); + res.end(html); + return; + } + + // ── console self-config (UI bootstraps mode/baseDir/gateway from this) ── + if (method === "GET" && pathname === "/api/config") { + sendJson(res, 200, { multiTenant, baseDir, gatewayUrl, hasApiKey: !!apiKey }); + return; + } + + // ── account picker (read-only disk) — real on-disk accounts + counts ──── + if (method === "GET" && pathname === "/api/accounts") { + try { + sendJson(res, 200, { accounts: listAccounts(baseDir, multiTenant), multiTenant }); + } catch (err) { + sendJson(res, 500, { error: String((err as Error)?.message ?? err) }); + } + return; + } + + // ── pyramid inspection (read-only disk) ──────────────────────────────── + if (method === "GET" && pathname === "/api/inspect") { + const sessionKey = url.searchParams.get("session_key") ?? ""; + if (multiTenant && !sessionKey.trim()) { + sendJson(res, 400, { error: "session_key is required in multi-tenant mode" }); + return; + } + try { + sendJson(res, 200, inspectAccount(sessionKey, baseDir, multiTenant)); + } catch (err) { + sendJson(res, 400, { error: String((err as Error)?.message ?? err) }); + } + return; + } + + // ── Gateway proxy ────────────────────────────────────────────────────── + if (pathname === "/api/gw/health" && method === "GET") { + const r = await proxyToGateway("GET", "/health"); + res.writeHead(r.status, { "Content-Type": r.contentType }); + res.end(r.text); + return; + } + + if (pathname.startsWith("/api/gw/") && method === "POST") { + const gwPath = pathname.slice("/api/gw".length); // → "/recall", "/search/memories", ... + if (!GW_POST_ROUTES.has(gwPath)) { + sendJson(res, 404, { error: `Unknown gateway route: ${gwPath}` }); + return; + } + const body = await readBody(req); + const r = await proxyToGateway("POST", gwPath, body); + res.writeHead(r.status, { "Content-Type": r.contentType }); + res.end(r.text); + return; + } + + sendJson(res, 404, { error: `Not found: ${method} ${pathname}` }); + } catch (err) { + sendJson(res, 500, { error: String((err as Error)?.message ?? err) }); + } +}); + +server.listen(consolePort, consoleHost, () => { + // eslint-disable-next-line no-console + console.log( + [ + `TDAI dev-console → http://${consoleHost}:${consolePort}`, + ` gateway : ${gatewayUrl}${apiKey ? " (Bearer auth on)" : ""}`, + ` baseDir : ${baseDir}`, + ` multiTenant: ${multiTenant}`, + ].join("\n"), + ); +}); diff --git a/scripts/import-psydt.ts b/scripts/import-psydt.ts new file mode 100644 index 00000000..da85eeba --- /dev/null +++ b/scripts/import-psydt.ts @@ -0,0 +1,209 @@ +/** + * import-psydt — bulk-import the PsyDTCorpus multi-turn counselling dialogues + * into the multi-tenant TDAI store, one account per conversation. + * + * Each corpus item is `{ id, normalizedTag, messages: [{role, content}] }`. + * We map: + * - account / username → `User{id}` (e.g. id 2993 → "User2993") + * - session_key → `psydt:User{id}` (namespaced, like `ai4all:alice`) + * - rounds → the user/assistant turns, paired; the `system` + * REBT-therapist prompt is dropped (it is a prompt, + * not the user's dialogue, and would pollute memory). + * + * Why a dedicated script and NOT the gateway `/seed` endpoint: + * `/seed` writes ALL sessions into a single `baseDir/seed-/` snapshot dir, + * so its output never lands in the per-account dirs that multi-tenant recall + * and the dev-console read. Here we call `executeSeed` once per account with + * `outputDir = baseDir/safeAccountDir(session_key)` — the EXACT directory the + * gateway's per-account core and the inspector resolve — so each imported user + * shows up as its own pyramid (L0→L1→L2→L3, with DashScope vectors). + * + * Config (LLM + embedding + baseDir) is loaded with the same `loadGatewayConfig` + * the gateway uses, so this imports against the identical DeepSeek + DashScope + * setup. Run it while the gateway is up — these are fresh keys, so no live core + * holds their dirs. + * + * Usage: + * node --env-file=.env --import tsx scripts/import-psydt.ts [--input ] + * [--limit N] [--only User2993[,User1977,...]] [--force] + * + * --limit N import only the first N conversations (cheap smoke test) + * --only ... import only the listed users (by "User" or bare id) + * --force re-import: delete an account's existing dir before seeding + * (default: skip accounts that already have L0 on disk) + */ + +import fs from "node:fs"; +import path from "node:path"; +import { loadGatewayConfig } from "../src/gateway/config.js"; +import { safeAccountDir } from "../src/gateway/core-registry.js"; +import { validateAndNormalizeRaw } from "../src/core/seed/input.js"; +import { executeSeed } from "../src/core/seed/seed-runtime.js"; +import type { Logger } from "../src/core/types.js"; + +const DEFAULT_INPUT = + "/Users/suchong/Data/PsyDTCorpus/PsyDTCorpus_train_mulit_turn_packing_longest20.json"; + +interface RawMsg { role: string; content: string } +interface CorpusItem { id: number | string; normalizedTag?: string; messages: RawMsg[] } + +// ── tiny arg parser ────────────────────────────────────────────────────── +function parseArgs(argv: string[]) { + const out: { input: string; limit?: number; only?: Set; force: boolean } = { + input: DEFAULT_INPUT, + force: false, + }; + for (let i = 0; i < argv.length; i++) { + const a = argv[i]; + if (a === "--input") out.input = argv[++i]!; + else if (a === "--limit") out.limit = Number(argv[++i]); + else if (a === "--force") out.force = true; + else if (a === "--only") { + out.only = new Set( + (argv[++i] ?? "") + .split(",") + .map((s) => s.trim()) + .filter(Boolean) + .map((s) => (s.startsWith("User") ? s : `User${s}`)), + ); + } + } + return out; +} + +// ── console logger (PipelineLogger = Logger) ───────────────────────────── +const logger: Logger = { + debug: () => {}, // seed is chatty at debug — mute + info: (m) => console.log(m), + warn: (m) => console.warn(m), + error: (m) => console.error(m), +}; + +/** + * Turn a flat corpus message list into seed "conversations" (a 2-D array of + * rounds). Drops `system`, drops empty content, and starts a new round at each + * `user` turn so each round is `[user, assistant]` (the corpus strictly + * alternates, but the boundary rule is robust to stray ordering). + */ +function toRounds(messages: RawMsg[]): RawMsg[][] { + const dialog = messages.filter( + (m) => (m.role === "user" || m.role === "assistant") && m.content && m.content.trim() !== "", + ); + const rounds: RawMsg[][] = []; + let cur: RawMsg[] = []; + for (const m of dialog) { + if (m.role === "user" && cur.length > 0) { + rounds.push(cur); + cur = []; + } + cur.push({ role: m.role, content: m.content }); + } + if (cur.length) rounds.push(cur); + return rounds; +} + +async function main(): Promise { + const args = parseArgs(process.argv.slice(2)); + const cfg = loadGatewayConfig(); + + if (!cfg.data.multiTenant) { + console.error("Refusing to import: gateway is in single-tenant mode (set TDAI_MULTI_TENANT=true)."); + process.exit(1); + } + if (!cfg.llm.apiKey) { + console.error("No LLM api key (TDAI_LLM_API_KEY) — L1/L2/L3 cannot build. Aborting."); + process.exit(1); + } + + // Mirror the gateway's /seed config assembly: memory config + injected llm. + const pluginConfig: Record = { + ...(cfg.memory as unknown as Record), + llm: { + enabled: true, + baseUrl: cfg.llm.baseUrl, + apiKey: cfg.llm.apiKey, + model: cfg.llm.model, + maxTokens: cfg.llm.maxTokens, + timeoutMs: cfg.llm.timeoutMs, + disableThinking: cfg.llm.disableThinking, + }, + }; + const embeddingOn = (cfg.memory as any)?.embedding?.enabled === true; + + const raw = JSON.parse(fs.readFileSync(args.input, "utf8")) as CorpusItem[]; + let items = Array.isArray(raw) ? raw : []; + if (args.only) items = items.filter((it) => args.only!.has(`User${it.id}`)); + if (args.limit != null) items = items.slice(0, args.limit); + + console.log( + `\n=== PsyDT import ===\n` + + ` input : ${args.input}\n` + + ` baseDir : ${cfg.data.baseDir}\n` + + ` llm : ${cfg.llm.model} @ ${cfg.llm.baseUrl}\n` + + ` embedding : ${embeddingOn ? "ON (vectors)" : "OFF (keyword only)"}\n` + + ` importing : ${items.length} conversation(s)\n` + + ` force : ${args.force}\n`, + ); + + let ok = 0; + let skipped = 0; + let failed = 0; + for (let i = 0; i < items.length; i++) { + const item = items[i]!; + const username = `User${item.id}`; + const sessionKey = `psydt:${username}`; + const outputDir = path.join(cfg.data.baseDir, safeAccountDir(sessionKey)); + const rounds = toRounds(item.messages); + const tag = item.normalizedTag ?? ""; + const header = `[${i + 1}/${items.length}] ${username} (${tag}, ${rounds.length} rounds)`; + + if (rounds.length === 0) { + console.warn(`${header} — no dialogue rounds, skipping`); + skipped++; + continue; + } + + // Idempotency: skip an account that already has L0 unless --force. + const dbPath = path.join(outputDir, "vectors.db"); + if (fs.existsSync(dbPath) && !args.force) { + console.log(`${header} — already imported (dir exists), skipping. Use --force to re-import.`); + skipped++; + continue; + } + if (args.force && fs.existsSync(outputDir)) { + fs.rmSync(outputDir, { recursive: true, force: true }); + } + + const input = validateAndNormalizeRaw([{ sessionKey, conversations: rounds }], { + sessionKey, + autoFillTimestamps: true, + }); + + console.log(`\n${header} → ${outputDir}`); + try { + const t0 = Date.now(); + const summary = await executeSeed(input, { + outputDir, + openclawConfig: {}, + pluginConfig, + logger, + }); + console.log( + `${header} ✓ l0=${summary.l0RecordedCount} rounds=${summary.roundsProcessed} ` + + `in ${((Date.now() - t0) / 1000).toFixed(1)}s`, + ); + ok++; + } catch (err) { + console.error(`${header} ✗ ${err instanceof Error ? err.message : String(err)}`); + failed++; + } + } + + console.log(`\n=== done: ${ok} imported, ${skipped} skipped, ${failed} failed ===`); + process.exit(failed > 0 ? 1 : 0); +} + +main().catch((err) => { + console.error(err); + process.exit(1); +}); diff --git a/scripts/smoke-recall.mjs b/scripts/smoke-recall.mjs new file mode 100755 index 00000000..26096108 --- /dev/null +++ b/scripts/smoke-recall.mjs @@ -0,0 +1,230 @@ +#!/usr/bin/env node +/** + * smoke-recall.mjs — §6 "is vector recall live?" one-click smoke test. + * + * Validates a REAL running TDAI Gateway end-to-end against its REAL configured + * embedding provider (e.g. Alibaba DashScope). It does exactly the manual §6 + * check from docs/tdai_gateway_integration.md, but repeatable and asserted: + * + * 1. GET /health — gateway up? embedding configured at all? + * 2. POST /capture — record a distinctive fact for a throwaway account + * 3. POST /search/memories — poll with a PARAPHRASE that shares NO keywords + * with the fact, until an L1 atom is recalled + * 4. assert strategy — PASS only if `hybrid`/`embedding` fired (vectors + * actually contributed); `fts`/`none` => FAIL + * 5. POST /namespace/wipe — clean up the throwaway account (multi-tenant) + * + * Why a paraphrase with no shared tokens: keyword (FTS/BM25) can't bridge it, so + * a hit can only come from vector similarity. That isolates the embedding path. + * + * No build, no deps — needs only Node >= 18 (global fetch). Run against a gateway + * that is already started with your real .env (DashScope key, TDAI_LLM_*, etc.): + * + * node scripts/smoke-recall.mjs + * TDAI_GATEWAY_URL=http://127.0.0.1:8420 \ + * TDAI_GATEWAY_API_KEY=*** node scripts/smoke-recall.mjs --timeout 120 + * + * Exit codes: 0 = PASS, 1 = FAIL (vectors not contributing / no recall), + * 2 = setup error (gateway unreachable, bad config, capture failed). + * + * Override the probe text for non-Chinese / domain-specific deployments via env: + * SMOKE_FACT_USER, SMOKE_FACT_ASSISTANT, SMOKE_KEYWORD, SMOKE_PARAPHRASE + * (keep KEYWORD a substring of the fact, and PARAPHRASE sharing no tokens with it). + */ + +// ── config ──────────────────────────────────────────────────────────────── +const argv = process.argv.slice(2); +const hasFlag = (name) => argv.includes(name); +const flagVal = (name) => { + const i = argv.indexOf(name); + return i >= 0 && i + 1 < argv.length ? argv[i + 1] : undefined; +}; + +const BASE = (process.env.TDAI_GATEWAY_URL || flagVal("--url") || "http://127.0.0.1:8420").replace(/\/$/, ""); +const API_KEY = process.env.TDAI_GATEWAY_API_KEY || flagVal("--api-key") || ""; +const TIMEOUT_MS = Number(flagVal("--timeout") ?? process.env.SMOKE_TIMEOUT_SEC ?? 90) * 1000; +const POLL_MS = Number(flagVal("--poll") ?? 4) * 1000; +const KEEP = hasFlag("--keep") || hasFlag("--no-wipe"); + +// A unique throwaway account so the test is isolated and cleanly wipeable. +const SESSION = + flagVal("--session") || + process.env.SMOKE_SESSION || + `smoke:recall:${Date.now()}:${Math.floor(Math.random() * 1e6)}`; + +// Distinctive fact + a paraphrase that shares NO surface tokens with it. +const FACT_USER = process.env.SMOKE_FACT_USER || "我每个周末都会去山里露营和钓鱼，这是我最喜欢的放松方式。"; +const FACT_ASSISTANT = process.env.SMOKE_FACT_ASSISTANT || "听起来很惬意，亲近大自然确实能让人充电。"; +const KEYWORD = process.env.SMOKE_KEYWORD || "露营"; // appears in the fact → formation probe +const PARAPHRASE = process.env.SMOKE_PARAPHRASE || "户外探险活动"; // no shared tokens, near in meaning + +const VECTOR_STRATEGIES = new Set(["hybrid", "embedding"]); + +// ── tiny http helper ──────────────────────────────────────────────────────── +const headers = { "content-type": "application/json", ...(API_KEY ? { authorization: `Bearer ${API_KEY}` } : {}) }; + +async function req(method, path, body) { + const ac = new AbortController(); + const timer = setTimeout(() => ac.abort(), 30_000); + try { + const res = await fetch(`${BASE}${path}`, { + method, + headers, + body: body === undefined ? undefined : JSON.stringify(body), + signal: ac.signal, + }); + const text = await res.text(); + let json; + try { + json = text ? JSON.parse(text) : {}; + } catch { + json = { _raw: text }; + } + return { status: res.status, json }; + } finally { + clearTimeout(timer); + } +} + +const log = (...a) => console.log("[smoke]", ...a); +const fail = (msg) => { + console.error("\n[smoke] ❌ FAIL —", msg); + process.exit(1); +}; +const setupError = (msg) => { + console.error("\n[smoke] ⚠️ SETUP ERROR —", msg); + process.exit(2); +}; +const delay = (ms) => new Promise((r) => setTimeout(r, ms)); + +// ── flow ───────────────────────────────────────────────────────────────────── +async function main() { + log(`gateway : ${BASE}`); + log(`auth : ${API_KEY ? "Bearer (set)" : "none"}`); + log(`session : ${SESSION}`); + log(`fact : "${FACT_USER}"`); + log(`paraphrase: "${PARAPHRASE}" (shares no keywords with the fact)`); + log(""); + + // 1. Health + embedding-intent gate ----------------------------------------- + let health; + try { + health = await req("GET", "/health"); + } catch (err) { + setupError(`cannot reach gateway at ${BASE} (${err?.cause?.code || err?.name || err}). Is it running?`); + } + if (health.status === 401 || health.status === 403) { + setupError(`/health returned ${health.status} — unexpected (health should be unauthenticated). Check the URL.`); + } + if (health.status !== 200 || health.json.status !== "ok") { + setupError(`/health not ok: status=${health.status} body=${JSON.stringify(health.json)}`); + } + const multiTenant = health.json.multi_tenant === true; + const emb = health.json.embedding; + log(`health : ok version=${health.json.version} multi_tenant=${multiTenant}`); + if (emb) { + log(`embedding : configured=${emb.configured} provider=${emb.provider} model=${emb.model ?? "?"} dims=${emb.dimensions ?? "?"} recall=${emb.recallStrategy}`); + } else { + log(`embedding : (no embedding block in /health — older gateway build)`); + } + // Fail fast on the single most common cause, with no waiting. + if (emb && emb.configured === false) { + fail( + `embedding is NOT configured (provider=${emb.provider}). Vector recall cannot work.\n` + + ` Fix the memory.embedding.* block in tdai-gateway.yaml (DashScope text-embedding-v3, the key),\n` + + ` set memory.recall.strategy: hybrid, and restart the gateway. (Embedding is yaml-only — there are no TDAI_EMBEDDING_* envs.)`, + ); + } + + // 2. Capture a distinctive fact ---------------------------------------------- + const cap = await req("POST", "/capture", { + session_key: SESSION, + session_id: `smoke-${Date.now()}`, + user_content: FACT_USER, + assistant_content: FACT_ASSISTANT, + }); + if (cap.status !== 200) { + setupError(`/capture failed: status=${cap.status} body=${JSON.stringify(cap.json)}`); + } + log(`capture : ok l0_recorded=${cap.json.l0_recorded} scheduler_notified=${cap.json.scheduler_notified}`); + log(""); + log(`polling /search/memories every ${POLL_MS / 1000}s for up to ${TIMEOUT_MS / 1000}s (waiting for L1 to form)…`); + + // 3. Poll the paraphrase search until a vector hit or timeout ----------------- + const search = (query) => req("POST", "/search/memories", { query, session_key: SESSION, limit: 5 }); + + const deadline = Date.now() + TIMEOUT_MS; + let lastStrategy = "none"; + let lastTotal = 0; + let sawVector = false; + let formationConfirmed = false; // keyword probe found the atom at all + + while (Date.now() < deadline) { + const para = await search(PARAPHRASE); + if (para.status !== 200) { + setupError(`/search/memories failed: status=${para.status} body=${JSON.stringify(para.json)}`); + } + lastStrategy = para.json.strategy ?? "none"; + lastTotal = para.json.total ?? 0; + + if (lastTotal > 0 && VECTOR_STRATEGIES.has(lastStrategy)) { + sawVector = true; + break; // PASS condition met — vectors fired on a no-keyword query + } + + // Diagnostic-only keyword probe: did L1 form at all yet? + if (!formationConfirmed) { + const probe = await search(KEYWORD); + if (probe.status === 200 && (probe.json.total ?? 0) > 0) formationConfirmed = true; + } + + const elapsed = Math.round((TIMEOUT_MS - (deadline - Date.now())) / 1000); + log(` …${elapsed}s paraphrase: total=${lastTotal} strategy=${lastStrategy} | L1 formed=${formationConfirmed}`); + await delay(POLL_MS); + } + + // 4. Verdict ------------------------------------------------------------------ + log(""); + const cleanup = async () => { + if (KEEP) { + log(`cleanup : skipped (--keep). Throwaway account left as "${SESSION}".`); + return; + } + if (!multiTenant) { + log(`cleanup : single-tenant mode — /namespace/wipe is refused; smoke data stays in the shared store.`); + log(` (Run a multi-tenant gateway, or use a disposable dataDir, to auto-clean.)`); + return; + } + const w = await req("POST", "/namespace/wipe", { session_key: SESSION }); + log(w.status === 200 && w.json.wiped ? `cleanup : wiped throwaway account ✓` : `cleanup : wipe returned status=${w.status} body=${JSON.stringify(w.json)}`); + }; + + if (sawVector) { + await cleanup(); + log(""); + log(`✅ PASS — vector recall is LIVE. A no-keyword paraphrase recalled the fact via strategy="${lastStrategy}".`); + process.exit(0); + } + + // Failed — distinguish the two root causes for an actionable message. + await cleanup(); + if (formationConfirmed) { + fail( + `L1 formed (the fact is searchable by keyword), but the paraphrase only ever returned ` + + `total=${lastTotal} strategy="${lastStrategy}" — vectors are NOT contributing.\n` + + ` => Embedding is enabled but not actually producing hits. Check the DashScope key/endpoint,\n` + + ` the model/dimensions in memory.embedding.*, and that memory.recall.strategy is hybrid/embedding.\n` + + ` (Watch the gateway logs for embedding errors during the search.)`, + ); + } else { + fail( + `could not confirm L1 formation within ${TIMEOUT_MS / 1000}s (keyword "${KEYWORD}" never matched, ` + + `paraphrase strategy="${lastStrategy}").\n` + + ` => Either extraction is slow/off (check TDAI_LLM_* and the gateway logs for L1 activity),\n` + + ` or the run needs longer — retry with --timeout 180. The fact may also have been\n` + + ` extracted with different wording; try SMOKE_KEYWORD/SMOKE_PARAPHRASE overrides.`, + ); + } +} + +main().catch((err) => setupError(err?.stack || String(err))); diff --git a/src/core/persona/persona-generator.ts b/src/core/persona/persona-generator.ts index cba05901..3caadbd6 100644 --- a/src/core/persona/persona-generator.ts +++ b/src/core/persona/persona-generator.ts @@ -5,6 +5,8 @@ import fs from "node:fs/promises"; import path from "node:path"; +import { atomicWriteFile } from "../../utils/atomic-write.js"; +import { fileWriteMutex } from "../../utils/keyed-mutex.js"; import { formatForLLM } from "../../utils/time.js"; import { CleanContextRunner } from "../../utils/clean-context-runner.js"; import { CheckpointManager } from "../../utils/checkpoint.js"; @@ -56,8 +58,23 @@ export class PersonaGenerator { /** * Execute local persona generation without advancing checkpoint. + * + * The entire read → LLM-write → read-back → final-write critical section runs + * under a per-file mutex (keyed by persona.md's absolute path) shared with the + * L2 scene-nav writer, so the two cannot lost-update each other. The lock is + * held for the full ~180s LLM run by design — see {@link fileWriteMutex}. */ async generateLocalPersona(triggerReason?: string): Promise { + const personaPath = path.join(this.dataDir, "persona.md"); + return fileWriteMutex.run(personaPath, () => + this.runGenerationLocked(triggerReason, personaPath), + ); + } + + private async runGenerationLocked( + triggerReason: string | undefined, + personaPath: string, + ): Promise { const startMs = Date.now(); this.logger?.debug?.(`${TAG} Starting generation: reason="${triggerReason ?? "none"}"`); @@ -65,8 +82,6 @@ export class PersonaGenerator { const cp = await cpManager.read(); this.logger?.debug?.(`${TAG} Checkpoint: total_processed=${cp.total_processed}, last_persona_at=${cp.last_persona_at}`); - const personaPath = path.join(this.dataDir, "persona.md"); - // 1. Read existing persona (strip navigation) let existingPersona: string | undefined; try { @@ -180,9 +195,19 @@ export class PersonaGenerator { } // 11. Append fresh scene navigation and write final content + // + // Lost-update safety: persona.md is written by two stages on DIFFERENT + // SerialQueues — this L3 generator and the L2 scene-extractor's + // updateSceneNavigation. atomicWriteFile() alone guards a TORN READ but NOT + // a lost update, so both writers' full read-modify-write now runs under the + // shared per-path fileWriteMutex (acquired in generateLocalPersona above, + // spanning the ~180s LLM run). That makes this regen and an interleaving L2 + // nav write mutually exclusive — neither can clobber the other. + // (profile-sync is a third writer but only on the tcvdb path, and + // tcvdb+multiTenant is rejected at config load.) const nav = generateSceneNavigation(index); const finalContent = nav ? `${personaText}\n\n${nav}\n` : personaText; - await fs.writeFile(personaPath, finalContent, "utf-8"); + await atomicWriteFile(personaPath, finalContent); const elapsedMs = Date.now() - startMs; this.logger?.info(`${TAG} Persona written (${finalContent.length} chars) in ${elapsedMs}ms`); diff --git a/src/core/profile/profile-sync.ts b/src/core/profile/profile-sync.ts index 3d931db8..7030230f 100644 --- a/src/core/profile/profile-sync.ts +++ b/src/core/profile/profile-sync.ts @@ -1,6 +1,7 @@ import { createHash } from "node:crypto"; import fs from "node:fs/promises"; import path from "node:path"; +import { atomicWriteFile } from "../../utils/atomic-write.js"; import type { IMemoryStore, ProfileRecord, ProfileSyncRecord } from "../store/types.js"; import { readSceneIndex, syncSceneIndex } from "../scene/scene-index.js"; import { generateSceneNavigation, stripSceneNavigation } from "../scene/scene-navigation.js"; @@ -58,7 +59,7 @@ async function refreshPersonaNavigation(dataDir: string): Promise { const index = await readSceneIndex(dataDir); const nav = generateSceneNavigation(index); const finalContent = nav ? `${body}\n\n${nav}\n` : `${body}\n`; - await fs.writeFile(personaPath, finalContent, "utf-8"); + await atomicWriteFile(personaPath, finalContent); } export async function listLocalProfiles(dataDir: string): Promise { diff --git a/src/core/prompts/persona-generation.ts b/src/core/prompts/persona-generation.ts index 92ec8c6b..2d84d7e8 100644 --- a/src/core/prompts/persona-generation.ts +++ b/src/core/prompts/persona-generation.ts @@ -32,13 +32,22 @@ export interface PersonaPromptResult { const PERSONA_SYSTEM_PROMPT = `# 🧬 Persona Architect - Incremental Evolution Protocol -**Output language contract**: -- Detect the dominant language from the changed scene content. -- \`persona.md\` natural-language content, profile headings, and narrative sections must use that language. -- For English scene content, output English persona headings and English body text. -- For non-Chinese scene content, do not emit Chinese persona headings. -- If the language is ambiguous, default to English. -- Keep Markdown syntax, file name \`persona.md\`, tool names, and structural markers in English. +**Output language contract (STRICT — read first)**: +- Detect the dominant natural language of the changed scene content below. +- EVERY piece of natural-language text in \`persona.md\` — the Archetype sentence, + all chapter/section headings, body paragraphs, bullet points, and trait notes — + MUST be written in that detected language. +- If the scenes are Chinese, the persona MUST be fully Chinese: Chinese headings + AND Chinese body. If the scenes are English, write it in English. Mirror the + user's language; never switch to a different one. +- The output template further below is written in English ONLY to convey its + structure. TRANSLATE its headings (Archetype, Basic Information, the Chapter + titles, etc.) into the detected language — do NOT copy the English words + verbatim when the scenes are not English. +- If the language is genuinely ambiguous, follow the language already used in the + existing \`persona.md\`; if there is none, default to the scene language. +- Keep ONLY the following in English: Markdown structural syntax, the file name + \`persona.md\`, and tool names (write/edit). Everything a human reads is localized. 请你结合已有的 persona.md 和新增/变化的 block 信息深度分析，然后使用文件工具将结果写入 \`persona.md\` 文件。 @@ -91,7 +100,9 @@ const PERSONA_SYSTEM_PROMPT = `# 🧬 Persona Architect - Incremental Evolution ## 📝 输出模板 (The Persona Template) -请参考以下格式，使用 **write** 工具写入最终内容。可以做自主调整（信息不足时可以减少或新增 chapter）（**必须保持 Markdown 格式**）： +请参考以下格式，使用 **write** 工具写入最终内容。可以做自主调整（信息不足时可以减少或新增 chapter）（**必须保持 Markdown 格式**）。 + +⚠️ **下面模板中的英文标题（Archetype / Basic Information / Chapter 标题等）仅用于示意结构。请按对话主语言翻译这些标题后再写入——中文对话就用中文标题与中文正文，不要照抄英文。** \`\`\`\`markdown # User Narrative Profile @@ -147,6 +158,29 @@ const PERSONA_SYSTEM_PROMPT = `# 🧬 Persona Architect - Incremental Evolution // User Prompt builder (dynamic data) // ============================ +/** + * Whether the text is predominantly Chinese (more CJK chars than Latin letters). + * Used to inject concrete Chinese headings — a weak / no-thinking model copies + * the English template headings verbatim despite the language contract, so for + * Chinese scenes we hand it the exact headings to use, data-adjacent. + */ +function isChineseDominant(text: string): boolean { + const cjk = (text.match(/[一-鿿]/g) ?? []).length; + const latin = (text.match(/[A-Za-z]/g) ?? []).length; + return cjk > latin; +} + +/** Mandatory Chinese heading set, injected when scenes are Chinese-dominant. */ +const ZH_HEADING_DIRECTIVE = `**【强制使用以下中文标题 — 禁止照抄模板里的英文标题】** +\`persona.md\` 的所有标题与正文必须为中文，结构标题严格使用： +- 一级标题：\`# 用户叙事档案\` +- 概要块标签：\`> **原型**:\` / \`> **基本信息**\` / \`> **长期偏好**\` +- 章节：\`## 📖 第一章：背景与当前状态\` / \`## 🎨 第二章：生活纹理\` / \`## 🤖 第三章：交互与认知协议\` / \`## 🧩 第四章：深层洞察与演化\` +- 三级子标题：\`### 3.1 如何沟通\` / \`### 3.2 如何思考\` +（只有 Markdown 语法、文件名 \`persona.md\`、工具名保持英文。） + +`; + export function buildPersonaPrompt(params: PersonaPromptParams): PersonaPromptResult { const { mode, @@ -176,9 +210,11 @@ export function buildPersonaPrompt(params: PersonaPromptParams): PersonaPromptRe `面对变化场景，自主判断处理方式：强化（佐证已有洞察）/ 补充（新维度）/ 修正（矛盾）/ 重构（结构调整）/ 不改（无有用新增内容）。\n` : ""; - const userPrompt = `**Output language**: \`persona.md\` headings and body text must use the dominant language of the changed scene content below. For English scene content, use English persona headings. + const headingDirective = isChineseDominant(changedScenesContent) ? ZH_HEADING_DIRECTIVE : ""; + + const userPrompt = `**Output language**: Write the ENTIRE \`persona.md\` (headings AND body) in the dominant language of the changed scene content below. Chinese scenes → Chinese headings and Chinese body; do not translate them into English. -**⏰ 更新时间**: ${currentTime} +${headingDirective}**⏰ 更新时间**: ${currentTime} **模式**: ${modeLabel} ${triggerSection} ## 📊 统计 diff --git a/src/core/scene/scene-extractor.ts b/src/core/scene/scene-extractor.ts index 6d8a4b84..9986ead2 100644 --- a/src/core/scene/scene-extractor.ts +++ b/src/core/scene/scene-extractor.ts @@ -18,6 +18,8 @@ import fs from "node:fs/promises"; import path from "node:path"; +import { atomicWriteFile } from "../../utils/atomic-write.js"; +import { fileWriteMutex } from "../../utils/keyed-mutex.js"; import { formatForLLM } from "../../utils/time.js"; import { CleanContextRunner } from "../../utils/clean-context-runner.js"; import { CheckpointManager } from "../../utils/checkpoint.js"; @@ -444,34 +446,39 @@ export class SceneExtractor { */ private async updateSceneNavigation(): Promise { const personaPath = path.join(this.dataDir, "persona.md"); - const index = await readSceneIndex(this.dataDir); - const nav = generateSceneNavigation(index); - - let existing = ""; - try { - existing = await fs.readFile(personaPath, "utf-8"); - } catch { - // No persona file yet — PersonaGenerator will create it with navigation. - // Don't write a navigation-only file. - this.logger?.debug?.(`${TAG} updateSceneNavigation() skipped: no persona file yet, waiting for PersonaGenerator`); - return; - } - - if (!existing.trim() && !nav) return; + // The whole read-modify-write runs under the shared per-path mutex so this + // L2 nav write and the L3 persona regen (which take the same lock, keyed by + // persona.md's absolute path) cannot lost-update each other — they run on + // different SerialQueues and would otherwise interleave. atomicWriteFile + // alone only prevents a torn read. See fileWriteMutex / persona-generator.ts. + await fileWriteMutex.run(personaPath, async () => { + const index = await readSceneIndex(this.dataDir); + const nav = generateSceneNavigation(index); + + let existing = ""; + try { + existing = await fs.readFile(personaPath, "utf-8"); + } catch { + // No persona file yet — PersonaGenerator will create it with navigation. + // Don't write a navigation-only file. + this.logger?.debug?.(`${TAG} updateSceneNavigation() skipped: no persona file yet, waiting for PersonaGenerator`); + return; + } - const stripped = stripSceneNavigation(existing).trimEnd(); + if (!existing.trim() && !nav) return; - // If the persona body is empty (only navigation existed), don't overwrite - // with a navigation-only file. Let PersonaGenerator handle full generation. - if (!stripped) { - this.logger?.debug?.(`${TAG} updateSceneNavigation() skipped: persona body is empty, waiting for PersonaGenerator`); - return; - } + const stripped = stripSceneNavigation(existing).trimEnd(); - const updated = nav ? `${stripped}\n\n${nav}\n` : `${stripped}\n`; + // If the persona body is empty (only navigation existed), don't overwrite + // with a navigation-only file. Let PersonaGenerator handle full generation. + if (!stripped) { + this.logger?.debug?.(`${TAG} updateSceneNavigation() skipped: persona body is empty, waiting for PersonaGenerator`); + return; + } - // persona.md is at dataDir root, no subdir needed - await fs.writeFile(personaPath, updated, "utf-8"); + const updated = nav ? `${stripped}\n\n${nav}\n` : `${stripped}\n`; + await atomicWriteFile(personaPath, updated); + }); } } diff --git a/src/core/scene/scene-index.ts b/src/core/scene/scene-index.ts index 84f7d471..ddccc21e 100644 --- a/src/core/scene/scene-index.ts +++ b/src/core/scene/scene-index.ts @@ -4,6 +4,7 @@ import fs from "node:fs/promises"; import path from "node:path"; +import { atomicWriteFile } from "../../utils/atomic-write.js"; import { parseSceneBlock } from "./scene-format.js"; export interface SceneIndexEntry { @@ -57,7 +58,7 @@ export async function writeSceneIndex( ): Promise { const indexPath = path.join(dataDir, ".metadata", "scene_index.json"); await fs.mkdir(path.dirname(indexPath), { recursive: true }); - await fs.writeFile(indexPath, JSON.stringify(entries, null, 2), "utf-8"); + await atomicWriteFile(indexPath, JSON.stringify(entries, null, 2)); } /** diff --git a/src/core/seed/seed-runtime.ts b/src/core/seed/seed-runtime.ts index 3cfeb89d..70d9116d 100644 --- a/src/core/seed/seed-runtime.ts +++ b/src/core/seed/seed-runtime.ts @@ -34,6 +34,14 @@ import type { const TAG = "[memory-tdai] [seed]"; +/** + * Flush budget for `pipeline.destroy()` in seed (ms). Unlike the live gateway — + * whose 2s default is bounded by the 3s gateway_stop hook — seed teardown has + * no such race, so allow enough time for the final L2 scene + L3 persona work + * to flush before the store closes. One LLM round-trip is ~10–15s; 30s covers it. + */ +const SEED_DESTROY_TIMEOUT_MS = 30_000; + // ============================ // Seed pipeline options // ============================ @@ -67,9 +75,27 @@ async function createSeedPipeline(opts: SeedRuntimeOptions): Promise<{ pipeline: // Parse config — all values come from pluginConfig (or parseConfig defaults) const cfg = parseConfig(pluginConfig); + // ── Seed-only pacing overrides ─────────────────────────────────────────── + // Seed feeds rounds synchronously and pauses every `everyNConversations` + // rounds to wait for that L1 batch to drain (`waitForL1Idle`). Two live-mode + // defaults sabotage that wait and make seeds ~10x slower than the actual LLM + // work: + // 1. `enableWarmup` doubles the L1 threshold (1→2→4→8…), so a fixed 5-round + // seed batch stops lining up with the threshold — L1 doesn't fire at the + // boundary and `conversation_count` stays > 0. + // 2. `l1IdleTimeoutSeconds` defaults to 600s, so the idle-timer fallback is + // far longer than waitForL1Idle's 120s/300s caps. + // The result: at most boundaries waitForL1Idle never sees idle and burns its + // full cap (~100s) doing nothing. Pin the threshold to everyN (warmup off) so + // L1 fires exactly at each boundary, and shrink the idle timer so the tail / + // any missed boundary still drains in seconds. Both are seed-local — the live + // gateway pipeline is untouched. + cfg.pipeline.enableWarmup = false; + cfg.pipeline.l1IdleTimeoutSeconds = Math.min(cfg.pipeline.l1IdleTimeoutSeconds, 5); + logger.info( `${TAG} Creating seed pipeline: outputDir=${outputDir}, ` + - `everyN=${cfg.pipeline.everyNConversations}, l1Idle=${cfg.pipeline.l1IdleTimeoutSeconds}s, ` + + `everyN=${cfg.pipeline.everyNConversations}, l1Idle=${cfg.pipeline.l1IdleTimeoutSeconds}s, warmup=off, ` + `l2Delay=${cfg.pipeline.l2DelayAfterL1Seconds}s, l2Min=${cfg.pipeline.l2MinIntervalSeconds}s, l2Max=${cfg.pipeline.l2MaxIntervalSeconds}s`, ); @@ -96,13 +122,17 @@ async function createSeedPipeline(opts: SeedRuntimeOptions): Promise<{ pipeline: logger.info(`${TAG} Seed using standalone LLM: model=${cfg.llm.model}`); } - // Use shared factory for everything: store init, L1 runner, persister, destroy + // Use shared factory for everything: store init, L1 runner, persister, destroy. + // Seed has no gateway_stop hook racing it, so give destroy a generous flush + // budget — the live default (2s) is tuned to stay under that hook and would + // cut off the final L2/L3 work (persona, last scene) mid-flight. const pipeline = await createPipeline({ pluginDataDir: outputDir, cfg, openclawConfig, logger, l1LlmRunner, + destroyTimeoutMs: SEED_DESTROY_TIMEOUT_MS, }); // Wire L2 runner via shared factory (same logic as index.ts live runtime) diff --git a/src/core/tdai-core.ts b/src/core/tdai-core.ts index 9185e51d..a8b3a34c 100644 --- a/src/core/tdai-core.ts +++ b/src/core/tdai-core.ts @@ -47,6 +47,7 @@ import { createL3Runner, } from "../utils/pipeline-factory.js"; import { MemoryPipelineManager } from "../utils/pipeline-manager.js"; +import type { ConcurrencyLimiter } from "../utils/async-semaphore.js"; import { CheckpointManager } from "../utils/checkpoint.js"; import { SessionFilter } from "../utils/session-filter.js"; import { StandaloneLLMRunnerFactory } from "../adapters/standalone/llm-runner.js"; @@ -66,6 +67,14 @@ export interface TdaiCoreOptions { sessionFilter?: SessionFilter; /** Plugin instance ID for metric reporting. */ instanceId?: string; + /** + * Optional shared limiter gating this core's L1/L2/L3 extraction runners. + * + * The multi-tenant CoreRegistry passes ONE limiter to every core so total + * concurrent background extraction across all accounts stays bounded + * (design §8.4 #5). Omit it for the unbounded single-core default. + */ + extractionLimiter?: ConcurrencyLimiter; } // ============================ @@ -80,6 +89,7 @@ export class TdaiCore { private runnerFactory: LLMRunnerFactory; private sessionFilter: SessionFilter; private instanceId?: string; + private extractionLimiter?: ConcurrencyLimiter; // Lazy-initialized resources private vectorStore?: IMemoryStore; @@ -130,6 +140,7 @@ export class TdaiCore { this.runnerFactory = opts.hostAdapter.getLLMRunnerFactory(); this.sessionFilter = opts.sessionFilter ?? new SessionFilter([]); this.instanceId = opts.instanceId; + this.extractionLimiter = opts.extractionLimiter; } // ============================ @@ -149,7 +160,7 @@ export class TdaiCore { // Create pipeline manager (sync — does not need store) if (this.cfg.extraction.enabled) { - this.scheduler = createPipelineManager(this.cfg, this.logger, this.sessionFilter); + this.scheduler = createPipelineManager(this.cfg, this.logger, this.sessionFilter, this.extractionLimiter); // Wire runners after store is ready (or after store init fails — runners // still work in degraded mode with JSONL fallback and no embedding) this.storeReady diff --git a/src/core/tools/conversation-search.ts b/src/core/tools/conversation-search.ts index d25ae178..cdd62410 100644 --- a/src/core/tools/conversation-search.ts +++ b/src/core/tools/conversation-search.ts @@ -221,6 +221,18 @@ export async function executeConversationSearch(params: { } // ── Apply session key filter ── + // This is a *post-filter* over the topK that the L0 vector/FTS search already + // returned — those store methods have no session pushdown. Two regimes: + // - structural multi-tenant: each account's core has its OWN store holding + // only that account's L0, so every row already matches and this is a no-op. + // Isolation there is physical (per-dataDir), NOT this filter. + // - single-tenant shared store: many sessions live in one store, so the + // filter is REQUIRED. Caveat: because it runs AFTER topK, a query whose + // topK is dominated by other sessions can yield few/zero rows post-filter + // (design gap #2). The real fix is to push session_key INTO the L0 + // SQL/vector query; deferred because the multi-tenant product gets + // isolation structurally, making the pushdown a single-tenant-only quality + // improvement. if (sessionFilter) { const preFilterCount = results.length; results = results.filter((r) => r.session_key === sessionFilter); diff --git a/src/gateway/config.test.ts b/src/gateway/config.test.ts new file mode 100644 index 00000000..893ae987 --- /dev/null +++ b/src/gateway/config.test.ts @@ -0,0 +1,45 @@ +import { describe, it, expect } from "vitest"; +import { loadGatewayConfig } from "./config.js"; +import { parseConfig } from "../config.js"; + +/** + * Guard: structural multi-tenant isolation is physical (per-account dataDir + + * its own SQLite). The TCVDB backend ignores dataDir and routes every core to + * ONE shared database/collection set, so the combination would leak L1/L0 + * recall and search across accounts. `loadGatewayConfig` must refuse it at load + * time. See the guard in config.ts and the review item (tcvdb isolation gap). + * + * A full `memory` override is passed so these assertions don't depend on the + * repo-root tdai-gateway.yaml that loadGatewayConfig picks up from CWD. + */ +describe("loadGatewayConfig — multi-tenant store-backend guard", () => { + const tcvdb = () => + parseConfig({ storeBackend: "tcvdb", tcvdb: { url: "https://x", apiKey: "k", database: "db" } }); + + it("throws when multiTenant + storeBackend=tcvdb (would leak across accounts)", () => { + expect(() => + loadGatewayConfig({ + data: { baseDir: "/tmp/tdai-test", multiTenant: true }, + memory: tcvdb(), + }), + ).toThrow(/multi-tenant/i); + }); + + it("allows multiTenant + sqlite (the supported structural-isolation backend)", () => { + expect(() => + loadGatewayConfig({ + data: { baseDir: "/tmp/tdai-test", multiTenant: true }, + memory: parseConfig({}), + }), + ).not.toThrow(); + }); + + it("allows single-tenant + tcvdb (one shared core, one database — no cross-account routing)", () => { + expect(() => + loadGatewayConfig({ + data: { baseDir: "/tmp/tdai-test", multiTenant: false }, + memory: tcvdb(), + }), + ).not.toThrow(); + }); +}); diff --git a/src/gateway/config.ts b/src/gateway/config.ts index 4fff5813..41b535c5 100644 --- a/src/gateway/config.ts +++ b/src/gateway/config.ts @@ -62,6 +62,65 @@ export interface GatewayConfig { data: { /** Base directory for TDAI data storage. */ baseDir: string; + /** + * Multi-tenant mode. When `true`, one Gateway process safely serves many + * end-user accounts: each `session_key` is routed to its own `TdaiCore` + * with a dedicated dataDir under `baseDir/{account}` (structural isolation + * — see design §8.4). When `false` (default), the Gateway behaves exactly + * as before: a single shared core rooted at `baseDir`, `session_key` only + * isolates L0/pipeline state. + * + * In multi-tenant mode `session_key` becomes **required** on every routed + * endpoint (`/recall`, `/capture`, `/search/*`, `/session/end`). + * + * env: `TDAI_MULTI_TENANT` + * yaml: `data.multiTenant` + */ + multiTenant: boolean; + /** + * Max concurrent background extraction runs (L1/L2/L3) across ALL per-account + * cores. Structural multi-tenant gives each account its own pipeline, so `N` + * active accounts otherwise fan out to up to `~3N` simultaneous LLM + * extraction calls (design §8.4 #5). One shared limiter caps the total. + * + * - `> 0` — hard cap. + * - `0` — unbounded. + * - unset — multi-tenant defaults to a safe internal cap; single-tenant + * stays unbounded (a lone core can't fan out). + * + * env: `TDAI_MAX_CONCURRENT_EXTRACTIONS` + * yaml: `data.maxConcurrentExtractions` + */ + maxConcurrentExtractions?: number; + /** + * Max number of per-account cores kept resident at once (multi-tenant LRU + * eviction). When a request would exceed this, the least-recently-used idle + * core is flushed + torn down to bound process memory. Must exceed the peak + * number of concurrently active accounts. + * + * - `> 0` — keep at most this many cores warm. + * - `0` / unset — unlimited (legacy: every account stays warm forever). + * + * Ignored in single-tenant mode. + * + * env: `TDAI_MAX_RESIDENT_CORES` + * yaml: `data.maxResidentCores` + */ + maxResidentCores?: number; + /** + * Idle time-to-live (ms) for resident per-account cores (multi-tenant). A + * periodic sweep evicts any idle (unpinned) core whose last request was + * longer ago than this, reclaiming long-tail accounts during quiet periods + * instead of waiting for an LRU push. Complements `maxResidentCores` (a count + * bound) with a time bound. + * + * - `> 0` — evict cores idle longer than this. + * - `0` / unset — disabled (default). Ignored in single-tenant mode. + * + * env: `TDAI_CORE_IDLE_TTL_MS` + * yaml: `data.coreIdleTtlMs` + */ + coreIdleTtlMs?: number; }; llm: StandaloneLLMConfig; /** Parsed memory-tdai plugin config (recall, capture, extraction, pipeline, etc.). */ @@ -124,6 +183,13 @@ export function loadGatewayConfig(overrides?: Partial): GatewayCo const rawBaseDir = env("TDAI_DATA_DIR") ?? str(dataConfig, "baseDir") ?? resolveDefaultDataDir(); const home = getEnv("HOME") ?? getEnv("USERPROFILE") ?? "/tmp"; const baseDir = rawBaseDir.startsWith("~/") ? path.join(home, rawBaseDir.slice(2)) : rawBaseDir; + const multiTenant = envBool("TDAI_MULTI_TENANT") ?? bool(dataConfig, "multiTenant") ?? true; + const maxConcurrentExtractions = + envInt("TDAI_MAX_CONCURRENT_EXTRACTIONS") ?? num(dataConfig, "maxConcurrentExtractions"); + const maxResidentCores = + envInt("TDAI_MAX_RESIDENT_CORES") ?? num(dataConfig, "maxResidentCores"); + const coreIdleTtlMs = + envInt("TDAI_CORE_IDLE_TTL_MS") ?? num(dataConfig, "coreIdleTtlMs"); // LLM config const llmConfig = obj(fileConfig, "llm"); @@ -144,7 +210,7 @@ export function loadGatewayConfig(overrides?: Partial): GatewayCo const base: GatewayConfig = { server: { port, host, apiKey, corsOrigins }, - data: { baseDir }, + data: { baseDir, multiTenant, maxConcurrentExtractions, maxResidentCores, coreIdleTtlMs }, llm, memory, }; @@ -152,14 +218,37 @@ export function loadGatewayConfig(overrides?: Partial): GatewayCo // Merge overrides one level deep so partial `server`/`data`/`llm` patches // (frequently used by e2e tests) don't accidentally drop sibling fields // such as `corsOrigins` introduced after they were written. - if (!overrides) return base; - return { - ...base, - ...overrides, - server: { ...base.server, ...(overrides.server ?? {}) }, - data: { ...base.data, ...(overrides.data ?? {}) }, - llm: { ...base.llm, ...(overrides.llm ?? {}) }, - }; + const effective: GatewayConfig = overrides + ? { + ...base, + ...overrides, + server: { ...base.server, ...(overrides.server ?? {}) }, + data: { ...base.data, ...(overrides.data ?? {}) }, + llm: { ...base.llm, ...(overrides.llm ?? {}) }, + } + : base; + + // Fail fast on the one config combo that silently breaks tenant isolation. + // + // Multi-tenant isolation in this fork is *structural*: each account gets its + // own dataDir + its own SQLite file (distinct L0/L1 tables, persona.md, …). + // The TCVDB backend ignores dataDir and routes EVERY core to one shared + // `${database}_l1_memories` / `_l0_conversations` / `_profiles` collection + // set (see core/store/factory.ts + tcvdb.ts), while the structural route does + // not push `session_key` into L1/L0 search — so `/recall` and + // `/search/memories` would return other accounts' memories. Refuse at load + // time rather than leak across tenants. (Single-tenant + TCVDB is fine: one + // shared core, one database, no cross-account routing.) + if (effective.data.multiTenant && effective.memory.storeBackend === "tcvdb") { + throw new Error( + "Multi-tenant mode is incompatible with storeBackend=tcvdb: every per-account " + + "core shares one TCVDB database/collection set, so L1/L0 recall and search would " + + "leak across accounts (structural isolation only holds for the per-dataDir SQLite " + + "backend). Use storeBackend=sqlite for multi-tenant, or run single-tenant " + + "(TDAI_MULTI_TENANT=false) with TCVDB.", + ); + } + return effective; } // ============================ @@ -237,6 +326,26 @@ function envInt(key: string): number | undefined { return Number.isFinite(n) ? n : undefined; } +/** + * Read a strict boolean env var. Accepts true/1/yes/on and false/0/no/off + * (case-insensitive); anything else (including unset) returns undefined so the + * caller can fall through to the next source / default. + */ +function envBool(key: string): boolean | undefined { + const v = env(key); + if (v === undefined) return undefined; + const s = v.toLowerCase(); + if (s === "true" || s === "1" || s === "yes" || s === "on") return true; + if (s === "false" || s === "0" || s === "no" || s === "off") return false; + return undefined; +} + +/** Read a boolean field from a config object (non-boolean → undefined). */ +function bool(src: Record, key: string): boolean | undefined { + const v = src[key]; + return typeof v === "boolean" ? v : undefined; +} + /** * Read an env var that may be a boolean ("true"/"false"/"1"/"0") * or a plain string (strategy name like "deepseek", "anthropic"). diff --git a/src/gateway/core-registry.test.ts b/src/gateway/core-registry.test.ts new file mode 100644 index 00000000..55f718d2 --- /dev/null +++ b/src/gateway/core-registry.test.ts @@ -0,0 +1,473 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { CoreRegistry, safeAccountDir } from "./core-registry.js"; +import { parseConfig } from "../config.js"; +import type { StandaloneLLMConfig } from "../adapters/standalone/llm-runner.js"; +import type { Logger } from "../core/types.js"; + +const silentLogger: Logger = { + debug: () => {}, + info: () => {}, + warn: () => {}, + error: () => {}, +}; + +const llmConfig: StandaloneLLMConfig = { + baseUrl: "http://127.0.0.1:0", + apiKey: "", + model: "test-model", + maxTokens: 256, + timeoutMs: 1000, + disableThinking: false, +}; + +// Extraction off → no background timers / LLM calls spun up by initialize(). +// provider "none" → no embedding service (keyword/FTS path), no network. +const memory = parseConfig({ extraction: { enabled: false }, embedding: { provider: "none" } }); + +function makeRegistry( + baseDir: string, + multiTenant: boolean, + maxResidentCores?: number, +): CoreRegistry { + return new CoreRegistry({ + baseDir, + llmConfig, + memory, + logger: silentLogger, + multiTenant, + maxResidentCores, + }); +} + +const delay = (ms: number) => new Promise((r) => setTimeout(r, ms)); + +describe("safeAccountDir", () => { + it("is deterministic for the same key", () => { + expect(safeAccountDir("ai4all:alice")).toBe(safeAccountDir("ai4all:alice")); + }); + + it("throws on empty / whitespace keys", () => { + expect(() => safeAccountDir("")).toThrow(); + expect(() => safeAccountDir(" ")).toThrow(); + }); + + it("never emits a path separator, traversal, or hidden segment", () => { + for (const key of ["ai4all:../../etc/passwd", "a/b/c", "..", ".", "../x", "\\\\evil", "..\\..\\x"]) { + const dir = safeAccountDir(key); + expect(dir).not.toContain("/"); + expect(dir).not.toContain("\\"); + expect(dir.startsWith(".")).toBe(false); + expect(dir).not.toBe(".."); + // Stays a single path segment. + expect(path.basename(dir)).toBe(dir); + } + }); + + it("maps slug-colliding keys to DISTINCT dirs (hash guards isolation)", () => { + // 'a/b' and 'a_b' sanitise to the same slug but must not share a directory. + expect(safeAccountDir("a/b")).not.toBe(safeAccountDir("a_b")); + expect(safeAccountDir("ai4all:x")).not.toBe(safeAccountDir("ai4all:y")); + }); +}); + +describe("CoreRegistry extraction cap", () => { + function reg(multiTenant: boolean, maxConcurrentExtractions?: number): CoreRegistry { + return new CoreRegistry({ + baseDir: "/tmp/tdai-cap-unused", // no core is created in these stats-only checks + llmConfig, + memory, + logger: silentLogger, + multiTenant, + maxConcurrentExtractions, + }); + } + + it("multi-tenant defaults to a bounded cap", () => { + expect(reg(true).extractionStats().limit).toBe(4); + }); + + it("single-tenant defaults to unbounded (0)", () => { + expect(reg(false).extractionStats().limit).toBe(0); + }); + + it("an explicit positive cap wins in either mode", () => { + expect(reg(true, 2).extractionStats().limit).toBe(2); + expect(reg(false, 3).extractionStats().limit).toBe(3); + }); + + it("an explicit 0 forces unbounded even in multi-tenant", () => { + expect(reg(true, 0).extractionStats().limit).toBe(0); + }); + + it("starts idle (no active or waiting permits)", () => { + const stats = reg(true).extractionStats(); + expect(stats.active).toBe(0); + expect(stats.waiting).toBe(0); + }); +}); + +describe("CoreRegistry LRU eviction (multi-tenant)", () => { + let baseDir: string; + const registries: CoreRegistry[] = []; + + function reg(maxResidentCores?: number, multiTenant = true): CoreRegistry { + const r = makeRegistry(baseDir, multiTenant, maxResidentCores); + registries.push(r); + return r; + } + + beforeEach(async () => { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-lru-")); + }); + + afterEach(async () => { + await Promise.all(registries.splice(0).map((r) => r.destroyAll())); + await fs.rm(baseDir, { recursive: true, force: true }); + }); + + it("defaults to unlimited resident cores (limit 0)", () => { + expect(reg().residentStats().limit).toBe(0); + }); + + it("single-tenant ignores the resident cap (always unlimited)", () => { + expect(reg(2, false).residentStats().limit).toBe(0); + }); + + it("honours an explicit positive cap", () => { + expect(reg(3).residentStats().limit).toBe(3); + }); + + it("unlimited keeps every account resident", async () => { + const r = reg(); // unlimited + await r.getCore("ai4all:a"); + await r.getCore("ai4all:b"); + await r.getCore("ai4all:c"); + expect(r.size).toBe(3); + expect(r.residentStats().count).toBe(3); + }); + + it("evicts the least-recently-used core past the cap, sparing the active one", async () => { + const r = reg(2); + await r.getCore("ai4all:a"); + await delay(5); + await r.getCore("ai4all:b"); + await delay(5); + await r.getCore("ai4all:a"); // touch a → b becomes the LRU + await delay(5); + await r.getCore("ai4all:c"); // size would be 3 (> 2) → evict b + + expect(r.size).toBe(2); + expect(r.peek("ai4all:b")).toBeUndefined(); + expect(r.peek("ai4all:a")).toBeDefined(); + expect(r.peek("ai4all:c")).toBeDefined(); + }); + + it("eviction flushes + closes the core but KEEPS the dataDir (unlike wipe)", async () => { + const r = reg(1); + const aDir = r.resolveDataDir("ai4all:a"); + await r.getCore("ai4all:a"); + await fs.writeFile(path.join(aDir, "marker.txt"), "x"); + await delay(5); + await r.getCore("ai4all:b"); // evicts a + + expect(r.peek("ai4all:a")).toBeUndefined(); + // dir + its contents survive eviction — eviction is not a hard delete. + expect((await fs.stat(aDir)).isDirectory()).toBe(true); + expect(await fs.readFile(path.join(aDir, "marker.txt"), "utf8")).toBe("x"); + }); + + it("re-requesting an evicted account rebuilds a working core on the same dataDir", async () => { + const r = reg(1); + const first = await r.getCore("ai4all:a"); + const aDir = r.resolveDataDir("ai4all:a"); + await delay(5); + await r.getCore("ai4all:b"); // evicts a (teardown in flight) + + // Re-request a: must wait for the old SQLite handle to close, then reopen. + const again = await r.getCore("ai4all:a"); // evicts b in turn + expect(again).not.toBe(first); // a fresh core, not the destroyed one + expect(r.resolveDataDir("ai4all:a")).toBe(aDir); // same dataDir reused + expect(r.peek("ai4all:a")).toBe(again); + expect(r.peek("ai4all:b")).toBeUndefined(); + expect(r.size).toBe(1); + }); +}); + +describe("CoreRegistry lease / refcount (LRU eviction safety)", () => { + let baseDir: string; + const registries: CoreRegistry[] = []; + + function reg(maxResidentCores?: number): CoreRegistry { + const r = makeRegistry(baseDir, true, maxResidentCores); + registries.push(r); + return r; + } + + beforeEach(async () => { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-lease-")); + }); + + afterEach(async () => { + await Promise.all(registries.splice(0).map((r) => r.destroyAll())); + await fs.rm(baseDir, { recursive: true, force: true }); + }); + + it("a leased core is NOT LRU-evicted even past the cap, and is evictable once released", async () => { + const r = reg(1); // cap of 1 — any second resident core would normally evict the first + const leaseA = await r.acquire("ai4all:a"); + // Touch a second account while a is still leased. With a refcount, a cannot + // be torn down mid-request, so the registry holds BOTH (transient over-limit) + // rather than destroy a's store underneath the live lease. + await r.getCore("ai4all:b"); + expect(r.size).toBe(2); + expect(r.peek("ai4all:a")).toBeDefined(); + expect(r.residentStats().pinned).toBe(1); + + // The leased core's store is still open: it can serve work. + const conv = await leaseA.core.searchConversations({ query: "x", limit: 1, sessionKey: "ai4all:a" }); + expect(conv.total).toBe(0); // empty store, but a LIVE one (no closed-handle throw) + + // Release a → it is now an eligible victim; the next get past the cap evicts it. + leaseA.release(); + expect(r.residentStats().pinned).toBe(0); + await delay(5); + await r.getCore("ai4all:c"); // size 3 > 1 → evict the now-unpinned LRU + expect(r.peek("ai4all:a")).toBeUndefined(); + }); + + it("wipe of a leased account defers teardown until the lease is released", async () => { + const r = reg(); + const dir = r.resolveDataDir("ai4all:erin"); + const lease = await r.acquire("ai4all:erin"); + + // Kick off wipe while the lease is held; it must not resolve before release. + let wiped = false; + const wipePromise = r.wipe("ai4all:erin").then((d) => { wiped = true; return d; }); + await delay(20); + expect(wiped).toBe(false); // teardown is parked on the in-flight lease + + lease.release(); + await wipePromise; + expect(wiped).toBe(true); + expect(r.peek("ai4all:erin")).toBeUndefined(); + await expect(fs.stat(dir)).rejects.toBeTruthy(); // dir gone only after drain + }); + + it("double-release is idempotent (does not under-count pins)", async () => { + const r = reg(); + const lease = await r.acquire("ai4all:a"); + lease.release(); + lease.release(); // second call must be a no-op + expect(r.residentStats().pinned).toBe(0); + // A fresh acquire still works and is correctly counted. + const again = await r.acquire("ai4all:a"); + expect(r.residentStats().pinned).toBe(1); + again.release(); + }); +}); + +describe("CoreRegistry idle-TTL eviction (multi-tenant)", () => { + let baseDir: string; + const registries: CoreRegistry[] = []; + + function reg(coreIdleTtlMs?: number, multiTenant = true): CoreRegistry { + const r = new CoreRegistry({ + baseDir, + llmConfig, + memory, + logger: silentLogger, + multiTenant, + coreIdleTtlMs, + }); + registries.push(r); + return r; + } + + beforeEach(async () => { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-ttl-")); + }); + + afterEach(async () => { + await Promise.all(registries.splice(0).map((r) => r.destroyAll())); + await fs.rm(baseDir, { recursive: true, force: true }); + }); + + // NB: a configured TTL also starts the background sweep timer, so tests assert + // observable state (peek/size), not exact sweepIdleCores() return counts — + // except where the outcome is deterministic regardless of the timer (a pinned + // core is always skipped; a disabled registry never sweeps). + + it("is disabled by default and in single-tenant mode (sweep is a no-op, no timer)", async () => { + const off = reg(undefined); // ttl resolves to 0 → no timer + await off.getCore("ai4all:a"); + await delay(10); + expect(off.sweepIdleCores()).toBe(0); + expect(off.size).toBe(1); + + const st = reg(20, false); // single-tenant ignores TTL → no timer + await st.getCore("ai4all:a"); + await delay(40); + expect(st.sweepIdleCores()).toBe(0); + expect(st.size).toBe(1); + }); + + it("evicts a core idle past the TTL, spares a recently-used one", async () => { + const r = reg(40); + await r.getCore("ai4all:idle"); + await delay(60); // idle core is now past its 40ms TTL + await r.getCore("ai4all:fresh"); // fresh: touched just now (within TTL) + r.sweepIdleCores(); // synchronous, runs before the next timer tick + expect(r.peek("ai4all:idle")).toBeUndefined(); + expect(r.peek("ai4all:fresh")).toBeDefined(); + }); + + it("never idle-evicts a pinned (in-flight) core; reclaims it after release", async () => { + const r = reg(40); + const lease = await r.acquire("ai4all:busy"); + await delay(70); // well past TTL, but a request still holds it + // Deterministic: pinned cores are skipped by both manual sweep and the timer. + expect(r.sweepIdleCores()).toBe(0); + expect(r.peek("ai4all:busy")).toBeDefined(); + + lease.release(); // now reclaimable — by a manual sweep or the background timer + const deadline = Date.now() + 1000; + while (Date.now() < deadline && r.peek("ai4all:busy")) { + r.sweepIdleCores(); + await delay(20); + } + expect(r.peek("ai4all:busy")).toBeUndefined(); + }); + + it("the background timer reclaims idle cores without a manual sweep", async () => { + const r = reg(40); + await r.getCore("ai4all:a"); + // sweepMs = max(50, min(40, 60000)) = 50ms; poll up to ~1s for the timer. + const deadline = Date.now() + 1000; + while (Date.now() < deadline && r.size > 0) await delay(25); + expect(r.size).toBe(0); + }); +}); + +describe("CoreRegistry wipe guards", () => { + it("refuses namespace wipe in single-tenant mode", async () => { + const baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-wipe-st-")); + const registry = makeRegistry(baseDir, false); + try { + await expect(registry.wipe("ai4all:alice")).rejects.toThrow(/multi-tenant/); + } finally { + await registry.destroyAll(); + await fs.rm(baseDir, { recursive: true, force: true }); + } + }); +}); + +describe("CoreRegistry (single-tenant)", () => { + let baseDir: string; + let registry: CoreRegistry; + + beforeEach(async () => { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-registry-st-")); + registry = makeRegistry(baseDir, false); + }); + + afterEach(async () => { + await registry.destroyAll(); + await fs.rm(baseDir, { recursive: true, force: true }); + }); + + it("returns one shared core rooted at baseDir regardless of session_key", async () => { + const a = await registry.getCore("ai4all:alice"); + const b = await registry.getCore("ai4all:bob"); + const c = await registry.getCore(""); + expect(a).toBe(b); + expect(b).toBe(c); + expect(registry.size).toBe(1); + expect(registry.resolveDataDir("anything")).toBe(baseDir); + }); +}); + +describe("CoreRegistry (multi-tenant)", () => { + let baseDir: string; + let registry: CoreRegistry; + + beforeEach(async () => { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-registry-mt-")); + registry = makeRegistry(baseDir, true); + }); + + afterEach(async () => { + await registry.destroyAll(); + await fs.rm(baseDir, { recursive: true, force: true }); + }); + + it("creates a distinct core + dataDir per account, both under baseDir", async () => { + const alice = await registry.getCore("ai4all:alice"); + const bob = await registry.getCore("ai4all:bob"); + + expect(alice).not.toBe(bob); + expect(registry.size).toBe(2); + + const aliceDir = registry.resolveDataDir("ai4all:alice"); + const bobDir = registry.resolveDataDir("ai4all:bob"); + expect(aliceDir).not.toBe(bobDir); + expect(path.dirname(aliceDir)).toBe(baseDir); + expect(path.dirname(bobDir)).toBe(baseDir); + + // initialize() creates the per-account directory tree on disk. + expect((await fs.stat(aliceDir)).isDirectory()).toBe(true); + expect((await fs.stat(bobDir)).isDirectory()).toBe(true); + const entries = (await fs.readdir(baseDir)).sort(); + expect(entries).toContain(path.basename(aliceDir)); + expect(entries).toContain(path.basename(bobDir)); + }); + + it("caches per account: repeated + concurrent gets return the same core", async () => { + const first = await registry.getCore("ai4all:alice"); + const again = await registry.getCore("ai4all:alice"); + expect(again).toBe(first); + + const [c1, c2] = await Promise.all([ + registry.getCore("ai4all:carol"), + registry.getCore("ai4all:carol"), + ]); + expect(c1).toBe(c2); + expect(registry.size).toBe(2); // alice + carol + }); + + it("requires a non-empty session_key", async () => { + await expect(registry.getCore("")).rejects.toThrow(); + expect(() => registry.resolveDataDir("")).toThrow(); + }); + + it("wipe removes the account core + its dataDir, idempotently", async () => { + const dir = registry.resolveDataDir("ai4all:erin"); + await registry.getCore("ai4all:erin"); + await fs.writeFile(path.join(dir, "marker.txt"), "x"); + expect((await fs.stat(dir)).isDirectory()).toBe(true); + + await registry.wipe("ai4all:erin"); + expect(registry.peek("ai4all:erin")).toBeUndefined(); + expect(registry.size).toBe(0); + await expect(fs.stat(dir)).rejects.toBeTruthy(); // dir gone + + // Wiping an already-wiped / never-seen account does not throw. + await expect(registry.wipe("ai4all:erin")).resolves.toBeTruthy(); + await expect(registry.wipe("ai4all:never")).resolves.toBeTruthy(); + }); + + it("peek does not create a core; evict tears one down and frees the slot", async () => { + expect(registry.peek("ai4all:dave")).toBeUndefined(); + const core = await registry.getCore("ai4all:dave"); + expect(registry.peek("ai4all:dave")).toBe(core); + + const dir = await registry.evict("ai4all:dave"); + expect(dir).toBe(registry.resolveDataDir("ai4all:dave")); + expect(registry.peek("ai4all:dave")).toBeUndefined(); + expect(registry.size).toBe(0); + // Evicting an absent account is a no-op. + expect(await registry.evict("ai4all:dave")).toBeUndefined(); + }); +}); diff --git a/src/gateway/core-registry.ts b/src/gateway/core-registry.ts new file mode 100644 index 00000000..12c0a77e --- /dev/null +++ b/src/gateway/core-registry.ts @@ -0,0 +1,571 @@ +/** + * CoreRegistry — routes Gateway requests to a per-account {@link TdaiCore}. + * + * This is the structural multi-tenant route from the design doc (§8.4): instead + * of one shared core + `session_key` filters on every query, each account gets + * its **own** `TdaiCore` rooted at a dedicated dataDir (`baseDir/{account}`). + * Isolation is then physical — distinct SQLite files, distinct L0/L1 tables, + * distinct `persona.md` / `scene_blocks/` — so L1/L2/L3 recall and search are + * isolated for free, without depending on "every SQL remembered its WHERE". + * + * Two modes: + * - **single-tenant (default)**: one shared core rooted at `baseDir`, + * `session_key` is ignored for routing. Behaviour is identical to the + * pre-multi-tenant Gateway. + * - **multi-tenant**: lazy `Map`; `session_key` is + * required and mapped to a collision-free, traversal-safe directory. + * + * The cross-core LLM extraction concurrency cap lives here too: one + * {@link AsyncSemaphore} is shared by every core so total background extraction + * (L1/L2/L3) across all accounts stays bounded (design §8.4 #5). + * + * **LRU eviction.** Resident cores are bounded by `maxResidentCores` (opt-in; + * `0`/unset = unlimited, legacy). When a `getCore` pushes the count over the + * limit, the least-recently-used *other* core is torn down — `TdaiCore.destroy` + * first **flushes** its pipeline queues (no buffered L1/L2/L3 work is lost), + * then releases its SQLite handle. Teardown runs fire-and-forget so it never + * adds drain latency to the triggering request, but is tracked per key in + * {@link closing} so a later `getCore`/`wipe` for an evicted account waits for + * the old handle to close before a fresh core re-opens the same dataDir. + */ + +import fs from "node:fs/promises"; +import path from "node:path"; +import { createHash } from "node:crypto"; +import { TdaiCore } from "../core/tdai-core.js"; +import { StandaloneHostAdapter } from "../adapters/standalone/host-adapter.js"; +import { SessionFilter } from "../utils/session-filter.js"; +import { AsyncSemaphore } from "../utils/async-semaphore.js"; +import type { StandaloneLLMConfig } from "../adapters/standalone/llm-runner.js"; +import type { MemoryTdaiConfig } from "../config.js"; +import type { Logger } from "../core/types.js"; + +/** Map key used for the single shared core when multi-tenant is off. */ +const SINGLE_TENANT_KEY = "__single__"; + +/** + * Default cap on concurrent background extraction runs (L1/L2/L3) across all + * resident cores, applied in multi-tenant mode when no explicit limit is + * configured. Without a cap, `N` active accounts fan out to up to `~3N` + * simultaneous LLM extraction calls (design §8.4 #5). Single-tenant mode keeps + * its legacy unbounded behaviour (one core can't fan out). + */ +const DEFAULT_MULTI_TENANT_EXTRACTION_CAP = 4; + +/** + * Default cap on resident per-account cores in multi-tenant mode when none is + * configured. `0` means **unlimited** — chosen as the default so existing + * deployments keep every warm core (no surprise evictions / re-init latency). + * Operators serving many accounts set `data.maxResidentCores` to bound memory. + */ +const DEFAULT_MAX_RESIDENT_CORES = 0; + +export interface CoreRegistryOptions { + /** Root data directory. Single-tenant: the core's dataDir. Multi-tenant: parent of per-account dirs. */ + baseDir: string; + /** LLM config passed to every core's host adapter. */ + llmConfig: StandaloneLLMConfig; + /** Parsed memory config shared by every core. */ + memory: MemoryTdaiConfig; + /** Logger shared by every core. */ + logger: Logger; + /** When true, route by `session_key` to per-account cores. */ + multiTenant: boolean; + /** Agents excluded from capture (forwarded to each core's SessionFilter). */ + excludeAgents?: string[]; + /** + * Max concurrent background extraction runs (L1/L2/L3) across ALL cores. + * + * - `> 0` — hard cap shared by every core. + * - `0` — unbounded (the single-tenant default — one core can't fan out). + * - `undefined` — multi-tenant falls back to + * {@link DEFAULT_MULTI_TENANT_EXTRACTION_CAP}; single-tenant stays unbounded. + */ + maxConcurrentExtractions?: number; + /** + * Max number of resident per-account cores kept warm at once (multi-tenant + * LRU eviction). When a `getCore` would exceed this, the least-recently-used + * *other* core is flushed + torn down. + * + * - `> 0` — keep at most this many cores resident. + * - `0` / `undefined` — unlimited (legacy: every account stays warm forever). + * + * Must exceed the peak number of *concurrently active* accounts, or a core + * could be evicted mid-request; it bounds idle warm cores, not live ones. + * Ignored in single-tenant mode (there is only ever one core). + */ + maxResidentCores?: number; + /** + * Idle time-to-live (ms) for resident cores (multi-tenant). A periodic sweep + * evicts any *unpinned* core whose last request was longer ago than this, so a + * long-tail of accounts that went quiet are reclaimed during idle periods + * instead of lingering until an LRU push. Complements {@link maxResidentCores} + * (count bound): TTL is a time bound. + * + * - `> 0` — evict cores idle longer than this. + * - `0` / `undefined` — disabled (no time-based reclamation; the default). + * + * Ignored in single-tenant mode. + */ + coreIdleTtlMs?: number; +} + +interface CoreEntry { + core: TdaiCore; + dataDir: string; + /** Resolves once the core has finished `initialize()`. */ + ready: Promise; + /** Last time this core served a request (epoch ms) — for LRU eviction. */ + lastUsedMs: number; + /** + * Active leases: in-flight requests currently holding this core (see + * {@link CoreRegistry.acquire}). A core with `pins > 0` is **never** an LRU + * eviction victim, and manual evict/wipe defers its teardown until pins drain + * — so a request mid-`capture`/`recall`/`search` can never have its SQLite + * handle closed underneath it. + */ + pins: number; + /** Set while a teardown is waiting for {@link pins} to reach 0; fired by the last release. */ + onIdle?: () => void; +} + +/** + * A held reference to an account's core. The core is guaranteed alive (its store + * open) until {@link release} is called. Always release exactly once — a + * `try/finally` around the handler body is the intended usage. Release is + * idempotent, so a double-release is harmless. + */ +export interface CoreLease { + core: TdaiCore; + release: () => void; +} + +/** + * Derive a filesystem-safe, collision-free directory name from a `session_key`. + * + * `session_key` is operator/business-supplied (`"ai4all:{account_id}"`), so it + * must never be trusted as a path segment. We combine: + * - a human-readable **slug** (ascii `[A-Za-z0-9._-]`, leading dots removed, + * length-capped) for debuggability, and + * - a **sha256 prefix** of the *original* key for injectivity. + * + * The hash is what guarantees isolation: two distinct keys that sanitise to the + * same slug (e.g. `a/b` and `a_b`) still get different directories, so they can + * never share a store. Path-traversal (`/`, `\`, `..`, leading `.`) is stripped + * from the slug, and the hash suffix means the result is never `.`/`..`/hidden. + */ +export function safeAccountDir(sessionKey: string): string { + const key = (sessionKey ?? "").trim(); + if (!key) throw new Error("session_key must be a non-empty string"); + + const hash = createHash("sha256").update(key).digest("hex").slice(0, 16); + let slug = key + .replace(/[^A-Za-z0-9._-]/g, "_") // drop path separators & non-ascii + .replace(/^\.+/, "_") // never a hidden/`.`/`..` dir + .slice(0, 48); + if (!slug) slug = "acct"; + + return `${slug}.${hash}`; +} + +export class CoreRegistry { + private readonly opts: CoreRegistryOptions; + private readonly cores = new Map(); + /** + * One limiter shared by every core created here, so background extraction + * (L1/L2/L3) is globally capped instead of fanning out per account. + */ + private readonly extractionLimiter: AsyncSemaphore; + /** Max resident cores (0 = unlimited). Resolved once from options. */ + private readonly maxResidentCores: number; + /** Idle TTL (ms) for resident cores; 0 = disabled. Resolved once from options. */ + private readonly idleTtlMs: number; + /** Background idle-sweep timer (multi-tenant + idleTtlMs > 0 only). */ + private idleTimer?: ReturnType; + /** + * In-flight teardowns keyed by account, so a re-request (or wipe) for an + * evicted account waits for its old core to finish closing — releasing the + * SQLite handle — before a fresh core opens on the same dataDir. + */ + private readonly closing = new Map>(); + + constructor(opts: CoreRegistryOptions) { + this.opts = opts; + this.extractionLimiter = new AsyncSemaphore(this.resolveExtractionCap()); + this.maxResidentCores = this.resolveMaxResidentCores(); + this.idleTtlMs = this.resolveIdleTtl(); + this.startIdleSweep(); + } + + /** Resolve the idle TTL. Multi-tenant + explicit positive value; else 0 (off). */ + private resolveIdleTtl(): number { + if (!this.opts.multiTenant) return 0; + const explicit = this.opts.coreIdleTtlMs; + return explicit !== undefined && Number.isFinite(explicit) && explicit > 0 + ? Math.floor(explicit) + : 0; + } + + /** + * Start the background idle sweep when an idle TTL is configured. The interval + * checks at least every `ttl` (capped at 60s so long TTLs still reclaim within + * a minute of going idle, floored so tiny test TTLs sweep promptly). `unref()` + * so the timer never keeps the process alive on its own. + */ + private startIdleSweep(): void { + if (this.idleTtlMs <= 0) return; + const sweepMs = Math.max(50, Math.min(this.idleTtlMs, 60_000)); + this.idleTimer = setInterval(() => this.sweepIdleCores(), sweepMs); + this.idleTimer.unref?.(); + } + + /** + * Evict every *unpinned* resident core idle longer than {@link idleTtlMs}. + * Pinned cores (in-flight requests) are spared — they'll be reclaimed on a + * later sweep once idle. Returns the number evicted (for tests/metrics). Safe + * to call directly; the sweep timer just invokes it on a schedule. + */ + sweepIdleCores(): number { + if (this.idleTtlMs <= 0) return 0; + const cutoff = Date.now() - this.idleTtlMs; + let evicted = 0; + for (const [key, entry] of [...this.cores]) { + if (entry.pins > 0) continue; // never evict a live core + if (entry.lastUsedMs > cutoff) continue; // still within TTL + this.cores.delete(key); + void this.beginTeardown(key, entry); + evicted++; + this.opts.logger.debug?.( + `[tdai-gateway] [registry] Idle-evicted ${key} (idle>${this.idleTtlMs}ms, active=${this.cores.size})`, + ); + } + return evicted; + } + + /** + * Resolve the effective extraction concurrency cap. An explicit value wins; + * otherwise multi-tenant gets a safe default and single-tenant stays + * unbounded (a lone core is already serialized by its own SerialQueues). + */ + private resolveExtractionCap(): number { + const explicit = this.opts.maxConcurrentExtractions; + if (explicit !== undefined && Number.isFinite(explicit)) { + return explicit > 0 ? Math.floor(explicit) : 0; + } + return this.opts.multiTenant ? DEFAULT_MULTI_TENANT_EXTRACTION_CAP : 0; + } + + /** + * Resolve the resident-core cap. Explicit positive value wins; anything else + * (0, undefined, negative, non-finite) means unlimited. Single-tenant always + * resolves to unlimited — there is only one core, so eviction is moot. + */ + private resolveMaxResidentCores(): number { + if (!this.opts.multiTenant) return 0; + const explicit = this.opts.maxResidentCores; + if (explicit !== undefined && Number.isFinite(explicit) && explicit > 0) { + return Math.floor(explicit); + } + return DEFAULT_MAX_RESIDENT_CORES; + } + + /** + * Live resident-core stats (for health/metrics). `limit` 0 = unlimited; + * `pinned` is how many resident cores currently have in-flight leases (a + * `pinned` near `count` while `count > limit` means the limiter is holding + * cores past the cap because they are all busy — transient, expected). + */ + residentStats(): { count: number; limit: number; pinned: number } { + let pinned = 0; + for (const e of this.cores.values()) if (e.pins > 0) pinned++; + return { count: this.cores.size, limit: this.maxResidentCores, pinned }; + } + + /** Live extraction-limiter stats (for health/metrics). */ + extractionStats(): { limit: number; active: number; waiting: number } { + return { + limit: this.extractionLimiter.capacity, + active: this.extractionLimiter.active, + waiting: this.extractionLimiter.waiting, + }; + } + + /** Whether this registry routes per-account. */ + get multiTenant(): boolean { + return this.opts.multiTenant; + } + + /** Number of currently-instantiated cores (for health/metrics/LRU). */ + get size(): number { + return this.cores.size; + } + + /** + * Resolve the dataDir a `session_key` maps to — **pure**, no side effects. + * Useful for the wipe API and tests. In single-tenant mode the argument is + * ignored and `baseDir` is returned. + */ + resolveDataDir(sessionKey: string): string { + return this.resolve(sessionKey).dataDir; + } + + /** + * Ensure an entry exists for `session_key` (lazily creating + scheduling its + * `initialize()`), returning it **synchronously** without awaiting `ready`. + * + * Synchronous on purpose: callers increment `pins` in the same tick before + * the first `await`, so a concurrent `evictLruIfNeeded` (which only runs from + * this method's own create branch) can never select a just-acquired core. + */ + private ensureEntry(sessionKey: string): CoreEntry { + const { key, dataDir } = this.resolve(sessionKey); + + let entry = this.cores.get(key); + if (!entry) { + // If this account is mid-eviction, gate the new core's `initialize()` + // (which opens SQLite) behind the old core's teardown so the same dataDir + // is never opened twice. The entry is still inserted *synchronously* below + // — the wait lives inside `ready`, not before the map write — so racing + // callers find this entry instead of creating a duplicate core. + const prevClosing = this.closing.get(key); + const core = this.createCore(dataDir); + const ready = prevClosing + ? prevClosing.then(() => core.initialize()) + : core.initialize(); + entry = { core, dataDir, ready, lastUsedMs: Date.now(), pins: 0 }; + this.cores.set(key, entry); + this.opts.logger.debug?.( + `[tdai-gateway] [registry] Core created for ${this.multiTenant ? key : "single-tenant"} (dataDir=${dataDir}, active=${this.cores.size})`, + ); + this.evictLruIfNeeded(key); + } + entry.lastUsedMs = Date.now(); + return entry; + } + + /** + * Get (lazily creating + initializing) the core for a `session_key`. + * + * Concurrent calls for the same key share a single `initialize()`. **Does not + * pin** — the returned core may be LRU-evicted at any time, so this is only + * safe for callers that use it synchronously or tolerate eviction (health, + * single-tenant eager warm-up, tests). Request handlers must use + * {@link acquire} so the core can't be torn down mid-request. + */ + async getCore(sessionKey: string): Promise { + const entry = this.ensureEntry(sessionKey); + await entry.ready; + return entry.core; + } + + /** + * Acquire a **lease** on the core for a `session_key`: like {@link getCore}, + * but pins the core so LRU eviction and manual teardown cannot close its store + * until the lease is released. This closes the race where one account's + * request triggers eviction of another account's core while a request is still + * using it (`maxResidentCores` below peak concurrency). Always release in a + * `finally`. + */ + async acquire(sessionKey: string): Promise { + const entry = this.ensureEntry(sessionKey); + entry.pins++; // synchronous, same tick as ensureEntry — no eviction can interleave + + let released = false; + const release = (): void => { + if (released) return; // idempotent: only the first call counts + released = true; + this.unpin(entry); + }; + + try { + await entry.ready; + } catch (err) { + release(); // init failed — don't leak the pin + throw err; + } + return { core: entry.core, release }; + } + + /** + * Drop one lease. When the last lease for a core that is awaiting teardown is + * released, the deferred teardown (set up in {@link beginTeardown}) is woken. + */ + private unpin(entry: CoreEntry): void { + if (entry.pins > 0) entry.pins--; + if (entry.pins === 0 && entry.onIdle) { + const wake = entry.onIdle; + entry.onIdle = undefined; + wake(); + } + } + + /** + * Return an already-instantiated core without creating one. Used by the + * health probe (single-tenant) so it doesn't spin up a core as a side effect. + */ + peek(sessionKey: string): TdaiCore | undefined { + return this.cores.get(this.resolve(sessionKey).key)?.core; + } + + /** + * Destroy and forget the core for a `session_key` (no-op if not loaded). + * Returns the dataDir it was bound to, so callers (e.g. the wipe API) can + * delete it on disk afterwards. Idempotent. + */ + async evict(sessionKey: string): Promise { + const { key } = this.resolve(sessionKey); + const entry = this.cores.get(key); + if (!entry) { + // Not resident — but it may be mid-eviction from the LRU path. Await that + // teardown so callers (e.g. wipe → rm) see the SQLite handle released. + await this.closing.get(key)?.catch(() => {}); + return undefined; + } + this.cores.delete(key); + await this.beginTeardown(key, entry); + return entry.dataDir; + } + + /** + * Hard-delete an account: tear down its core (closing the SQLite handle) and + * remove its dataDir from disk. This is the structural "namespace wipe" that + * backs the host's account hard-delete (design §8.4 wipe/unbind). + * + * - **Multi-tenant only**: in single-tenant mode there is no per-account + * dataDir to delete, so this throws rather than risk wiping the shared store. + * - **Idempotent**: wiping an account with no resident core (or whose dir was + * already deleted) still succeeds. + * - **Bounded**: refuses to delete anything that is not strictly inside + * `baseDir`, as a defence-in-depth backstop on top of {@link safeAccountDir}. + * + * The core is destroyed *before* the `rm` so the store releases its file + * handle and {@link TdaiCore.destroy} clears the per-dataDir store cache, + * letting a later `getCore` for the same key rebuild a fresh empty store. + */ + async wipe(sessionKey: string): Promise { + if (!this.opts.multiTenant) { + throw new Error("namespace wipe requires multi-tenant mode"); + } + const dataDir = this.resolveDataDir(sessionKey); // throws on empty key + + const root = path.resolve(this.opts.baseDir); + const target = path.resolve(dataDir); + if (target === root || !target.startsWith(root + path.sep)) { + throw new Error(`refusing to wipe path outside baseDir: ${target}`); + } + + await this.evict(sessionKey); // close core + free slot (no-op if absent) + await fs.rm(target, { recursive: true, force: true }); + this.opts.logger.debug?.(`[tdai-gateway] [registry] Wiped account dataDir ${target}`); + return target; + } + + /** Destroy every loaded core. Call on Gateway shutdown. */ + async destroyAll(): Promise { + if (this.idleTimer) { + clearInterval(this.idleTimer); + this.idleTimer = undefined; + } + const entries = [...this.cores.values()]; + this.cores.clear(); + // Include teardowns already in flight from LRU eviction so shutdown waits + // for every core (resident + closing) to release its SQLite handle. + const inFlight = [...this.closing.values()]; + await Promise.allSettled([ + ...inFlight.map((p) => p.catch(() => {})), + ...entries.map(async (e) => { + await e.ready.catch(() => {}); + await e.core.destroy(); + }), + ]); + } + + // ── internals ─────────────────────────────────────────────────────────── + + /** + * Evict least-recently-used cores until the resident count is back within + * {@link maxResidentCores}. `protectKey` (the core just served) is never a + * victim, so the caller's own request always survives. No-op when unlimited. + */ + private evictLruIfNeeded(protectKey: string): void { + if (this.maxResidentCores <= 0) return; // unlimited + while (this.cores.size > this.maxResidentCores) { + let lruKey: string | undefined; + let lru: CoreEntry | undefined; + for (const [k, e] of this.cores) { + if (k === protectKey) continue; + if (e.pins > 0) continue; // never evict a core with in-flight leases + if (!lru || e.lastUsedMs < lru.lastUsedMs) { + lruKey = k; + lru = e; + } + } + // No evictable (unpinned) victim: every other core is busy. Stop and allow + // a transient over-limit rather than tear down a live request — the cap + // bounds *idle* warm cores, and correctness wins over the memory bound. + if (!lruKey || !lru) break; + this.cores.delete(lruKey); + void this.beginTeardown(lruKey, lru); + this.opts.logger.debug?.( + `[tdai-gateway] [registry] LRU-evicted ${lruKey} (active=${this.cores.size}, limit=${this.maxResidentCores})`, + ); + } + } + + /** + * Flush + destroy a core out of band, tracking the teardown in {@link closing} + * so re-creation / wipe of the same key waits for the SQLite handle to close. + * Caller must have already removed the entry from {@link cores}. + */ + private beginTeardown(key: string, entry: CoreEntry): Promise { + const done = (async () => { + // Wait for in-flight leases to release before closing the store, so a + // request holding this core never hits a closed SQLite handle. LRU + // eviction only ever tears down UNPINNED cores, so this await is a no-op + // there; it matters for manual evict()/wipe() of a still-busy account. + if (entry.pins > 0) { + await new Promise((resolve) => { + entry.onIdle = resolve; + }); + } + await entry.ready.catch(() => {}); + try { + await entry.core.destroy(); // flushes pipeline queues, then closes store + } catch (err) { + this.opts.logger.warn?.( + `[tdai-gateway] [registry] Teardown of ${key} failed: ${String(err)}`, + ); + } + })(); + this.closing.set(key, done); + void done.finally(() => { + // Only clear if this exact teardown is still the tracked one (a newer + // getCore→evict cycle may have replaced it). + if (this.closing.get(key) === done) this.closing.delete(key); + }); + return done; + } + + private resolve(sessionKey: string): { key: string; dataDir: string } { + if (!this.opts.multiTenant) { + return { key: SINGLE_TENANT_KEY, dataDir: this.opts.baseDir }; + } + const dir = safeAccountDir(sessionKey); // throws on empty — enforced upstream + return { key: dir, dataDir: path.join(this.opts.baseDir, dir) }; + } + + private createCore(dataDir: string): TdaiCore { + const adapter = new StandaloneHostAdapter({ + dataDir, + llmConfig: this.opts.llmConfig, + logger: this.opts.logger, + platform: "gateway", + }); + return new TdaiCore({ + hostAdapter: adapter, + config: this.opts.memory, + sessionFilter: new SessionFilter(this.opts.excludeAgents ?? []), + extractionLimiter: this.extractionLimiter, + }); + } +} diff --git a/src/gateway/server.e2e.test.ts b/src/gateway/server.e2e.test.ts new file mode 100644 index 00000000..4c9e9434 --- /dev/null +++ b/src/gateway/server.e2e.test.ts @@ -0,0 +1,204 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { TdaiGateway } from "./server.js"; +import type { GatewayConfig } from "./config.js"; +import { parseConfig } from "../config.js"; + +/** + * HTTP-level end-to-end tests for the Gateway + CoreRegistry multi-tenant + * routing. These boot a real HTTP server backed by per-account SQLite stores, + * so they live in the e2e suite (run via `vitest --config vitest.e2e.config.ts`) + * rather than the default unit run — booting ~a dozen stores in parallel with + * the fast unit files starves the background L0 indexer. No real LLM/embedding: + * extraction off + provider "none" (keyword/FTS path). + */ + +// Pin the memory config explicitly so these tests are hermetic: without it, +// loadGatewayConfig() picks up the repo-root tdai-gateway.yaml from CWD, which +// enables DashScope embedding — so a machine with DASHSCOPE_API_KEY set would +// silently exercise real embedding/network instead of the keyword/FTS path +// these assertions assume. extraction off → no background LLM timers. +const baseOverrides = (baseDir: string, multiTenant: boolean): Partial => ({ + server: { port: 0, host: "127.0.0.1", apiKey: undefined, corsOrigins: [] }, + data: { baseDir, multiTenant }, + memory: parseConfig({ + extraction: { enabled: false }, + embedding: { provider: "none" }, + recall: { strategy: "keyword" }, + }), +}); + +async function post(url: string, body: unknown): Promise<{ status: number; json: any }> { + const res = await fetch(url, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(body), + }); + return { status: res.status, json: await res.json().catch(() => null) }; +} + +async function getJson(url: string): Promise<{ status: number; json: any }> { + const res = await fetch(url); + return { status: res.status, json: await res.json().catch(() => null) }; +} + +/** + * Poll conversation search until the owner's just-captured L0 is FTS-indexed. + * Capture indexes L0 in a fire-and-forget background task, so a search issued + * immediately after /capture can race ahead of indexing — poll the *positive* + * (owner) case so the subsequent cross-tenant negative assertion is meaningful. + */ +async function pollSearchTotal( + origin: string, + sessionKey: string, + query: string, + timeoutMs = 30_000, +): Promise { + const deadline = Date.now() + timeoutMs; + let total = 0; + while (Date.now() < deadline) { + const r = await post(`${origin}/search/conversations`, { session_key: sessionKey, query }); + total = r.json?.total ?? 0; + if (total > 0) return total; + await new Promise((res) => setTimeout(res, 100)); + } + return total; +} + +describe("TdaiGateway HTTP wiring", () => { + let baseDir: string; + let gateway: TdaiGateway; + let origin: string; + + afterEach(async () => { + await gateway?.stop(); + if (baseDir) await fs.rm(baseDir, { recursive: true, force: true }); + }); + + async function boot(multiTenant: boolean): Promise { + baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "tdai-gw-")); + gateway = new TdaiGateway(baseOverrides(baseDir, multiTenant)); + await gateway.start(); + const addr = gateway.address(); + if (!addr) throw new Error("gateway did not bind"); + origin = `http://127.0.0.1:${addr.port}`; + } + + // One multi-tenant boot covers health/routing/capture/isolation/wipe so the + // suite tears down few in-process gateways (each full teardown drains a SQLite + // store + background indexer; many boots in one process starve each other). + it("multi-tenant: end-to-end routing, L0 + persona isolation, and wipe", async () => { + await boot(true); + + // Health: lazy, no cores yet. + const health0 = await getJson(`${origin}/health`); + expect(health0.json.multi_tenant).toBe(true); + expect(health0.json.active_cores).toBe(0); + // Embedding *config intent* is reported even with no cores resident. This + // boot forces provider "none" + keyword recall, so vector recall is off. + expect(health0.json.embedding.configured).toBe(false); + expect(health0.json.embedding.provider).toBe("none"); + expect(health0.json.embedding.recallStrategy).toBe("keyword"); + + // Routed endpoints require session_key. + for (const ep of ["/recall", "/capture", "/search/memories", "/search/conversations"]) { + const { status } = await post(`${origin}${ep}`, { query: "q", user_content: "u", assistant_content: "a" }); + expect(status, `${ep} should 400 without session_key`).toBe(400); + } + + // /seed is refused in multi-tenant mode: it writes a shared snapshot dir, + // not the per-account store, so a "successful" seed would be invisible to + // recall/search. Even a well-formed body (session_key + data) must 400. + const seedMt = await post(`${origin}/seed`, { + session_key: "ai4all:alice", + data: [{ sessionKey: "ai4all:alice", conversations: [[{ role: "user", content: "x" }, { role: "assistant", content: "y" }]] }], + }); + expect(seedMt.status, "/seed should 400 in multi-tenant mode").toBe(400); + + // Capture two accounts. This is the FIRST capture to each freshly-created + // account core, which is exactly the cold-start path: the gateway must stamp + // the turn's messages strictly after the cold-start L0 cursor floor, so BOTH + // the user and assistant message land. A regression here records 0 or 1 (the + // user message silently filtered by a floor sitting in the same millisecond) + // — assert the exact count, not just >0, to catch that. + const capA = await post(`${origin}/capture`, { + session_key: "ai4all:alice", + user_content: "alice fact pineapple", + assistant_content: "ok", + }); + expect(capA.status).toBe(200); + expect(capA.json.l0_recorded).toBe(2); + const capB = await post(`${origin}/capture`, { + session_key: "ai4all:bob", + user_content: "bob fact coffee", + assistant_content: "ok", + }); + expect(capB.status).toBe(200); + expect(capB.json.l0_recorded).toBe(2); + + // Two resident cores; two on-disk account dirs under baseDir. + expect((await getJson(`${origin}/health`)).json.active_cores).toBe(2); + const dirs = (await fs.readdir(baseDir)).filter((d) => !d.startsWith("seed-")); + expect(dirs).toHaveLength(2); + + // Both accounts' L0 indexes and is searchable by its owner. + expect(await pollSearchTotal(origin, "ai4all:alice", "pineapple")).toBeGreaterThan(0); + expect(await pollSearchTotal(origin, "ai4all:bob", "coffee")).toBeGreaterThan(0); + + // Isolation: neither account can see the other's L0. + const bobSeesAlice = await post(`${origin}/search/conversations`, { session_key: "ai4all:bob", query: "pineapple" }); + expect(bobSeesAlice.json.total).toBe(0); + expect(bobSeesAlice.json.results).not.toContain("pineapple"); + const aliceSeesBob = await post(`${origin}/search/conversations`, { session_key: "ai4all:alice", query: "coffee" }); + expect(aliceSeesBob.json.total).toBe(0); + + // ── L3 (persona) isolation via /recall (design §8.4 #5) ────────────────── + // Persona is a per-account file at `/persona.md`; recall reads it + // fresh and injects it into `context` (appendSystemContext, ). + // Seed a distinct persona per account directly on disk (no LLM needed) and + // assert each /recall sees ONLY its own — the structural per-account dataDir + // is what keeps the upper Markdown layers isolated, not a query filter. + const aliceDirEarly = (await fs.readdir(baseDir)).find((d) => d.startsWith("ai4all_alice"))!; + const bobDirEarly = (await fs.readdir(baseDir)).find((d) => d.startsWith("ai4all_bob"))!; + await fs.writeFile(path.join(baseDir, aliceDirEarly, "persona.md"), "Alice is a PINEAPPLE farmer."); + await fs.writeFile(path.join(baseDir, bobDirEarly, "persona.md"), "Bob is a COFFEE roaster."); + + const aliceRecall = await post(`${origin}/recall`, { session_key: "ai4all:alice", query: "hello" }); + expect(aliceRecall.status).toBe(200); + expect(aliceRecall.json.context).toContain(""); + expect(aliceRecall.json.context).toContain("PINEAPPLE farmer"); + expect(aliceRecall.json.context).not.toContain("COFFEE roaster"); // no leak from bob + + const bobRecall = await post(`${origin}/recall`, { session_key: "ai4all:bob", query: "hello" }); + expect(bobRecall.status).toBe(200); + expect(bobRecall.json.context).toContain("COFFEE roaster"); + expect(bobRecall.json.context).not.toContain("PINEAPPLE"); // no leak from alice + + // Wipe alice → alice dir gone, bob intact and still searchable. + const aliceDir = (await fs.readdir(baseDir)).find((d) => d.startsWith("ai4all_alice"))!; + const bobDir = (await fs.readdir(baseDir)).find((d) => d.startsWith("ai4all_bob"))!; + const wipe = await post(`${origin}/namespace/wipe`, { session_key: "ai4all:alice" }); + expect(wipe.status).toBe(200); + expect(wipe.json.wiped).toBe(true); + + const dirsAfter = await fs.readdir(baseDir); + expect(dirsAfter).not.toContain(aliceDir); + expect(dirsAfter).toContain(bobDir); + expect((await post(`${origin}/search/conversations`, { session_key: "ai4all:bob", query: "coffee" })).json.total).toBeGreaterThan(0); + expect((await post(`${origin}/search/conversations`, { session_key: "ai4all:alice", query: "pineapple" })).json.total).toBe(0); + }); + + it("single-tenant: health reports the shared store, and wipe is rejected", async () => { + await boot(false); + + const { status, json } = await getJson(`${origin}/health`); + expect(status).toBe(200); + expect(json.multi_tenant).toBe(false); + expect(json.active_cores).toBeUndefined(); + + const wipe = await post(`${origin}/namespace/wipe`, { session_key: "ai4all:alice" }); + expect(wipe.status).toBe(400); + }); +}); diff --git a/src/gateway/server.ts b/src/gateway/server.ts index 1da5592e..29e71aad 100644 --- a/src/gateway/server.ts +++ b/src/gateway/server.ts @@ -15,14 +15,14 @@ */ import http from "node:http"; +import fs from "node:fs"; import { URL } from "node:url"; import { timingSafeEqual } from "node:crypto"; -import { TdaiCore } from "../core/tdai-core.js"; -import { StandaloneHostAdapter } from "../adapters/standalone/host-adapter.js"; +import { CoreRegistry } from "./core-registry.js"; +import type { CoreLease } from "./core-registry.js"; import { loadGatewayConfig } from "./config.js"; import type { GatewayConfig } from "./config.js"; import { initDataDirectories } from "../utils/pipeline-factory.js"; -import { SessionFilter } from "../utils/session-filter.js"; import type { HealthResponse, RecallRequest, @@ -35,6 +35,8 @@ import type { ConversationSearchResponse, SessionEndRequest, SessionEndResponse, + WipeRequest, + WipeResponse, SeedRequest, SeedResponse, GatewayErrorResponse, @@ -114,39 +116,67 @@ function safeEqual(a: string, b: string): boolean { export class TdaiGateway { private config: GatewayConfig; private logger: Logger; - private core: TdaiCore; + private registry: CoreRegistry; + private multiTenant: boolean; private server: http.Server | null = null; private startTime = Date.now(); constructor(configOverrides?: Partial) { this.config = loadGatewayConfig(configOverrides); this.logger = createConsoleLogger(); + this.multiTenant = this.config.data.multiTenant; - // Create host adapter - const adapter = new StandaloneHostAdapter({ - dataDir: this.config.data.baseDir, + // Route requests to a per-account TdaiCore (or one shared core in + // single-tenant mode). The registry owns core lifecycle + dataDir binding. + this.registry = new CoreRegistry({ + baseDir: this.config.data.baseDir, llmConfig: this.config.llm, + memory: this.config.memory, logger: this.logger, - platform: "gateway", + multiTenant: this.multiTenant, + excludeAgents: this.config.memory.capture.excludeAgents, + maxConcurrentExtractions: this.config.data.maxConcurrentExtractions, + maxResidentCores: this.config.data.maxResidentCores, + coreIdleTtlMs: this.config.data.coreIdleTtlMs, }); + } - // Create core - this.core = new TdaiCore({ - hostAdapter: adapter, - config: this.config.memory, - sessionFilter: new SessionFilter(this.config.memory.capture.excludeAgents), - }); + /** + * Lease the core for a request, enforcing the multi-tenant `session_key` + * contract. Returns `null` (after writing a 400) when multi-tenant mode is on + * but the caller omitted `session_key`, so handlers must short-circuit. + * + * The returned lease pins the core for the duration of the request — handlers + * MUST `release()` it in a `finally` so it becomes eligible for LRU eviction + * again. The pin is what prevents another account's request from evicting + + * destroying this core while we're mid-`capture`/`recall`/`search`. + */ + private async leaseFor( + sessionKey: string | undefined, + res: http.ServerResponse, + ): Promise { + if (this.multiTenant && !sessionKey) { + sendError(res, 400, "Missing required field in multi-tenant mode: session_key"); + return null; + } + return this.registry.acquire(sessionKey ?? ""); } /** * Start the Gateway HTTP server. */ async start(): Promise { - // Initialize data directories - initDataDirectories(this.config.data.baseDir); - - // Initialize core - await this.core.initialize(); + if (this.multiTenant) { + // baseDir is only the *parent* of per-account dataDirs; each account core + // builds its own subdir layout lazily. Just ensure the parent exists. + fs.mkdirSync(this.config.data.baseDir, { recursive: true }); + } else { + // Single-tenant: baseDir IS the shared core's dataDir. Build the full + // layout and eagerly create + initialize the one shared core so the first + // request (and /health) sees a ready store, matching legacy startup. + initDataDirectories(this.config.data.baseDir); + await this.registry.getCore(""); + } // Create HTTP server this.server = http.createServer((req, res) => this.handleRequest(req, res)); @@ -212,6 +242,15 @@ export class TdaiGateway { } } + /** + * The actual bound address after {@link start}. Returns `null` before the + * server is listening. Primarily for tests / orchestration that bind port 0. + */ + address(): { host: string; port: number } | null { + const a = this.server?.address(); + return a && typeof a === "object" ? { host: a.address, port: a.port } : null; + } + /** * Gracefully stop the Gateway. */ @@ -224,7 +263,7 @@ export class TdaiGateway { }); } - await this.core.destroy(); + await this.registry.destroyAll(); this.logger.info("Gateway stopped"); } @@ -270,6 +309,8 @@ export class TdaiGateway { return await this.handleSearchConversations(req, res); case "POST /session/end": return await this.handleSessionEnd(req, res); + case "POST /namespace/wipe": + return await this.handleWipe(req, res); case "POST /seed": return await this.handleSeed(req, res); default: @@ -355,15 +396,57 @@ export class TdaiGateway { // Route handlers // ============================ + /** + * Summarise embedding *configuration intent* for /health — config-only, no + * network probe (health is a hot liveness path). `configured` is true only + * when embedding is enabled, the provider is not the "none" disable sentinel, + * and the config carries no error. The live signal stays the `strategy` field + * on /search/memories responses. + */ + private embeddingSummary(): NonNullable { + const emb = this.config.memory.embedding; + return { + configured: emb.enabled && emb.provider !== "none" && !emb.configError, + provider: emb.provider, + model: emb.model || undefined, + dimensions: emb.dimensions || undefined, + recallStrategy: this.config.memory.recall.strategy, + }; + } + private handleHealth(res: http.ServerResponse): void { + if (this.multiTenant) { + // No single shared store to probe — cores are per-account and lazy. Report + // liveness, resident-core stats, and embedding *config intent* (the only + // embedding signal possible without spinning up a core), since + // stores.embeddingService cannot reflect per-account lazy services here. + const response: HealthResponse = { + status: "ok", + version: VERSION, + uptime: Math.floor((Date.now() - this.startTime) / 1000), + stores: { vectorStore: false, embeddingService: false }, + multi_tenant: true, + active_cores: this.registry.size, + extraction: this.registry.extractionStats(), + resident: this.registry.residentStats(), + embedding: this.embeddingSummary(), + }; + sendJson(res, 200, response); + return; + } + + // Single-tenant: probe the one shared core (created eagerly in start()). + const core = this.registry.peek(""); const response: HealthResponse = { - status: this.core.getVectorStore() ? "ok" : "degraded", + status: core?.getVectorStore() ? "ok" : "degraded", version: VERSION, uptime: Math.floor((Date.now() - this.startTime) / 1000), stores: { - vectorStore: !!this.core.getVectorStore(), - embeddingService: !!this.core.getEmbeddingService(), + vectorStore: !!core?.getVectorStore(), + embeddingService: !!core?.getEmbeddingService(), }, + multi_tenant: false, + embedding: this.embeddingSummary(), }; sendJson(res, 200, response); } @@ -376,18 +459,29 @@ export class TdaiGateway { return; } - const startMs = Date.now(); - const result = await this.core.handleBeforeRecall(body.query, body.session_key); - const elapsed = Date.now() - startMs; + const lease = await this.leaseFor(body.session_key, res); + if (!lease) return; - this.logger.info(`Recall completed in ${elapsed}ms: context=${(result.appendSystemContext?.length ?? 0)} chars`); + try { + const startMs = Date.now(); + const result = await lease.core.handleBeforeRecall(body.query, body.session_key); + const elapsed = Date.now() - startMs; - const response: RecallResponse = { - context: result.appendSystemContext ?? "", - strategy: result.recallStrategy, - memory_count: result.recalledL1Memories?.length ?? 0, - }; - sendJson(res, 200, response); + this.logger.info( + `Recall completed in ${elapsed}ms: context=${(result.appendSystemContext?.length ?? 0)} chars, ` + + `prepend=${(result.prependContext?.length ?? 0)} chars`, + ); + + const response: RecallResponse = { + context: result.appendSystemContext ?? "", + prepend_context: result.prependContext ?? "", + strategy: result.recallStrategy, + memory_count: result.recalledL1Memories?.length ?? 0, + }; + sendJson(res, 200, response); + } finally { + lease.release(); + } } private async handleCapture(req: http.IncomingMessage, res: http.ServerResponse): Promise { @@ -398,26 +492,48 @@ export class TdaiGateway { return; } + // Capture the turn-start timestamp BEFORE resolving the core. On the first + // capture to a brand-new account the core is created lazily here (open DB, + // warm store), which puts real wall-clock distance between startMs and the + // moment messages are stamped during extraction — so startMs is reliably + // earlier than any message timestamp, even for caller-supplied messages + // that lack one. const startMs = Date.now(); - const result = await this.core.handleTurnCommitted({ - userText: body.user_content, - assistantText: body.assistant_content, - messages: body.messages ?? [ - { role: "user", content: body.user_content }, - { role: "assistant", content: body.assistant_content }, - ], - sessionKey: body.session_key, - sessionId: body.session_id, - }); - const elapsed = Date.now() - startMs; - this.logger.info(`Capture completed in ${elapsed}ms: l0=${result.l0RecordedCount}`); + const lease = await this.leaseFor(body.session_key, res); + if (!lease) return; - const response: CaptureResponse = { - l0_recorded: result.l0RecordedCount, - scheduler_notified: result.schedulerNotified, - }; - sendJson(res, 200, response); + try { + // Stamp the synthesized messages explicitly (startMs+1/+2) and pass + // startedAt=startMs as the cold-start L0 cursor floor. The recorder keeps + // messages with timestamp strictly greater than the floor, so the floor + // MUST be below the turn's own messages. Previously no startedAt was passed, + // so the floor fell back to Date.now() inside TdaiCore — landing in the same + // millisecond as the messages and silently dropping the first turn's L0 rows + // on every freshly-created account (see cold-start capture regression test). + const result = await lease.core.handleTurnCommitted({ + userText: body.user_content, + assistantText: body.assistant_content, + messages: body.messages ?? [ + { role: "user", content: body.user_content, timestamp: startMs + 1 }, + { role: "assistant", content: body.assistant_content, timestamp: startMs + 2 }, + ], + sessionKey: body.session_key, + sessionId: body.session_id, + startedAt: startMs, + }); + const elapsed = Date.now() - startMs; + + this.logger.info(`Capture completed in ${elapsed}ms: l0=${result.l0RecordedCount}`); + + const response: CaptureResponse = { + l0_recorded: result.l0RecordedCount, + scheduler_notified: result.schedulerNotified, + }; + sendJson(res, 200, response); + } finally { + lease.release(); + } } private async handleSearchMemories(req: http.IncomingMessage, res: http.ServerResponse): Promise { @@ -428,19 +544,26 @@ export class TdaiGateway { return; } - const result = await this.core.searchMemories({ - query: body.query, - limit: body.limit, - type: body.type, - scene: body.scene, - }); + const lease = await this.leaseFor(body.session_key, res); + if (!lease) return; - const response: MemorySearchResponse = { - results: result.text, - total: result.total, - strategy: result.strategy, - }; - sendJson(res, 200, response); + try { + const result = await lease.core.searchMemories({ + query: body.query, + limit: body.limit, + type: body.type, + scene: body.scene, + }); + + const response: MemorySearchResponse = { + results: result.text, + total: result.total, + strategy: result.strategy, + }; + sendJson(res, 200, response); + } finally { + lease.release(); + } } private async handleSearchConversations(req: http.IncomingMessage, res: http.ServerResponse): Promise { @@ -451,17 +574,24 @@ export class TdaiGateway { return; } - const result = await this.core.searchConversations({ - query: body.query, - limit: body.limit, - sessionKey: body.session_key, - }); + const lease = await this.leaseFor(body.session_key, res); + if (!lease) return; - const response: ConversationSearchResponse = { - results: result.text, - total: result.total, - }; - sendJson(res, 200, response); + try { + const result = await lease.core.searchConversations({ + query: body.query, + limit: body.limit, + sessionKey: body.session_key, + }); + + const response: ConversationSearchResponse = { + results: result.text, + total: result.total, + }; + sendJson(res, 200, response); + } finally { + lease.release(); + } } private async handleSessionEnd(req: http.IncomingMessage, res: http.ServerResponse): Promise { @@ -472,15 +602,58 @@ export class TdaiGateway { return; } - await this.core.handleSessionEnd(body.session_key); + // Only flush a session that already has a resident core; never spin one up + // just to tear its buffers down. Unknown/evicted sessions are a no-op. + const core = this.registry.peek(body.session_key); + if (core) await core.handleSessionEnd(body.session_key); const response: SessionEndResponse = { flushed: true }; sendJson(res, 200, response); } + private async handleWipe(req: http.IncomingMessage, res: http.ServerResponse): Promise { + const body = await parseJsonBody(req); + + if (!body.session_key) { + sendError(res, 400, "Missing required field: session_key"); + return; + } + + if (!this.multiTenant) { + // No per-account dataDir exists in single-tenant mode; refuse rather than + // risk deleting the shared store out from under the process. + sendError(res, 400, "namespace wipe is only supported in multi-tenant mode"); + return; + } + + const dataDir = await this.registry.wipe(body.session_key); + this.logger.info(`Wiped account namespace for ${body.session_key} (${dataDir})`); + + const response: WipeResponse = { wiped: true }; + sendJson(res, 200, response); + } + private async handleSeed(req: http.IncomingMessage, res: http.ServerResponse): Promise { const body = await parseJsonBody(req); + if (this.multiTenant) { + // /seed writes to a shared `baseDir/seed-` snapshot dir, NOT the + // per-account store under `baseDir/{account}` that recall/search read — so + // in multi-tenant mode a "successful" seed (200, l0_recorded > 0) is + // invisible to every core. Refuse rather than silently no-op. Backfill an + // account by running `executeSeed` into `registry.resolveDataDir(key)` + // (the exact per-account dir); see scripts/import-psydt.ts for the pattern. + sendError( + res, + 400, + "POST /seed is not supported in multi-tenant mode: it writes a shared snapshot " + + "dir, not the per-account store, so seeded memory is invisible to /recall and " + + "/search. Seed per-account into registry.resolveDataDir(session_key) — see " + + "scripts/import-psydt.ts.", + ); + return; + } + if (!body.data) { sendError(res, 400, "Missing required field: data"); return; diff --git a/src/gateway/types.ts b/src/gateway/types.ts index 50b2ff4c..861697db 100644 --- a/src/gateway/types.ts +++ b/src/gateway/types.ts @@ -23,6 +23,48 @@ export interface HealthResponse { vectorStore: boolean; embeddingService: boolean; }; + /** Whether the Gateway is routing per-account (structural multi-tenant). */ + multi_tenant?: boolean; + /** + * Multi-tenant only: number of per-account cores currently resident in + * memory. Omitted in single-tenant mode (there is exactly one shared core). + */ + active_cores?: number; + /** + * Multi-tenant only: live state of the shared background-extraction limiter + * that caps concurrent L1/L2/L3 runs across all cores (design §8.4 #5). + * `limit` is the configured cap (0 = unbounded), `active` the permits in use, + * `waiting` the runs currently blocked on a permit. + */ + extraction?: { limit: number; active: number; waiting: number }; + /** + * Multi-tenant only: resident-core LRU state. `count` is how many per-account + * cores are warm right now, `limit` the configured cap (0 = unlimited), and + * `pinned` how many are currently serving a request (held by a lease). A + * `count` above `limit` with a matching `pinned` means the cap is being held + * past its bound because every candidate core is busy — transient and safe. + */ + resident?: { count: number; limit: number; pinned: number }; + /** + * Embedding **configuration intent** — answers "is vector recall supposed to + * be on?" without a network probe (health must stay a cheap liveness check). + * + * `configured` is `true` only when embedding is enabled, the provider is not + * the `"none"` disable sentinel, and the config carries no error. It does NOT + * confirm the embedding endpoint is reachable — for the live runtime signal, + * read the `strategy` field returned by `POST /search/memories` (`hybrid` / + * `embedding` mean vectors actually fired). In multi-tenant mode this is the + * embedding signal health can give, since cores (and their embedding services) + * are lazy and per-account — `stores.embeddingService` stays `false` there. + */ + embedding?: { + configured: boolean; + provider: string; + model?: string; + dimensions?: number; + /** Configured recall strategy: "hybrid" | "embedding" | "keyword". */ + recallStrategy: string; + }; } // ============================ @@ -36,7 +78,20 @@ export interface RecallRequest { } export interface RecallResponse { + /** + * Stable recall context for the system prompt end (persona, scene nav, + * tools guide). Mirrors `RecallResult.appendSystemContext`. + */ context: string; + /** + * Query-time L1 relevant memories, meant to be prepended to the user prompt + * (dynamic, per-turn). Mirrors `RecallResult.prependContext`. Empty string + * when no L1 memories were recalled this turn. + * + * Without this, callers get persona/scene but never the query-relevant + * memories, while `memory_count` still reports L1 hits — see design §5.3/§8.4#6. + */ + prepend_context: string; strategy?: string; memory_count?: number; } @@ -68,6 +123,13 @@ export interface MemorySearchRequest { limit?: number; type?: string; scene?: string; + /** + * Account/session to scope the search to. **Required in multi-tenant mode** + * (the Gateway returns 400 without it); ignored in single-tenant mode. In + * the structural multi-tenant route this routes to the account's own core, so + * L1 isolation is physical — see design §8.4 #3. + */ + session_key?: string; } export interface MemorySearchResponse { @@ -104,6 +166,24 @@ export interface SessionEndResponse { flushed: boolean; } +// ============================ +// /namespace/wipe +// ============================ + +/** + * Request body for `POST /namespace/wipe` — account hard-delete. + * + * Multi-tenant only. Removes the account's core and its entire on-disk + * dataDir (L0/L1/L2/L3). Backs the host's `unbind_and_wipe_account()`. + */ +export interface WipeRequest { + session_key: string; +} + +export interface WipeResponse { + wiped: boolean; +} + // ============================ // /seed // ============================ diff --git a/src/utils/async-semaphore.test.ts b/src/utils/async-semaphore.test.ts new file mode 100644 index 00000000..260b6e04 --- /dev/null +++ b/src/utils/async-semaphore.test.ts @@ -0,0 +1,149 @@ +import { describe, it, expect } from "vitest"; +import { AsyncSemaphore, PASSTHROUGH_LIMITER } from "./async-semaphore.js"; + +/** Resolve after the given ms (real timer — these tests exercise async timing). */ +const delay = (ms: number) => new Promise((r) => setTimeout(r, ms)); + +/** + * Run `n` tasks through `limiter`, each holding for `holdMs`, and report the + * peak number observed running at once. With a cap of `k`, peak must be ≤ k. + */ +async function peakConcurrency( + limiter: { run(fn: () => Promise): Promise }, + n: number, + holdMs: number, +): Promise { + let active = 0; + let peak = 0; + await Promise.all( + Array.from({ length: n }, () => + limiter.run(async () => { + active++; + peak = Math.max(peak, active); + await delay(holdMs); + active--; + }), + ), + ); + return peak; +} + +describe("AsyncSemaphore", () => { + it("caps concurrency at the configured limit", async () => { + const sem = new AsyncSemaphore(2); + expect(await peakConcurrency(sem, 6, 15)).toBe(2); + // Fully drained afterwards. + expect(sem.active).toBe(0); + expect(sem.waiting).toBe(0); + }); + + it("serializes when the limit is 1", async () => { + const sem = new AsyncSemaphore(1); + expect(await peakConcurrency(sem, 5, 10)).toBe(1); + }); + + it("treats limit <= 0 as unlimited (no gating)", async () => { + for (const lim of [0, -1, Number.NaN]) { + const sem = new AsyncSemaphore(lim); + expect(sem.unlimited).toBe(true); + expect(sem.capacity).toBe(0); + // All 8 run together — peak equals the task count. + expect(await peakConcurrency(sem, 8, 10)).toBe(8); + } + }); + + it("PASSTHROUGH_LIMITER runs everything immediately", async () => { + expect(await peakConcurrency(PASSTHROUGH_LIMITER, 5, 10)).toBe(5); + }); + + it("serves waiters in FIFO order", async () => { + const sem = new AsyncSemaphore(1); + const order: number[] = []; + const release0Holder: Promise<() => void> = sem.acquire(); + const release0 = await release0Holder; // hold the only permit + + // Queue three waiters; they must run in the order they queued. + const waiters = [1, 2, 3].map((i) => + sem.run(async () => { + order.push(i); + await delay(5); + }), + ); + + expect(sem.waiting).toBe(3); + release0(); // let them through one at a time + await Promise.all(waiters); + expect(order).toEqual([1, 2, 3]); + }); + + it("never over-subscribes when an acquire races a release", async () => { + // Regression guard for the classic handoff bug: releasing while a waiter is + // queued must hand the permit over without transiently freeing a slot that + // a synchronously-racing acquire() could grab. + const sem = new AsyncSemaphore(1); + const r1 = await sem.acquire(); // inUse=1 + const w = sem.acquire(); // queued waiter + expect(sem.waiting).toBe(1); + + r1(); // hand permit to the queued waiter + // Synchronously race a third acquire before the waiter's microtask resolves. + let thirdGotPermit = false; + const third = sem.acquire().then((rel) => { + thirdGotPermit = true; + return rel; + }); + + const r2 = await w; // waiter got the permit + expect(sem.active).toBe(1); + expect(thirdGotPermit).toBe(false); // third still blocked — no over-subscription + + r2(); + const r3 = await third; + expect(thirdGotPermit).toBe(true); + r3(); + expect(sem.active).toBe(0); + }); + + it("release is idempotent (double-release does not free extra permits)", async () => { + const sem = new AsyncSemaphore(1); + const rel = await sem.acquire(); + expect(sem.active).toBe(1); + rel(); + rel(); // no-op + expect(sem.active).toBe(0); + + // A fresh acquire still works and the limit is intact. + expect(await peakConcurrency(sem, 3, 5)).toBe(1); + }); + + it("propagates task errors while still releasing the permit", async () => { + const sem = new AsyncSemaphore(1); + await expect( + sem.run(async () => { + throw new Error("boom"); + }), + ).rejects.toThrow("boom"); + // Permit was released despite the throw. + expect(sem.active).toBe(0); + expect(await peakConcurrency(sem, 2, 5)).toBe(1); + }); + + it("reports live active / waiting counts", async () => { + const sem = new AsyncSemaphore(2); + const rels = await Promise.all([sem.acquire(), sem.acquire()]); + expect(sem.active).toBe(2); + expect(sem.available).toBe(0); + + const queued = sem.acquire(); + expect(sem.waiting).toBe(1); + + rels[0]!(); + const rel3 = await queued; + expect(sem.waiting).toBe(0); + expect(sem.active).toBe(2); + + rels[1]!(); + rel3(); + expect(sem.active).toBe(0); + }); +}); diff --git a/src/utils/async-semaphore.ts b/src/utils/async-semaphore.ts new file mode 100644 index 00000000..36e65f16 --- /dev/null +++ b/src/utils/async-semaphore.ts @@ -0,0 +1,126 @@ +/** + * AsyncSemaphore — a small counting semaphore for bounding async concurrency. + * + * ## Why this exists (multi-tenant extraction cap) + * + * In the structural multi-tenant route (see `gateway/core-registry.ts`) every + * account gets its **own** {@link MemoryPipelineManager}, each with its own L1/L2/L3 + * `SerialQueue`s. A single account is therefore naturally capped (one L1, one L2, + * one L3 at a time), but `N` resident accounts fan out to up to `~3N` + * **concurrent background LLM extraction calls** with nothing holding them back — + * the real cost red line called out in design §8.4 #5. + * + * The fix is one semaphore *shared across all cores*: each pipeline manager runs + * its L1/L2/L3 runner through {@link run}, so the total number of in-flight + * extraction runs across every tenant can never exceed the configured limit. + * + * ## Semantics + * + * - `limit <= 0` (or non-finite) means **unlimited** — {@link acquire}/{@link run} + * resolve immediately and impose no ordering. This is the single-tenant default + * (one core can't fan out, so there is nothing to cap) and keeps legacy + * behaviour byte-for-byte. + * - Waiters are served strictly FIFO. + * - A permit is **handed directly** from a releaser to the next waiter without + * ever dropping `active` below the in-use count, so a synchronous `acquire()` + * racing in between a release and the woken waiter cannot over-subscribe the + * limit. (The naive "decrement then resolve" implementation has exactly that + * bug, because resolving a promise only schedules a microtask.) + */ + +/** Minimal interface the pipeline manager depends on (keeps it test-friendly). */ +export interface ConcurrencyLimiter { + /** Run `fn` once a permit is available, releasing the permit when it settles. */ + run(fn: () => Promise): Promise; +} + +/** A limiter that imposes no bound — used when concurrency capping is disabled. */ +export const PASSTHROUGH_LIMITER: ConcurrencyLimiter = { + run: (fn) => fn(), +}; + +const NOOP_RELEASE = (): void => {}; + +export class AsyncSemaphore implements ConcurrencyLimiter { + /** Configured permit count; `<= 0` means unlimited. */ + private readonly limit: number; + /** Permits currently held (acquired but not yet released). */ + private inUse = 0; + /** FIFO queue of resolvers for callers waiting on a permit. */ + private readonly waiters: Array<() => void> = []; + + constructor(limit: number) { + this.limit = Number.isFinite(limit) && limit > 0 ? Math.floor(limit) : 0; + } + + /** Whether this semaphore imposes no bound. */ + get unlimited(): boolean { + return this.limit <= 0; + } + + /** Configured permit count (0 = unlimited). */ + get capacity(): number { + return this.limit; + } + + /** Permits currently held. */ + get active(): number { + return this.inUse; + } + + /** Callers currently blocked waiting for a permit. */ + get waiting(): number { + return this.waiters.length; + } + + /** Free permits right now (`Infinity` when unlimited). */ + get available(): number { + return this.unlimited ? Infinity : Math.max(0, this.limit - this.inUse); + } + + /** + * Acquire a permit, resolving with an idempotent `release` function. Always + * call `release()` exactly once (a `try/finally` is safest — or use + * {@link run}, which does it for you). + */ + async acquire(): Promise<() => void> { + if (this.unlimited) return NOOP_RELEASE; + + if (this.inUse < this.limit) { + this.inUse++; + return this.makeRelease(); + } + + // At capacity: wait for a permit to be handed to us. The handoff in + // makeRelease() leaves `inUse` unchanged, so we must NOT increment here. + await new Promise((resolve) => this.waiters.push(resolve)); + return this.makeRelease(); + } + + /** Acquire a permit, run `fn`, and release the permit when it settles. */ + async run(fn: () => Promise): Promise { + if (this.unlimited) return fn(); + const release = await this.acquire(); + try { + return await fn(); + } finally { + release(); + } + } + + private makeRelease(): () => void { + let released = false; + return () => { + if (released) return; + released = true; + const next = this.waiters.shift(); + if (next) { + // Hand the permit straight to the next waiter: `inUse` stays the same, + // closing the window where a racing acquire() could over-subscribe. + next(); + } else { + this.inUse--; + } + }; + } +} diff --git a/src/utils/atomic-write.test.ts b/src/utils/atomic-write.test.ts new file mode 100644 index 00000000..ad1ffc17 --- /dev/null +++ b/src/utils/atomic-write.test.ts @@ -0,0 +1,60 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { atomicWriteFile } from "./atomic-write.js"; + +describe("atomicWriteFile", () => { + let dir: string; + + beforeEach(async () => { + dir = await fs.mkdtemp(path.join(os.tmpdir(), "atomic-write-test-")); + }); + + afterEach(async () => { + await fs.rm(dir, { recursive: true, force: true }); + }); + + it("writes content to a new file", async () => { + const p = path.join(dir, "persona.md"); + await atomicWriteFile(p, "hello"); + expect(await fs.readFile(p, "utf-8")).toBe("hello"); + }); + + it("overwrites an existing file", async () => { + const p = path.join(dir, "persona.md"); + await fs.writeFile(p, "old", "utf-8"); + await atomicWriteFile(p, "new"); + expect(await fs.readFile(p, "utf-8")).toBe("new"); + }); + + it("leaves no temp files behind on success", async () => { + const p = path.join(dir, "scene_index.json"); + await atomicWriteFile(p, JSON.stringify({ a: 1 })); + const entries = await fs.readdir(dir); + expect(entries).toEqual(["scene_index.json"]); + }); + + it("does not corrupt the existing file when the write throws", async () => { + const p = path.join(dir, "persona.md"); + await atomicWriteFile(p, "original"); + // Target a non-existent nested dir to force a write failure. + const bad = path.join(dir, "no-such-subdir", "persona.md"); + await expect(atomicWriteFile(bad, "doomed")).rejects.toBeTruthy(); + // Original untouched. + expect(await fs.readFile(p, "utf-8")).toBe("original"); + }); + + it("survives concurrent writes to different files without temp collisions", async () => { + await Promise.all( + Array.from({ length: 20 }, (_, i) => + atomicWriteFile(path.join(dir, `f${i}.md`), `content-${i}`), + ), + ); + const entries = (await fs.readdir(dir)).sort(); + expect(entries).toHaveLength(20); + for (let i = 0; i < 20; i++) { + expect(await fs.readFile(path.join(dir, `f${i}.md`), "utf-8")).toBe(`content-${i}`); + } + }); +}); diff --git a/src/utils/atomic-write.ts b/src/utils/atomic-write.ts new file mode 100644 index 00000000..4a0ca546 --- /dev/null +++ b/src/utils/atomic-write.ts @@ -0,0 +1,46 @@ +/** + * Atomic file write — write to a temp file in the same directory, then rename + * over the target. rename(2) is atomic on POSIX within a single filesystem, so + * a reader never observes a half-written file and a crash mid-write leaves the + * previous version intact (or, worst case, an orphan temp file). + * + * Why this matters for TDAI: L3 persona (`persona.md`) and L2 scene index are + * derived state written via a raw `fs.writeFile` read-modify-write, with no + * lock or atomicity. Under the multi-tenant retrofit a single dataDir is still + * owned by one process, but concurrent pipeline stages (or a crash) can corrupt + * a plain write. See design §8.5 red line 3 / issue §2. + * + * Contract reminder: this guards against torn writes within one process/host. + * It does NOT make two processes sharing one dataDir safe — that remains a hard + * "single dataDir, single process" constraint. + */ + +import fs from "node:fs/promises"; +import path from "node:path"; + +/** + * Atomically write `content` to `filePath`. + * + * The temp file is created in the same directory as the target (rename across + * filesystems is not atomic and would throw EXDEV). The temp name embeds the pid + * and a counter so concurrent writers in the same process don't collide. + */ +let counter = 0; +export async function atomicWriteFile( + filePath: string, + content: string | Uint8Array, + encoding: BufferEncoding = "utf-8", +): Promise { + const dir = path.dirname(filePath); + const base = path.basename(filePath); + // eslint-disable-next-line no-plusplus + const tmpPath = path.join(dir, `.${base}.${process.pid}.${counter++}.tmp`); + try { + await fs.writeFile(tmpPath, content, encoding); + await fs.rename(tmpPath, filePath); + } catch (err) { + // Best-effort cleanup of the temp file if rename never happened. + await fs.rm(tmpPath, { force: true }).catch(() => {}); + throw err; + } +} diff --git a/src/utils/keyed-mutex.test.ts b/src/utils/keyed-mutex.test.ts new file mode 100644 index 00000000..3fc7ff37 --- /dev/null +++ b/src/utils/keyed-mutex.test.ts @@ -0,0 +1,117 @@ +import { describe, it, expect } from "vitest"; +import { KeyedAsyncMutex } from "./keyed-mutex.js"; + +const tick = () => new Promise((r) => setTimeout(r, 0)); + +describe("KeyedAsyncMutex", () => { + it("serializes critical sections sharing a key (no interleave)", async () => { + const m = new KeyedAsyncMutex(); + const trace: string[] = []; + + const section = (id: string) => + m.run("k", async () => { + trace.push(`${id}:start`); + await tick(); + await tick(); + trace.push(`${id}:end`); + }); + + await Promise.all([section("a"), section("b"), section("c")]); + + // Each section's start is immediately followed by its own end — never + // another section's start in between. + expect(trace).toEqual([ + "a:start", "a:end", + "b:start", "b:end", + "c:start", "c:end", + ]); + }); + + it("runs in strict FIFO (call) order for the same key", async () => { + const m = new KeyedAsyncMutex(); + const order: number[] = []; + const tasks = [0, 1, 2, 3, 4].map((i) => + m.run("k", async () => { + await tick(); + order.push(i); + }), + ); + await Promise.all(tasks); + expect(order).toEqual([0, 1, 2, 3, 4]); + }); + + it("lets different keys proceed concurrently", async () => { + const m = new KeyedAsyncMutex(); + let aInside = false; + let bSawAInside = false; + + const a = m.run("a", async () => { + aInside = true; + await tick(); + await tick(); + aInside = false; + }); + // Give "a" a chance to enter its section first. + await tick(); + const b = m.run("b", async () => { + bSawAInside = aInside; // a different key must not block on a's lock + }); + + await Promise.all([a, b]); + expect(bSawAInside).toBe(true); + }); + + it("returns the critical section's value to the caller", async () => { + const m = new KeyedAsyncMutex(); + await expect(m.run("k", async () => 42)).resolves.toBe(42); + }); + + it("a throwing section rejects only that caller and does not poison the key", async () => { + const m = new KeyedAsyncMutex(); + const ran: string[] = []; + + const bad = m.run("k", async () => { + ran.push("bad"); + throw new Error("boom"); + }); + const good = m.run("k", async () => { + ran.push("good"); + return "ok"; + }); + + await expect(bad).rejects.toThrow("boom"); + await expect(good).resolves.toBe("ok"); // next waiter still runs + expect(ran).toEqual(["bad", "good"]); + }); + + it("releases the key when idle (map does not grow unbounded)", async () => { + const m = new KeyedAsyncMutex(); + expect(m.activeKeys).toBe(0); + + const p = m.run("k", async () => { + // While held, the key is tracked. + expect(m.activeKeys).toBe(1); + }); + await p; + // Once the last holder releases, the key is dropped. + expect(m.activeKeys).toBe(0); + + // A fresh acquire on the same key still works after cleanup. + await expect(m.run("k", async () => "again")).resolves.toBe("again"); + expect(m.activeKeys).toBe(0); + }); + + it("keeps the key while waiters remain, drops it after the last drains", async () => { + const m = new KeyedAsyncMutex(); + const p1 = m.run("k", async () => { + await tick(); + }); + const p2 = m.run("k", async () => { + await tick(); + }); + // Two holders queued on one key → exactly one key tracked. + expect(m.activeKeys).toBe(1); + await Promise.all([p1, p2]); + expect(m.activeKeys).toBe(0); + }); +}); diff --git a/src/utils/keyed-mutex.ts b/src/utils/keyed-mutex.ts new file mode 100644 index 00000000..cc286de4 --- /dev/null +++ b/src/utils/keyed-mutex.ts @@ -0,0 +1,80 @@ +/** + * KeyedAsyncMutex — a per-key async mutex (one critical section at a time per key). + * + * ## Why this exists (persona.md single-writer) + * + * `persona.md` is written by two background stages that run on **separate** + * `SerialQueue`s (`utils/pipeline-manager.ts`), so they are NOT serialized + * against each other: + * + * - **L3** `PersonaGenerator.generateLocalPersona` — a read → ~180s LLM + * tool-write → read-back → final write cycle. + * - **L2** `SceneExtractor.updateSceneNavigation` — a read → strip-nav → + * append-nav → write cycle. + * + * Both are read-modify-write. `atomicWriteFile` (temp+rename) prevents a *torn + * read* but NOT a *lost update*: if the two interleave, whichever writes last + * overwrites the other's body with content built from a now-stale read (e.g. an + * L2 nav write clobbering a freshly regenerated L3 body). Routing both writers + * through one mutex keyed by the file's absolute path makes each whole RMW + * mutually exclusive, so neither can clobber the other. + * + * The L3 critical section spans the full LLM run by design ("Method B"): the LLM + * writes `persona.md` directly mid-run, so a correct critical section must cover + * it. The cost is that a same-account L2 nav update can wait behind an in-flight + * persona regen — acceptable because regen is infrequent and nav is cheap and + * deferrable. Recall *reads* `persona.md` without taking this lock, so the read + * hot path is unaffected (atomic writes keep reads whole). + * + * ## Semantics + * + * - Keys are independent: different keys run concurrently, same key serializes. + * - Critical sections sharing a key run strictly in call (FIFO) order. + * - A failing critical section does NOT poison the key — the next waiter still + * runs, and `run` rejects only to the caller whose `fn` threw. + * - In-process only. This relies on the "one process per dataDir" invariant + * (see the integration guide §9.1); it does not guard against two processes + * writing the same file. + */ +export class KeyedAsyncMutex { + /** Tail of the release-chain per key. Absence === currently unlocked. */ + private readonly tails = new Map>(); + + /** Critical sections sharing `key` run one at a time, in call order. */ + async run(key: string, fn: () => Promise): Promise { + const prev = this.tails.get(key) ?? Promise.resolve(); + + // Our gate resolves when WE release; the next caller for this key waits on + // it (chained after every earlier holder). The chain only ever resolves + // (never rejects), so one failing section cannot block the queue. + let release!: () => void; + const gate = new Promise((resolve) => { + release = resolve; + }); + const tail = prev.then(() => gate); + this.tails.set(key, tail); + + await prev; // our turn, once all earlier holders have released + try { + return await fn(); + } finally { + release(); + // Drop the key once we are the last in line, so the map does not grow + // unbounded across many one-shot keys. + if (this.tails.get(key) === tail) this.tails.delete(key); + } + } + + /** Number of keys with an active or queued holder (for tests/diagnostics). */ + get activeKeys(): number { + return this.tails.size; + } +} + +/** + * Process-wide mutex for files written by more than one pipeline stage, keyed by + * the file's **absolute path**. Per-account isolation is automatic because each + * account's `persona.md` lives at a distinct path, so different accounts never + * contend. Shared by {@link PersonaGenerator} (L3) and {@link SceneExtractor} (L2). + */ +export const fileWriteMutex = new KeyedAsyncMutex(); diff --git a/src/utils/no-think-fetch.ts b/src/utils/no-think-fetch.ts index 6f68c2cb..c4307bbd 100644 --- a/src/utils/no-think-fetch.ts +++ b/src/utils/no-think-fetch.ts @@ -8,7 +8,9 @@ * * Strategies: * - `"vllm"`: vLLM / SGLang — `chat_template_kwargs.enable_thinking = false` - * - `"deepseek"`: DeepSeek official API — top-level `enable_thinking: false` + * - `"deepseek"`: DeepSeek V3-series API — top-level `enable_thinking: false` + * - `"deepseek-v4"`: DeepSeek V4 hybrid models (e.g. deepseek-v4-flash) — they + * ignore `enable_thinking`; honor `thinking: { type: "disabled" }` * - `"dashscope"`: Alibaba DashScope (Qwen) — top-level `enable_thinking: false` * - `"openai"`: OpenAI o-series — `reasoning_effort: "low"` (cannot fully disable) * - `"anthropic"`: Anthropic Claude — `thinking: { type: "disabled" }` @@ -22,6 +24,7 @@ export type DisableThinkingStrategy = | false | "vllm" | "deepseek" + | "deepseek-v4" | "dashscope" | "openai" | "anthropic" @@ -29,7 +32,7 @@ export type DisableThinkingStrategy = | "gemini"; export const VALID_DISABLE_THINKING_STRATEGIES: readonly DisableThinkingStrategy[] = [ - false, "vllm", "deepseek", "dashscope", "openai", "anthropic", "kimi", "gemini", + false, "vllm", "deepseek", "deepseek-v4", "dashscope", "openai", "anthropic", "kimi", "gemini", ] as const; /** Check if a value is a valid DisableThinkingStrategy. */ @@ -52,7 +55,7 @@ export function normalizeDisableThinking(raw: boolean | string | undefined): Dis if (isValidDisableThinkingStrategy(raw)) return raw; console.warn( `[memory-tdai] Unknown disableThinking strategy "${raw}", ` + - `valid values: false, true, "vllm", "deepseek", "dashscope", "openai", "anthropic", "kimi", "gemini". ` + + `valid values: false, true, "vllm", "deepseek", "deepseek-v4", "dashscope", "openai", "anthropic", "kimi", "gemini". ` + `Thinking will NOT be disabled.`, ); return false; @@ -72,6 +75,13 @@ function applyDeepSeek(body: Record): void { body.enable_thinking = false; } +// DeepSeek V4 hybrid models (deepseek-v4-flash, …) ignore the top-level +// `enable_thinking` flag the V3 API used; they only suppress reasoning when +// given the object form `thinking: { type: "disabled" }` (verified 2026-06). +function applyDeepSeekV4(body: Record): void { + body.thinking = { type: "disabled" }; +} + function applyDashScope(body: Record): void { body.enable_thinking = false; } @@ -94,6 +104,7 @@ const STRATEGY_TRANSFORMERS: Record< > = { vllm: applyVllm, deepseek: applyDeepSeek, + "deepseek-v4": applyDeepSeekV4, dashscope: applyDashScope, openai: applyOpenAI, anthropic: applyAnthropic, diff --git a/src/utils/pipeline-factory.ts b/src/utils/pipeline-factory.ts index ce87801c..cc6023cb 100644 --- a/src/utils/pipeline-factory.ts +++ b/src/utils/pipeline-factory.ts @@ -70,6 +70,12 @@ export interface PipelineFactoryOptions { l1LlmRunner?: import("../core/types.js").LLMRunner; /** Host-neutral LLM runner for L2/L3 (tool-call enabled, enableTools=true). */ l2l3LlmRunner?: import("../core/types.js").LLMRunner; + /** + * Max time (ms) `pipeline.destroy()` waits to flush pending L1/L2/L3 work. + * Defaults to the manager's live-safe 2s. Seed raises it (no gateway_stop + * hook to race) so the final persona/scene flush isn't cut off. + */ + destroyTimeoutMs?: number; } // ============================ @@ -651,11 +657,18 @@ export function createL3Runner(opts: { /** * Create a MemoryPipelineManager with the standard config mapping. + * + * `extractionLimiter`, when provided, gates L1/L2/L3 runner execution. Pass the + * same instance to every manager (as the multi-tenant CoreRegistry does) to cap + * total concurrent extraction across cores; omit it for the unbounded + * single-core default. */ export function createPipelineManager( cfg: MemoryTdaiConfig, logger: PipelineLogger, sessionFilter?: SessionFilter, + extractionLimiter?: import("./async-semaphore.js").ConcurrencyLimiter, + destroyTimeoutMs?: number, ): MemoryPipelineManager { return new MemoryPipelineManager( { @@ -668,9 +681,11 @@ export function createPipelineManager( maxIntervalSeconds: cfg.pipeline.l2MaxIntervalSeconds, sessionActiveWindowHours: cfg.pipeline.sessionActiveWindowHours, }, + ...(destroyTimeoutMs !== undefined ? { destroyTimeoutMs } : {}), }, logger, sessionFilter ?? new SessionFilter([]), + extractionLimiter, ); } @@ -687,7 +702,7 @@ export function createPipelineManager( * and `createL3Runner()` from this module. */ export async function createPipeline(opts: PipelineFactoryOptions): Promise { - const { pluginDataDir, cfg, openclawConfig, logger, sessionFilter, l1LlmRunner } = opts; + const { pluginDataDir, cfg, openclawConfig, logger, sessionFilter, l1LlmRunner, destroyTimeoutMs } = opts; // Ensure data directories exist initDataDirectories(pluginDataDir); @@ -697,7 +712,7 @@ export async function createPipeline(opts: PipelineFactoryOptions): Promise new Promise((r) => setTimeout(r, ms)); + +/** Minimal pipeline config: warm-up on → the first conversation triggers L1. */ +function testPipelineConfig(): PipelineConfig { + return { + everyNConversations: 1, + enableWarmup: true, + l1: { idleTimeoutSeconds: 999 }, + l2: { + delayAfterL1Seconds: 999, + minIntervalSeconds: 999, + maxIntervalSeconds: 999, + sessionActiveWindowHours: 24, + }, + }; +} + +const msg = (content: string): CapturedMessage => ({ + role: "user", + content, + timestamp: new Date(0).toISOString(), +}); + +/** Poll until every manager's L1 queue is idle (or time out). */ +async function waitForL1Idle(managers: MemoryPipelineManager[], timeoutMs = 5000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (managers.every((m) => m.getQueueSizes().l1Idle)) return; + await delay(10); + } + throw new Error("timed out waiting for L1 queues to drain"); +} + +/** + * Build `count` managers sharing `limiter`, each with an L1 runner that records + * peak overlap into the shared `tracker`. Returns the managers + the tracker. + */ +function buildManagers(count: number, limiter: AsyncSemaphore | undefined, holdMs: number) { + const tracker = { active: 0, peak: 0, runs: 0 }; + const managers = Array.from({ length: count }, () => { + const mgr = new MemoryPipelineManager(testPipelineConfig(), undefined, undefined, limiter); + mgr.setL1Runner(async () => { + tracker.active++; + tracker.runs++; + tracker.peak = Math.max(tracker.peak, tracker.active); + await delay(holdMs); + tracker.active--; + return { processedCount: 1 }; + }); + return mgr; + }); + return { managers, tracker }; +} + +describe("MemoryPipelineManager — shared extraction limiter", () => { + it("caps concurrent L1 across managers at the shared limit", async () => { + const limiter = new AsyncSemaphore(1); + const { managers, tracker } = buildManagers(4, limiter, 25); + + // Fire one conversation per manager (each triggers L1 immediately via warm-up). + await Promise.all(managers.map((m, i) => m.notifyConversation(`acct-${i}`, [msg(`hi ${i}`)]))); + await waitForL1Idle(managers); + + expect(tracker.runs).toBe(4); // every account's L1 ran + expect(tracker.peak).toBe(1); // but never more than one at a time + + await Promise.all(managers.map((m) => m.destroy())); + }); + + it("allows up to the cap to run together (cap = 2)", async () => { + const limiter = new AsyncSemaphore(2); + const { managers, tracker } = buildManagers(6, limiter, 25); + + await Promise.all(managers.map((m, i) => m.notifyConversation(`acct-${i}`, [msg(`hi ${i}`)]))); + await waitForL1Idle(managers); + + expect(tracker.runs).toBe(6); + expect(tracker.peak).toBe(2); + + await Promise.all(managers.map((m) => m.destroy())); + }); + + it("without a shared limiter, managers fan out (peak > 1)", async () => { + // Control: omitting the limiter restores the unbounded per-core behaviour, + // proving the cap above is the limiter's doing and not incidental timing. + const { managers, tracker } = buildManagers(4, undefined, 25); + + await Promise.all(managers.map((m, i) => m.notifyConversation(`acct-${i}`, [msg(`hi ${i}`)]))); + await waitForL1Idle(managers); + + expect(tracker.runs).toBe(4); + expect(tracker.peak).toBeGreaterThan(1); + + await Promise.all(managers.map((m) => m.destroy())); + }); +}); diff --git a/src/utils/pipeline-manager.ts b/src/utils/pipeline-manager.ts index bccf2936..40ef3bc5 100644 --- a/src/utils/pipeline-manager.ts +++ b/src/utils/pipeline-manager.ts @@ -80,6 +80,8 @@ import type { PipelineSessionState } from "./checkpoint.js"; import { SessionFilter } from "./session-filter.js"; import { ManagedTimer } from "./managed-timer.js"; import { SerialQueue } from "./serial-queue.js"; +import { PASSTHROUGH_LIMITER } from "./async-semaphore.js"; +import type { ConcurrencyLimiter } from "./async-semaphore.js"; import { report } from "../core/report/reporter.js"; import type { Logger } from "../core/types.js"; @@ -138,6 +140,14 @@ export interface PipelineConfig { */ sessionActiveWindowHours: number; }; + + /** + * Max time (ms) `destroy()` waits to flush pending L1/L2/L3 work before + * persisting state and closing. Default: 2000 — tuned to stay under the live + * gateway's 3s gateway_stop hook. Offline callers (seed) raise it so the + * final L2/L3 flush isn't cut off mid-run. + */ + destroyTimeoutMs?: number; } /** Result returned by the L1 runner. */ @@ -233,6 +243,14 @@ export class MemoryPipelineManager { // Unified session filter (internal sessions + excludeAgents) private readonly sessionFilter: SessionFilter; + /** + * Gate around L1/L2/L3 runner execution. In multi-tenant mode every core + * shares ONE limiter so total concurrent extraction across all accounts is + * bounded (design §8.4 #5). Defaults to a passthrough (no bound) — correct + * for single-tenant, where one core can't fan out. + */ + private readonly extractionLimiter: ConcurrencyLimiter; + // Lifecycle private destroyed = false; @@ -247,7 +265,12 @@ export class MemoryPipelineManager { /** Counter for GC scheduling. */ private notifyCounter = 0; - constructor(config: PipelineConfig, logger?: Logger, sessionFilter?: SessionFilter) { + constructor( + config: PipelineConfig, + logger?: Logger, + sessionFilter?: SessionFilter, + extractionLimiter?: ConcurrencyLimiter, + ) { this.l1IdleTimeoutMs = config.l1.idleTimeoutSeconds * 1000; this.everyNConversations = config.everyNConversations; this.enableWarmup = config.enableWarmup; @@ -255,8 +278,10 @@ export class MemoryPipelineManager { this.l2MinIntervalMs = config.l2.minIntervalSeconds * 1000; this.l2MaxIntervalMs = config.l2.maxIntervalSeconds * 1000; this.sessionActiveWindowMs = config.l2.sessionActiveWindowHours * 60 * 60 * 1000; + this.DESTROY_TIMEOUT_MS = config.destroyTimeoutMs ?? 2_000; this.logger = logger; this.sessionFilter = sessionFilter ?? new SessionFilter(); + this.extractionLimiter = extractionLimiter ?? PASSTHROUGH_LIMITER; this.logger?.debug?.( `${TAG} Initialized: everyNConversations=${config.everyNConversations}, ` + @@ -500,10 +525,11 @@ export class MemoryPipelineManager { /** * Maximum time (ms) to wait for pipeline flush during destroy. - * Must be shorter than the gateway_stop hook timeout (3 s) to leave - * headroom for VectorStore / EmbeddingService cleanup that runs after. + * Live default (2 s) must stay shorter than the gateway_stop hook timeout + * (3 s) to leave headroom for VectorStore / EmbeddingService cleanup that + * runs after. Offline callers (seed) override via `config.destroyTimeoutMs`. */ - private readonly DESTROY_TIMEOUT_MS = 2_000; + private readonly DESTROY_TIMEOUT_MS: number; /** * Graceful shutdown with timeout protection: @@ -690,12 +716,17 @@ export class MemoryPipelineManager { return; } + const l1Runner = this.l1Runner; try { - await this.l1Runner({ - sessionKey, - msg: buffer, - bg_msg: [], // reserved for future use - }); + // Gate L1 extraction (LLM-bearing) through the shared limiter so that, in + // multi-tenant mode, concurrent L1 runs across all accounts stay capped. + await this.extractionLimiter.run(() => + l1Runner({ + sessionKey, + msg: buffer, + bg_msg: [], // reserved for future use + }), + ); this.logger?.debug?.( `${TAG} [${sessionKey}] L1 complete: processed ${buffer.length} messages`, @@ -882,10 +913,12 @@ export class MemoryPipelineManager { ); const cursor = state.last_extraction_updated_time || undefined; + const l2Runner = this.l2Runner; let result: L2RunnerResult | void; try { - result = await this.l2Runner(sessionKey, cursor); + // Shared-limiter gated (see runL1) — caps L2 fan-out across accounts. + result = await this.extractionLimiter.run(() => l2Runner(sessionKey, cursor)); } catch (err) { this.logger?.error( `${TAG} [${sessionKey}] L2 runner failed: ${err instanceof Error ? err.stack ?? err.message : String(err)}`, @@ -988,9 +1021,11 @@ export class MemoryPipelineManager { return; } + const l3Runner = this.l3Runner; this.logger?.debug?.(`${TAG} L3 running`); try { - await this.l3Runner(); + // Shared-limiter gated (see runL1) — caps L3 fan-out across accounts. + await this.extractionLimiter.run(() => l3Runner()); this.logger?.debug?.(`${TAG} L3 complete`); } catch (err) { this.logger?.error( diff --git a/tdai-gateway.yaml b/tdai-gateway.yaml new file mode 100644 index 00000000..b6005209 --- /dev/null +++ b/tdai-gateway.yaml @@ -0,0 +1,27 @@ +# TDAI Gateway config — picked up automatically from the project root (CWD) +# by resolveConfigPath() in src/gateway/config.ts. +# +# Scope: this file ONLY carries what env vars cannot express — the embedding +# block (the gateway has no TDAI_EMBEDDING_* vars). server / data / llm keep +# coming from .env (TDAI_DATA_DIR, TDAI_MULTI_TENANT, TDAI_LLM_*), and env +# always wins over yaml, so there is nothing to keep in sync here. +# +# ${VAR} placeholders are expanded from process.env at load time. Because the +# gateway is launched with `node --env-file=.env ...`, ${DASHSCOPE_API_KEY} +# resolves to the value in .env — the key is never written into this file +# (which, unlike .env, is NOT gitignored). + +memory: + embedding: + enabled: true + # Any non-"none"/"local"/"qclaw" provider is treated as OpenAI-compatible + # and hits ${baseUrl}/embeddings. Aliyun Bailian / DashScope exposes exactly + # that shape under its compatible-mode endpoint. + provider: dashscope + baseUrl: https://dashscope.aliyuncs.com/compatible-mode/v1 + apiKey: ${DASHSCOPE_API_KEY} + model: text-embedding-v3 + dimensions: 1024 # text-embedding-v3 default output dim + sendDimensions: true # compat mode accepts the dimensions param (verified) + recall: + strategy: hybrid # vector + BM25; falls back to keyword if embedding is down