Skip to content

feat(skill-memory): per-skill cross-session recall + historian auto-extraction#181

Draft
iceteaSA wants to merge 5 commits into
cortexkit:masterfrom
iceteaSA:skill-memory-pr
Draft

feat(skill-memory): per-skill cross-session recall + historian auto-extraction#181
iceteaSA wants to merge 5 commits into
cortexkit:masterfrom
iceteaSA:skill-memory-pr

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds skill-memory — per-skill "motor memory" that gives a skill cross-session recall of its own hard-won lessons (gotchas, discoveries, fixes, workflow steps). When a skill's SKILL.md declares skill-memory: { enabled: true }, accumulated notes for that skill surface automatically in a <skill-memory> block appended to the skill tool's result on every load — and, with the historian extension, the historian writes those notes automatically during compaction (no agent action required).

It's fully opt-in per skill and cache-safe by construction: the block rides the tool-result tail (conversation), never the cached system/m[0] prefix, so it can't bust the prompt cache.

How it works

  • Transparent recall — three-hook augmentation: tool.definition advertises an intent param; tool.execute.before stashes the intent (bounded TTL); the after-hook parses the skill's Base directory, reads its SKILL.md frontmatter, and formats the recall block. Lands in the tool RESULT (cache-safe).
  • Write pathsctx_skill_note (agent-authored) and the historian (auto-extracted). Both dedup on a normalized hash.
  • Intent-scoped ranking — a recall cascade: model-matched embeddings → cosine blend over intent_embedding + delta_embedding (relevance/recency/hit weights tunable per skill via ranking_* frontmatter); FTS5 fallback over a content-linked skill_memory_fts vtable; flat recency×hit fallback.
  • Historian auto-extraction — the historian sees TC: skill(<name>) markers in its chunk and emits a <skill_observations> block; both the OpenCode and Pi runners promote those post-commit as global notes (project_identity='*', source_type='historian'), recallable from any project.

Review units (4 commits)

The branch is organized into four coherent, reviewable phases:

  1. P1 — transparent per-skill recall: skill_memory table migration, the three-hook augmentation, flat recall + storage, ctx_skill_note/ctx_skill_recall tools, opt-in distill-skill-memory dreamer task, TUI/ctx-status stats, docs.
  2. P2 — embeddings + intent-scoped recall: delta_embedding + recall_count columns + skill_memory_fts FTS5 vtable; embed-on-write; the cosine/FTS recall cascade; a programmatic (no-LLM) reembed pre-step.
  3. P3a — historian-extraction foundation: the TC: skill(<name>) marker keystone; origin_project + source_type columns; global-tier notes unified under project_identity='*' (collision-merge); partitionKey routing.
  4. P3b — historian auto-extraction pipeline: prompt emits <skill_observations>, parser, validated-result threading, both runners promote via the shared promoteSkillObservations helper, plus an initializeDatabase self-heal net (re-creates skill_memory + ensureColumn so an upgraded DB recovers even if a migration row is lost).

Schema / migrations

Three migrations (numbered after the current master ceiling): skill_memory table, then delta_embedding/recall_count/FTS, then origin_project/source_type/* unification. LATEST_SUPPORTED_VERSION bumped in lockstep (the schema-version-fence test enforces it). Migration bodies are ensureColumn/IF NOT EXISTS idempotent.

Testing

Full plugin + Pi suites pass (one unrelated pre-existing full-suite ordering flake in tui-config.test.ts that passes in isolation), tsc clean both packages, lint clean, build produces all bundles. Dedicated coverage for the migrations (coexistence + fence), recall rungs, the tools, FTS triggers/backfill, the '*' collision-merge, and the historian promotion path on both runners. Verified working live: the historian auto-extraction writes genuine source_type='historian' notes, embeddings populate, and the read-side recall_count increments on surfacing.

Notes for reviewers

  • Migration version numbers are placeholders relative to this fork's base — happy to renumber to whatever slots are free at merge time.
  • The four commits are independently meaningful; if you'd prefer this as stacked PRs (P1 / P2 / historian), I can split it.

View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


Summary by cubic

Adds per-skill cross-session memory with opt-in via SKILL.md and a cache-safe <skill-memory> block appended to skill tool results. Also adds intent-scoped recall with embeddings, historian auto-extraction to global notes, and a fix to prevent opencode 1.17 from breaking plugin load.

  • New Features

    • Per-skill recall: ctx_skill_note writes notes; ctx_skill_recall fetches without reloading. The skill tool advertises an optional intent; a before-hook stashes it and an after-hook formats recall into the tool RESULT (cache-safe).
    • Intent-scoped ranking: cosine blend over intent_embedding + delta_embedding with FTS5 fallback; flat recency×hit fallback; read-side recall_count. Programmatic re-embed backfills stale vectors; opt-in dreamer task distill-skill-memory runs a read-only health pass.
    • Historian pipeline: prompt emits <skill_observations>, parser extracts them, and both runners promote them as global '*' notes (source_type='historian') when not discarded. TC: skill(<name>) markers anchor extraction.
    • Schema/migrations: add skill_memory table + FTS vtable, delta_embedding/recall_count, origin_project/source_type, and unify global notes under '*'. LATEST_SUPPORTED_VERSION set to 44 with init-time self-heal.
    • UX: Status/TUI show per-project skill-memory totals and pinned counts. Config/docs include opt-in distill-skill-memory.
  • Bug Fixes

    • Move buildHiddenAgentConfig to a non-entry module so only the default export remains; stops opencode 1.17 from invoking it as a plugin factory and breaking hook/tool registration.

Written for commit 952e736. Summary will update on new commits.

Review in cubic

Tehan added 5 commits June 23, 2026 14:49
Per-skill "motor memory": when a skill's SKILL.md declares
`skill-memory: { enabled: true }`, accumulated gotchas/discoveries/fixes/
workflow-steps surface in a <skill-memory> block appended to the skill
tool's RESULT on every load (cache-safe — rides the tool-result tail).
Agents write back via ctx_skill_note; ctx_skill_recall is the explicit
companion to the transparent after-hook.

- migration: skill_memory table (per-skill; tier project/global; UNIQUE on
  skill_id/tier/project_identity/normalized_hash) + lookup indexes.
- three-hook augmentation: tool.definition advertises an `intent` param;
  tool.execute.before stashes intent (bounded TTL); after-hook parses the
  skill's Base directory, reads SKILL.md frontmatter, formats the block.
- flat recency×hit recall + storage layer; ctx_skill_note / ctx_skill_recall.
- opt-in distill-skill-memory dreamer task; agent-prompt guidance; TUI/ctx-status stats.
- docs: ARCHITECTURE / STRUCTURE / CONFIGURATION / README.
Upgrade recall from flat recency×hit to a multi-rung cascade: intent +
model-matched embeddings → cosine blend across intent_embedding +
delta_embedding (relevance/recency/hit weights tunable per skill via
ranking_* frontmatter); intent + no model match → FTS5 fallback over the
content-linked skill_memory_fts vtable; empty → flat fallback.

- migration: delta_embedding + recall_count columns + skill_memory_fts FTS5 vtable.
- embed-on-write in insertSkillMemoryNote; delta-only semantic dedup.
- programmatic, no-LLM reembed pre-step for the distill-skill-memory dreamer task.
- read-side recall_count (distinct from write-side hit_count).
- canonical vector serde + dedup/ranking/FTS query helpers.
…ication (P3a)

Foundation for the historian to auto-capture skill notes cross-project.

- surface the skill name in the historian chunk as a `TC: skill(<name>)`
  marker (the keystone — the tool input name was previously dropped).
- migration: origin_project + source_type columns; unify global-tier notes
  under project_identity='*' (collision-merge) so a global note is one row
  recallable from any repo.
- partitionKey helper routes global write/recall/reembed/stats through '*';
  recall reads global-tier from '*' (cross-project); reembed sweeps '*'.
Close the loop so the historian writes skill notes during compaction
without an agent volunteering ctx_skill_note.

- historian prompt emits a <skill_observations> block; parser extracts it;
  threaded through the validated historian result.
- both runners (OpenCode + Pi) promote skill observations post-commit via
  the shared promoteSkillObservations helper, gated by
  promotionActive && !discardedLast, writing global '*' notes with
  source_type='historian'.
- self-heal net: initializeDatabase re-creates skill_memory + ensureColumn
  so an upgraded DB recovers even if a migration row is lost.
…encode 1.17 invokes every entry export as a plugin factory

opencode 1.17 calls every exported function in a plugin's entry module
(index.ts) as its own plugin factory: fn(ctx). The helper buildHiddenAgentConfig
was exported from index.ts (introduced upstream in 29a49cb 'D19 resilience
safe parts'), so opencode invoked it as buildHiddenAgentConfig(ctx) — passing
the plugin context as `prompt` and undefined as `allowedTools` — which threw
'undefined is not an object (evaluating allowedTools)' during plugin load.
Result: no hooks/tools registered, all ctx_* tools dead, empty MC runtime log,
migration churn each boot.

The entry module must export ONLY default (the plugin factory). Move the helper
to a sibling module (hidden-agent-config.ts) where it can still be exported and
unit-tested without being mis-invoked as a factory. index-refresh.test.ts now
imports it from there.

Verified: dist export keys = ['default']; full plugin suite green (the lone
tui-config ordering flake is pre-existing on master); tsc + lint clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant