diff --git a/.claude/skills/noetic-agent-builder/references/api-reference.md b/.claude/skills/noetic-agent-builder/references/api-reference.md index f1621097..5b4a1bf3 100644 --- a/.claude/skills/noetic-agent-builder/references/api-reference.md +++ b/.claude/skills/noetic-agent-builder/references/api-reference.md @@ -439,6 +439,7 @@ interface ProjectionPolicy { overflow: 'truncate' | 'summarize' | 'sliding_window'; overflowModel?: string; windowSize?: number; + compactAt?: number; // folded-history threshold that arms compaction (default 80% of budget − reserve) } // Fallback when neither step nor harness configures one: @@ -459,7 +460,7 @@ interface StepCallModel { } ``` -- A single allocator (`allocateBudgets`) splits the recall budget: each layer's `budget.min` is satisfied first, then ~60% of the remainder funds a proportional pool across layers (by headroom `max − min`; `'auto'` and **omitted** budgets have infinite headroom and split the pool after finite layers take their share — the pool is fully conserved) and ~40% is reserved for conversation history. A layer never exceeds its `max`. NaN inputs throw `NoeticConfigError` (`INVALID_BUDGET_INPUT`); `Infinity` = uncapped. +- A single allocator (`allocateBudgets`) splits the recall budget deterministically: each layer's `budget.min` is satisfied first from the full available window, then only the discretionary remainder is rationed as `min(available × 0.25, available − totalMin)` across headroom (`cap − min`). `'auto'` and **omitted** budgets use a fixed 2000-token cap; explicit `Infinity` stays uncapped. A layer never exceeds its cap. NaN inputs throw `NoeticConfigError` (`INVALID_BUDGET_INPUT`); `Infinity` = uncapped. - `assembleView` then holds the final view to a hard cap (`tokenBudget − responseReserve`) and lays it out in bands: ``` @@ -469,6 +470,8 @@ interface StepCallModel { Both layer bands arrive slot-ascending. The budget is claimed in this order: system items (never dropped), anchor output, live output, the tail, then the supersedes — with history taking whatever is left and keeping the most recent turns. Within a layer band each non-fitting item is dropped **individually** (later-slot items that still fit are kept); history is trimmed as a contiguous recent window and orphan tool calls are stripped at the boundary. Supersedes are never dropped — each corrects a pinned block already in the view, so dropping one would leave the model reading content known to be stale. History absorbs the cost instead. - `forceAtomicRecall: true` makes every layer atomic regardless of `recallMode`. +**Compaction** is the explicit alternative to the assembler's silent front-drop: append a `CompactionItem` (`'noetic:compaction'`) to the log and the folded view collapses the replaced prefix to a `` developer message while the raw log (checkpoints, forks) keeps everything. Helpers: `historyPressure(items, policy)` (measures the folded view against `compactAt`), `compactHistory({ log, keepRecent, summarize })` / `createCompaction(...)` (build the record — the caller supplies the summary and appends it via `compactionAsItem`), `foldCompactions(items)` (project the model view; run before `assembleView`), `hasCompaction(items)`. + ### Prompt-cache anchoring (`placement`) A prompt cache matches on a prefix, so the first changed token invalidates everything after it. Putting volatile layer output ahead of a large stable history re-bills the whole window every turn. The bands fix that: stable output sits in the **anchor** band ahead of history where the cache can hold it, volatile output sits in the **live** band after history where re-rendering costs almost nothing. diff --git a/docs/plans/2026-08-12-001-openrouter-fork-upstreaming-plan.md b/docs/plans/2026-08-12-001-openrouter-fork-upstreaming-plan.md new file mode 100644 index 00000000..dbff2ec2 --- /dev/null +++ b/docs/plans/2026-08-12-001-openrouter-fork-upstreaming-plan.md @@ -0,0 +1,162 @@ +# OpenRouter fork upstreaming plan + +Status: active +Owner: Pi orchestrator in Herdr workspace `w10` +Upstream base: `origin/main` at `8a6665ba` +Source: `fork/port/openrouter-fixes` at `6df9b785` (old base `ead54108`) + +## Objective + +Semantically port the non-Standard-Schema improvements from `fork/port/openrouter-fixes` onto current `mattapperson/noetic` architecture as focused, independently reviewable pull requests. Historical commits are provenance, not patches: current source, specs, package boundaries, Standard Schema support from #67, and naming cutover from #68 are authoritative. + +The program is complete only when every retained ledger item is either: + +1. merged upstream; +2. an open, green, review-ready upstream PR with no unresolved feedback and explicit dependency state; or +3. dropped with concrete supersession or current-main evidence. + +## Current-main overlap audit + +Audit performed after fetching both remotes. `HEAD` and `origin/main` were both `8a6665ba`; source tip was `6df9b785`. Four independent read-only Pi audits compared the ledger against current symbols, specs, #67, #68, and PR #69. + +### Naming and architecture adaptations + +Historical names must not return: + +| Historical source | Current main | +|---|---| +| `llm` / `step.llm` | `callModel` | +| `provide` | `withContext` | +| `every` | `schedule` | +| `branch` | `conditional` | +| `fork` | `inParallel` | +| `run` | `runCode` | +| memory / working-memory symbols | context / `scratchpad` / current layer names | +| `patterns/dynamic-workflow` | `builders/dynamic-workflow` | + +`packages/context` remains dependent only on `packages/types`. Core must not import platform, OpenUI, or sub-harness adapter packages. Workflow ports must not recreate the historical builder-to-adapter Sentrux violation. + +### Already upstream, superseded, or intentionally dropped + +| Source/change | Disposition | Evidence | +|---|---|---| +| Standard Schema work | Dropped | Merged in #67 (`1751b6a9`); current OpenRouter adapter uses the shared Standard Schema validation path. | +| Tool-argument validation from `0e63664d` | Validation half dropped | Equivalent validation is in #67. Only resolved-tool lookup/memoization remains eligible under item 10. | +| `a792e90a` baseline cleanup | Dropped as a commit | Its old `fork` test hunk is superseded by #68 `inParallel`/`frameworkCast`; eval provider setup is now `callModelDefaults` + `resolveEnvLlm`; the lifecycle type widening is unrelated and has no demonstrated current failure. | +| Historical `UPSTREAMING.md` | Not ported | Used only for intent/provenance; durable facts belong in owning PR bodies, specs, docs, and tests. | +| Branch-specific drift tests | Not ported | Current behavior receives focused regression tests in each owning PR. | +| Unsupported `~60x` checkpoint claim | Omitted | Item 13 may make complexity claims; quantified claims require a committed reproducible benchmark. | +| Public bundled pattern layout from item 19 | Not ported without design approval | #68 removed `packages/core/src/patterns`; `specs/13-patterns.md` normatively says core ships no bundled agent patterns. | +| PR #69 behavior | Separate | [#69](https://github.com/mattapperson/noetic/pull/69) owns EventBroadcaster watermark trimming and latest-only generator progress. Item 11 must not touch those behaviors/files. | + +### Item-by-item audit and PR ledger + +States: `queued`, `implementing`, `review`, `open`, `merged`, `dropped`, `design-review`, `blocked`. + +| # | Planned PR | Current-main audit | Dependencies | State | Branch / PR / evidence | +|---:|---|---|---|---|---| +| 1 | File-storage encoding, async writes, legacy reads | Absent: `packages/platform-node/src/file-storage.ts` still uses lossy single-phase encoding and sync filesystem operations; no legacy fallback. | none | queued | source `b9905ac5` + `653b3986` | +| 2 | Durable queue and subprocess IPC ordering | Absent: queue `clear()` resets sequence, ack scans storage, IPC dispatch is unsequenced, error frames are uncorrelated, subprocess identity uses spawned `ps`. Keep as one platform PR unless review shows queue vs IPC is materially clearer split. | soft after 1 to reduce platform conflicts | queued | remainder of `b9905ac5` | +| 3 | Sub-harness session/turn reliability | Absent: session cache stores sessions rather than in-flight promises; no turn idle watchdog, error-finish failure, or reasoning item retention. | none | queued | `0fd1b9ef` | +| 4 | Eval dirty-write guard and bounded case concurrency | Absent: no write guard/run pool, suite cases remain serial, dead `--budget` remains. | none | queued | first part of `aedf712f` | +| 5 | GEPA discovery, traversal, judge reuse | Absent: scope discovery, stable tool paths, composite traversal, and harness reuse gaps remain. Port traversal with current step kinds only. | 4 merged | blocked | remainder of `aedf712f` | +| 6 | OpenUI transport/state hardening | Absent: parser lacks prose-safe statement detection; surface has one global sequence watermark and unrestricted set events; state reads are not thread-keyed. | none | queued | `b095d1f5` | +| 7 | Result-aware configurable doom-loop protection | Absent: no round fingerprint or configurable identical-round threshold. | none | queued | `3fc95caf` + `33f44721` | +| 8 | Deferred observation distillation | Absent: `observations()` awaits its observer in the append path and carries long append timeouts. | none | queued | relevant `fb22ac35`; adapt observation naming | +| 9 | Opt-in filesystem/file-reference LLM scoring | Absent: `filesystem()` still defaults to Haiku scoring. Explicit breaking default: heuristic unless a model is configured. | none | queued | relevant `fb22ac35` | +| 10 | Memoized unified tools and SDK conversion | Partial: validation is upstream via #67, but tool collection/conversion is repeated and resolution remains linear. | none | queued | relevant `fb22ac35`; validation excluded | +| 11 | Channel reaping, loop snapshot efficiency, deterministic park jitter, inline dispatch | Absent in channel store/control/subprocess paths. Must not duplicate #69 EventBroadcaster or latest-yield changes. | coordinate with #69 only | queued | `27ddda8e` | +| 12 | Session-owned logs and warm layer hydration | Absent: sessions copy `accumulatedItems`; no session-owned log, truncation rollback, or scope-keyed warm hydration. | none | queued | `fc89dbab` + relevant `796f7ec2` fixes | +| 13 | Delta checkpoint batches, rollback-safe stitching, ledger continuity | Absent: checkpoints still embed the full item log; no batch stitching/truncation repair. | 12 merged | blocked | remaining `fc89dbab` + `796f7ec2`; benchmark before numeric claims | +| 14 | Declarative workflow runtime hardening | Absent: tool nodes bypass the shared execution path; no exact route mode, duplicate-id protection, dynamic path cache, size cap, or hydration-error revision loop. | none; blocks 20 | queued | `4cb90e35`; Sentrux-safe seam; regenerate both schemas when Zod changes | +| 15 | Compaction records, projection, pressure helpers | Absent: no compaction item/schema/projector helpers. | none | queued | `3d65b3e2` + `135ac3fd` | +| 16 | Fold compactions into model calls; emit context pressure | Absent. Folding must occur before system/history partitioning and pressure latch only on emission. | 15 merged | blocked | `1761e6b7` + `b83bdd91` | +| 17 | Deterministic minimum-first allocator | Absent: allocator remains 60/40 with unbounded `auto` and a `historyBudget` result. Explicit breaking/RFC review for the 2,000-token cap and interface removal. | 16 merged | blocked | `d8fb4f34` + `59497c48` | +| 18 | Assignable `context()` output at attachment seams | Absent: seams still use invariant `ContextConfig | ContextLayer[]`; port structural `ContextInput` to `withContext` and current names. | none | queued | `efc2b689` | +| 19 | Native multi-agent patterns | Primitives remain expressive, but a literal core pattern module conflicts with #68 and `specs/13-patterns.md`. Start with a proposal choosing examples/docs, a separate package, or a deliberate policy reversal. Standard Schema and current names are mandatory. | primitives stable; design approval | design-review | source `4cccf95d` | +| 20 | Declarative multi-agent nodes/hydrator seam | Absent and intentionally blocked. JSON nodes need an accepted item 19 registry/API and item 14's hardened hydrator seam. | 14 + accepted 19 merged | blocked | `77cb290d` + `928d63c3`; regenerate both schema artifacts and inspector glyphs | + +## Dependency graph + +```text +origin/main @ 8a6665ba +├─ 1 ──(soft conflict reduction)──► 2 +├─ 3 +├─ 4 ─────────────────────────────► 5 +├─ 6 +├─ 7 +├─ 8 +├─ 9 +├─ 10 +├─ 11 (parallel to #69; disjoint behavior) +├─ 12 ────────────────────────────► 13 +├─ 14 ───────────────┐ +├─ 15 ───────────────► 16 ────────► 17 +├─ 18 │ +└─ 19 design review ─┴────────────► 20 +``` + +Dependent tranches `12→13`, `15→16→17`, and `14 + 19→20` sequence through merged upstream `main`; they are not maintained as long stacked diffs. + +## First independent implementation wave + +Delegate in parallel to isolated general-profile Pi workers, each based semantically on current main: + +1. item 1 — platform-node file storage; +2. item 3 — sub-harness reliability; +3. item 4 — eval guard/concurrency; and +4. item 6 — OpenUI hardening. + +These touch different primary packages and have no hard dependencies. Each worker implements and verifies only its bounded item. The orchestrator inspects every diff, runs a separate clean-context review, applies/fixes confirmed findings, performs fresh verification, creates signed DCO commits, pushes to `fork`, opens an upstream PR, and attaches monitoring. + +The next independent wave is selected from items 7, 8, 9, 10, 11, 12, 14, 15, and 18 based on review/CI bandwidth. Item 5 waits for 4; item 2 follows 1 to avoid avoidable platform-node conflicts. + +## Per-PR acceptance criteria + +Every retained implementation PR must: + +- be based on current merged `origin/main`, with dependent tranches rebased only after prerequisites merge; +- contain a semantic port, not a blind cherry-pick; +- use current public names and package boundaries; +- include focused regression tests and required spec/docs/skill changes; +- regenerate both workflow schema artifacts in the same commit when the workflow Zod schema changes; +- pass affected package tests and typecheck, root lint, relevant root/full tests, `sentrux check .`, and applicable local `agent-ci` workflows; +- receive a separate review-profile Pi review for standards, spec, correctness, and port completeness, with confirmed findings fixed and reverified; +- use Conventional Commit PR titles and signed, DCO-signed-off commits; +- explain what/why, source provenance, behavior or breaking changes, and test evidence; +- make no quantified performance claim without a reproducible benchmark; +- remain monitored for CI, merge conflicts, and review feedback; agent-authored GitHub messages begin with `Agent:`; and +- be recorded below with branch, URL, CI/review state, and merge SHA or concrete drop evidence. + +## Live PR ledger + +| Item | Branch | PR | CI | Reviews | Merge SHA / dependency | Last update | +|---:|---|---|---|---|---|---| +| 1 | `lukeparke/openrouter-file-storage` | [#70](https://github.com/mattapperson/noetic/pull/70) | CI/DCO/structural pass; compat blocked by missing upstream `OPENROUTER_API_KEY` | no feedback | review-ready; external compat dependency documented | 2026-08-12 opened | +| 2 | `lukeparke/openrouter-queue-ipc` | [#81](https://github.com/mattapperson/noetic/pull/81) | pending | no feedback | independent of #70 after split | 2026-08-12 opened | +| 3 | `lukeparke/openrouter-sub-harness-reliability` | [#71](https://github.com/mattapperson/noetic/pull/71) | pending | no feedback | independent | 2026-08-12 opened | +| 4 | `lukeparke/openrouter-eval-safety` | [#72](https://github.com/mattapperson/noetic/pull/72) | pending | no feedback | blocks 5 | 2026-08-12 opened | +| 5 | — | — | — | — | blocked by 4 merge | 2026-08-12 audit complete | +| 6 | `lukeparke/openrouter-openui-hardening` | [#73](https://github.com/mattapperson/noetic/pull/73) | pending | no feedback | independent | 2026-08-12 opened | +| 7 | `lukeparke/openrouter-doom-loop` | [#74](https://github.com/mattapperson/noetic/pull/74) | pending | no feedback | independent | 2026-08-12 opened | +| 8 | `lukeparke/openrouter-deferred-observations` | [#75](https://github.com/mattapperson/noetic/pull/75) | pending | no feedback | independent | 2026-08-12 opened | +| 9 | `lukeparke/openrouter-filesystem-scoring` | [#76](https://github.com/mattapperson/noetic/pull/76) | pending | no feedback | breaking default | 2026-08-12 opened | +| 10 | — | — | — | — | deferred after review found function-tool correctness and mutation-contract gaps; validation dropped via #67 | 2026-08-12 implementation rejected pending redesign | +| 11 | `lukeparke/openrouter-runtime-efficiency` | [#78](https://github.com/mattapperson/noetic/pull/78) | pending | no feedback | excludes [#69](https://github.com/mattapperson/noetic/pull/69) behavior | 2026-08-12 opened | +| 12 | `lukeparke/openrouter-session-log` | [#79](https://github.com/mattapperson/noetic/pull/79) | pending | no feedback | blocks 13 | 2026-08-12 opened | +| 13 | `lukeparke/openrouter-delta-checkpoints` | [#86](https://github.com/mattapperson/noetic/pull/86) | pending | clean-context review recorded namespace/contract boundaries | depends on #79; rebase through main after merge | 2026-08-12 opened | +| 14 | `lukeparke/openrouter-workflow-hardening` | [#84](https://github.com/mattapperson/noetic/pull/84) | pending | clean-context review addressed | blocks 20 | 2026-08-12 opened | +| 15 | `lukeparke/openrouter-compaction-primitives` | [#80](https://github.com/mattapperson/noetic/pull/80) | pending | no feedback | blocks 16 | 2026-08-12 opened | +| 16 | `lukeparke/openrouter-compaction-runtime` | [#85](https://github.com/mattapperson/noetic/pull/85) | pending | clean-context review addressed | depends on #80; rebase through main after merge | 2026-08-12 opened | +| 17 | — | — | — | — | blocked by 16; RFC/breaking | 2026-08-12 audit complete | +| 18 | `lukeparke/openrouter-context-input` | [#77](https://github.com/mattapperson/noetic/pull/77) | pending | no feedback | independent | 2026-08-12 opened | +| 19 | — | — | — | — | proposal committed at `docs/plans/2026-08-12-002-multi-agent-patterns-proposal.md`; recommends examples/docs and rejects core policy reversal | 2026-08-12 design proposal complete | +| 20 | — | — | — | — | deferred/redesign required: examples cannot be declarative hydration targets; requires accepted package/registry plus item 14 | 2026-08-12 explicitly blocked | + +## Program risks + +- The plan is intentionally broad; concurrency is bounded by review and CI capacity rather than worker count. +- Public behavior changes in items 9 and 17 require explicit reviewer attention and release notes. +- Items 12–17 modify state/persistence semantics; compatibility and rollback tests are required, not just happy-path coverage. +- Item 14 must use or extract an allowed shared tool-dispatch seam instead of importing upward across Sentrux layers. +- Item 19 may be rejected as a core API on policy grounds. That is an acceptable documented drop; item 20 is then deferred or redesigned rather than forced through. diff --git a/docs/plans/2026-08-12-002-multi-agent-patterns-proposal.md b/docs/plans/2026-08-12-002-multi-agent-patterns-proposal.md new file mode 100644 index 00000000..d1d418c7 --- /dev/null +++ b/docs/plans/2026-08-12-002-multi-agent-patterns-proposal.md @@ -0,0 +1,98 @@ +# Design Proposal: Multi-Agent Patterns (Item 19) + +- **Status:** proposal — design review, no runtime code +- **Source:** fork commit `4cccf95d` (`feat(core): native multi-agent patterns — defineAgent, asTool, handoff, quorum, teammate`, ported from OpenRouter `fa8a9d24`) +- **Plan item:** 19 in `2026-08-12-001-openrouter-fork-upstreaming-plan.md` +- **Constraints:** PR #68 removed `packages/core/src/patterns` and cut naming over; `specs/13-patterns.md` normatively states **core ships no bundled agent patterns**; PR #67 made Standard Schema v1 the schema contract. + +## Problem + +The fork shipped a bundled multi-agent module (`defineAgent`, `asTool`, `handoff`, `quorum`, `teammate`) inside `@noetic-tools/core`. Upstream policy (#68 + spec 13) forbids exactly this: a pattern baked into the framework fixes its termination rules, context boundaries, and feedback shape, and becomes an obstacle when users need to change one of them. Item 19 is therefore a design decision, not a port: where — if anywhere — does this functionality live? This document inventories what the source actually contains and recommends one of three resolutions: examples/docs, a separate package, or a deliberate policy reversal. + +## Source inventory (4cccf95d) + +773 lines, one file, zero new runtime machinery — every export compiles to existing primitives: + +| Export | Shape | Composed from | Reduces to existing runnable composition? | +|---|---|---|---| +| `defineAgent(def)` | The "agent noun": name/description/model/instructions/tools/context/until/output, compiled once to a reusable `StepLoop` | `loop` + `step.llm` + `any(noToolCalls, maxSteps(10))`, optional `spawn` | Yes — this is the ReAct recipe with a config bag | +| `asTool(agent, opts)` | Agent exposed as a `Tool` (`{task: string}` in, text out); sync via `harness.run`, or `detached: true` via `detachedSpawn` + optional result channel | `tool` + `spawn` + `detachedSpawn` + `channel` | Yes — `sync-delegate.ts` and `async-delegate.ts` already demonstrate both halves | +| `handoff(agents, opts)` | Routing swarm: per-agent `transfer_to_` tools, active agent swapped via `ctx.state` + `Lazy` model/instructions/tools, shared ItemLog | `loop` + `step.llm` + custom `until` + `prepareNext` | Partially — the `Lazy`-re-resolution trick over `ctx.state` is novel and worth documenting | +| `quorum(agents, {vote})` | Fan-out panel with `majority` / `first` / `all` / `judge` reduction | `fork('settle')` + merge fn; judge = `loop` of fan-out then one judge turn | Mostly — `parallel-research.ts` + `dynamic-judge-workflow.ts` cover fan-out and judge separately | +| `teammate(agent, task, toolCtx)` | Named background worker: detached thread, queue-mode inbox/outbox channels, `send`/`status`/`result` handle, failures surface on outbox | `detachedSpawn` + `spawn` + two `channel`s + inbox-parked `loop` | Partially — `async-delegate.ts` shows detached+inbox; the parked-on-inbox loop and status handle are the delta | + +The genuinely reusable deltas over today's examples are: (a) the handoff swarm's per-iteration `Lazy` swap, (b) the teammate's parked inbox loop with a status handle, (c) the judge-vote reducer. Everything else is recipe composition the spec already demonstrates. + +## Required adaptation if any code ships + +- **Naming cutover (#68):** `step.llm` → `callModel`; `fork({mode:'settle'})` → `inParallel(..., 'settle')`; `frameworkCast` only where genuinely needed. Loop `inbox`/`parkTimeout` and `detachedSpawn` survive unchanged. +- **Standard Schema (#67):** `AgentDef.output?: ZodType` → `StandardSchemaV1` (spec 02); `tool({input/output})` likewise. The `z.object({task})` internal input schemas can stay Zod (fast path) but the public types must accept any Standard Schema validator — and `defineAgent` must propagate `outputJsonSchema` for validation-only schemas or hit `MISSING_JSON_SCHEMA` at runtime. +- **No `packages/core/src/patterns` directory** under any option except a policy reversal; `.sentrux/rules.toml` and `specs/13-patterns.md` would both need edits in the same commit. + +### Naming / CLI collision check + +`teammate` collides with an established CLI-domain concept: the Noetic CLI (noetic-internal, spec 22, `packages/web/content/docs/code-agent-cli/`) already uses "teammate" for sub-agent presets spawned via its `agent` tool, with `send_message` / `check_agent` companions, and `spawn.mdx` documents per-teammate session logs. A core export named `teammate()` would shadow that vocabulary at a different layer with different semantics (channel-addressable worker vs. CLI sub-agent preset). If the shape ships, rename to `backgroundAgent()` / `worker()`; keep `teammate` reserved for the CLI domain. `handoff` and `quorum` are collision-free; `asTool` reads fine; `defineAgent` is the most builder-flavored name and the one spec 13's philosophy most objects to (see options). + +## Options + +### Option A — Examples + docs recipes (recommended) + +Port the four shapes as **runnable compositions**: `defineAgent` dissolved back into the ReAct recipe (it is one), plus new examples `handoff-swarm.ts`, `quorum-panel.ts`, `background-agent.ts` under `packages/core/examples/`, and matching rows + prose in `specs/13-patterns.md` ("Runnable Compositions" table). Web docs get a "Multi-agent patterns" guide under `packages/web/content/docs/framework/`. + +- **Acceptance criteria:** each example compiles and has a test; spec 13 table updated in the same commit; no new core exports; docs use post-#68 names and Standard Schema. +- **Rejection criteria:** the examples duplicate >100 lines of non-trivial shared logic across copies, or consumers demonstrably need stable cross-version behavior (semver) for the vote reducers/handle semantics — that's the signal for Option B. + +### Option B — Separate `@noetic-tools/patterns` package + +Ship the compositions as **copy-ready source with tests** (not re-exported builders), per spec 13's own "Future considerations". Depends on `core` only; new `[[layers]]` entry + boundary in `.sentrux/rules.toml`. + +- **Acceptance criteria:** Option A's rejection criteria trigger, OR two or more downstream consumers vendor the same recipe. Public surface is small and explicitly recipe-flavored: `reactAgent()`, `handoffSwarm()`, `panel()` (not `quorum`, to avoid implying consensus machinery it doesn't have), `backgroundAgent()`. No `defineAgent` noun — pass plain config objects. +- **Rejection criteria:** the package just re-exports thin wrappers users could paste; publish/maintenance cost exceeds the value; or it drifts into fixing termination/context policy per user (spec 13's original objection). + +### Option C — Policy reversal: restore bundled patterns in core + +Re-add `packages/core/src/patterns/agents.ts` behind `@public` exports, edit spec 13 and `.sentrux/rules.toml` in the same commit. + +- **Acceptance criteria (all required):** written rationale superseding spec 13's "patterns are compositions" argument; maintainer sign-off that core's API surface should grow by 5 exports + 3 types; a demonstrated consumer that cannot use examples or a side package. +- **Rejection criteria:** any of the above missing. On current evidence none are met — this option exists to make the rejection explicit and documented, which the plan (line 162) already sanctions as an acceptable outcome. + +## Recommendation + +**Option A now, Option B as a triggered evolution, reject C on the record.** The source commit itself is the proof: 773 lines composing only builders, with no interpreter/runtime change — which is spec 13's thesis. The novel deltas (Lazy handoff swap, parked-inbox teammate, judge reducer) are exactly the kind of thing runnable examples teach better than frozen builders. + +## Concrete API alternatives (if B ever triggers) + +```ts +// @noetic-tools/patterns — recipes, not builders; Standard Schema throughout +import { callModel, inParallel, loop, spawn, until } from '@noetic-tools/core'; +import type { StandardSchemaV1 } from '@standard-schema/spec'; + +reactAgent({ model, instructions, tools, maxSteps?, maxCost?, context? }); // = today's ReAct recipe +handoffSwarm(agents: AgentConfig[], { entry?, until?, maxIterations? }); // Lazy swap over ctx.state +panel(agents: AgentConfig[], { vote: 'majority' | 'first' | 'all' | { judge } }); +backgroundAgent(agent: AgentConfig, task, toolCtx): { send, status, result, outbox }; + +interface AgentConfig { + name: string; // tool-name-safe + description?: string; + model: Lazy; + instructions?: Lazy; + tools?: Tool[]; + context?: ContextConfig | ContextLayer[]; + output?: StandardSchemaV1; // + outputJsonSchema for validation-only schemas +} +``` + +Note what is absent vs. the fork: no `Agent` wrapper type, no memoized `agent.step`, no `defineAgent`. The recipe takes config and returns a `Step`; the user owns the noun. + +## Implications for item 20 (declarative multi-agent nodes) + +Item 20 (JSON workflow nodes for multi-agent shapes) is blocked on this decision because a JSON node must hydrate against a **registered, versioned API** — examples cannot be hydration targets. + +- **Option A (chosen):** item 20 is deferred or redesigned. A JSON `handoff`/`quorum` node has nothing stable to hydrate to; the honest resolution is to extend the JSON schema's existing node set only when/if Option B lands. This matches plan line 162 ("item 20 is then deferred or redesigned rather than forced through"). +- **Option B (if triggered later):** item 20 unblocks cleanly — add `handoff`/`panel`/`background-agent` node types whose hydrator resolves against `@noetic-tools/patterns` via the registry seam from item 14, regenerating both `noetic-workflow.schema.json` artifacts (`bun run gen:schema`) and inspector glyphs in the same commit. +- **Option C:** same as B but against core; rejected for the reasons above. + +## Decision requested + +Approve Option A and record the rejection of Option C, or direct Option B with the naming adjustments above. diff --git a/packages/context/README.md b/packages/context/README.md index 495807e9..7eaec5e2 100644 --- a/packages/context/README.md +++ b/packages/context/README.md @@ -10,7 +10,9 @@ provides: - The **`ContextLayer` contract** — the interface every layer implements. - The **lifecycle, budget, and projection machinery** that converges layer outputs into the assembled LLM context (`assembleView`, `allocateBudgets`, - layer state stores, scoping). + layer state stores, scoping), plus the **compaction helpers** + (`foldCompactions`, `historyPressure`, `createCompaction`, `compactHistory`) + for replacing an old history prefix with a logged summary. - The **built-in layers**: instructions, history, scratchpad, observations, temporal, filesystem, plan, task state, tool calls, and steering. diff --git a/packages/context/src/context/budget.ts b/packages/context/src/context/budget.ts index f96d6b63..003adccd 100644 --- a/packages/context/src/context/budget.ts +++ b/packages/context/src/context/budget.ts @@ -17,6 +17,22 @@ export const DEFAULT_PROJECTION: ProjectionPolicy = { overflow: 'sliding_window', }; +/** + * Share of the post-reserve budget offered to context layers as *discretionary* + * headroom each turn. Layer recall competes with the conversation for the + * window, and the conversation has to win by default: a layer set that can + * claim most of the budget starves history and pushes the assembler into + * trimming turns. `historyPressure` handles the history side. + * + * This share bounds the remainder pool only. Declared `min` values are a floor + * satisfied out of the full available window before this share applies — see + * `allocateBudgets`. + */ +const LAYER_POOL_SHARE = 0.25; + +/** Default cap for layers that declare `'auto'` (or omit) a budget. */ +const AUTO_LAYER_CAP = 2_000; + function extractMin(config: BudgetConfig | undefined): number { if (config && typeof config === 'object' && 'min' in config) { return config.min; @@ -24,41 +40,22 @@ function extractMin(config: BudgetConfig | undefined): number { return 0; } -function extractMax(config: BudgetConfig | undefined): number { - // 'auto' AND omitted budgets have infinite headroom (spec 11): a layer that - // declares no budget splits the proportional pool with the 'auto' layers - // rather than being starved at 0. +/** + * A layer's declared cap. `'auto'` and omitted budgets get a fixed default cap + * instead of infinite headroom: an allocation that scales with the model's + * context length makes the rendered block a different size on a different + * model — and a different size turn to turn as layers come and go — which is + * exactly what a prompt cache cannot tolerate. A layer that needs more than + * the default declares it. + */ +function extractCap(config: BudgetConfig | undefined): number { if (config === 'auto' || config === undefined) { - return Number.POSITIVE_INFINITY; + return AUTO_LAYER_CAP; } if (typeof config === 'number') { return config; } - if (typeof config === 'object' && 'min' in config) { - return config.max; - } - return 0; -} - -/** - * Single-priced finite shares: each finite layer's share is computed ONCE and - * that same value is subtracted from the pool before infinite-headroom layers - * split the remainder — so the pool is conserved (Σ shares === layerPool when - * any infinite layer exists). With no infinite layers, finite layers split the - * whole pool proportionally; with a mix, finite layers share half the pool, - * clamped to their headroom. - */ -function computeFiniteShares(headrooms: number[], layerPool: number): number[] { - const infiniteCount = headrooms.filter((h) => !Number.isFinite(h)).length; - const finiteTotal = headrooms.reduce((sum, h) => (Number.isFinite(h) ? sum + h : sum), 0); - const finitePool = infiniteCount === 0 ? layerPool : layerPool * 0.5; - return headrooms.map((h) => { - if (!Number.isFinite(h) || finiteTotal === 0) { - return 0; - } - const proportional = (h / finiteTotal) * finitePool; - return infiniteCount === 0 ? proportional : Math.min(h, proportional); - }); + return config.max; } const BUDGET_INPUT_FIELDS = [ @@ -70,9 +67,9 @@ const BUDGET_INPUT_FIELDS = [ /** * NaN in any budget input silently poisons every allocation downstream * (NaN fails the `available <= 0` guard, then every arithmetic op yields - * NaN). Reject it loudly at the boundary. `Infinity` stays allowed — it is a - * coherent "uncapped" budget — and fractional values are fine (allocations - * floor where it matters). + * NaN). Reject it loudly at the boundary. `Infinity` stays allowed on layer + * caps — it is a coherent "uncapped" declaration — and fractional values are + * fine (allocations floor where it matters). */ function assertBudgetInputs(opts: AllocateBudgetsOpts): void { for (const field of BUDGET_INPUT_FIELDS) { @@ -94,11 +91,27 @@ interface AllocateBudgetsOpts { } /** - * Split the recall budget across layers: minimums first, then 60% of the - * remainder as a proportional pool (by headroom `max − min`; `'auto'` and - * omitted budgets have infinite headroom and split the pool after finite - * layers take their share), 40% reserved for history. The pool is conserved — - * finite shares plus the infinite layers' split always sum to the pool. + * Split the layer budget across layers, deterministically: + * + * 1. **Minimums are satisfied first**, out of the *full* available window + * (`totalBudget − responseReserve − systemPromptTokens`) — not out of the + * discretionary pool. A layer that declares `{ min: 10_000, max: 12_000 }` + * because 10k is what it takes to render a coherent block gets its 10k on a + * 32k model. Mins are scaled down proportionally only when the mins alone + * overcommit the available window, and in that case nothing else is + * distributed. + * 2. The remainder is distributed proportionally to remaining headroom + * (`cap − min`), clamped to each cap. `'auto'`/omitted budgets use a fixed + * default cap — no infinite-headroom special cases, so the same layer set + * gets the same allocation on a 32k model and a 1M one. + * + * Only that *discretionary remainder* is rationed by `LAYER_POOL_SHARE`: the + * remainder pool is `min(available × LAYER_POOL_SHARE, available − totalMin)`, + * so opportunistic recall cannot claim most of the window, while a declared + * floor is never silently cut to a fraction of itself. History does not get a + * per-turn budget line here: it is append-only between explicit compactions, + * and what the assembler can actually fit is decided in `assembleView` against + * the same policy (see `historyPressure` for the compaction signal). * * Input contract: `totalBudget` / `systemPromptTokens` / `responseReserve` * MUST NOT be NaN (throws `NoeticConfigError` code `INVALID_BUDGET_INPUT`). @@ -111,7 +124,6 @@ export function allocateBudgets({ responseReserve, }: AllocateBudgetsOpts): { allocations: BudgetAllocation[]; - historyBudget: number; } { assertBudgetInputs({ layers, @@ -126,69 +138,59 @@ export function allocateBudgets({ layerId: l.id, allocated: 0, })), - historyBudget: 0, }; } - // Phase 1: satisfy minimums - let remaining = available; - const allocations: BudgetAllocation[] = []; - - for (const layer of layers) { - const min = extractMin(layer.budget); - allocations.push({ - layerId: layer.id, - allocated: min, - }); - remaining -= min; - } - - // Declared minimums overcommit the available budget: scale them down - // proportionally so the total fits `available` (never negative, never over). - // Nothing is left for the proportional pool or history. - if (remaining < 0) { - const totalMin = available - remaining; - const scale = totalMin > 0 ? available / totalMin : 0; - for (const alloc of allocations) { - alloc.allocated = Math.floor(alloc.allocated * scale); + // Phase 1: guarantee minimums out of the FULL available window — a declared + // floor is a floor, not a share of the discretionary pool. Scale down only + // when the mins alone overcommit what is actually available. + const mins = layers.map((l) => extractMin(l.budget)); + const totalMin = mins.reduce((sum, m) => sum + m, 0); + const minScale = totalMin > available ? available / totalMin : 1; + const allocations: BudgetAllocation[] = layers.map((l, i) => ({ + layerId: l.id, + allocated: Math.floor(mins[i] * minScale), + })); + if (minScale < 1) { + let unallocated = Math.floor(available) - allocations.reduce((sum, a) => sum + a.allocated, 0); + for (let i = 0; i < allocations.length && unallocated > 0; i++, unallocated--) { + allocations[i].allocated += 1; } return { allocations, - historyBudget: 0, }; } - // Phase 2: distribute 60% of remaining to layers proportionally, 40% to history - const layerPool = remaining * 0.6; - const historyBudget = remaining * 0.4; - - // Compute headroom per layer (how much above the minimum each layer can absorb) - let totalMax = 0; - const headrooms: number[] = []; - for (let i = 0; i < layers.length; i++) { - const max = extractMax(layers[i].budget); - const headroom = Math.max(0, max - allocations[i].allocated); - headrooms.push(headroom); - totalMax += headroom; + // Phase 2: distribute the discretionary remainder proportionally to headroom, + // clamped to caps. Only this pool is rationed by LAYER_POOL_SHARE, and it can + // never exceed what the mins left behind. + const remainder = Math.min(available * LAYER_POOL_SHARE, available - totalMin); + const headrooms = layers.map((layer, i) => Math.max(0, extractCap(layer.budget) - mins[i])); + const finiteTotal = headrooms.reduce((sum, h) => (Number.isFinite(h) ? sum + h : sum), 0); + const infiniteCount = headrooms.filter((h) => !Number.isFinite(h)).length; + // Explicit `Infinity` caps split whatever the finite headrooms leave over. + const finitePool = infiniteCount === 0 ? remainder : remainder * 0.5; + let finiteUsed = 0; + for (let i = 0; i < headrooms.length; i++) { + if (!Number.isFinite(headrooms[i]) || finiteTotal === 0) { + continue; + } + const share = Math.min(headrooms[i], (headrooms[i] / finiteTotal) * finitePool); + const allocatedShare = Math.floor(share); + allocations[i].allocated += allocatedShare; + finiteUsed += allocatedShare; } - - if (totalMax > 0) { - const infiniteCount = headrooms.filter((h) => !Number.isFinite(h)).length; - const finiteShares = computeFiniteShares(headrooms, layerPool); - const finiteUsed = finiteShares.reduce((sum, s) => sum + s, 0); - // Infinite-headroom layers split exactly what the finite shares left over, - // so no part of the pool is silently lost. - const infiniteShare = infiniteCount > 0 ? (layerPool - finiteUsed) / infiniteCount : 0; - + if (infiniteCount > 0) { + const perInfinite = (remainder - finiteUsed) / infiniteCount; for (let i = 0; i < headrooms.length; i++) { - const share = Number.isFinite(headrooms[i]) ? finiteShares[i] : infiniteShare; - allocations[i].allocated += Math.min(share, headrooms[i]); + if (!Number.isFinite(headrooms[i])) { + allocations[i].allocated += Math.floor(perInfinite); + } } } return { allocations, - historyBudget: Math.max(0, historyBudget), }; } diff --git a/packages/context/src/context/projector.ts b/packages/context/src/context/projector.ts index 25571c0c..2b9e02fa 100644 --- a/packages/context/src/context/projector.ts +++ b/packages/context/src/context/projector.ts @@ -1,5 +1,10 @@ -import type { Item, ProjectionPolicy } from '@noetic-tools/types'; -import { estimateTokens } from '@noetic-tools/types'; +import type { CompactionItem, Item, ProjectionPolicy } from '@noetic-tools/types'; +import { + COMPACTION_ITEM_TYPE, + createMessage, + estimateTokens, + frameworkCast, +} from '@noetic-tools/types'; import { stripUnresolvedToolCalls } from './strip-unresolved'; //#region Types @@ -18,6 +23,36 @@ interface AssembleViewParams { policy?: ProjectionPolicy; } +/** @public Result of measuring folded history against a compaction threshold. */ +export interface HistoryPressure { + /** Estimated tokens of the folded history. */ + historyTokens: number; + /** The `compactAt` threshold in effect. */ + compactAt: number; + /** True when historyTokens > compactAt — compaction should run. */ + overThreshold: boolean; +} + +/** @public Parameters for `createCompaction`. */ +export interface CreateCompactionParams { + /** The RAW item log the compaction indexes into. */ + items: ReadonlyArray; + /** How many leading raw-log items the summary replaces. */ + replacesUntil: number; + /** The summary that stands in for the replaced prefix. */ + summary: string; +} + +/** @public Parameters for `compactHistory`. */ +export interface CompactHistoryParams { + /** The RAW item log to compact. */ + log: ReadonlyArray; + /** How many trailing raw-log items to leave uncompacted. */ + keepRecent: number; + /** Produces the summary for the covered prefix. */ + summarize: (replaced: ReadonlyArray) => string | Promise; +} + //#endregion //#region Helpers @@ -27,6 +62,18 @@ function itemTokens(item: Item): number { return estimateTokens(JSON.stringify(item)); } +function isCompactionItem(item: Item): item is Item & CompactionItem { + return item.type === COMPACTION_ITEM_TYPE; +} + +/** Render a compaction summary as a developer message the model can read. */ +function renderCompaction(compaction: CompactionItem): Item { + return createMessage( + `\n${compaction.summary}\n`, + 'developer', + ); +} + function totalTokens(items: ReadonlyArray): number { let total = 0; for (const item of items) { @@ -161,4 +208,139 @@ export function assembleView({ ]; } +/** + * @public Fold recorded compactions into a history view. + * + * The item log is append-only; a compaction is an ordinary logged item that + * declares "the first `replacesUntil` items are summarized by `summary`". + * Folding keeps the log immutable (checkpoints, forks, and audits see the full + * record) while the model sees `[summary, ...items after replacesUntil]`. + * + * When multiple compactions exist, the one with the highest `replacesUntil` + * wins (later compactions subsume earlier ones — their summaries were produced + * with the earlier summary already in view). Compaction items themselves never + * appear in the folded view; the winning one renders as a developer message. + * + * Runs BEFORE the band assembler, so a compaction genuinely reduces what + * `assembleView` has to fit — and therefore how much history it has to trim. + */ +export function foldCompactions(items: ReadonlyArray): Item[] { + let winner: CompactionItem | null = null; + for (const item of items) { + if (isCompactionItem(item) && (!winner || item.replacesUntil > winner.replacesUntil)) { + winner = item; + } + } + if (!winner) { + return items.filter((i) => !isCompactionItem(i)); + } + const kept: Item[] = [ + renderCompaction(winner), + ]; + for (let i = winner.replacesUntil; i < items.length; i++) { + const item = items[i]; + if (isCompactionItem(item)) { + continue; + } + kept.push(item); + } + // A fold boundary can strand a tool call whose output was compacted away + // (or vice versa); repair the seam the same way the trimmer does. + return stripUnresolvedToolCalls(kept); +} + +/** + * @public Whether a log carries any compaction record. + * + * Lets a caller skip `foldCompactions` (and its copy) on the overwhelmingly + * common no-compaction path while still guaranteeing that a compaction item + * never reaches a provider un-folded. + */ +export function hasCompaction(items: ReadonlyArray): boolean { + return items.some(isCompactionItem); +} + +/** + * @public Measure history pressure against the policy's compaction threshold. + * + * The caller surfaces `overThreshold` to whatever drives compaction. That is + * the complementary half of the band assembler's budget enforcement: + * `assembleView` still trims the oldest history when the view will not fit, but + * it does so silently. This says when the trim is coming, so the caller can + * compact — replacing the prefix with a summary — instead of losing it. + */ +export function historyPressure( + historyItems: ReadonlyArray, + policy: ProjectionPolicy, +): HistoryPressure { + const historyTokens = totalTokens(foldCompactions(historyItems)); + const compactAt = + policy.compactAt ?? + Math.max(0, Math.floor((policy.tokenBudget - policy.responseReserve) * 0.8)); + return { + historyTokens, + compactAt, + overThreshold: historyTokens > compactAt, + }; +} + +/** + * @public Build the compaction record for a history prefix. + * + * The caller supplies the summary (an LLM call, a heuristic, or a verbatim + * digest — compaction is a composition point, not engine policy) and appends + * the returned item to the log. Idempotent with respect to prior compactions: + * `replacesUntil` indexes the RAW log, including any earlier compaction items + * in the prefix, so stacking compactions is well-defined. + */ +export function createCompaction(params: CreateCompactionParams): CompactionItem { + const replaced = params.items.slice(0, params.replacesUntil); + const summaryTokens = estimateTokens(params.summary); + return { + id: crypto.randomUUID(), + type: COMPACTION_ITEM_TYPE, + status: 'completed', + replacesUntil: params.replacesUntil, + summary: params.summary, + replacedCount: replaced.length, + tokensSaved: Math.max(0, totalTokens(replaced) - summaryTokens), + }; +} + +/** + * @public Compact a log down to its most recent `keepRecent` items. + * + * A thin convenience over `createCompaction`: it works out `replacesUntil` from + * `keepRecent`, calls `summarize` with exactly the items being replaced, and + * returns the record for the CALLER to append (`ctx.itemLog.append(...)`). + * Appending is left to the caller because compaction is an explicit, logged + * decision — the projector never mutates a log behind the runtime's back. + * + * Returns `null` when there is nothing to compact. + */ +export async function compactHistory(params: CompactHistoryParams): Promise { + const replacesUntil = params.log.length - Math.max(0, params.keepRecent); + if (replacesUntil <= 0) { + return null; + } + const summary = await params.summarize(params.log.slice(0, replacesUntil)); + return createCompaction({ + items: params.log, + replacesUntil, + summary, + }); +} + +/** + * @public Append-safe view of a compaction record. + * + * `CompactionItem` is deliberately outside the `Item` union — nothing renders + * it directly, the projector folds it — but it must reach `ItemLog.append`, + * which takes an `Item`. This is the one sanctioned bridge; the item type is + * registered with the schema registry so the append validates. + */ +export function compactionAsItem(compaction: CompactionItem): Item { + return frameworkCast(compaction); +} + //#endregion diff --git a/packages/core/src/harness/agent-harness.ts b/packages/core/src/harness/agent-harness.ts index 2f90ad29..e719b44b 100644 --- a/packages/core/src/harness/agent-harness.ts +++ b/packages/core/src/harness/agent-harness.ts @@ -21,6 +21,8 @@ import { DEFAULT_PROJECTION, disposeLayers, executeRerender, + foldCompactions, + hasCompaction, initLayers, projectHistoryLayers, recallLayers, @@ -1054,8 +1056,11 @@ export class AgentHarness = Record = Record( lastError = e instanceof Error ? e : new Error(String(e)); if (attempt < maxAttempts - 1 && retry) { - const delay = computeDelay(retry, attempt); + const delay = computeRetryDelay(retry, attempt); await new Promise((r) => setTimeout(r, delay)); } } @@ -125,7 +128,8 @@ export async function executeRunCode( }); } -function computeDelay(retry: RetryPolicy, attempt: number): number { +/** @internal Pure retry delay calculation for deterministic tests. */ +export function computeRetryDelay(retry: RetryPolicy, attempt: number): number { let delay: number; switch (retry.backoff) { case 'fixed': @@ -517,12 +521,128 @@ async function gatherRecallResults(params: { return recallResults; } +/** + * Emit `context_pressure` when the folded history has crossed the policy's + * `compactAt` threshold, at most once per step execution. + * + * `assembleView` still enforces the token budget by dropping the oldest turns, + * but it does so silently. This is the signal that lets an agent (or the host + * app) record a compaction — replacing that prefix with a summary — instead of + * losing it. Measured post-fold, so a compaction genuinely turns the signal off. + * + * Returns the new "already emitted" state, so the caller tracks the once-only + * guarantee without branching on it: a steering retry re-sends the same view, so + * re-emitting would only repeat itself. + * + * The latch closes only on an ACTUAL emission. An assembly that stayed under the + * threshold (or whose event a `shouldEmit` filter rejected) leaves the flag as it + * found it — otherwise a first assembly with room to spare would disarm the event + * for the rest of the step, and a steering retry that appended enough to cross + * `compactAt` would trim the oldest turns in the silence the event exists to break. + */ +function emitContextPressureOnce(params: { + historyItems: ReadonlyArray; + policy: ProjectionPolicy; + nodeId: string; + emit: EmitOption | undefined; + ctx: Context; + alreadyEmitted: boolean; +}): boolean { + if (params.alreadyEmitted) { + return true; + } + const pressure = historyPressure(params.historyItems, params.policy); + if (!pressure.overThreshold) { + return params.alreadyEmitted; + } + const data = { + nodeId: params.nodeId, + historyTokens: pressure.historyTokens, + compactAt: pressure.compactAt, + }; + if (!shouldEmit(params.emit, 'context_pressure', data)) { + return params.alreadyEmitted; + } + emitFrameworkEvent({ + broadcaster: getBroadcaster(params.ctx), + agentName: params.ctx.harness.config.name, + eventType: 'context_pressure', + data, + }); + return true; +} + +/** The layer-bearing path's two history bands, split out of the projected log. */ +interface PartitionedHistory { + /** System messages, hoisted to the front. Never dropped by the budget. */ + systemItems: Item[]; + /** Everything else, compactions folded. */ + historyItems: Item[]; +} + +/** + * Split the projected log into the system band and the history band, folding any + * recorded compaction on the way. + * + * Order of operations matters twice over: + * + * 1. The fold runs on the WHOLE projected array, because + * `CompactionItem.replacesUntil` indexes the log the record was created + * against (`ctx.itemLog.items`). Folding a system/tail-stripped array would + * apply that index to a shorter list, so the cut point lands past its + * intended position and silently eats one live turn per stripped item — and + * the same log would then produce a different history depending on whether + * layers are configured, since the no-layers path folds unstripped. + * 2. System items are hoisted from the array as it stood BEFORE the fold, so a + * compaction whose covered prefix happens to include the system prompt still + * leaves the model its instructions. They are then skipped when the folded + * result is walked, so each appears exactly once — in the band the budget + * never trims. + * + * The fold renders the winning compaction as a `developer` message carrying a + * fresh id, so neither the system test nor the tail test below can claim it and + * the summary always reaches history. + */ +function partitionProjectedHistory(params: { + projected: ReadonlyArray; + tailIds: ReadonlySet; +}): PartitionedHistory { + const systemItems: Item[] = []; + for (const item of params.projected) { + if (item.type === 'message' && item.role === 'system') { + systemItems.push(item); + } + } + const folded = hasCompaction(params.projected) + ? foldCompactions(params.projected) + : params.projected; + const historyItems: Item[] = []; + for (const item of folded) { + // Hoisted above, into a band that is never dropped. + if (item.type === 'message' && item.role === 'system') { + continue; + } + // Steering guidance is re-placed at the tail; skip it here so the retry + // does not see the same correction twice. + if ('id' in item && typeof item.id === 'string' && params.tailIds.has(item.id)) { + continue; + } + historyItems.push(item); + } + return { + systemItems, + historyItems, + }; +} + /** * Builds the items the model sees for one attempt: system messages stay at the * front, the banded anchor/live/delta output rides between them and history, * and steering guidance from a prior retry lands at the very tail. The * projector enforces the token budget (drops highest-slot layer output, then - * oldest history). + * oldest history). Recorded compactions are folded BEFORE the bands claim + * their budget, and a real threshold crossing emits `context_pressure` once + * per step execution. */ async function assembleTurnItems(params: { ctx: Context; @@ -530,37 +650,60 @@ async function assembleTurnItems(params: { banded: BandedView; steeringTail: InputMessageItem[]; viewPolicy: ProjectionPolicy; -}): Promise> { + nodeId: string; + emit: EmitOption | undefined; + pressureEmitted: boolean; +}): Promise<{ + items: ReadonlyArray; + pressureEmitted: boolean; +}> { const { ctx, layers, banded, steeringTail, viewPolicy } = params; const rawHistoryItems: ReadonlyArray = ctx.itemLog.items; if (layers === undefined || layers.length === 0) { - return rawHistoryItems; + // Fold even here: the model must read the summary, never the raw record. + // Preserve array identity on the common no-compaction path. + const items = hasCompaction(rawHistoryItems) + ? foldCompactions(rawHistoryItems) + : rawHistoryItems; + const pressureEmitted = emitContextPressureOnce({ + historyItems: items, + policy: viewPolicy, + nodeId: params.nodeId, + emit: params.emit, + ctx, + alreadyEmitted: params.pressureEmitted, + }); + return { + items, + pressureEmitted, + }; } const projectedHistoryItems = await ctx.harness.projectHistory(layers, rawHistoryItems, ctx); - const systemItems: Item[] = []; - const nonSystemHistory: Item[] = []; - const tailIds = new Set(steeringTail.map((i) => i.id)); - for (const item of projectedHistoryItems) { - if (item.type === 'message' && item.role === 'system') { - systemItems.push(item); - continue; - } - // Steering guidance is re-placed at the tail; skip it here so the retry - // does not see the same correction twice. - if ('id' in item && typeof item.id === 'string' && tailIds.has(item.id)) { - continue; - } - nonSystemHistory.push(item); - } - return assembleView({ - systemPromptItems: systemItems, - layerOutputItems: banded.anchorItems, - historyItems: nonSystemHistory, - liveLayerItems: banded.liveItems, - deltaItems: banded.deltaItems, - tailItems: steeringTail, + const { systemItems, historyItems } = partitionProjectedHistory({ + projected: projectedHistoryItems, + tailIds: new Set(steeringTail.map((i) => i.id)), + }); + // Announce the trim that is coming, before the assembler performs it. + const pressureEmitted = emitContextPressureOnce({ + historyItems, policy: viewPolicy, + nodeId: params.nodeId, + emit: params.emit, + ctx, + alreadyEmitted: params.pressureEmitted, }); + return { + items: assembleView({ + systemPromptItems: systemItems, + layerOutputItems: banded.anchorItems, + historyItems, + liveLayerItems: banded.liveItems, + deltaItems: banded.deltaItems, + tailItems: steeringTail, + policy: viewPolicy, + }), + pressureEmitted, + }; } interface BuildModelRequestParams { @@ -772,15 +915,21 @@ export async function executeCallModel( // live band and the supersedes, rather than wherever history happens to put it. let steeringTail: InputMessageItem[] = []; let cacheJudged = false; + let pressureEmitted = false; while (retries <= MAX_STEERING_RETRIES) { - const assembledItems = await assembleTurnItems({ + const assembled = await assembleTurnItems({ ctx: baseCtx, layers, banded, steeringTail, viewPolicy, + nodeId: step.id, + emit: step.emit, + pressureEmitted, }); + pressureEmitted = assembled.pressureEmitted; + const assembledItems = assembled.items; const request = buildModelRequest({ step, diff --git a/packages/core/test/context/_audit/lifecycle.test.ts b/packages/core/test/context/_audit/lifecycle.test.ts index aa96868c..34f8cd8a 100644 --- a/packages/core/test/context/_audit/lifecycle.test.ts +++ b/packages/core/test/context/_audit/lifecycle.test.ts @@ -13,9 +13,13 @@ import { readFileSync } from 'node:fs'; import { allocateBudgets, assembleView, + compactionAsItem, completeLayers, + createCompaction, createLayerStateStore, disposeLayers, + foldCompactions, + historyPressure, initLayers, recallLayers, returnLayers, @@ -23,7 +27,7 @@ import { spawnLayers, storeLayers, } from '@noetic-tools/context'; -import type { ContextLayer, Item } from '@noetic-tools/types'; +import type { ContextLayer, Item, ProjectionPolicy } from '@noetic-tools/types'; import { makeCtx, makeItemLog, @@ -226,6 +230,37 @@ describe('AUDIT-3 assembleView token cap', () => { // only branches on 'sliding_window' → 'truncate' is a silent no-op. expect(view.length).toBeLessThan(500); }); + + it('3c: the trim that AUDIT-3 relies on is observable, and compaction relieves it', () => { + // The complement of 3a: assembleView is allowed to drop history, but the + // caller must be able to see it coming and do something cheaper about it. + const history: Item[] = Array.from( + { + length: 500, + }, + (_, i) => makeMessage('user', `msg-${i}`), + ); + const policy: ProjectionPolicy = { + tokenBudget: 1_000, + responseReserve: 0, + overflow: 'sliding_window', + }; + // 1. The pressure is visible BEFORE the assembler silently drops anything. + expect(historyPressure(history, policy).overThreshold).toBe(true); + // 2. A logged compaction relieves it... + const compaction = createCompaction({ + items: history, + replacesUntil: 495, + summary: 'first 495 messages: greetings', + }); + const withCompaction = [ + ...history, + compactionAsItem(compaction), + ]; + expect(historyPressure(withCompaction, policy).overThreshold).toBe(false); + // 3. ...by replacing the covered prefix with the summary, not discarding it. + expect(foldCompactions(withCompaction)).toHaveLength(1 + 5); + }); }); //#endregion diff --git a/packages/core/test/context/adversarial-context.test.ts b/packages/core/test/context/adversarial-context.test.ts index ce0f5045..2c001c6e 100644 --- a/packages/core/test/context/adversarial-context.test.ts +++ b/packages/core/test/context/adversarial-context.test.ts @@ -1027,13 +1027,14 @@ describe('Budget allocation: edge cases', () => { const allocB = result.allocations.find((a) => a.layerId === 'b'); const allocC = result.allocations.find((a) => a.layerId === 'c'); - expect(allocA!.allocated).toBe(1.6e3); - expect(allocB!.allocated).toBe(1.6e3); - expect(allocC!.allocated).toBe(1.6e3); - expect(result.historyBudget).toBe(3.2e3); + // available = 8000, pool = 25% = 2000; three auto caps of 2000 each split + // the pool proportionally (666 each after flooring). + expect(allocA!.allocated).toBe(666); + expect(allocB!.allocated).toBe(666); + expect(allocC!.allocated).toBe(666); }); - it('single layer with budget gets 60% of available', () => { + it('single layer with headroom absorbs the whole pool', () => { const layers: ContextLayer[] = [ { id: 'only', @@ -1054,7 +1055,10 @@ describe('Budget allocation: edge cases', () => { responseReserve: 1e3, }); - expect(result.allocations[0].allocated).toBe(4840); + // available = 8000. The min (100) is guaranteed out of the full window, then + // the sole layer's headroom absorbs the whole discretionary pool + // (25% of 8000 = 2000) on top of that floor. + expect(result.allocations[0].allocated).toBe(2.1e3); }); it('zero total budget yields zero for all', () => { @@ -1079,7 +1083,6 @@ describe('Budget allocation: edge cases', () => { }); expect(result.allocations[0].allocated).toBe(0); - expect(result.historyBudget).toBe(0); }); it('mixed finite and infinite layers do not over-allocate', () => { @@ -1116,9 +1119,9 @@ describe('Budget allocation: edge cases', () => { expect(finiteAlloc!.allocated).toBeLessThanOrEqual(1e3); expect(infiniteAlloc!.allocated).toBeGreaterThan(0); - // Total allocated to layers should not exceed layerPool (4800) + // Total allocated to layers must not exceed the pool (25% of 8000 = 2000) const totalAllocated = result.allocations.reduce((sum, a) => sum + a.allocated, 0); - expect(totalAllocated).toBeLessThanOrEqual(4.8e3); + expect(totalAllocated).toBeLessThanOrEqual(2e3); }); }); diff --git a/packages/core/test/context/budget.test.ts b/packages/core/test/context/budget.test.ts index 696c63b8..6d183e9f 100644 --- a/packages/core/test/context/budget.test.ts +++ b/packages/core/test/context/budget.test.ts @@ -17,6 +17,16 @@ function makeLayer(id: string, budget?: BudgetConfig): ContextLayer { }; } +// Declared mins are satisfied out of the full available window +// (totalBudget - responseReserve - systemPromptTokens); only the discretionary +// remainder on top of them is 25% of that window. +// History no longer takes a per-turn budget line from the allocator: what the +// assembler can fit is decided in assembleView, and compaction is how history +// shrinks (see historyPressure). +const POOL_SHARE = 0.25; +// Cap assigned to 'auto'/omitted budgets — deterministic, not infinite. +const AUTO_CAP = 2_000; + describe('allocateBudgets', () => { it('satisfies minimums first', () => { const layers = [ @@ -39,22 +49,6 @@ describe('allocateBudgets', () => { expect(allocations[1].allocated).toBeGreaterThanOrEqual(300); }); - it('reserves 40% for history', () => { - const layers = [ - makeLayer('a', { - min: 0, - max: 5_000, - }), - ]; - const { historyBudget } = allocateBudgets({ - layers, - totalBudget: 10_000, - systemPromptTokens: 0, - responseReserve: 0, - }); - expect(historyBudget).toBe(4_000); - }); - it('handles zero available budget', () => { const layers = [ makeLayer('a', { @@ -62,25 +56,24 @@ describe('allocateBudgets', () => { max: 1_000, }), ]; - const { allocations, historyBudget } = allocateBudgets({ + const { allocations } = allocateBudgets({ layers, totalBudget: 100, systemPromptTokens: 50, responseReserve: 50, }); expect(allocations[0].allocated).toBe(0); - expect(historyBudget).toBe(0); }); - it('distributes proportionally to max headroom', () => { + it('distributes proportionally to cap headroom, clamped to caps', () => { const layers = [ makeLayer('a', { min: 0, - max: 3_000, + max: 300, }), makeLayer('b', { min: 0, - max: 1_000, + max: 100, }), ]; const { allocations } = allocateBudgets({ @@ -89,12 +82,12 @@ describe('allocateBudgets', () => { systemPromptTokens: 0, responseReserve: 0, }); - // 60% of 10000 = 6000 for layers; a gets 3/4 = 4500 capped to 3000, b gets 1/4 = 1500 capped to 1000 - expect(allocations[0].allocated).toBe(3_000); - expect(allocations[1].allocated).toBe(1_000); + // pool = 2500; headrooms 300/100 → both fully satisfiable within the pool. + expect(allocations[0].allocated).toBe(300); + expect(allocations[1].allocated).toBe(100); }); - it('handles numeric budget config', () => { + it('handles numeric budget config as a cap', () => { const layers = [ makeLayer('a', 500), ]; @@ -107,7 +100,7 @@ describe('allocateBudgets', () => { expect(allocations[0].allocated).toBe(500); }); - it('Infinity headroom layers split equally', () => { + it('auto layers get the default cap, split when pool-constrained', () => { const layers = [ makeLayer('a', 'auto'), makeLayer('b', 'auto'), @@ -118,9 +111,26 @@ describe('allocateBudgets', () => { systemPromptTokens: 0, responseReserve: 0, }); - // 60% of 10000 = 6000 for layers, split equally between 2 auto layers - expect(allocations[0].allocated).toBe(3_000); - expect(allocations[1].allocated).toBe(3_000); + // pool = 2500; two auto caps of 2000 each (4000 total headroom) → + // proportional split of the pool: 1250 each. + expect(allocations[0].allocated).toBe(1_250); + expect(allocations[1].allocated).toBe(1_250); + }); + + it('a single auto layer is clamped to the default cap, not the pool', () => { + const layers = [ + makeLayer('a', 'auto'), + ]; + const { allocations } = allocateBudgets({ + layers, + totalBudget: 100_000, + systemPromptTokens: 0, + responseReserve: 0, + }); + // pool = 25000 but the auto cap bounds the layer at 2000 — the rendered + // block is the same size whatever the model's context length, which is what + // makes it cacheable. + expect(allocations[0].allocated).toBe(AUTO_CAP); }); it('negative available budget yields all zero allocations', () => { @@ -132,7 +142,7 @@ describe('allocateBudgets', () => { makeLayer('b', 'auto'), ]; // systemPromptTokens + responseReserve > totalBudget - const { allocations, historyBudget } = allocateBudgets({ + const { allocations } = allocateBudgets({ layers, totalBudget: 100, systemPromptTokens: 5_000, @@ -140,27 +150,33 @@ describe('allocateBudgets', () => { }); expect(allocations[0].allocated).toBe(0); expect(allocations[1].allocated).toBe(0); - expect(historyBudget).toBe(0); }); - it('handles auto budget config', () => { + it('a layer that omits budget gets the auto cap treatment, not 0', () => { const layers = [ - makeLayer('a', 'auto'), + makeLayer('no-budget'), + makeLayer('auto', 'auto'), ]; const { allocations } = allocateBudgets({ layers, - totalBudget: 10_000, + totalBudget: 100_000, systemPromptTokens: 0, responseReserve: 0, }); - // 60% of 10000 (layerPool ratio) allocated to single auto layer - expect(allocations[0].allocated).toBe(6_000); + expect(allocations[0].allocated).toBe(AUTO_CAP); + expect(allocations[1].allocated).toBe(AUTO_CAP); }); - it('a layer that omits budget gets an auto share, not 0', () => { + it('mins that exceed the pool but fit the window are still satisfied in full', () => { const layers = [ - makeLayer('no-budget'), - makeLayer('auto', 'auto'), + makeLayer('a', { + min: 2_000, + max: 3_000, + }), + makeLayer('b', { + min: 6_000, + max: 8_000, + }), ]; const { allocations } = allocateBudgets({ layers, @@ -168,18 +184,84 @@ describe('allocateBudgets', () => { systemPromptTokens: 0, responseReserve: 0, }); - // Omitted budget = infinite headroom: splits the 6000 pool with 'auto'. - expect(allocations[0].allocated).toBe(3_000); + // mins total 8000 — over the 2500 pool, but well inside the 10000 window. + // The pool bounds the discretionary remainder only, never the floors. + expect(allocations[0].allocated).toBeGreaterThanOrEqual(2_000); + expect(allocations[1].allocated).toBeGreaterThanOrEqual(6_000); + }); + + it('scales minimums down proportionally only when they overcommit the window', () => { + const layers = [ + makeLayer('a', { + min: 2_000, + max: 3_000, + }), + makeLayer('b', { + min: 6_000, + max: 8_000, + }), + ]; + const { allocations } = allocateBudgets({ + layers, + totalBudget: 4_000, + systemPromptTokens: 0, + responseReserve: 0, + }); + // available = 4000, mins total 8000 → scale = 4000/8000, nothing else split. + const sum = allocations.reduce((s, a) => s + a.allocated, 0); + expect(sum).toBeLessThanOrEqual(4_000); + expect(allocations[0].allocated).toBe(1_000); expect(allocations[1].allocated).toBe(3_000); }); - it('conserves the pool when finite and auto layers mix (verifier repro)', () => { + it('a min larger than the pool survives a small window (32k/4k regression)', () => { const layers = [ - makeLayer('finite', { - min: 0, - max: 4_800, + makeLayer('a', { + min: 10_000, + max: 12_000, + }), + ]; + const { allocations } = allocateBudgets({ + layers, + totalBudget: 32_000, + systemPromptTokens: 0, + responseReserve: 4_000, + }); + // available = 28000; the 25% pool is 7000, below the declared 10000 min. + // The floor comes out of the window first, then the remainder tops it up to + // the cap — squeezing the min into the pool would silently render a + // truncated block. + expect(allocations[0].allocated).toBe(12_000); + }); + + it('two min-declaring layers both keep their floors on a small window', () => { + const layers = [ + makeLayer('a', { + min: 3_000, + max: 4_000, + }), + makeLayer('b', { + min: 5_000, + max: 6_000, + }), + ]; + const { allocations } = allocateBudgets({ + layers, + totalBudget: 32_000, + systemPromptTokens: 0, + responseReserve: 4_000, + }); + // mins total 8000 > the 7000 pool, but available is 28000. + expect(allocations[0].allocated).toBe(4_000); + expect(allocations[1].allocated).toBe(6_000); + }); + + it('the discretionary remainder never exceeds what the mins left behind', () => { + const layers = [ + makeLayer('a', { + min: 9_000, + max: Number.POSITIVE_INFINITY, }), - makeLayer('auto', 'auto'), ]; const { allocations } = allocateBudgets({ layers, @@ -187,38 +269,21 @@ describe('allocateBudgets', () => { systemPromptTokens: 0, responseReserve: 0, }); - // layerPool = 6000; finite gets min(4800, half-pool 3000) = 3000; - // auto gets the REST of the pool — nothing vanishes. - expect(allocations[0].allocated).toBe(3_000); - expect(allocations[1].allocated).toBe(3_000); - const sum = allocations.reduce((s, a) => s + a.allocated, 0); - expect(sum).toBe(6_000); + // available = 10000, min = 9000 → remainder is capped at 1000, not the + // 2500 pool share, so the total stays inside the window. + expect(allocations[0].allocated).toBe(10_000); }); - it.each([ - // [finiteMax, expectedFinite, expectedAuto] — half-pool clamp boundary at 3000 - [ - 2_999, - 2_999, - 3_001, - ], - [ - 3_000, - 3_000, - 3_000, - ], - [ - 3_001, - 3_000, - 3_000, - ], - ])('clamp switchover boundary: finite max %d → finite %d / auto %d (pool conserved)', (finiteMax, expectedFinite, expectedAuto) => { + it('explicit Infinity caps split the remainder after finite caps', () => { const layers = [ makeLayer('finite', { min: 0, - max: finiteMax, + max: 400, + }), + makeLayer('uncapped', { + min: 0, + max: Number.POSITIVE_INFINITY, }), - makeLayer('auto', 'auto'), ]; const { allocations } = allocateBudgets({ layers, @@ -226,10 +291,12 @@ describe('allocateBudgets', () => { systemPromptTokens: 0, responseReserve: 0, }); - expect(allocations[0].allocated).toBeCloseTo(expectedFinite, 6); - expect(allocations[1].allocated).toBeCloseTo(expectedAuto, 6); + // pool = 2500; finite gets its full 400 cap; the uncapped layer absorbs + // the rest. Pool is conserved (floor() may shave a token). const sum = allocations.reduce((s, a) => s + a.allocated, 0); - expect(sum).toBeCloseTo(6_000, 6); + expect(allocations[0].allocated).toBe(400); + expect(sum).toBeGreaterThanOrEqual(2_499); + expect(sum).toBeLessThanOrEqual(2_500); }); it.each([ @@ -264,7 +331,7 @@ describe('allocateBudgets', () => { }), makeLayer('auto', 'auto'), ]; - const { allocations, historyBudget } = allocateBudgets({ + const { allocations } = allocateBudgets({ layers, totalBudget: Number.POSITIVE_INFINITY, systemPromptTokens: 0, @@ -274,22 +341,38 @@ describe('allocateBudgets', () => { expect(Number.isNaN(a.allocated)).toBe(false); } expect(allocations[0].allocated).toBe(1_000); // capped at max - expect(allocations[1].allocated).toBe(Number.POSITIVE_INFINITY); - expect(historyBudget).toBe(Number.POSITIVE_INFINITY); + expect(allocations[1].allocated).toBe(AUTO_CAP); // capped at the auto default }); it('fractional budgets are accepted (pinned)', () => { const layers = [ makeLayer('auto', 'auto'), ]; - const { allocations, historyBudget } = allocateBudgets({ + const { allocations } = allocateBudgets({ layers, totalBudget: 0.5, systemPromptTokens: 0, responseReserve: 0, }); - expect(allocations[0].allocated).toBeCloseTo(0.3, 9); - expect(historyBudget).toBeCloseTo(0.2, 9); + // pool = 0.125 — sub-token pools floor to 0. + expect(allocations[0].allocated).toBe(0); + }); + + it('pool share constant is pinned', () => { + const layers = [ + makeLayer('a', { + min: 0, + max: Number.POSITIVE_INFINITY, + }), + ]; + const { allocations } = allocateBudgets({ + layers, + totalBudget: 40_000, + systemPromptTokens: 0, + responseReserve: 0, + }); + // A single uncapped layer absorbs the whole pool: 25% of 40000. + expect(allocations[0].allocated).toBe(40_000 * POOL_SHARE); }); }); diff --git a/packages/core/test/context/compaction.test.ts b/packages/core/test/context/compaction.test.ts new file mode 100644 index 00000000..b4d05b28 --- /dev/null +++ b/packages/core/test/context/compaction.test.ts @@ -0,0 +1,439 @@ +import { describe, expect, it } from 'bun:test'; +import assert from 'node:assert'; +import { + compactHistory, + compactionAsItem, + createCompaction, + foldCompactions, + hasCompaction, + historyPressure, +} from '@noetic-tools/context'; +import type { Item, ProjectionPolicy } from '@noetic-tools/types'; +import { + COMPACTION_ITEM_TYPE, + estimateTokens, + frameworkCast, + isNoeticError, +} from '@noetic-tools/types'; +import { ItemLogImpl } from '../../src/runtime/item-log-impl'; +import { makeFunctionCall, makeFunctionCallOutput, makeMessage } from '../_helpers'; + +/** Extract the text of every message item in the view (for order assertions). */ +function viewTexts(view: Item[]): string[] { + const texts: string[] = []; + for (const item of view) { + assert(item.type === 'message'); + const part = item.content[0]; + assert('text' in part && typeof part.text === 'string'); + texts.push(part.text.slice(0, 12)); + } + return texts; +} + +/** Read the rendered summary text out of a folded view's leading item. */ +function summaryTextOf(view: Item[]): string { + const head = view[0]; + assert(head.type === 'message'); + const part = head.content[0]; + assert('text' in part && typeof part.text === 'string'); + return part.text; +} + +describe('foldCompactions', () => { + const history = [ + makeMessage('user', 'old-1'), + makeMessage('assistant', 'old-2'), + makeMessage('user', 'old-3'), + makeMessage('user', 'recent-1'), + ]; + + it('passes history through when no compaction exists', () => { + expect(foldCompactions(history)).toEqual(history); + }); + + it('replaces the covered prefix with a rendered summary', () => { + const compaction = createCompaction({ + items: history, + replacesUntil: 3, + summary: 'the user discussed old things', + }); + const folded = foldCompactions([ + ...history, + compactionAsItem(compaction), + ]); + expect(folded).toHaveLength(2); // summary + recent-1 + expect(summaryTextOf(folded)).toContain('the user discussed old things'); + expect(viewTexts(folded.slice(1))).toEqual([ + 'recent-1', + ]); + }); + + it('the highest replacesUntil wins when compactions stack', () => { + const first = createCompaction({ + items: history, + replacesUntil: 2, + summary: 'first summary', + }); + const log: Item[] = [ + ...history, + compactionAsItem(first), + makeMessage('user', 'after-first'), + ]; + const second = createCompaction({ + items: log, + replacesUntil: 5, // covers history + first compaction + summary: 'second summary', + }); + const folded = foldCompactions([ + ...log, + compactionAsItem(second), + ]); + const summary = summaryTextOf(folded); + expect(summary).toContain('second summary'); + expect(summary).not.toContain('first summary'); + expect(viewTexts(folded.slice(1))).toEqual([ + 'after-first', + ]); + }); + + it('compaction items never leak into the folded view', () => { + const compaction = createCompaction({ + items: history, + replacesUntil: 2, + summary: 's', + }); + const folded = foldCompactions([ + ...history, + compactionAsItem(compaction), + ]); + expect(folded.some((i) => i.type === COMPACTION_ITEM_TYPE)).toBe(false); + }); + + it('indexes the array it is given, so a caller must not pre-strip it', () => { + // `replacesUntil` is a RAW-log index. Removing items before folding shifts + // the cut point forward by one per removal and eats live turns past the + // boundary — the reason the interpreter folds before it partitions. + const log: Item[] = [ + makeMessage('system', 'sys'), + makeMessage('user', 'live-1'), + makeMessage('user', 'live-2'), + ]; + const compaction = compactionAsItem( + createCompaction({ + items: log, + replacesUntil: 1, + summary: 's', + }), + ); + + const whole = foldCompactions([ + ...log, + compaction, + ]); + expect(viewTexts(whole)).toEqual([ + ' !(i.type === 'message' && i.role === 'system')), + compaction, + ]); + expect(viewTexts(preStripped)).toEqual([ + ' { + // The call survives the fold boundary but its output does not — an + // unresolved call would make the provider reject the request. + const log: Item[] = [ + makeFunctionCall('search', '{}', 'call-1'), + makeFunctionCallOutput('call_call-1', 'results'), + makeMessage('user', 'next'), + ]; + const compaction = createCompaction({ + items: log, + replacesUntil: 2, + summary: 'searched and got results', + }); + const folded = foldCompactions([ + ...log, + compactionAsItem(compaction), + ]); + expect(folded.some((i) => i.type === 'function_call')).toBe(false); + expect(folded.some((i) => i.type === 'function_call_output')).toBe(false); + }); +}); + +describe('hasCompaction', () => { + it('is false for an uncompacted log and true once a record is present', () => { + const history = [ + makeMessage('user', 'a'), + ]; + expect(hasCompaction(history)).toBe(false); + const compaction = createCompaction({ + items: history, + replacesUntil: 1, + summary: 's', + }); + expect( + hasCompaction([ + ...history, + compactionAsItem(compaction), + ]), + ).toBe(true); + }); +}); + +describe('createCompaction', () => { + it('records replaced count and token savings', () => { + const items = [ + makeMessage('user', 'a'.repeat(400)), + makeMessage('assistant', 'b'.repeat(400)), + ]; + const compaction = createCompaction({ + items, + replacesUntil: 2, + summary: 'short', + }); + expect(compaction.type).toBe(COMPACTION_ITEM_TYPE); + expect(compaction.replacedCount).toBe(2); + expect(compaction.summary).toBe('short'); + expect(compaction.tokensSaved).toBeGreaterThan(0); + }); + + it('tokensSaved floors at 0 when the summary is longer than what it replaces', () => { + const compaction = createCompaction({ + items: [ + makeMessage('user', 'hi'), + ], + replacesUntil: 1, + summary: 'x'.repeat(4_000), + }); + expect(compaction.tokensSaved).toBe(0); + }); +}); + +describe('compactHistory', () => { + const log = [ + makeMessage('user', 'turn-1'), + makeMessage('assistant', 'turn-2'), + makeMessage('user', 'turn-3'), + makeMessage('user', 'turn-4'), + ]; + + it('summarizes everything but the most recent N items', async () => { + const compaction = await compactHistory({ + log, + keepRecent: 1, + summarize: (replaced) => `summarized ${replaced.length}`, + }); + assert(compaction !== null); + expect(compaction.replacesUntil).toBe(3); + expect(compaction.summary).toBe('summarized 3'); + const folded = foldCompactions([ + ...log, + compactionAsItem(compaction), + ]); + expect(viewTexts(folded.slice(1))).toEqual([ + 'turn-4', + ]); + }); + + it('returns null when keepRecent covers the whole log', async () => { + const compaction = await compactHistory({ + log, + keepRecent: log.length, + summarize: () => 'unused', + }); + expect(compaction).toBeNull(); + }); + + it('awaits an async summarizer (an LLM step is the intended shape)', async () => { + const compaction = await compactHistory({ + log, + keepRecent: 2, + summarize: async () => { + await Promise.resolve(); + return 'async summary'; + }, + }); + assert(compaction !== null); + expect(compaction.summary).toBe('async summary'); + }); +}); + +describe('historyPressure', () => { + const policy: ProjectionPolicy = { + tokenBudget: 1_000, + responseReserve: 200, + overflow: 'sliding_window', + }; + + it('reports under-threshold for small histories', () => { + const pressure = historyPressure( + [ + makeMessage('user', 'hi'), + ], + policy, + ); + expect(pressure.overThreshold).toBe(false); + expect(pressure.compactAt).toBe(Math.floor((1_000 - 200) * 0.8)); + }); + + it('reports over-threshold when folded history exceeds compactAt', () => { + const big = Array.from( + { + length: 40, + }, + (_, i) => makeMessage('user', `msg-${i} ${'y'.repeat(120)}`), + ); + const pressure = historyPressure(big, policy); + expect(pressure.overThreshold).toBe(true); + expect(pressure.historyTokens).toBeGreaterThan(pressure.compactAt); + }); + + it('measures the FOLDED history — compaction relieves pressure', () => { + const big = Array.from( + { + length: 40, + }, + (_, i) => makeMessage('user', `msg-${i} ${'z'.repeat(120)}`), + ); + expect(historyPressure(big, policy).overThreshold).toBe(true); + const compaction = createCompaction({ + items: big, + replacesUntil: 38, + summary: 'earlier messages summarized', + }); + const relieved = historyPressure( + [ + ...big, + compactionAsItem(compaction), + ], + policy, + ); + expect(relieved.overThreshold).toBe(false); + }); + + it('respects an explicit compactAt override', () => { + const pressure = historyPressure( + [ + makeMessage('user', 'x'.repeat(4_000)), + ], + { + tokenBudget: 1_000_000, + responseReserve: 0, + overflow: 'sliding_window', + compactAt: 100, + }, + ); + expect(pressure.compactAt).toBe(100); + expect(pressure.overThreshold).toBe(true); + }); +}); + +// A compaction record is only worth writing down if it can actually be written +// down: it has to pass the strict item-schema registry that guards every append, +// or the "compaction survives checkpoint/resume" claim is false. +describe('compaction in a real ItemLog', () => { + it('appends to a log under the default strict schema registry', () => { + const log = new ItemLogImpl(); + log.append(makeMessage('user', 'turn-1')); + log.append(makeMessage('assistant', 'turn-2')); + const compaction = createCompaction({ + items: log.items, + replacesUntil: 2, + summary: 'the opening exchange', + }); + expect(() => log.append(compactionAsItem(compaction))).not.toThrow(); + expect(log.items).toHaveLength(3); + expect(log.items[2].type).toBe(COMPACTION_ITEM_TYPE); + }); + + it('the compaction allowance did not open the gate for other unknown types', () => { + const log = new ItemLogImpl(); + try { + // Structurally identical to a compaction record apart from the `type` + // string, so only the registry's allow-list can be what rejects it. + log.append( + frameworkCast({ + id: 'y', + type: 'noetic:not_a_real_type', + status: 'completed', + replacesUntil: 0, + summary: '', + replacedCount: 0, + }), + ); + expect.unreachable('should have thrown'); + } catch (e) { + assert(isNoeticError(e)); + expect(e.noeticError.kind).toBe('item_schema_mismatch'); + } + }); + + it('a compacted log round-trips: the raw log keeps every item, the view shrinks', () => { + const log = new ItemLogImpl(); + for (let i = 0; i < 10; i++) { + log.append(makeMessage('user', `turn-${i}`)); + } + const before = foldCompactions(log.items); + expect(before).toHaveLength(10); + + const compaction = createCompaction({ + items: log.items, + replacesUntil: 8, + summary: 'turns 0 through 7', + }); + log.append(compactionAsItem(compaction)); + + // The log is the durable record — nothing was destroyed, which is what makes + // this survive a checkpoint. + expect(log.items).toHaveLength(11); + // The model's view is what shrank. + const after = foldCompactions(log.items); + expect(after).toHaveLength(3); // summary + turn-8 + turn-9 + expect(summaryTextOf(after)).toContain('turns 0 through 7'); + expect(viewTexts(after.slice(1))).toEqual([ + 'turn-8', + 'turn-9', + ]); + }); + + it('serializing and rehydrating a compacted log preserves the folded view', () => { + const log = new ItemLogImpl(); + for (let i = 0; i < 6; i++) { + log.append(makeMessage('user', `turn-${i}`)); + } + const compaction = createCompaction({ + items: log.items, + replacesUntil: 4, + summary: 'the first four turns', + }); + log.append(compactionAsItem(compaction)); + const expected = foldCompactions(log.items); + + // Round-trip through JSON the way a checkpoint does, then re-append through + // the registry — the compaction has to survive validation a second time. + const revived = new ItemLogImpl(); + for (const item of JSON.parse(JSON.stringify(log.items))) { + revived.append(item); + } + expect(revived.items).toHaveLength(7); + const folded = foldCompactions(revived.items); + expect(folded).toHaveLength(expected.length); + expect(summaryTextOf(folded)).toContain('the first four turns'); + }); +}); + +// estimateTokens participates in the fold contract (tokensSaved) — pin its shape. +describe('token estimation contract', () => { + it('longer content estimates more tokens', () => { + expect(estimateTokens('a'.repeat(1_000))).toBeGreaterThan(estimateTokens('a')); + }); +}); diff --git a/packages/core/test/interpreter/execute-llm-compaction.test.ts b/packages/core/test/interpreter/execute-llm-compaction.test.ts new file mode 100644 index 00000000..e43d5107 --- /dev/null +++ b/packages/core/test/interpreter/execute-llm-compaction.test.ts @@ -0,0 +1,529 @@ +/** + * Compaction reaching the model, through the interpreter. + * + * The unit suite (test/context/compaction.test.ts) proves `foldCompactions` is + * correct in isolation. This proves the interpreter actually applies it: a + * compaction record written to the item log must shrink what `callModel` + * receives, and crossing `compactAt` must surface as a `context_pressure` + * framework event rather than a silent trim. + */ + +import { describe, expect, it } from 'bun:test'; +import assert from 'node:assert'; +import type { ContextData, ContextLayer } from '@noetic-tools/context'; +import { compactionAsItem, createCompaction } from '@noetic-tools/context'; +import type { + CallModelRequest, + Item, + ProjectionPolicy, + StepCallModel, + StreamEvent, +} from '@noetic-tools/types'; +import { frameworkCast, SteeringAction } from '@noetic-tools/types'; +import { executeCallModel } from '../../src/interpreter/execute-action'; +import { ContextImpl } from '../../src/runtime/context-impl'; +import { EventBroadcaster } from '../../src/runtime/event-broadcaster'; +import { makeLLMResponse, makeMockHarness } from '../_helpers'; + +//#region Helpers + +/** A real broadcaster that also records what it was asked to emit. */ +class RecordingBroadcaster extends EventBroadcaster { + readonly events: StreamEvent[] = []; + + override emit(event: StreamEvent): void { + this.events.push(event); + super.emit(event); + } +} + +/** One inert layer — enough to put executeCallModel on the layer-bearing assembly path. */ +const LAYERS: ContextLayer[] = [ + { + id: 'inert', + slot: 275, + scope: 'execution', + hooks: {}, + }, +]; + +const STEP: StepCallModel = { + kind: 'callModel', + id: 'compaction-step', + model: 'gpt-4', +}; + +interface Rig { + ctx: ContextImpl; + events: StreamEvent[]; + request: () => CallModelRequest; +} + +/** An executeCallModel rig with a recording broadcaster and a captured model request. */ +function makeRig(): Rig { + let captured: CallModelRequest | undefined; + const harness = makeMockHarness(); + harness.callModel = async (request) => { + captured = request; + return makeLLMResponse('done'); + }; + const broadcaster = new RecordingBroadcaster(); + const ctx = new ContextImpl({ + harness, + _broadcaster: broadcaster, + }); + return { + ctx, + events: broadcaster.events, + request: () => { + assert(captured !== undefined, 'callModel was never invoked'); + return captured; + }, + }; +} + +/** Append one system message to the log — the band the budget never trims. */ +function seedSystem(ctx: ContextImpl, text: string): void { + ctx.itemLog.append( + frameworkCast({ + id: `sys-${text}`, + type: 'message', + role: 'system', + status: 'completed', + content: [ + { + type: 'input_text', + text, + }, + ], + }), + ); +} + +/** Every text part of every message item in a request, in order. */ +function requestTexts(items: ReadonlyArray): string[] { + return items.flatMap((item) => + item.type === 'message' + ? item.content.flatMap((part) => + 'text' in part + ? [ + part.text, + ] + : [], + ) + : [], + ); +} + +/** Seed `count` user turns of roughly `pad` characters each. */ +function seedTurns(ctx: ContextImpl, count: number, pad = 0): void { + for (let i = 0; i < count; i++) { + ctx.itemLog.append( + frameworkCast({ + id: `u-${i}`, + type: 'message', + role: 'user', + status: 'completed', + content: [ + { + type: 'input_text', + text: `q-${i} ${'x'.repeat(pad)}`, + }, + ], + }), + ); + } +} + +/** Framework events of one type, unwrapped to their data payloads. */ +function frameworkData(events: StreamEvent[], type: string): Array> { + const matches: Array> = []; + for (const event of events) { + if (event.source !== 'framework' || !event.type.endsWith(`:${type}`)) { + continue; + } + assert(typeof event.data === 'object' && event.data !== null); + matches.push(frameworkCast>(event.data)); + } + return matches; +} + +//#endregion + +describe('executeCallModel — compaction folded into the request', () => { + it('a compaction record in the log shrinks what the model receives', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 10); + const beforeCount = rig.ctx.itemLog.items.length; + + const compaction = createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 8, + summary: 'the first eight turns, summarized', + }); + rig.ctx.itemLog.append(compactionAsItem(compaction)); + + await executeCallModel(STEP, '', rig.ctx, LAYERS); + + const sent = rig.request().items; + // The log GREW (compaction is append-only) but the view SHRANK. + expect(rig.ctx.itemLog.items.length).toBeGreaterThan(beforeCount); + expect(sent.length).toBeLessThan(beforeCount); + // The summary reached the model in place of the covered prefix... + const texts = requestTexts(sent); + expect(texts.some((t) => t.includes('the first eight turns, summarized'))).toBe(true); + expect(texts.some((t) => t.includes('q-0'))).toBe(false); + // ...and the uncompacted tail survived. + expect(texts.some((t) => t.includes('q-9'))).toBe(true); + // The raw record itself never goes to the provider. + expect(sent.some((i) => i.type === compaction.type)).toBe(false); + }); + + it('folds on the no-layers path too', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 6); + const compaction = createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 5, + summary: 'earlier turns', + }); + rig.ctx.itemLog.append(compactionAsItem(compaction)); + + await executeCallModel(STEP, '', rig.ctx); + + const sent = rig.request().items; + expect(sent.some((i) => i.type === compaction.type)).toBe(false); + const texts = requestTexts(sent); + expect(texts.some((t) => t.includes('earlier turns'))).toBe(true); + expect(texts.some((t) => t.includes('q-0'))).toBe(false); + }); + + it('leaves an uncompacted log untouched', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 4); + + await executeCallModel(STEP, '', rig.ctx, LAYERS); + + const texts = requestTexts(rig.request().items); + for (let i = 0; i < 4; i++) { + expect(texts.some((t) => t.includes(`q-${i}`))).toBe(true); + } + }); +}); + +describe('executeCallModel — the fold index survives the system/tail partition', () => { + /** + * `replacesUntil` indexes the RAW log. The layered path splits system and + * steering-tail items out of that log before assembling, so folding the + * shortened array applies the index to the wrong list — the cut lands one + * position further along per stripped item and eats live turns past the + * compaction boundary, silently. + */ + it('keeps every turn past replacesUntil when a system message precedes them', async () => { + const rig = makeRig(); + // Raw log: [SYSTEM, q-0..q-5, compaction(replacesUntil: 5)]. + seedSystem(rig.ctx, 'you are a helpful assistant'); + seedTurns(rig.ctx, 6); + const compaction = createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 5, + summary: 'the earliest turns, summarized', + }); + rig.ctx.itemLog.append(compactionAsItem(compaction)); + + await executeCallModel(STEP, '', rig.ctx, LAYERS); + + const texts = requestTexts(rig.request().items); + // replacesUntil 5 covers raw[0..4] = SYSTEM + q-0..q-3, so q-4 and q-5 live. + expect(texts.some((t) => t.includes('the earliest turns, summarized'))).toBe(true); + expect(texts.some((t) => t.includes('q-4'))).toBe(true); + expect(texts.some((t) => t.includes('q-5'))).toBe(true); + // The system prompt is never dropped, even though the fold covered it. + expect(texts.some((t) => t.includes('you are a helpful assistant'))).toBe(true); + // ...and the covered prefix really is gone. + expect(texts.some((t) => t.includes('q-0'))).toBe(false); + }); + + it('agrees with the bare no-layer path on the same log', async () => { + const seed = (rig: Rig): void => { + seedSystem(rig.ctx, 'system prompt'); + seedTurns(rig.ctx, 6); + rig.ctx.itemLog.append( + compactionAsItem( + createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 5, + summary: 'summary', + }), + ), + ); + }; + + const layered = makeRig(); + seed(layered); + await executeCallModel(STEP, '', layered.ctx, LAYERS); + + const bare = makeRig(); + seed(bare); + await executeCallModel(STEP, '', bare.ctx); + + // The same log must not yield a different history just because layers exist. + const layeredTexts = requestTexts(layered.request().items).filter((t) => t.startsWith('q-')); + const bareTexts = requestTexts(bare.request().items).filter((t) => t.startsWith('q-')); + expect(layeredTexts).toEqual(bareTexts); + }); + + it('holds with several system items ahead of the boundary', async () => { + const rig = makeRig(); + // Raw log: [SYS-a, SYS-b, SYS-c, q-0..q-5, compaction(replacesUntil: 6)]. + seedSystem(rig.ctx, 'rule-a'); + seedSystem(rig.ctx, 'rule-b'); + seedSystem(rig.ctx, 'rule-c'); + seedTurns(rig.ctx, 6); + rig.ctx.itemLog.append( + compactionAsItem( + createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 6, + summary: 'covered', + }), + ), + ); + + await executeCallModel(STEP, '', rig.ctx, LAYERS); + + const texts = requestTexts(rig.request().items); + // replacesUntil 6 covers raw[0..5] = 3 system items + q-0..q-2; q-3..q-5 live. + // Three stripped system items = three turns lost to the old index drift. + for (const live of [ + 'q-3', + 'q-4', + 'q-5', + ]) { + expect(texts.some((t) => t.includes(live))).toBe(true); + } + for (const rule of [ + 'rule-a', + 'rule-b', + 'rule-c', + ]) { + expect(texts.some((t) => t.includes(rule))).toBe(true); + } + expect(texts.some((t) => t.includes('q-2'))).toBe(false); + }); + + it('resolves stacked compactions against the raw index space', async () => { + const rig = makeRig(); + seedSystem(rig.ctx, 'system prompt'); + seedTurns(rig.ctx, 8); + // An early compaction, then a later one that subsumes it. + rig.ctx.itemLog.append( + compactionAsItem( + createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 3, + summary: 'first summary', + }), + ), + ); + rig.ctx.itemLog.append( + compactionAsItem( + createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 7, + summary: 'second summary', + }), + ), + ); + + await executeCallModel(STEP, '', rig.ctx, LAYERS); + + const texts = requestTexts(rig.request().items); + // The higher replacesUntil (7) wins: raw[0..6] = SYSTEM + q-0..q-5 covered, + // so q-6 and q-7 must survive. + expect(texts.some((t) => t.includes('second summary'))).toBe(true); + expect(texts.some((t) => t.includes('first summary'))).toBe(false); + expect(texts.some((t) => t.includes('q-6'))).toBe(true); + expect(texts.some((t) => t.includes('q-7'))).toBe(true); + expect(texts.some((t) => t.includes('q-5'))).toBe(false); + }); + + it('a steering retry replays the same history rather than a shorter one', async () => { + // Each Guide retry appends a guidance item and adds its id to tailIds, so a + // partition-then-fold order drops one more turn on every pass. + let calls = 0; + const views: string[][] = []; + const harness = makeMockHarness(); + harness.callModel = async (request) => { + calls++; + views.push(requestTexts(request.items).filter((t) => t.startsWith('q-'))); + return makeLLMResponse('done'); + }; + harness.afterModelCall = async () => + calls === 1 + ? { + action: SteeringAction.Guide, + guidance: 'try again', + } + : { + action: SteeringAction.Allow, + }; + const ctx = new ContextImpl({ + harness, + _broadcaster: new RecordingBroadcaster(), + }); + seedSystem(ctx, 'system prompt'); + seedTurns(ctx, 6); + ctx.itemLog.append( + compactionAsItem( + createCompaction({ + items: ctx.itemLog.items, + replacesUntil: 5, + summary: 'summary', + }), + ), + ); + + await executeCallModel(STEP, '', ctx, LAYERS); + + expect(calls).toBe(2); + // The retry is a replay of the same history, not a shortened view of it. + expect(views[1]).toEqual(views[0]); + expect(views[1].some((t) => t.startsWith('q-5'))).toBe(true); + }); +}); + +describe('executeCallModel — context_pressure event', () => { + // A budget tight enough that 40 padded turns cross the default 80% compactAt. + const tightPolicy: ProjectionPolicy = { + tokenBudget: 2_000, + responseReserve: 200, + overflow: 'sliding_window', + }; + const tightStep: StepCallModel = { + ...STEP, + projection: tightPolicy, + }; + + it('emits context_pressure when folded history crosses compactAt', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 40, 120); + + await executeCallModel(tightStep, '', rig.ctx, LAYERS); + + const emitted = frameworkData(rig.events, 'context_pressure'); + expect(emitted).toHaveLength(1); + expect(emitted[0].nodeId).toBe(tightStep.id); + const historyTokens = emitted[0].historyTokens; + const compactAt = emitted[0].compactAt; + assert(typeof historyTokens === 'number' && typeof compactAt === 'number'); + expect(historyTokens).toBeGreaterThan(compactAt); + }); + + it('stays silent under the threshold', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 2); + + await executeCallModel(tightStep, '', rig.ctx, LAYERS); + + expect(frameworkData(rig.events, 'context_pressure')).toHaveLength(0); + }); + + it('a compaction that relieves the pressure silences the event', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 40, 120); + const compaction = createCompaction({ + items: rig.ctx.itemLog.items, + replacesUntil: 39, + summary: 'everything before the last turn', + }); + rig.ctx.itemLog.append(compactionAsItem(compaction)); + + await executeCallModel(tightStep, '', rig.ctx, LAYERS); + + // Measured POST-fold, so relieving the pressure genuinely turns the signal + // off — otherwise an agent that compacted would keep being told to compact. + expect(frameworkData(rig.events, 'context_pressure')).toHaveLength(0); + }); + + it('still fires when the threshold is crossed on a steering retry', async () => { + // The once-per-step latch must close on an actual EMISSION, not on the first + // evaluation. A first assembly with room to spare that latched the flag would + // leave a retry which crosses compactAt to trim the oldest turns in silence. + let calls = 0; + const broadcaster = new RecordingBroadcaster(); + const harness = makeMockHarness(); + const ctx = new ContextImpl({ + harness, + _broadcaster: broadcaster, + }); + harness.callModel = async () => { + calls++; + return makeLLMResponse('done'); + }; + harness.afterModelCall = async () => { + if (calls > 1) { + return { + action: SteeringAction.Allow, + }; + } + // Between the two assemblies, grow the log past compactAt. + seedTurns(ctx, 40, 120); + return { + action: SteeringAction.Guide, + guidance: 'try again', + }; + }; + // First assembly: two short turns, comfortably under the threshold. + seedTurns(ctx, 2); + + await executeCallModel(tightStep, '', ctx, LAYERS); + + expect(calls).toBe(2); + const emitted = frameworkData(broadcaster.events, 'context_pressure'); + expect(emitted).toHaveLength(1); + const historyTokens = emitted[0].historyTokens; + const compactAt = emitted[0].compactAt; + assert(typeof historyTokens === 'number' && typeof compactAt === 'number'); + expect(historyTokens).toBeGreaterThan(compactAt); + }); + + it('honours step.emit === false', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 40, 120); + + await executeCallModel( + { + ...tightStep, + emit: false, + }, + '', + rig.ctx, + LAYERS, + ); + + expect(frameworkData(rig.events, 'context_pressure')).toHaveLength(0); + }); + + it('honours an emit predicate that filters the event out', async () => { + const rig = makeRig(); + seedTurns(rig.ctx, 40, 120); + const asked: string[] = []; + + await executeCallModel( + { + ...tightStep, + emit: (eventType) => { + asked.push(eventType); + return eventType !== 'context_pressure'; + }, + }, + '', + rig.ctx, + LAYERS, + ); + + expect(asked).toContain('context_pressure'); + expect(frameworkData(rig.events, 'context_pressure')).toHaveLength(0); + }); +}); diff --git a/packages/core/test/interpreter/execute-run.test.ts b/packages/core/test/interpreter/execute-run.test.ts index 4426fc0b..dc414fb6 100644 --- a/packages/core/test/interpreter/execute-run.test.ts +++ b/packages/core/test/interpreter/execute-run.test.ts @@ -1,46 +1,14 @@ -import { afterEach, describe, expect, it } from 'bun:test'; +import { describe, expect, it } from 'bun:test'; import assert from 'node:assert'; import type { ContextData } from '@noetic-tools/context'; -import type { Context, StepRunCode } from '@noetic-tools/types'; +import type { Context, RetryPolicy, StepRunCode } from '@noetic-tools/types'; import { isNoeticError, NoeticErrorImpl } from '@noetic-tools/types'; -import { executeRunCode } from '../../src/interpreter/execute-action'; +import { computeRetryDelay, executeRunCode } from '../../src/interpreter/execute-action'; import { ContextImpl } from '../../src/runtime/context-impl'; import { makeMockContext, makeMockHarness } from '../_helpers'; const mockCtx: Context = makeMockContext(); -// Safety net: ensure setTimeout is always restored -const _originalSetTimeout = globalThis.setTimeout; -afterEach(() => { - globalThis.setTimeout = _originalSetTimeout; -}); - -/** Patch setTimeout to capture delay values and execute callbacks instantly. */ -function interceptDelays(): { - delays: number[]; - restore: () => void; -} { - const delays: number[] = []; - // Wrap the original to intercept delay values while preserving the full overloaded signature - const handler: ProxyHandler = { - apply(_target, thisArg, argsList: unknown[]) { - const delay = argsList[1]; - if (typeof delay === 'number' && delay > 0) { - delays.push(delay); - } - argsList[1] = 1; - return Reflect.apply(_originalSetTimeout, thisArg, argsList); - }, - }; - globalThis.setTimeout = new Proxy(_originalSetTimeout, handler); - return { - delays, - restore: () => { - globalThis.setTimeout = _originalSetTimeout; - }, - }; -} - describe('executeRunCode', () => { it('calls execute function and returns output', async () => { const s: StepRunCode = { @@ -88,105 +56,81 @@ describe('executeRunCode', () => { }); it('retries with fixed backoff', async () => { - const { delays, restore } = interceptDelays(); - let attempts = 0; - const s: StepRunCode = { + const retry: RetryPolicy = { + maxAttempts: 3, + backoff: 'fixed', + initialDelay: 1, + }; + const step: StepRunCode = { kind: 'runCode', id: 'retry-test', - execute: async (_input) => { + retry, + execute: async () => { attempts++; if (attempts < 3) { throw new Error('not yet'); } return 'success'; }, - retry: { - maxAttempts: 3, - backoff: 'fixed', - initialDelay: 10, - }, }; - try { - const result = await executeRunCode(s, 'test', mockCtx); - expect(result).toBe('success'); - expect(attempts).toBe(3); - // Fixed backoff: all delays should be 10 - expect(delays).toEqual([ - 10, - 10, - ]); - } finally { - restore(); - } + expect(await executeRunCode(step, 'test', mockCtx)).toBe('success'); + expect(attempts).toBe(3); + expect( + [ + 0, + 1, + ].map((attempt) => computeRetryDelay(retry, attempt)), + ).toEqual([ + 1, + 1, + ]); }); it('retries with exponential backoff and exhausts', async () => { - const { delays, restore } = interceptDelays(); let attempts = 0; - const s: StepRunCode = { + const retry: RetryPolicy = { + maxAttempts: 3, + backoff: 'exponential', + initialDelay: 1, + }; + const step: StepRunCode = { kind: 'runCode', id: 'exhaust-test', + retry, execute: async () => { attempts++; throw new Error('always fails'); }, - retry: { - maxAttempts: 3, - backoff: 'exponential', - initialDelay: 10, - }, }; - try { - await executeRunCode(s, 'test', mockCtx); - expect.unreachable('should have thrown'); - } catch (e) { - assert(isNoeticError(e)); - const oe = e.noeticError; - assert(oe.kind === 'step_failed'); - expect(oe.retriesExhausted).toBe(true); - expect(attempts).toBe(3); - expect(delays).toEqual([ - 10, - 20, - ]); - } finally { - restore(); - } + await expect(executeRunCode(step, 'test', mockCtx)).rejects.toThrow('always fails'); + expect(attempts).toBe(3); + expect( + [ + 0, + 1, + ].map((attempt) => computeRetryDelay(retry, attempt)), + ).toEqual([ + 1, + 2, + ]); }); - it('caps exponential backoff delay at maxDelay', async () => { - const { delays, restore } = interceptDelays(); - - let attempts = 0; - const s: StepRunCode = { - kind: 'runCode', - id: 'cap-test', - execute: async () => { - attempts++; - if (attempts < 5) { - throw new Error('fail'); - } - return 'ok'; - }, - retry: { - maxAttempts: 5, - backoff: 'exponential', - initialDelay: 100, - maxDelay: 500, - }, + it('caps exponential backoff delay at maxDelay', () => { + const retry: RetryPolicy = { + maxAttempts: 5, + backoff: 'exponential', + initialDelay: 100, + maxDelay: 500, }; - try { - await executeRunCode(s, 'test', mockCtx); - } finally { - restore(); - } - // Delays: 100, 200, 400, 500 (capped from 800) - for (const d of delays) { - expect(d).toBeLessThanOrEqual(500); - } - expect(delays.length).toBeGreaterThan(0); - expect(delays).toEqual([ + expect( + [ + 0, + 1, + 2, + 3, + ].map((attempt) => computeRetryDelay(retry, attempt)), + ).toEqual([ 100, 200, 400, @@ -194,46 +138,26 @@ describe('executeRunCode', () => { ]); }); - it('defaults maxDelay to 30000', async () => { - // With exponential backoff, delay = 100 * 2^attempt - // For attempt 9: 100 * 512 = 51200, should be capped at 30000 - const { delays, restore } = interceptDelays(); - - let attempts = 0; - const s: StepRunCode = { - kind: 'runCode', - id: 'default-cap-test', - execute: async () => { - attempts++; - if (attempts < 11) { - throw new Error('fail'); - } - return 'ok'; - }, - retry: { - maxAttempts: 11, - backoff: 'exponential', - initialDelay: 100, - }, + it('defaults maxDelay to 30000', () => { + const retry: RetryPolicy = { + maxAttempts: 11, + backoff: 'exponential', + initialDelay: 100, }; - try { - await executeRunCode(s, 'test', mockCtx); - } finally { - restore(); - } - for (const d of delays) { - expect(d).toBeLessThanOrEqual(30_000); - } - expect(delays.length).toBeGreaterThan(0); + expect(computeRetryDelay(retry, 9)).toBe(30_000); }); it('retries with linear backoff', async () => { - const { delays, restore } = interceptDelays(); - let attempts = 0; - const s: StepRunCode = { + const retry: RetryPolicy = { + maxAttempts: 3, + backoff: 'linear', + initialDelay: 1, + }; + const step: StepRunCode = { kind: 'runCode', id: 'linear-test', + retry, execute: async () => { attempts++; if (attempts < 3) { @@ -241,24 +165,18 @@ describe('executeRunCode', () => { } return 'ok'; }, - retry: { - maxAttempts: 3, - backoff: 'linear', - initialDelay: 10, - }, }; - try { - const result = await executeRunCode(s, 'test', mockCtx); - expect(result).toBe('ok'); - expect(attempts).toBe(3); - // Linear backoff: delay = initialDelay * attempt - expect(delays).toEqual([ - 10, - 20, - ]); - } finally { - restore(); - } + expect(await executeRunCode(step, 'test', mockCtx)).toBe('ok'); + expect(attempts).toBe(3); + expect( + [ + 0, + 1, + ].map((attempt) => computeRetryDelay(retry, attempt)), + ).toEqual([ + 1, + 2, + ]); }); describe('cancellation (not retriable)', () => { diff --git a/packages/types/src/schemas/item.ts b/packages/types/src/schemas/item.ts index 3d38cb30..3f040e93 100644 --- a/packages/types/src/schemas/item.ts +++ b/packages/types/src/schemas/item.ts @@ -21,6 +21,7 @@ import { z } from 'zod'; import { NoeticErrorImpl } from '../errors/noetic-error'; import type { Item, ItemSchemaExtensions } from '../types/items'; +import { COMPACTION_ITEM_TYPE } from '../types/items'; function isItemLike(value: unknown): value is Item { if (typeof value !== 'object' || value === null) { @@ -100,6 +101,16 @@ function relevantCategories(value: unknown): ItemSchemaCategory[] { return categories; } +const CompactionItemSchema = z.object({ + id: z.string().min(1), + type: z.literal(COMPACTION_ITEM_TYPE), + status: z.literal('completed'), + replacesUntil: z.number().int().nonnegative(), + summary: z.string(), + replacedCount: z.number().int().nonnegative(), + tokensSaved: z.number().nonnegative().optional(), +}); + function isKnownBaseType(type: string): boolean { return ( type === 'message' || @@ -109,6 +120,12 @@ function isKnownBaseType(type: string): boolean { type === 'web_search_call' || type === 'file_search_call' || type === 'image_generation_call' || + // A compaction record is a framework item, not a provider item: it is + // deliberately outside the `Item` union (nothing renders it directly — the + // projector folds it), but it MUST be appendable to a real item log or + // compaction cannot survive a checkpoint, which is the whole point of + // recording it in the log rather than in engine state. + type === COMPACTION_ITEM_TYPE || type.startsWith('openrouter:') ); } @@ -153,6 +170,10 @@ export class ItemSchemaRegistry { parse(value: unknown): Item { const base = ItemSchema.parse(value); + if (base.type === COMPACTION_ITEM_TYPE) { + CompactionItemSchema.parse(value); + return base; + } const categories = relevantCategories(base); const schemas = categories.flatMap((category) => schemasForCategory(this.extensions, category)); diff --git a/packages/types/src/types/context-layer.ts b/packages/types/src/types/context-layer.ts index 645e18d3..62e0a790 100644 --- a/packages/types/src/types/context-layer.ts +++ b/packages/types/src/types/context-layer.ts @@ -477,4 +477,14 @@ export interface ProjectionPolicy { overflow: 'truncate' | 'summarize' | 'sliding_window'; overflowModel?: string; windowSize?: number; + /** + * History-token threshold that arms compaction. When the folded history + * exceeds this, `historyPressure` reports `overThreshold` so an agent (or + * the host app) can compact instead of letting the assembler silently drop + * the oldest turns. Compaction itself stays explicit: record a + * `CompactionItem` in the log (see `createCompaction` / `compactHistory`). + * + * Defaults to 80% of `tokenBudget - responseReserve` when omitted. + */ + compactAt?: number; } diff --git a/packages/types/src/types/items.ts b/packages/types/src/types/items.ts index 307dc99c..992b7e92 100644 --- a/packages/types/src/types/items.ts +++ b/packages/types/src/types/items.ts @@ -152,6 +152,33 @@ export interface FunctionCallOutputItem { /** @public Item produced by an extension schema registered by a tool, context layer, or harness. */ export type ExtensionItem = ItemBase & Record; +/** @public Item type identifying a recorded history compaction. */ +export const COMPACTION_ITEM_TYPE = 'noetic:compaction' as const; + +/** + * @public A recorded compaction: a summary that REPLACES a contiguous prefix of + * conversation history. Lives in the item log like any other item, so + * compaction survives checkpoints/resume for free and forked logs share it. + * + * `replacesUntil` is the item-log index (exclusive) of the last item the + * summary covers. Folding the log (`foldCompactions`) drops items before that + * index and renders the summary in their place. Compactions may stack: a later + * compaction with a higher `replacesUntil` subsumes an earlier one. + */ +export interface CompactionItem { + readonly id: string; + readonly type: typeof COMPACTION_ITEM_TYPE; + readonly status: 'completed'; + /** Item-log index (exclusive) up to which history is replaced by `summary`. */ + readonly replacesUntil: number; + /** The compacted summary of the replaced items. */ + readonly summary: string; + /** Count of items replaced, for diagnostics. */ + readonly replacedCount: number; + /** Estimated tokens saved by this compaction, for diagnostics. */ + readonly tokensSaved?: number; +} + /** @public Developer-role message item refined by a context-layer extension schema. */ export type DeveloperMessageExtensionItem = InputMessageItem & { readonly role: 'developer'; diff --git a/packages/web/content/docs/framework/api/context-layer-types.mdx b/packages/web/content/docs/framework/api/context-layer-types.mdx index 35667f1a..50be98b8 100644 --- a/packages/web/content/docs/framework/api/context-layer-types.mdx +++ b/packages/web/content/docs/framework/api/context-layer-types.mdx @@ -70,7 +70,7 @@ type BudgetConfig = | 'auto'; ``` -An **omitted** `budget` behaves like `'auto'`: the layer has infinite headroom and splits the proportional pool with the other auto layers after finite layers take their share. +An **omitted** `budget` behaves like `'auto'`: the layer gets the same fixed 2000-token default cap as an explicit `'auto'` budget. Declare a number or `{ min, max }` when a layer needs more. ## Slot Constants @@ -284,6 +284,7 @@ interface ProjectionPolicy { overflow: 'truncate' | 'summarize' | 'sliding_window'; overflowModel?: string; windowSize?: number; + compactAt?: number; } ``` @@ -294,6 +295,7 @@ interface ProjectionPolicy { | `overflow` | `'truncate' \| 'summarize' \| 'sliding_window'` | yes | Strategy when total recall exceeds budget | | `overflowModel` | `string` | no | Model for summarization overflow | | `windowSize` | `number` | no | Items to keep for sliding window overflow | +| `compactAt` | `number` | no | Folded-history token threshold that arms compaction; `historyPressure` reports `overThreshold` above it. Defaults to 80% of `tokenBudget − responseReserve` | ## ContextCacheConfig diff --git a/packages/web/content/docs/framework/api/items-and-events.mdx b/packages/web/content/docs/framework/api/items-and-events.mdx index 61c982f8..75750ee1 100644 --- a/packages/web/content/docs/framework/api/items-and-events.mdx +++ b/packages/web/content/docs/framework/api/items-and-events.mdx @@ -254,6 +254,31 @@ interface FunctionCallOutputItem { | `output` | `string` | JSON-encoded tool output | | `status` | `string` | Lifecycle status. Includes `failed` as a Noetic extension. | +### CompactionItem + +A recorded history compaction: a summary that replaces a contiguous prefix of conversation history. Deliberately **outside** the `Item` union — nothing renders it directly; the projector folds it (`foldCompactions`). It is registered with the strict item-schema registry so it can live in the `ItemLog`, which is what lets compaction survive checkpoint/resume and be shared by forked logs. See [History](../context-layers/history.mdx#compaction). + +```ts validate=none +const COMPACTION_ITEM_TYPE = 'noetic:compaction'; + +interface CompactionItem { + readonly id: string; + readonly type: typeof COMPACTION_ITEM_TYPE; + readonly status: 'completed'; + readonly replacesUntil: number; + readonly summary: string; + readonly replacedCount: number; + readonly tokensSaved?: number; +} +``` + +| Field | Type | Description | +|---|---|---| +| `replacesUntil` | `number` | Item-log index (exclusive) up to which history is replaced by `summary` | +| `summary` | `string` | The compacted summary of the replaced items | +| `replacedCount` | `number` | Count of items replaced (diagnostics) | +| `tokensSaved` | `number` | Estimated tokens saved, floored at 0 (diagnostics, optional) | + ### Item Union All item types that can appear in an `ItemLog`: @@ -412,6 +437,7 @@ In addition to OpenResponses streaming events, Noetic emits framework-level even | `{name}:tool_call_started` | Before each tool call | `name`, `callId` | | `{name}:tool_call_completed` | After each tool call | `name`, `callId`, `error` | | `{name}:tool_round_completed` | After all tool calls in a round | `round`, `toolCount` | +| `{name}:context_pressure` | Folded history crossed the projection policy's `compactAt` threshold — emitted once per `callModel` step execution, only on a real crossing | `nodeId`, `historyTokens`, `compactAt` | Framework events are available through `harness.execute(input).getFullStream()` alongside SDK events. They can be distinguished by the `source` field: diff --git a/packages/web/content/docs/framework/context-layers/history.mdx b/packages/web/content/docs/framework/context-layers/history.mdx index 6ceedf28..a7bdaa9a 100644 --- a/packages/web/content/docs/framework/context-layers/history.mdx +++ b/packages/web/content/docs/framework/context-layers/history.mdx @@ -50,6 +50,34 @@ The projection runs against `ctx.itemLog.items` but never mutates it. As a resul Within a single `callModel` invocation's tool loop, that round's own `function_call` and `function_call_output` items accumulate in the wire payload until the call returns. The cap fires at **turn boundaries**, not mid-call — capping mid-flight would corrupt the active tool flow. +## Compaction + +Capping bounds item *count*; it cannot preserve what falls outside the window. A **recorded compaction** is the explicit alternative: a `CompactionItem` appended to the log declares "the first `replacesUntil` items are summarized by `summary`". The log stays append-only (checkpoints, forks, and audits keep the full record) while the folded view the model sees collapses the replaced prefix to a `` developer message. + +The helpers live in `@noetic-tools/core` (re-exported from `@noetic-tools/context`): + +- `foldCompactions(items)` — project the model view from the raw log. Highest `replacesUntil` wins; compaction records never reach a provider; the fold seam strips orphan tool calls. Run it on history before `assembleView` — and before this layer's cap — so a compaction genuinely shrinks what has to fit. +- `hasCompaction(items)` — cheap check to skip the fold on the common no-compaction path. +- `historyPressure(historyItems, policy)` — measures the **folded** history against `policy.compactAt` (default 80% of `tokenBudget − responseReserve`), so writing a compaction genuinely relieves pressure. +- `createCompaction({ items, replacesUntil, summary })` / `compactHistory({ log, keepRecent, summarize })` — build the record. The caller supplies the summary (an LLM step, a heuristic, a digest) and appends the record; the projector never mutates a log. +- `compactionAsItem(compaction)` — the sanctioned bridge to `ctx.itemLog.append(...)`. + +The runtime folds for you: every model request path (`callModel` assembly and `previewRequestItems`) folds compactions out of history **before** the system/history partition and before `assembleView` claims the budget, so a recorded compaction demonstrably shrinks what the model receives while the raw log stays intact. When the folded history still exceeds `compactAt`, the step emits one `context_pressure` framework event (`{nodeId, historyTokens, compactAt}`) per execution — the signal to record a compaction instead of letting the assembler silently drop the oldest turns. + +```ts validate=none +import { compactHistory, compactionAsItem, historyPressure } from '@noetic-tools/core'; + +const pressure = historyPressure(ctx.itemLog.items, policy); +if (pressure.overThreshold) { + const compaction = await compactHistory({ + log: ctx.itemLog.items, + keepRecent: 20, + summarize: (replaced) => summarizeSomehow(replaced), // your LLM step or heuristic + }); + if (compaction) ctx.itemLog.append(compactionAsItem(compaction)); +} +``` + ## CLI configuration The CLI installs `history` only when `AgentConfig.history.maxItems` is set. When unset, history is uncapped (the pre-existing default behaviour). Configure it via: diff --git a/packages/web/content/docs/framework/context-layers/index.mdx b/packages/web/content/docs/framework/context-layers/index.mdx index 8fc70595..46a0a046 100644 --- a/packages/web/content/docs/framework/context-layers/index.mdx +++ b/packages/web/content/docs/framework/context-layers/index.mdx @@ -85,7 +85,7 @@ type BudgetConfig = | 'auto'; // let the runtime decide ``` -When using a `{ min, max }` range, the budget allocator guarantees at least `min` tokens and distributes remaining capacity up to `max`. An omitted `budget` behaves like `'auto'` — the layer splits the proportional pool with the other auto layers after finite layers take their share. The pool is conserved: finite shares plus the auto layers' split always account for the full layer pool. NaN budget inputs throw a `NoeticConfigError` (`INVALID_BUDGET_INPUT`); `Infinity` means uncapped. +When using a `{ min, max }` range, the budget allocator guarantees at least `min` tokens and distributes discretionary capacity up to `max`. Those minimums are satisfied from the full available window first; only the remainder is rationed. An omitted `budget` behaves like `'auto'` and gets the same fixed 2000-token default cap. NaN budget inputs throw a `NoeticConfigError` (`INVALID_BUDGET_INPUT`); explicit `Infinity` still means uncapped. ## ProjectionPolicy diff --git a/specs/07-context-and-event-log.md b/specs/07-context-and-event-log.md index 175990b3..7bfa4441 100644 --- a/specs/07-context-and-event-log.md +++ b/specs/07-context-and-event-log.md @@ -1,7 +1,7 @@ # Context and Item Log > **Depends On:** `06-channels` (Channel — for send/recv/tryRecv signatures), `08-runtime` (AgentHarness — for harness reference), `10-observability` (Span) -> **Exports:** `Context`, `ItemLog`, `Item`, `OutputItem`, `MessageItem`, `FunctionCallItem`, `FunctionCallOutputItem`, `ReasoningItem`, `WebSearchItem`, `FileSearchItem`, `ImageGenerationItem`, `ServerToolItem`, `InputMessageItem`, `ContentPart`, `OutputTextPart`, `RefusalPart`, `InputTextPart`, `InputImagePart`, `InputFilePart`, `InputContentPart`, `ReasoningTextPart`, `SummaryTextPart`, `StepMeta`, `TokenUsage`, `LLMResponse`, `LayerUsageEntry`, `LastLayerUsage` +> **Exports:** `Context`, `ItemLog`, `Item`, `OutputItem`, `MessageItem`, `FunctionCallItem`, `FunctionCallOutputItem`, `ReasoningItem`, `WebSearchItem`, `FileSearchItem`, `ImageGenerationItem`, `ServerToolItem`, `InputMessageItem`, `ContentPart`, `OutputTextPart`, `RefusalPart`, `InputTextPart`, `InputImagePart`, `InputFilePart`, `InputContentPart`, `ReasoningTextPart`, `SummaryTextPart`, `CompactionItem`, `COMPACTION_ITEM_TYPE`, `StepMeta`, `TokenUsage`, `LLMResponse`, `LayerUsageEntry`, `LastLayerUsage` --- @@ -221,6 +221,28 @@ type Item = OutputItem | InputMessageItem | FunctionCallOutputItem; Server tool items use the `prefix:name` convention established by the OpenResponses `ResponsesServerToolOutput` type. The prefix identifies the vendor or domain (e.g., `openrouter`, `noetic`, `myapp`) and the name identifies the specific item kind. None of the standard item types contain a colon, so the presence of `:` in the type string cleanly distinguishes server tool outputs from standard items. +### Framework-owned Records: `CompactionItem` + +```typescript +const COMPACTION_ITEM_TYPE = 'noetic:compaction'; + +interface CompactionItem { + readonly id: string; + readonly type: typeof COMPACTION_ITEM_TYPE; + readonly status: 'completed'; + /** Item-log index (exclusive) up to which history is replaced by `summary`. */ + readonly replacesUntil: number; + /** The compacted summary of the replaced items. */ + readonly summary: string; + /** Count of items replaced, for diagnostics. */ + readonly replacedCount: number; + /** Estimated tokens saved by this compaction, for diagnostics. */ + readonly tokensSaved?: number; +} +``` + +A compaction record is a framework item, deliberately **outside** the `Item` union — nothing renders it directly; the projector folds it (see spec 11, *History Compaction*). It is registered with the strict item-schema registry (`isKnownBaseType`) so it can be appended to a real `ItemLog` — recording compaction in the log rather than in engine state is what makes it survive checkpoint/resume and shared by forked logs. The allowance is a single-type exception: other unknown `noetic:*` types are still rejected. + --- ## `StepMeta` diff --git a/specs/08-runtime.md b/specs/08-runtime.md index aa92927e..c35e92b7 100644 --- a/specs/08-runtime.md +++ b/specs/08-runtime.md @@ -511,6 +511,7 @@ interface FrameworkStreamEvent { | `{name}:model_call_completed` | Emitted after the provider's response is fully received. Data: `{ round, itemCount }` | | `{name}:model_call_stalled` | Emitted when the stream-idle watchdog fires (fixed 120s idle window). The turn then aborts. Data: `{ round, idleTimeoutMs }` | | `{name}:stream_pipe_error` | Emitted when SDK stream piping fails. Data: `{ error }` | +| `{name}:context_pressure` | Emitted when the folded history crosses the projection policy's `compactAt` threshold during `callModel` assembly — the signal to record a compaction (spec 11, *History Compaction*) before the assembler silently drops the oldest turns. Emitted at most once per step execution, and only on an actual threshold crossing: an assembly under the threshold does not disarm the event for a later steering retry that crosses it. Data: `{ nodeId, historyTokens, compactAt }` | ### Streaming Scope @@ -571,7 +572,7 @@ type DeliveryMode = 'next-turn' | 'between-rounds' | 'interrupt'; - **`getChannelHandle`** returns a `ChannelHandle` for external code to write into a running execution. The handle is typed, lifecycle-aware, and scoped to the root execution. External handles route to the correct execution via `executionId`. `AgentHarness` uses in-process handles; `DurableAgentHarness` translates to durable signals (e.g., Temporal signals, Inngest events). - **`getChannelStream`** is the read-side counterpart: an `AsyncIterable` over values the agent sends on an external channel. Delivery is channel-scoped; `executionId` bounds only the subscription's lifetime (it ends when that root execution completes, and a never-run id yields a harness-lifetime stream). Per-mode delivery and lifecycle semantics are specified in `06-channels` (External Subscriptions). - **Context layer methods** manage the full lifecycle defined in `11-context-layer-system`. `initLayers` runs `init()` sequentially. `recallLayers` runs `recall()` in slot order and returns `Item[]`. `storeLayers` runs `store()` concurrently via `Promise.allSettled` and receives `LLMResponse` (with items + usage). `disposeLayers` runs `dispose()` in reverse order. Error handling follows the per-hook policy. -- **`previewRequestItems(scope?)`** returns the `Item[]` that would be sent to the model on the next turn for `scope.threadId` (or the default thread): the session's accumulated history with harness-level context-layer recall outputs prepended via `assembleView`. Runs layer `init` on a throwaway preview context first — the same hydration the next real turn would perform, re-loading thread/resource-scoped state from storage — because the recall lifecycle skips init-bearing layers that were never initialized. The preview execution's layer-state entries are flushed and dropped afterwards, so repeated previews neither grow the store nor leak into real turns. Does not allocate a session for unknown thread ids — returns an empty history in that case. +- **`previewRequestItems(scope?)`** returns the `Item[]` that would be sent to the model on the next turn for `scope.threadId` (or the default thread): the session's accumulated history — compactions folded, exactly as the real turn folds them — with harness-level context-layer recall outputs prepended via `assembleView`. Runs layer `init` on a throwaway preview context first — the same hydration the next real turn would perform, re-loading thread/resource-scoped state from storage — because the recall lifecycle skips init-bearing layers that were never initialized. The preview execution's layer-state entries are flushed and dropped afterwards, so repeated previews neither grow the store nor leak into real turns. Does not allocate a session for unknown thread ids — returns an empty history in that case. - **`beforeToolCall(layers, toolName, toolArgs, ctx)`** runs each layer's `beforeToolCall` hook sequentially in slot order before a tool is executed. Returns a `SteeringDecision` — `Allow` proceeds normally, `Deny` short-circuits and blocks the tool call, `Guide` returns guidance text to the model. Short-circuits on the first `Deny`. When multiple layers return `Guide`, their guidance is concatenated. - **`afterModelCall(layers, response, ctx)`** runs each layer's `afterModelCall` hook sequentially in slot order immediately after the LLM responds. Returns a `SteeringDecision` — `Allow` proceeds normally, `Deny` throws `steering_denied`, `Guide` injects guidance as a developer message and retries the model call (up to 3 times). Short-circuits on the first `Deny`. - **`fs`** exposes the `FsAdapter` that the harness was constructed with via `environment.fs` (or the default in-memory adapter). All filesystem operations — CLI tools (read, write, edit, ls, grep, find), skill discovery, and context layers — use `ctx.harness.fs` rather than importing `fs/promises` directly. This enables sandboxed or virtualized filesystems (e.g., in-memory FS for testing, remote FS for cloud execution). `Context` exposes a `readonly fs: FsAdapter` getter that delegates to the harness. `ToolExecutionContext` and memory `ExecutionContext` also expose `readonly fs: FsAdapter`. diff --git a/specs/11-context-layer-system.md b/specs/11-context-layer-system.md index 25d88761..7e32e275 100644 --- a/specs/11-context-layer-system.md +++ b/specs/11-context-layer-system.md @@ -2,7 +2,7 @@ > **Module:** `@noetic-tools/context` (source at `packages/context/src/**`); the `ContextLayer` contract is owned by `@noetic-tools/types` (`packages/types/src/types/context-layer.ts`, also at the `@noetic-tools/types/contract` subpath). Both are re-exported by `@noetic-tools/core`. > **Depends On:** `07-context-and-event-log` (ItemLog, Item — type import only), `10-observability` (LayerTraceSpan, trace conventions), `04-spawn` (SpawnOpts — referenced in SpawnParams) -> **Exports:** `ContextLayer`, `ContextLayerHooks`, `ContextScope`, `BudgetConfig`, `Slot`, `InitParams`, `InitResult`, `RecallParams`, `RecallResult`, `StoreParams`, `StoreResult`, `SpawnParams`, `SpawnResult`, `ReturnParams`, `ReturnResult`, `CompleteParams`, `DisposeParams`, `BeforeToolCallParams`, `BeforeToolCallResult`, `AfterModelCallParams`, `AfterModelCallResult`, `OnItemAppendParams`, `OnItemAppendResult`, `RerenderScope`, `ParentUpdateParams`, `ParentUpdateResult`, `ExecutionOutcome`, `ExecutionContext`, `ScopedStorage`, `StorageAdapter`, `ProjectionPolicy`, `LayerTimeouts`, `LayerProvides`, `LayerDataDecl`, `LayerFunctionDecl`, `ContextConfig`, `InferContext`, `InferContextShape`, `layerData`, `layerFunction`, `context`, `storageGetMany`, `LayerPlacement`, `RenderDeltaParams`, `ContextCacheConfig`, `ContextCacheStore`, `ContextEpoch`, `AnchorPin`, `LayerChurn`, `ReanchorReason` +> **Exports:** `ContextLayer`, `ContextLayerHooks`, `ContextScope`, `BudgetConfig`, `Slot`, `InitParams`, `InitResult`, `RecallParams`, `RecallResult`, `StoreParams`, `StoreResult`, `SpawnParams`, `SpawnResult`, `ReturnParams`, `ReturnResult`, `CompleteParams`, `DisposeParams`, `BeforeToolCallParams`, `BeforeToolCallResult`, `AfterModelCallParams`, `AfterModelCallResult`, `OnItemAppendParams`, `OnItemAppendResult`, `RerenderScope`, `ParentUpdateParams`, `ParentUpdateResult`, `ExecutionOutcome`, `ExecutionContext`, `ScopedStorage`, `StorageAdapter`, `ProjectionPolicy`, `LayerTimeouts`, `LayerProvides`, `LayerDataDecl`, `LayerFunctionDecl`, `ContextConfig`, `InferContext`, `InferContextShape`, `layerData`, `layerFunction`, `context`, `storageGetMany`, `LayerPlacement`, `RenderDeltaParams`, `ContextCacheConfig`, `ContextCacheStore`, `ContextEpoch`, `AnchorPin`, `LayerChurn`, `ReanchorReason`, `foldCompactions`, `hasCompaction`, `historyPressure`, `HistoryPressure`, `createCompaction`, `CreateCompactionParams`, `compactHistory`, `CompactHistoryParams`, `compactionAsItem` ## Module Boundary @@ -14,7 +14,7 @@ The context layer system lives in `@noetic-tools/context` (`packages/context/src | `ContextScope`, `ScopedStorage`, `StorageAdapter` | Projector (View assembly algorithm) in `projector.ts` | | `BudgetConfig`, `Slot` | budget algorithm; `allocateBudgets` in `budget.ts` | | `ExecutionContext` (layer-facing read-only view) | built-in layer factories under `context/layers/` | -| `ProjectionPolicy` | Projector implementation in `projector.ts` | +| `ProjectionPolicy` | Projector implementation in `projector.ts` (incl. the compaction helpers: `foldCompactions`, `historyPressure`, `createCompaction`, `compactHistory`, `hasCompaction`, `compactionAsItem`) | `Context` (the full execution object) lives in `@noetic-tools/core`'s `runtime/`; the `contextToExecCtx` mapping (Context → ExecutionContext) bridges core to the context contract. @@ -428,41 +428,37 @@ function allocateBudgets(opts: { totalBudget: number; // policy.tokenBudget systemPromptTokens: number; responseReserve: number; // policy.responseReserve -}): { allocations: { layerId: string; allocated: number }[]; historyBudget: number } { +}): { allocations: { layerId: string; allocated: number }[] } { // Input validation: NaN in totalBudget/systemPromptTokens/responseReserve // throws NoeticConfigError (code INVALID_BUDGET_INPUT). Infinity is allowed // (= uncapped budget); fractional values are accepted. const available = opts.totalBudget - opts.responseReserve - opts.systemPromptTokens; if (available <= 0) { - // Every layer gets 0; history gets 0. + // Every layer gets 0. } - // Phase 1: satisfy each layer's minimum first. - let remaining = available; - for (const layer of opts.layers) { - const min = extractMin(layer.budget); // {min,max}.min, else 0 - allocate(layer.id, min); - remaining -= min; - } + // Phase 1: satisfy each layer's minimum first, from the FULL available + // window. Scale down proportionally only when the mins alone overcommit it. - // Phase 2: distribute a proportional pool above the minimums. - // 60% of what remains funds the layers (by headroom = max - min, - // where 'auto'/undefined max is +Infinity), 40% is reserved for history. - const layerPool = remaining * 0.6; - const historyBudget = remaining * 0.4; - // each layer's share is its headroom proportion of layerPool, clamped to headroom + // Phase 2: distribute only the discretionary remainder. + // The remainder pool is min(available * 0.25, available - totalMin). + // Each layer's share is proportional to headroom = cap - min, clamped to + // cap. 'auto' and omitted budgets use a fixed 2000-token cap. Explicit + // Infinity caps still absorb whatever finite layers leave behind. } ``` -- **Minimums are satisfied first**, in array order. -- The remaining budget is split: **60% into a proportional pool** distributed across layers by headroom (`max − min`; `'auto'`/`undefined` budgets have infinite headroom and split the pool among themselves after finite layers take their share), and **40% reserved for conversation history** (`historyBudget`). -- **The pool is conserved.** Finite shares are single-priced: each finite layer's share (in a mixed finite/infinite population, `min(headroom, half-pool proportional)`) is computed once, and the infinite-headroom layers split exactly `layerPool − Σ finiteShare`. No part of the pool is silently lost. -- A layer's final allocation never exceeds its `max`. +- **Minimums are satisfied first from the full available window** (`totalBudget − responseReserve − systemPromptTokens`), not from the discretionary pool. Declaring `{ min: 10000, max: 12000 }` means "this block needs 10k to be coherent," and the allocator honors that floor whenever the window can. +- **Declared mins scale down proportionally only when they alone overcommit the available window.** In that case, nothing else is distributed. +- **Only the discretionary remainder is rationed.** The allocator offers layers at most **25% of the available window** above their floors, capped again by what the mins left behind: `min(available × 0.25, available − totalMin)`. +- **`'auto'` and omitted budgets are deterministic.** They use a fixed **2000-token cap**, not infinite headroom. Layers that need more must declare it explicitly. +- **Explicit `Infinity` caps remain uncapped.** In a mixed finite/uncapped set, finite layers take their proportional/clamped share and uncapped layers split the remainder. +- A layer's final allocation never exceeds its cap. - **Input contract.** `totalBudget`, `systemPromptTokens`, and `responseReserve` MUST NOT be NaN — the allocator throws `NoeticConfigError` (code `INVALID_BUDGET_INPUT`). `Infinity` is a coherent "uncapped" budget and is accepted; fractional values are accepted. ### Budget Yielding -When `recall()` returns `tokenCount` less than allocated, the difference goes to conversation history. The Projector MUST NOT reallocate to other layers (prevents cascading re-recalls). +When `recall()` returns `tokenCount` less than allocated, the difference simply remains available to history when `assembleView` fits the final prompt. The Projector MUST NOT reallocate it to other layers (prevents cascading re-recalls). ### Budget Verification @@ -758,6 +754,8 @@ interface ProjectionPolicy { overflow: 'truncate' | 'summarize' | 'sliding_window'; overflowModel?: string; windowSize?: number; + compactAt?: number; // folded-history token threshold that arms compaction; + // defaults to 80% of (tokenBudget - responseReserve) } ``` @@ -774,6 +772,19 @@ interface ProjectionPolicy { 7. Result is Item[] — directly passable to the LLM provider ``` +### History Compaction + +The item log is append-only; a **compaction** is an ordinary logged item (a `CompactionItem`, spec 07) declaring "the first `replacesUntil` items are summarized by `summary`". Folding keeps the log immutable — checkpoints, forks, and audits see the full record — while the model sees `[summary, ...items after replacesUntil]`. The projector owns the pure helpers; recording a compaction is always an explicit, caller-driven decision, never something the projector does behind the runtime's back. + +- `foldCompactions(items)` — projects the model view from the raw log. When multiple compactions exist the highest `replacesUntil` wins (later compactions subsume earlier ones). The replaced prefix collapses to a rendered `` developer message; compaction records themselves never appear in the folded view, so one never reaches a provider un-folded. The fold seam strips unresolved tool calls, the same repair the history trimmer applies. Run it on history **before** the band assembler so a compaction genuinely reduces what `assembleView` has to fit. +- `hasCompaction(items)` — cheap check that lets a caller skip the fold (and its copy) on the common no-compaction path. +- `historyPressure(historyItems, policy)` — measures the **folded** history against `policy.compactAt` (default 80% of `tokenBudget − responseReserve`) and reports `{ historyTokens, compactAt, overThreshold }`. Measuring the folded view means writing a compaction genuinely relieves pressure. This is the observable complement of the assembler's silent front-drop: it says when the trim is coming so the caller can compact instead of losing the prefix. +- `createCompaction({ items, replacesUntil, summary })` — builds the record (`replacedCount` + `tokensSaved` bookkeeping, floored at 0). The caller supplies the summary — an LLM call, a heuristic, or a verbatim digest; summarization is a composition point, not engine policy. `replacesUntil` indexes the RAW log, including earlier compaction items, so stacking compactions is well-defined. +- `compactHistory({ log, keepRecent, summarize })` — thin convenience: works out `replacesUntil` from `keepRecent`, awaits `summarize` on exactly the replaced prefix, and returns the record for the **caller** to append (`ctx.itemLog.append(...)`). Returns `null` when there is nothing to compact. +- `compactionAsItem(compaction)` — the one sanctioned bridge from `CompactionItem` to the `Item` that `ItemLog.append` takes; the type is registered with the schema registry so the append validates. + +The runtime applies the fold on every model request path. `callModel` assembly folds the projected log **before** the system/history partition — `replacesUntil` indexes the raw log, so folding a system-stripped array would drift the cut point and silently eat live turns; system items are hoisted from pre-fold positions so a compaction whose covered prefix includes the system prompt still leaves the model its instructions. The no-layers path folds the raw log directly, and `previewRequestItems` folds identically, so a preview never shows the raw record in place of the summary the model would actually read. When the folded history still exceeds `compactAt`, the step emits one `context_pressure` framework event (`{ nodeId, historyTokens, compactAt }`, spec 08) per execution, measured post-fold so writing a compaction genuinely relieves the pressure. + --- ## Prompt-Cache Anchoring @@ -886,7 +897,7 @@ Set on the harness as `contextCache`. Anchoring is **on by default**; `enabled: ### Limitation: History Overflow -Once conversation history exceeds its budget, the projector drops from the front, which moves the anchor/history boundary and loses the history portion of the cache on every subsequent turn. The `[system][anchor]` prefix still caches. Pair anchoring with `history` (spec 12) to keep the boundary still. +Once conversation history exceeds its budget, the projector drops from the front, which moves the anchor/history boundary and loses the history portion of the cache on every subsequent turn. The `[system][anchor]` prefix still caches. Pair anchoring with `history` (spec 12) to keep the boundary still, or record a compaction so the prefix collapses to a stable summary instead of sliding. ### Hard Token Cap (`assembleView`) diff --git a/specs/12-builtin-context-layers.md b/specs/12-builtin-context-layers.md index e6517653..8c6cc559 100644 --- a/specs/12-builtin-context-layers.md +++ b/specs/12-builtin-context-layers.md @@ -331,6 +331,8 @@ function history(config?: { maxItems?: number }): ContextLayer The CLI exposes the cap via `AgentConfig.history.maxItems`. When unset, the layer is not installed and history is uncapped. +**Compaction composes with the cap**: `history()` bounds item *count*; a recorded compaction (spec 11, *History Compaction*) replaces the old prefix with a summary instead of dropping it. Fold first (`foldCompactions`), then cap — the fold seam and the slice boundary both strip orphan tool calls. + ```typescript // Direct usage in core const layers = [