diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7a27ed1529..132deaf456 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,7 +4,6 @@ on: push: branches: [main] pull_request: - branches: [main] concurrency: group: ci-${{ github.ref }} diff --git a/AGENTS.md b/AGENTS.md index 6d1666bc6a..8442f2b26f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,6 +22,7 @@ ## Source Organization +- Use the [refactoring guide](docs/refactoring.md) for structural changes across packages. Plan complete feature ownership, compatibility, migration coverage and validation before splitting files. - Follow the [source ownership rules](packages/coding-agent/docs/architecture.md#source-ownership-and-module-boundaries) when adding or extracting modules. The [source map](packages/coding-agent/src/README.md) documents current owners and ordering invariants. - Before a structural change, identify the feature owner, its state and lifecycle, its public API, and its allowed dependencies. Update the source map when ownership changes. - Group complete feature responsibilities into reviewable PRs. File size alone does not justify an abstraction or a separate PR. diff --git a/docs/refactoring.md b/docs/refactoring.md new file mode 100644 index 0000000000..24c637c8a6 --- /dev/null +++ b/docs/refactoring.md @@ -0,0 +1,78 @@ +# Refactoring existing packages + +Use this workflow for coding-agent and later cleanups of ai, agent, tui and the Python runtime. A package's own architecture and contributor instructions still apply. This guide records lessons from the coding-agent extraction: reducing one class left older parts of the same features in different folders, and several mixed files needed responsibility changes before relocation. + +## Start with the whole ownership map + +Inventory the package's tracked source, direct files in mixed directories, largest handwritten files, assets, executable entry points and public exports. Distinguish generated code, vendor code, fixtures and tests. Report physical LOC honestly; source size does not establish runtime speed or package weight. + +For each responsibility, record: + +| Question | Required answer | +| --- | --- | +| Who owns it? | The feature and its canonical directory. | +| What state does it own? | Authoritative state, derived views and invariants. | +| How long does it live? | Package/process, session, turn or individual operation. | +| How does it end? | Cancellation, replacement, failure, disposal and pending work. | +| What does it expose? | Public operations, lightweight contracts and actual consumers. | +| What may it depend on? | Named capabilities and the owner of cross-feature ordering. | +| What moves together? | Existing helpers as well as newly extracted code, plus tests/assets/scripts. | +| What remains? | Exact deferred files, their destination and the workstream that will move them. | + +Read complete implementations before editing them. An import/size inventory can establish scope but cannot prove semantic ownership. Mark provisional assignments explicitly and revisit them during implementation. Do not let “transitional” folders remain without destinations. + +## Organize by feature, with explicit integration boundaries + +Keep a feature's algorithms, state, contracts, persistence adapters and cleanup together. Separate files inside that feature when they have distinct responsibilities. A pure function, isolated test or several type consumers does not automatically justify a new top-level capability. + +Use a separate capability when it has an independently useful API and real consumers outside the parent feature. Name the session/application integration separately and document the direction of dependency. For example, Python process transport and environment provisioning can operate without a conversation; a session's replacement order and transcript notices remain session-owned. + +At grouping directories, keep direct files limited to entry points, composition and explicitly named cross-feature contracts. At leaf feature directories, put implementation files directly in the folder. Do not force one folder per file, arbitrary LOC limits or a class per method. Avoid large generic core, shared, helpers and services buckets. + +Split mixed responsibilities before moving a file. Examples found in coding-agent: + +- Input queue state was bundled with daemon worker eviction policy. +- A tool implementation owned the reusable kernel provisioner. +- Kernel contracts shared a module with process-wide cleanup registration. +- Refinement formatting shared a barrel with model planning, creating a message-conversion cycle. +- Session lease support exposed generic process identity helpers used by daemon ownership. + +Moving those files unchanged would preserve the underlying dependency problems. A smaller facade also fails if every extracted object receives the entire old class or a universal services container. + +## Establish the compatibility contract before implementation + +Honor the user's compatibility requirements. For Kevin's September 11 coding-agent cleanup, full feature and backward compatibility are required. Preserve CLI/SDK exports, supported import paths, method signatures, extension hooks, configurable shortcuts, wire payloads and persisted data behavior. + +Keep one canonical implementation. If an existing import path must remain usable, retain an explicit forwarding facade with no state or copied logic; migrate internal consumers to the owner. Record such facades separately from unfinished implementation. Do not remove them merely to improve directory or LOC statistics. A later removal requires an explicit compatibility decision. + +Test representative old and new entry points against the same exported objects and behavior. Check type declarations as well as runtime exports. Avoid importing a broad public barrel from its own implementation; preserve existing dependency injection rather than introducing a new factory cycle. + +Structural changes should preserve event order, completion settlement, callback receivers, synchronous/asynchronous boundaries, cancellation timing and late-result handling. Define these from existing behavior rather than from the cleaner design one might prefer. Separate intentional behavior changes for review. + +Protocol/schema changes need the package's compatibility process. File moves alone should not change protocol versions or disk formats. Preserve old-client/new-server and new-client/old-server behavior, including optional-capability fallback and startup. Keep recovery data readable and failures visible. + +## Implement a few coherent changes + +Use isolated worktrees with an exact shared base. Delegate independent ownership areas; explicitly identify shared composition files and integration order. Share the accepted map and compatibility contract with every implementer. Prefer substantial feature PRs with small reviewable commits over one PR per helper. + +Integrate one lane at a time, inspecting conflicts semantically. Import relocation, helper extraction and behavior fixes should remain distinguishable. Preserve a single queue, transcript, registry, lifecycle authority and accounting path. Do not rewrite published stacks or merge automatically as a shortcut. + +Update consumers, source maps, tests, examples, scripts, assets, packaging and declarations in the same change. Search for old paths in executable strings and asset resolution, not only static imports. Check workspace dependency realpaths so tests cannot accidentally exercise another checkout. + +## Validate behavior and shipped artifacts + +Choose tests from the ownership risk, not from the number of changed lines. Cover success and failure, duplicate delivery/completion, cancellation at awaited boundaries, replacement while callbacks are pending, persistence rejection, resource cleanup and reentrancy where applicable. + +Use deterministic providers, injected clocks and isolated I/O for regression coverage. Preserve existing regression assertions; diagnose failures against the exact base before changing an assertion. Run every modified test file and the repository's required checks with full output. Record failures and missing-environment skips; test counts are not a coverage guarantee. + +For process or kernel changes, validate real processes in the supported environment. Run paired base/head scenarios in Prime Sandboxes when requested: identical fixtures, toolchain and test harness; explicit source SHAs and archive hashes; isolated temporary homes and sockets; no personal histories or unrelated secrets. Use separate lanes for heavy kernel startup and process stress when verified resource limits require it. Reconcile and clean up all created sandboxes, including ambiguous creation outcomes. + +Exercise shipped entry points as well as source tests: CLI, SDK, TUI, JSON/RPC/ACP as applicable, fresh startup, attach/reconnect, save/reload, extensions, tools and native assets. Check packaged Python sidecars and import resolution separately from TypeScript checking. Existing credentials may be used only within the user's authorized scope; do not silently substitute paid external inference for an internal route. + +Benchmark identical workloads with the same trusted harness and complete observations. Distinguish timing, RSS, serialized bytes, compressed/installed size and build/check time. A successful small reproduction does not complete an older partial report. Make no performance claim based solely on moved code or noisy scores. + +## Finish with an accurate source map + +A responsibility is migrated when its implementation has one canonical home, consumers use that home, compatibility facades are explicit, all path/asset references work, invariants remain covered and required validation passes. Update the migration ledger and contributor rules in the same PR. + +Report exact PR bases/heads, tests actually run, platform/environment limits, remaining review gates and deferred work. Distinguish implementation, integration checkout and installed executable provenance. Do not call a whole package clean because one class is smaller, or an installation updated because only its source manifest changed. diff --git a/packages/coding-agent/.changes/session-idle-pump-wait.md b/packages/coding-agent/.changes/session-idle-pump-wait.md new file mode 100644 index 0000000000..05ae259a24 --- /dev/null +++ b/packages/coding-agent/.changes/session-idle-pump-wait.md @@ -0,0 +1 @@ +- Fixed session idle waits consuming CPU while queued input is blocked by bash, compaction, or retry work. diff --git a/packages/coding-agent/docs/architecture.md b/packages/coding-agent/docs/architecture.md index 394c98acf0..d7e9fdd14b 100644 --- a/packages/coding-agent/docs/architecture.md +++ b/packages/coding-agent/docs/architecture.md @@ -100,6 +100,12 @@ Place code inside the smallest feature that owns its behavior. Promote it outsid Use feature names and keep the structure shallow: files directly inside a feature directory are the default. Add another directory only for a coherent subfeature. Application source stays under `src/`; tests, scripts, docs, and build output stay at the package root. `core/` is a transitional location for existing code, not the default home for new shared code. +A directory containing feature subdirectories should keep only its entry points, composition, and explicitly named cross-feature contracts directly inside it. Feature implementation belongs with its owner, even when other features import its contracts. Do not create a folder for every file or prohibit useful files at a feature root merely to make the tree uniform. + +Complete a feature's placement across old and new files. Extracting its controller does not leave its algorithms, persistence, or contracts ownerless in `core/`. Separate files can express those responsibilities inside one feature. If similar directory names represent an independent capability and its session integration, document the distinct APIs and consumers; neither naming nor hypothetical reuse establishes that boundary. Mixed files require a responsibility split before relocation. + +Every transitional location needs a named destination and migration scope. The [source organization completion plan](source-organization-plan.md) records the current gaps, proposed destinations, and validation requirements. Its proposed tree is not a claim that those moves have shipped. + Independent testability, a pure function, or a small dependency interface does not require a top-level directory. Multiple importers are evidence to inspect, not a promotion rule: a UI reading a session goal's status does not make goal execution independent of sessions. Do not promote code for hypothetical future reuse. ### Keeping a feature together diff --git a/packages/coding-agent/docs/source-organization-plan.md b/packages/coding-agent/docs/source-organization-plan.md new file mode 100644 index 0000000000..11adf6729b --- /dev/null +++ b/packages/coding-agent/docs/source-organization-plan.md @@ -0,0 +1,182 @@ +# Completing coding-agent source organization + +Status: Kevin approved implementation on September 11, 2026, with GPT-Astra delegation, full feature/backward compatibility and extensive Prime Sandbox validation. Audited stack base: `254666f2cfb70b4740ea5b5a67703a34ada44ffc`. Session and kernel ownership implementation is in progress; later package-wide assignments remain subject to detailed review. This supersedes the older flat, top-level goals/compaction/refinement proposal in ENG-5934; the architecture guide remains the placement rule. The repository-wide [refactoring guide](../../../docs/refactoring.md) records the reusable method. + +## What the first stack missed + +The five PRs extracted behavior from AgentSession, but left existing parts of the same features in `core/`. The plan did not specify how to finish those migrations or organize the rest of the package. A smaller facade is useful, but it is not completion of the codebase cleanup. + +The audited stack base contains 343 TypeScript files. The following table records that baseline, before the ownership completion below. Physical LOC includes comments and blank lines. Assets, JavaScript templates/vendor files, Python runtime, tests, and generated output are outside that count. + +| Current location | Direct TypeScript files | Physical LOC in those files | Problem | +| --- | ---: | ---: | --- | +| `src/` | 8 | 5,002 | CLI implementation, configuration and migrations mixed with entry points. | +| `src/core/` | 64 | 28,211 | Session, configuration, authentication, models, resources, process support and presentation have no consistent grouping. | +| `src/session/` | 1 | 277 | Prepared action contracts/factories sit outside their input owner. | +| `src/modes/daemon/` | 30 | 23,349 | Worker lifecycle, public/private protocol, catalog, scheduling and snapshots are flat; two large classes still combine them. | +| `src/modes/interactive/` | 11 | 11,740 | The 10,175-line interactive shell still owns substantial feature behavior. | +| `src/modes/interactive/components/` | 59 | 12,263 | Transcript, editor, authentication, settings and generic UI components share one bucket. | + +This is a full path/size inventory and import/export survey, with detailed implementation review of compaction, refinement, kernel/provisioner, runtime/services, input actions, messaging and related lifecycle code. Other destinations below are planning assignments, not claims that every implementation has been fully reviewed or is ready to move. + +## Boundary to standardize + +1. A session feature owns its contracts, algorithms, state, persistence adapters and cleanup together. Goals stay in `session/goals/`; compaction and refinement follow the same rule. Reading a goal or compaction result from the SDK/UI does not transfer ownership. +2. Independent capabilities have a named home outside session. Python environment setup and process transport work without a session; model catalogs, credentials and package discovery also have independent consumers. Their session-specific selection/bindings remain session-owned. +3. Cross-feature composition is explicit. `session/agent-session.ts` may remain beside feature directories because it composes them. It must not accumulate their state again. `session/runtime/` is reserved for construction, replacement, configuration and owned resource setup/disposal, not queues, turns or arbitrary helpers. +4. A parent directory primarily groups features. Its direct files are entry points, composition, or clearly named contracts spanning those children. A leaf feature directory contains implementation files directly. Add a subdirectory only for a coherent responsibility; neither zero loose files nor a fixed maximum LOC is the goal. +5. Use descriptive filenames within a feature: `controller.ts`, `execution.ts`, `summary.ts`, `persistence.ts`. Avoid repetitive paths such as `compaction/compaction-execution.ts` and vague files such as `shared.ts` that mix contracts with side effects. +6. A feature owns its public contracts even with several consumers. Use lightweight direct imports; do not load runtime barrels for types or introduce a global contracts dumping ground. Kevin explicitly requires backward compatibility for this work: retain thin explicit old-path export facades where needed, with no duplicated implementation, and migrate internal consumers to canonical paths. Keep existing supported package exports intact. +7. Every move includes consumers, tests, assets, build references and documentation. Record remaining exceptions by exact responsibility and workstream; `core/` cannot remain an indefinite destination. + +## Proposed package map + +Kevin approved the kernel/session-runtime distinction with the implementation request. The map applies the existing session ownership rule. Omitted files under each leaf remain with that feature; this is a boundary map, not a file-per-function prescription. Compatibility facades at legacy paths are documented exceptions and must contain no feature implementation. + +```text +src/ + index.ts public package exports + cli.ts executable entry point + postinstall.ts installation entry point + sdk/ public factories and service composition + cli/ CLI parsing, commands and startup + config/ paths, settings and config migrations + auth/ credential storage and login services + models/ catalog, resolution and provider policy + resources/ package discovery, skills and prompt templates + kernel/ Python provisioning, transport and snapshots + tools/ tool execution adapters + extensions/ extension API, loading and execution + mcp/ MCP integration + session/ + agent-session.ts public session composition + runtime/ construction, replacement and kernel binding + input/ admission, action store, queues and recovery + turns/ execution, events, retry and continuation order + goals/ goal state, contracts and accounting + autonomy/ autonomous budgets, gates and continuation + compaction/ controller, execution and summary generation + refinement/ controller, planning and harness updates + context/ model-facing messages, projections and prompts + history/ persisted transcript/tree, paths and leases + children/ child records, runs, contracts and accounting + side-questions/ temporary questions over cloned conversation context + tools/ session tool selection and shell lifecycle + extensions/ this session's extension bindings + models/ this session's model preferences and selection + coordination/ agent-family messaging, observation and schedules + diagnostics/ logs, traces, telemetry and request ledgers + modes/ execution/presentation entry points + daemon/ worker lifecycle, catalog, protocol and snapshots + agent-connection/ client connection contracts and implementations + interactive/ shell, transcript, composer, dialogs and status + agents-view/ catalog view, selection and rendering + acp/ ACP adapter + rpc/ RPC adapter + bun/ Bun entry adapters + node/ Node entry adapters + utils/ small domain-independent primitives only +``` + +Keep `modes/` in this pass: it names the existing application boundary. Its internal feature organization matters more than removing another layer. Do not add generic `core/`, `services/` or `helpers/` layers beneath the new owners. + +Some repeated names still represent deliberate capability/integration pairs: `tools/` defines executable tools while `session/tools/` selects them and owns session shell state; `extensions/` implements the extension system while `session/extensions/` binds it to one session; `models/` owns the catalog while `session/models/` owns the selected model. These are explicit exceptions, not permission to split algorithms from controllers arbitrarily. An adapter whose sole responsibility is construction/binding may belong in runtime composition. Stateless feature parsing or policy still belongs with its feature. + +## First consolidated stack update + +Implemented in one consolidated follow-up PR on the stack, with context and input/child changes in separate reviewable commits. The five existing PRs remain unchanged. Historical paths in this table remain explicit compatibility exports; canonical application consumers use the destinations. + +| Current source | Destination/responsibility | +| --- | --- | +| `core/agent-session.ts` | `session/agent-session.ts`; retain public composition and ordering. | +| `core/session-action-store.ts` | Action queue, transitions and tickets to `session/input/action-store.ts`; daemon eviction/passivation policy to the daemon residency owner. Separate these responsibilities before moving. | +| `session/prepared-actions.ts` | `session/input/prepared-actions.ts`; contracts and factories belong to input even when turns/context consume them. | +| `core/prompt-admission.ts` | `session/input/prompt-admission.ts`. | +| `core/compaction/compaction.ts` | Summary preparation/generation to `session/compaction/summary.ts`; shared context token estimation to `session/context/token-estimate.ts`. | +| `core/compaction/branch-summarization.ts` | `session/context/branch-summary.ts`, consumed by branch navigation. | +| `core/compaction/utils.ts` | Purpose-named helpers in `session/context/`: conversation serialization is shared with refinement; file tracking is shared by compaction and branch summarization. | +| `session/compaction/compaction.ts`, `compaction-execution.ts` | `session/compaction/controller.ts`, `execution.ts`. | +| `core/refinement/refinement.ts` | Planning/review, harness state persistence/application, and lightweight contracts/formatting within `session/refinement/`. Split by those responsibilities rather than relocating the entire mixed module. | +| `session/refinement/refinement.ts`, `auto-refinement.ts`, `refinement-execution.ts` | `session/refinement/controller.ts`, `automatic.ts`, `execution.ts`. | +| `core/autonomous.ts`, `session/turns/autonomous-continuation.ts` | `session/autonomy/`; same budget/gate/continuation ownership rule as goals. General turn scheduling stays in turns. | +| `core/rlm-runtime.ts`, `rlm-max-depth.ts` | Child contracts/spawn adapters to children; model search to models; async bash notice adapters to input. No wholesale move of this mixed file into runtime. | +| `core/context-tree.ts`, `messages.ts`, `session-stats.ts`, `usage.ts`, `system-prompt.ts`, `prompts/` | `session/context/`; message, usage and prompt contracts remain lightweight and callable by history, UI and SDK consumers. | + +Compaction/refinement functions are not independent just because they can be unit tested: they interpret persisted session entries and operate on session context. Keep the public SDK functions exported from the package root, redirecting them to their owner. + +Refinement now separates lightweight outcome formatting from planning. Message conversion imports formatting/contracts directly; planning may consume message conversion without creating the former cycle. Harness persistence retains its shared on-disk contract with Python's `rlm.harness`, including reread-before-apply behavior. + +## Kernel and runtime completion + +This is the second consolidated follow-up PR. It completes kernel/runtime and SDK ownership together; it is pending integration and validation after the session feature PR. + +| Current source | Destination/responsibility | +| --- | --- | +| `core/kernel/repl-manager.ts`, `bootstrap.ts`, `boot-gate.ts`, `state-snapshot.ts` | `kernel/`; subprocess/protocol, environment setup, startup concurrency and snapshots. | +| `core/kernel/shared.ts` | Separate `kernel/contracts.ts`, `protocol.ts`, `process-registry.ts`: importing a type must not import global cleanup registration. | +| `core/kernel/bootstrap-cli.ts` | `cli/bootstrap-kernel.ts`; update executable references and runtime path resolution. | +| Provisioner and Python skill bootstrap in `core/tools/ipython.ts` | `kernel/provisioner.ts`, `skill-bootstrap.ts`; keep the tool adapter separate from provisioning. | +| `core/agent-session-runtime.ts`, `agent-session-config.ts` | `session/runtime/runtime.ts`, `config.ts`. | +| `core/agent-session-services.ts`, construction in `core/sdk.ts` | Factories/services in `sdk/`, creation/service contracts in lightweight owner modules. Preserve the runtime's injected `CreateAgentSessionRuntimeFactory`; construction implementations use direct imports, and public re-exports stay outward-facing. Remove the runtime → SDK barrel → runtime cycle without introducing a new concrete factory dependency. | +| `session/kernel/kernel.ts`, `kernel-environment.ts` | `session/runtime/kernel-lifecycle.ts`, `kernel-environment.ts`. | +| `session/kernel/kernel-host-handlers.ts` | `session/runtime/host-bridge.ts`; composition only. | +| Session kernel message/observe/heartbeat request handlers | Respective coordination features' request adapters; host bridge supplies current session operations. | + +`SessionKernel.build()` currently constructs all built-in tools. Tool assembly belongs to the existing session tool owner; kernel lifecycle should expose its provisioner/client. Preserve readiness and event ordering while changing this dependency. + +Keep the independent kernel capability usable by postinstall/bootstrap without a session or terminal UI. Preserve process-wide startup limiting, error repair, old-snapshot flush before replacement restore, reinstalling live skill handles after restore, and final disposal. Do not fold input/turn/child policy into `session/runtime/` to empty other directories. + +## Remaining core migration ledger + +The session and kernel tables above plus this ledger account for all 64 direct `core/*.ts` files and all eight existing `core/` subdirectories. These later assignments require full implementation review before edits. They are proposed ownership destinations covering the inventory, not authorization for blanket renames or deletion of functionality. + +| Current core files/folders | Owner and required boundary | +| --- | --- | +| `auth-storage.ts`, `auth-guidance.ts`, `prime-inference-auth.ts`, `websearch-credential.ts` | `auth/`; credential storage, recovery guidance and provider/service login. | +| `model-registry.ts`, `model-resolver.ts`, `prime-inference-model-catalog.ts`, `prime-inference-models.ts`, `provider-display-names.ts`, `provider-retry.ts`, `thinking-levels.ts`, `defaults.ts` | `models/`; separate model parsing from CLI validation/presentation. A shared model helper must not import CLI argument handling. | +| `prime-inference-model-selection.ts` | Interactive auth flow; its result decides whether to open a model picker after login. | +| `settings-manager.ts`, `resolve-config-value.ts` | `config/`; settings/credential-value resolution, with dependencies directed toward primitives. | +| `package-manager.ts`, `resource-loader.ts`, `skills.ts`, `skill-blocks.ts`, `prompt-templates.ts`, `source-info.ts`, `diagnostics.ts` | `resources/`; package resolution/install and discovered resources, with skills/templates grouped by feature. Remove resource-loader's concrete theme dependency by composing theme registration at the UI boundary. | +| `session-manager.ts`, `session-file-actions.ts`, `session-id.ts`, `session-resolver.ts`, `session-import-errors.ts`, `session-lease.ts` | `session/history/`; persisted session discovery/tree/writes/artifacts and exclusive ownership. Extract generic process identity from leases first because daemon ownership and orphan reaping consume it. | +| `session-cwd.ts` | Runtime working-directory validation/error data to `session/runtime/`; interactive confirmation formatting to the terminal owner. | +| `side-question.ts` | `session/side-questions/`; temporary answer runs over cloned conversation context, with their own cancellation and no writes to parent history. | +| `agent-messages.ts`, `agent-observe.ts`, `cron-jobs.ts` | `coordination/messaging/`, `observation/`, `scheduling/` with agent-family contracts owned by coordination. Split message/run-boundary classification into session input/turns. Keep daemon dispatch and actual worker residency in daemon. | +| `agent-traces.ts`, `telemetry.ts`, `semantic-edges.ts`, `event-log.ts`, `logging.ts`, `timings.ts` | `diagnostics/`, with traces/telemetry/ledger responsibilities grouped when needed; contracts must not load a session implementation. | +| `orphan-process-journal.ts` | Kernel/process journal support under `kernel/`; retain its journal environment contract and kernel-scoped reaping. Extract domain-independent identity/tree-kill primitives to utils; preserve kernel and daemon consumers. | +| `event-bus.ts` | `extensions/`; currently used for extension/resource discovery, not a universal application bus. | +| `exec.ts` | Extension execution support under `extensions/`; generic subprocess primitives remain in utils. | +| `bash-executor.ts`, `tools/` | `tools/`, after extracting kernel provisioning and terminal rendering from tool implementations. Session shell state remains session-owned. | +| `extensions/` | `extensions/`; separate execution contracts from optional terminal UI types/bindings. | +| `mcp/` | `mcp/`; protocol-specific ACP adapters stay with ACP when they have no independent MCP responsibility. | +| `export-html/` | `session/history/export-html/`; export adapter and its templates/vendor assets travel together. | +| `footer-data-provider.ts`, `keybindings.ts`, `output-guard.ts` | Terminal presentation/input support; footer stays under interactive/status, shared terminal keybindings/stdout plumbing under a named modes terminal owner. Execution receives operations rather than importing UI globals. | +| `slash-commands.ts`, `new-session-command.ts` | Named command definitions/parsers under `cli/commands/`; session-specific parsing/contracts stay with the session feature when split. Shared definitions must not import executable CLI startup or UI. | +| `index.ts` | Redirect real consumers/public root exports; retire the legacy barrel only after checking repo consumers and supported package entry points. | + +Top-level `config.ts` and `migrations.ts` move to config; `main.ts`, `cli-main.ts` and `package-manager-cli.ts` belong to CLI startup/package commands. Keep tiny supported executable wrappers where packaging requires them. `themes/` needs review together with UI theme resources; do not assume its one module is redundant. + +## Mode and utility organization + +Keep these in the existing daemon/UI workstreams, not in a 343-file mechanical move: + +- Daemon: `protocol/` for wire contracts/validation; `workers/` for process ownership/recovery/residency; `catalog/` for saved/live discovery; `scheduling/` for dispatch; `snapshots/` for transfer/cache. The daemon and supervisor entry files compose these. Public client transport remains clearly separated from supervisor implementation. +- Interactive: `transcript/`, `composer/`, `commands/`, `dialogs/`, `status/`, `theme/`; move feature-specific components alongside those owners. Reserve `components/` for genuinely shared UI primitives, not every rendered object. Keep terminal lifecycle in the interactive shell. +- Agents view: catalog/view model, selection/navigation, rendering. Connection, ACP and RPC retain their existing named owners unless their own audit identifies a mixed responsibility. +- Utilities: retain domain-independent file/async/process primitives. Rehome daemon socket paths with daemon, clipboard with terminal input, and release/changelog/update helpers with their CLI owner. Review image conversion/orientation/resizing as a coherent media capability. Do not replace `core/` with a new 64-file `utils/`. + +No prescribed file-count cap substitutes for ownership. The 10,175-line interactive shell, 7,843-line daemon mode, 7,007-line supervisor, 2,443-line package manager and 2,429-line session manager still need responsibility extraction; putting them in folders does not complete that work. + +## Safety and completion gates + +1. Before each commit group, fully read every implementation to edit, list consumers including tests/examples/scripts, record state and cleanup owners, and confirm the proposed map against actual APIs. Start from the current stack head and inspect worktree changes before editing. +2. Split mixed responsibilities first in reviewable commits, then move files/imports. Keep feature behavior, protocol shapes and stored formats unchanged. No new universal context object, service locator, dependency or runtime migration. +3. Preserve exactly-once action settlement, commit fences and passivation behavior; goal/compaction continuation order; refinement plan claiming and concurrent harness writes; child cleanup before kernel teardown; live controller lookup and callback receivers; kernel replacement/restore order. In compaction, request accounting completes before transcript append, and failed persistence retains the existing live outcome disclosure. The detailed ordering rules in `src/README.md` remain required. +4. Use existing focused action/queue/goal/compaction/refinement/runtime/kernel suites and the faux-provider integration harness. Run every modified test file from its package root. Kernel/process integration uses the dedicated environment lanes. Record skips and failures explicitly; do not claim every edge case is proven. +5. Add an architecture check for the agreed boundaries, using the existing parser/tooling: prevent new core paths, implementation in parent grouping folders outside a small documented exception list, runtime imports of public SDK barrels, feature contracts importing controllers, and execution imports of terminal renderers. Distinguish type-only edges and use an explicit shrinking baseline for existing violations. The test must reject intentional bad-import fixtures and permit legitimate contract consumers. +6. Update package exports, extension loading, test mappings, bootstrap paths, copy-assets scripts and compiled sidecar resolution. The current `./hooks` export and example mapping reference `core/hooks`; inspect that existing discrepancy before changing exports, rather than silently deleting a supported entry. Validate the assembled artifact and SDK import separately from type checking. +7. Run `npm run check` with full output after code changes. Follow AGENTS.md for focused tests; do not run prohibited blanket dev/build/test commands. A documentation-only plan does not need application tests. +8. Benchmark identical base/head workloads using the same trusted harness and complete reports. A move alone is not evidence of faster startup or smaller artifacts. Keep the existing partial benchmark disclosed; do not substitute a successful small reproduction for its missing trial. +9. Update this ledger and `src/README.md` in the same PR as each implemented boundary. A moved owner is complete only when internal imports and assets use canonical paths, required legacy export facades are explicit and tested, conflicting conventions are gone, and affected validation passes. Report compatibility facades separately from unfinished implementation. +10. Keep local cleanup activation held while this design/implementation is pending. Rebuild from the latest UI/native integration only after the agreed gates clear; preserve current installed changes. No automatic merges or release publication. + +Track session work in ENG-5938, kernel/children in ENG-5939, dependency separation in ENG-5936, daemon/UI in ENG-5940/ENG-5941, and build/artifact validation in ENG-5944. ENG-5934 is the overall plan. Keep Kevin involved at the ownership-map review and the first concrete implementation diff, rather than asking about each file move. diff --git a/packages/coding-agent/src/README.md b/packages/coding-agent/src/README.md index 92b7d3f337..98b36665a6 100644 --- a/packages/coding-agent/src/README.md +++ b/packages/coding-agent/src/README.md @@ -2,7 +2,9 @@ `src/` contains application source; tests, scripts, docs, examples, and build output stay at the package root. Follow the [source ownership rules](../docs/architecture.md#source-ownership-and-module-boundaries) when choosing module boundaries. Session features, including goals, live below `session/`; each feature groups its state, dependencies, and lifecycle. -`core/` currently contains most execution logic. Migrate responsibilities into their owning feature folders as their boundaries are established. `AgentSession` remains the public entry point and coordinates work across features. Each feature owner keeps its state and transitions together and receives only the dependencies it uses. +`session/agent-session.ts` is the public session composition point. Each feature owner keeps its state and transitions together and receives only the dependencies it uses. `core/` still contains unmigrated capabilities and compatibility exports; it is not a destination for new implementation. + +The session ownership follow-up consolidates context algorithms, input actions, autonomy and child request contracts with their owners. The [completion plan](../docs/source-organization-plan.md) distinguishes implemented boundaries from remaining package work. Historical module paths forward explicit exports to canonical modules; application imports use the canonical paths. These compatibility files do not own duplicate state or behavior. ## Goals @@ -36,23 +38,24 @@ Apply the architecture guide's placement and dependency rules to each extraction | --- | --- | | `session/input/` | Submission normalization, admission, scheduling, commit fencing, action queues, delivery, and recovery. | | `session/goals/` | Goal state and contracts, accounting, persistence, command parsing, and goal-specific continuation. | -| `session/turns/` | Turn preparation and execution, session commands, ordered events, retry, and autonomous/shared continuation. | -| `session/compaction/` | Session compaction lifecycle and execution. | +| `session/turns/` | Turn preparation and execution, session commands, ordered events, retry, and shared continuation. | +| `session/autonomy/` | Autonomous budgets, gates, continuation messages and rollback. | +| `session/compaction/` | Session compaction lifecycle, execution and summary generation. | | `session/refinement/` | Refinement planning/application lifecycle, automatic review, and execution. | -| `session/context/` | Pending context, harness context, transcript views, branch navigation, and export. | +| `session/context/` | Model-facing messages, usage, token estimates, prompts, pending context, transcript views, branch navigation, and export. | | `session/children/` | Child records, runtime creation, execution, projections, and usage accounting. | | `session/kernel/` | Kernel lifecycle, environment, host-handler composition, and stateless host-request adapters. | | `session/models/` | Model selection, thinking preferences, and authenticated availability. | | `session/tools/` | Tool selection and shell-command execution. | | `session/extensions/` | Extension bindings, resource reload, and tool hooks. | -`session/prepared-actions.ts` remains the shared action and recovery contract used across input, turns, and context. Shared continuation belongs to `session/turns/`, including continuation after compaction. Child usage belongs to `session/children/` because its accounting and cleanup follow child records. Core compaction/refinement algorithms and their public exports remain in `core/` for their broader callers. +`session/input/prepared-actions.ts` contains action and recovery contracts used across input, turns, and context. Shared continuation belongs to `session/turns/`, including continuation after compaction. Child usage belongs to `session/children/` because its accounting and cleanup follow child records. Compaction/refinement implementations live with their session features; their historical `core/` paths contain only explicit compatibility exports. ## Session input scheduling `session/input/input-scheduler.ts` owns the serialized pump, its preparation epoch, pause leases, and abort/restart suspension. It receives two callbacks: whether the session has work eligible for scheduling, and the operation that runs that work. The scheduler exposes read-only state and named operations; callers cannot change its pause sets or scheduling flags. -The existing `ActionStore` in `core/session-action-store.ts` owns queued actions, their transitions, and delivery/completion tickets. `session/input/input-dispatcher.ts` selects and batches those actions, reconciles durable delivery after dispatch, rolls undelivered work back, and settles completion or failure. `AgentSession` supplies turn execution and session-command operations and coordinates goals, child agents, and compaction. The dispatcher shares the existing `ActionStore`; it does not create a second queue or copy the transcript. +`ActionStore` in `session/input/action-store.ts` owns queued actions, their transitions, and delivery/completion tickets. `session/input/input-dispatcher.ts` selects and batches those actions, reconciles durable delivery after dispatch, rolls undelivered work back, and settles completion or failure. `AgentSession` supplies turn execution and session-command operations and coordinates goals, child agents, and compaction. The dispatcher shares the existing `ActionStore`; it does not create a second queue or copy the transcript. Worker eviction and passivation predicates belong to `modes/daemon/workers/residency-policy.ts` and consume action-state views without owning the queue. Preserve these distinctions when extending the scheduler: @@ -105,7 +108,7 @@ Execution and recording callbacks preserve dispatch through the public session m Preserve the policy differences: direct prompts flush shell output before validation and compact after model selection; queued turns validate before flushing and compact before model selection. Conditional refinement barriers are checked when reached, so a refinement started during preparation is still awaited. Withdrawing prepared work skips the final barrier and commit. The exported `TurnExecutionPolicy` shape remains available from the session facade. -`session/prepared-actions.ts` contains prepared action types, delivery records, recovery contracts, input copying, action factories, and queue projections. It has no session dependency. Primary messages retain their identity for durable-delivery checks; separately stored input blocks and prefix messages retain their existing copy behavior. Recovery format version 1 and the public exports from `AgentSession` stay unchanged. +`session/input/prepared-actions.ts` contains prepared action types, delivery records, recovery contracts, input copying, action factories, and queue projections. It has no session implementation dependency. Primary messages retain their identity for durable-delivery checks; separately stored input blocks and prefix messages retain their existing copy behavior. Recovery format version 1 and the public exports from `AgentSession` stay unchanged. ## Session input and turns @@ -123,7 +126,7 @@ Preserve the policy differences: direct prompts flush shell output before valida | `session/turns/events.ts` | Ordered agent-event processing, listener delivery, and transcript/accounting coordination. | | `session/context/pending-context.ts` | Pending messages, notices, and retention for the next turn. | | `session/goals/continuation.ts` | Goal continuation admission, budget notices, child-wait coordination, and rollback. | -| `session/turns/autonomous-continuation.ts` | Autonomous continuation messages, snapshots, and rollback. | +| `session/autonomy/continuation.ts` | Autonomous continuation messages, snapshots, and rollback. | | `session/turns/turn-policy.ts` | Turn stopping, threshold compaction, and continuation decisions from current session state. | Input has one durable ActionStore, one scheduling pump, one commit fence, and one ordered event queue. Queue operations, admission, preparation, execution, and recovery use these same owners; they do not maintain parallel queues or transcripts. Dependencies are named operations and small state views. Callbacks read the current model, controllers, and runtime where the original operation did. @@ -137,14 +140,19 @@ Input has one durable ActionStore, one scheduling pump, one commit fence, and on | File | Responsibility | | --- | --- | -| `session/compaction/compaction.ts` | Manual and automatic compaction lifecycle, pending requests, cancellation, thresholds, and overflow recovery. | -| `session/compaction/compaction-execution.ts` | Summary generation, extension interception, request accounting, persistence, and context rebuild ordering. | -| `session/refinement/refinement.ts` | Refinement admission, planning and application barriers, serialized plan ownership, and disposal drains. | -| `session/refinement/auto-refinement.ts` | Review triggers, cooldowns, pending reviews, timers, and automatic operation cleanup. | -| `session/refinement/refinement-execution.ts` | Planning against current dependencies, applying harness edits, and persisting outcomes and notices. | +| `session/compaction/controller.ts` | Manual and automatic compaction lifecycle, pending requests, cancellation, thresholds, and overflow recovery. | +| `session/compaction/execution.ts` | Summary generation, extension interception, request accounting, persistence, and context rebuild ordering. | +| `session/compaction/summary.ts`, `types.ts` | Summary preparation/generation and compaction contracts. | +| `session/context/token-estimate.ts`, `conversation-text.ts`, `file-tracking.ts` | Shared context estimation, serialization and file tracking. | +| `session/context/branch-summary.ts` | Branch summary preparation and generation for history navigation. | +| `session/refinement/controller.ts` | Refinement admission, planning and application barriers, serialized plan ownership, and disposal drains. | +| `session/refinement/automatic.ts` | Review triggers, cooldowns, pending reviews, timers, and automatic operation cleanup. | +| `session/refinement/execution.ts` | Planning against current dependencies, applying harness edits, and persisting outcomes and notices. | +| `session/refinement/planning.ts`, `harness-state.ts` | Review/planning and persisted harness application/history. | +| `session/refinement/types.ts`, `format.ts` | Lightweight contracts and outcome formatting, usable without loading planning. | | `session/turns/continuation.ts` | Resuming work after compaction, settlement, cancellation, and ownership of continuation messages. | -Each owner keeps its mutable state and cleanup together. Typed host operations connect the owners to current model, authentication, extensions, storage, and scheduling. `AgentSession` composes them and retains public methods, events, and decisions that cross features, including goal and autonomous continuation admission. The summary algorithms and harness storage remain in their existing `core/` feature modules. +Each owner keeps its mutable state and cleanup together. Typed host operations connect the owners to current model, authentication, extensions, storage, and scheduling. `AgentSession` composes them and retains public methods, events, and decisions that cross features, including goal and autonomous continuation admission. Message conversion imports only refinement formatting/contracts, avoiding the former message-conversion/planning import cycle. Harness persistence retains Python's on-disk contract, atomic writes and reread-before-apply behavior. Preserve these boundaries when changing context behavior: @@ -164,9 +172,12 @@ Preserve these boundaries when changing context behavior: | `session/children/child-usage.ts` | Child usage attribution, origin batches, flush timers, and retry bookkeeping. | | `session/children/child-projection.ts` | Read-only child list and snapshot projections. | | `session/children/child-types.ts` | Child contracts and shared child data helpers. | +| `session/children/runtime-contracts.ts`, `spawn-options.ts`, `host-requests.ts` | Child host contracts, spawn option validation and child request adapters. | The registry owns child identity and lifecycle transitions. Execution and usage components operate on the same child records; they do not create competing copies of run state. Child state exposes read-only properties and named mutations. Runtime hosts, inherited depth, model selection, and event queues are read through live operations supplied by the session. +The former RLM runtime module's model search belongs to `session/models/model-search.ts`; background shell completion request handling belongs to `session/input/bash-host-requests.ts`. Their legacy combined module forwards exports but has no implementation. Child maximum-depth policy stays with children. + - Reserve and publish children in the original order. A late completion cannot replace a newer run or clear another run's cancellation state. - Retain deletion reservations, retryable cleanup state, and descendant quiescence until their existing completion conditions hold. Parent continuation still waits for the appropriate child work. - Flush child usage once at the existing parent event boundary. Origin batches and timers have one cleanup owner. diff --git a/packages/coding-agent/src/cli/daemon-command.ts b/packages/coding-agent/src/cli/daemon-command.ts index 0154275727..3a69fc7050 100644 --- a/packages/coding-agent/src/cli/daemon-command.ts +++ b/packages/coding-agent/src/cli/daemon-command.ts @@ -4,7 +4,6 @@ import { setTimeout as delay } from "node:timers/promises"; import type { AgentMessage } from "@earendil-works/pi-agent-core"; import chalk from "chalk"; import { expandTildePath } from "../config.js"; -import type { AgentSessionEvent } from "../core/agent-session.js"; import type { AgentSessionRuntimeConfig } from "../core/agent-session-config.js"; import { type AgentCronJob, formatAgentCronJob } from "../core/cron-jobs.js"; import { looksLikeSessionPath } from "../core/session-resolver.js"; @@ -13,6 +12,7 @@ import type { DaemonOutbound, DaemonResponse } from "../modes/daemon/daemon-prot import { matchesSessionIdSuffix } from "../modes/daemon/daemon-session-id.js"; import type { SessionSummary } from "../modes/daemon/daemon-session-list.js"; import { defaultDaemonSocketPath, normalizeSocketPath } from "../modes/daemon/daemon-socket.js"; +import type { AgentSessionEvent } from "../session/agent-session.js"; import { spawnHidden } from "../utils/child-process.js"; import { isLocalPath } from "../utils/paths.js"; import { isValidThinkingLevel } from "./args.js"; diff --git a/packages/coding-agent/src/cli/owned-session-worker.ts b/packages/coding-agent/src/cli/owned-session-worker.ts index aa6467ed35..2e56b8df84 100644 --- a/packages/coding-agent/src/cli/owned-session-worker.ts +++ b/packages/coding-agent/src/cli/owned-session-worker.ts @@ -3,7 +3,6 @@ import { randomUUID } from "node:crypto"; import { chmodSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import type { AgentSession } from "../core/agent-session.js"; import type { AgentSessionRuntime } from "../core/agent-session-runtime.js"; import { clearOrphanProcessJournal, @@ -14,6 +13,7 @@ import { } from "../core/orphan-process-journal.js"; import { SESSION_LEASE_OWNER_ID_ENV, SESSION_LEASES_ENABLED_ENV } from "../core/session-lease.js"; import { attachJsonlLineReader, serializeJsonLine } from "../modes/rpc/jsonl.js"; +import type { AgentSession } from "../session/agent-session.js"; import { spawnHidden } from "../utils/child-process.js"; import { isHelpCommandRequest, PUBLIC_COMMAND_NAMES, REMOVED_COMMAND_NAMES } from "./command-registry.js"; import { type CliSubprocessLaunchSpec, createCliSubprocessLaunchSpec } from "./subprocess-launch.js"; diff --git a/packages/coding-agent/src/core/agent-messages.ts b/packages/coding-agent/src/core/agent-messages.ts index f49df9195d..e087f546c1 100644 --- a/packages/coding-agent/src/core/agent-messages.ts +++ b/packages/coding-agent/src/core/agent-messages.ts @@ -1,12 +1,12 @@ import { randomUUID } from "node:crypto"; import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { HostRequestHandler } from "./kernel/index.js"; -import type { CustomMessage } from "./messages.js"; +import type { CustomMessage } from "../session/context/messages.js"; import { ASYNC_BASH_COMPLETION_CUSTOM_TYPE, HEARTBEAT_PROMPT_CUSTOM_TYPE, sanitizeMessageHeaderValue, -} from "./messages.js"; +} from "../session/context/messages.js"; +import type { HostRequestHandler } from "./kernel/index.js"; import { canonicalSessionPath } from "./session-lease.js"; export const AGENT_MESSAGE_CUSTOM_TYPE = "agent_message"; diff --git a/packages/coding-agent/src/core/agent-session-config.ts b/packages/coding-agent/src/core/agent-session-config.ts index f855f4c075..b0140eb97a 100644 --- a/packages/coding-agent/src/core/agent-session-config.ts +++ b/packages/coding-agent/src/core/agent-session-config.ts @@ -1,5 +1,5 @@ import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { AgentAutonomousConfig } from "./autonomous.js"; +import type { AgentAutonomousConfig } from "../session/autonomy/autonomous.js"; export type AgentExecutionMode = "interactive" | "print" | "json" | "rpc" | "acp"; diff --git a/packages/coding-agent/src/core/agent-session-runtime.ts b/packages/coding-agent/src/core/agent-session-runtime.ts index 113f6ed92e..97a93ef67e 100644 --- a/packages/coding-agent/src/core/agent-session-runtime.ts +++ b/packages/coding-agent/src/core/agent-session-runtime.ts @@ -1,6 +1,11 @@ import { copyFileSync, existsSync, mkdirSync } from "node:fs"; import { basename, join, resolve } from "node:path"; -import type { AgentSession } from "./agent-session.js"; +import type { AgentSession } from "../session/agent-session.js"; +import type { + CreateRlmSubagentRuntimeOptions, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "../session/children/runtime-contracts.js"; import type { AgentSessionRuntimeConfig } from "./agent-session-config.js"; import type { AgentSessionCreationOptions, @@ -10,7 +15,6 @@ import type { import { isNoModelsAvailableMessage } from "./auth-guidance.js"; import type { ReplacedSessionContext, SessionShutdownEvent, SessionStartEvent } from "./extensions/index.js"; import { emitSessionShutdownEvent } from "./extensions/runner.js"; -import type { CreateRlmSubagentRuntimeOptions, RlmSubagentRuntime, SubagentRuntimeHost } from "./rlm-runtime.js"; import type { CreateAgentSessionResult } from "./sdk.js"; import { assertSessionCwdExists } from "./session-cwd.js"; import { SessionImportFileNotFoundError } from "./session-import-errors.js"; diff --git a/packages/coding-agent/src/core/agent-session-services.ts b/packages/coding-agent/src/core/agent-session-services.ts index 8da9b04bc0..7e229b56d0 100644 --- a/packages/coding-agent/src/core/agent-session-services.ts +++ b/packages/coding-agent/src/core/agent-session-services.ts @@ -2,19 +2,19 @@ import { join } from "node:path"; import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; import type { Model, ServiceTier } from "@earendil-works/pi-ai"; import { getAgentDir } from "../config.js"; +import type { AgentAutonomousConfig } from "../session/autonomy/autonomous.js"; +import type { SubagentRuntimeHost } from "../session/children/runtime-contracts.js"; import type { AgentSessionMessageController } from "./agent-messages.js"; import type { AgentObserveController } from "./agent-observe.js"; import type { AgentExecutionMode } from "./agent-session-config.js"; import { installAgentTraceUpload } from "./agent-traces.js"; import { AuthStorage } from "./auth-storage.js"; -import type { AgentAutonomousConfig } from "./autonomous.js"; import type { AgentRlmHeartbeatController } from "./cron-jobs.js"; import { createHerdrAgentStateExtension } from "./extensions/builtin/herdr-agent-state.js"; import type { SessionStartEvent, ToolDefinition } from "./extensions/index.js"; import { McpManager } from "./mcp/mcp-manager.js"; import { ModelRegistry } from "./model-registry.js"; import { DefaultResourceLoader, type DefaultResourceLoaderOptions, type ResourceLoader } from "./resource-loader.js"; -import type { SubagentRuntimeHost } from "./rlm-runtime.js"; import { type CreateAgentSessionResult, createAgentSession } from "./sdk.js"; import { semanticEdgeLedgerPath } from "./semantic-edges.js"; import type { SessionManager } from "./session-manager.js"; diff --git a/packages/coding-agent/src/core/agent-session.ts b/packages/coding-agent/src/core/agent-session.ts index e46f24bfcc..1e9c5990cd 100644 --- a/packages/coding-agent/src/core/agent-session.ts +++ b/packages/coding-agent/src/core/agent-session.ts @@ -1,2936 +1,33 @@ -import type { - Agent, - AgentContext, - AgentMessage, - AgentState, - AgentTool, - ThinkingLevel, -} from "@earendil-works/pi-agent-core"; -import type { AssistantMessage, Model, ServiceTier } from "@earendil-works/pi-ai"; -import { clampThinkingLevel, cleanupSessionResources, supportsFastMode } from "@earendil-works/pi-ai"; -import { createChildSessionDir, createInlineChildRuntime } from "../session/children/child-runtime.js"; -import { SessionChildState } from "../session/children/child-state.js"; -import { +export { + AgentSession, + type AgentSessionConfig, + type AgentSessionEvent, + type AgentSessionEventListener, + type AutoRefineReviewer, + type AutoRefineReviewRequest, + type CompactionReason, + CompactionSkippedError, compactRlmText, + type ExtensionBindings, + type GoalState, + type GoalStatus, + type ModelCycleResult, + type ParsedSkillBlock, + type PromptOptions, + parseSkillBlock, + RefineSkippedError, + type RlmChildAgentActivity, type RlmChildAgentSnapshot, type RlmChildAgentStatus, + type RlmMaxDepthSource, + type RlmMaxDepthStatus, rlmChildLabel, -} from "../session/children/child-types.js"; -import { SessionChildUsage } from "../session/children/child-usage.js"; -import { SessionChildren } from "../session/children/children.js"; -import { SessionCompaction } from "../session/compaction/compaction.js"; -import { - type CompactionExecutionHost, - type CompactionExecutionOptions, - performSessionCompaction, -} from "../session/compaction/compaction-execution.js"; -import { type ContextViewChild, SessionContextView } from "../session/context/context-view.js"; -import { SessionExport } from "../session/context/export.js"; -import { SessionHarnessContext } from "../session/context/harness-context.js"; -import { SessionHistoryNavigation } from "../session/context/history-navigation.js"; -import { SessionPendingContext } from "../session/context/pending-context.js"; -import { - type ExtensionBindings, - installExtensionToolHooks, - SessionExtensions, -} from "../session/extensions/extensions.js"; -import { SessionGoalContinuation } from "../session/goals/continuation.js"; -import { - createGoalContextMessage, - GOAL_CONTEXT_CUSTOM_TYPE, - GOAL_SKILL_NAME, - type GoalState, -} from "../session/goals/contracts.js"; -import { GoalController } from "../session/goals/controller.js"; -import { createGoalPersistence } from "../session/goals/persistence.js"; -import { SessionActionQueue } from "../session/input/action-queue.js"; -import { SessionActionRecovery } from "../session/input/action-recovery.js"; -import { SessionCommitFence, type SessionCommitLease } from "../session/input/commit-fence.js"; -import { SessionInputAdmission } from "../session/input/input-admission.js"; -import { SessionInputCheckpoints } from "../session/input/input-checkpoints.js"; -import { SessionInputDispatcher } from "../session/input/input-dispatcher.js"; -import { SessionInputScheduler } from "../session/input/input-scheduler.js"; -import { SessionMessageDelivery } from "../session/input/message-delivery.js"; -import { type PromptOptions, SessionPromptSubmission } from "../session/input/prompt-submission.js"; -import { SubmissionNormalizer } from "../session/input/submission-normalization.js"; -import { handleRlmHeartbeatHostRequest } from "../session/kernel/heartbeat-host-requests.js"; -import { SessionKernel } from "../session/kernel/kernel.js"; -import { KernelEnvironment } from "../session/kernel/kernel-environment.js"; -import { createSessionKernelHostHandlers } from "../session/kernel/kernel-host-handlers.js"; -import { handleAgentMessageHostRequest } from "../session/kernel/message-host-requests.js"; -import { handleAgentObserveHostRequest } from "../session/kernel/observe-host-requests.js"; -import { SessionModelSelection } from "../session/models/model-selection.js"; -import { type QueuedSessionAction, visibleSessionActionProjection } from "../session/prepared-actions.js"; -import { type AutoRefineReviewer, SessionRefinement } from "../session/refinement/refinement.js"; -import { type ExecuteBashOptions, type RunUserBashOptions, SessionBash } from "../session/tools/bash.js"; -import { SessionTools } from "../session/tools/tools.js"; -import { SessionAutonomousContinuation } from "../session/turns/autonomous-continuation.js"; -import { SessionCommandExecution } from "../session/turns/command-execution.js"; -import { SessionContinuation } from "../session/turns/continuation.js"; -import { SessionEvents } from "../session/turns/events.js"; -import { SessionRetry } from "../session/turns/retry.js"; -import { SessionTurnExecution } from "../session/turns/turn-execution.js"; -import { SessionTurnPolicy } from "../session/turns/turn-policy.js"; -import { TurnPreparer } from "../session/turns/turn-preparation.js"; -import { - AGENT_MESSAGE_SKILL_NAME, - type AgentSessionMessageController, - type AgentSessionMessageReceipt, -} from "./agent-messages.js"; -import { - AGENT_OBSERVE_SKILL_NAME, - type AgentObserveAgentSnapshot, - type AgentObserveController, - type AgentObserveListResult, - type AgentObserveRecentMessagesResult, - ORCHESTRATION_HEARTBEAT_SKILL_NAME, -} from "./agent-observe.js"; -import type { AgentAutonomousConfig } from "./autonomous.js"; -import type { BashResult } from "./bash-executor.js"; -import { COMPACT_SKILL_NAME, type CompactionResult, calculateContextTokens } from "./compaction/index.js"; -import type { AgentCronJob, AgentRlmHeartbeatController } from "./cron-jobs.js"; -import type { - ExtensionRunner, - ReplacedSessionContext, - SessionStartEvent, - ToolDefinition, - ToolInfo, -} from "./extensions/index.js"; -import type { HostRequestHandlers } from "./kernel/index.js"; -import type { AcpMcpServerConfig } from "./mcp/acp-mcp-types.js"; -import type { McpManager } from "./mcp/mcp-manager.js"; -import { type CustomMessage, createHeartbeatPromptMessage, type RefinementSource } from "./messages.js"; -import type { ModelRegistry } from "./model-registry.js"; -import type { PromptTemplate } from "./prompt-templates.js"; -import { providerRetryPolicy } from "./provider-retry.js"; -import { REFINE_SKILL_NAME, type RefinementResult } from "./refinement/index.js"; -import type { ResourceLoader } from "./resource-loader.js"; -import type { - CreateRlmSubagentRuntimeOptions, - RlmCreateSessionResult, - RlmDeleteSubagentResult, - RlmListSubagentsResult, - RlmSpawnHandle, - RlmSubagentRuntime, - SubagentRuntimeHost, -} from "./rlm-runtime.js"; -import { SemanticEdgeRecorder, semanticEdgeLedgerPath, wrapStreamFnWithSemanticEdges } from "./semantic-edges.js"; -import { ActionStore, type RuntimeActivity } from "./session-action-store.js"; -import type { SessionManager } from "./session-manager.js"; -import type { SettingsManager } from "./settings-manager.js"; -import { getPythonSkillRuntimeInfo, type Skill } from "./skills.js"; -import type { BuildSystemPromptOptions } from "./system-prompt.js"; -import type { IpythonKernelProvisioner } from "./tools/ipython.js"; - -export type { - RlmChildAgentActivity, - RlmChildAgentSnapshot, - RlmChildAgentStatus, -} from "../session/children/child-types.js"; -export { compactRlmText, rlmChildLabel } from "../session/children/child-types.js"; -export type { CompactionReason } from "../session/compaction/compaction.js"; -export { CompactionSkippedError } from "../session/compaction/compaction-execution.js"; -export type { GoalState, GoalStatus } from "../session/goals/contracts.js"; -export { SESSION_ACTION_RECOVERY_FORMAT_VERSION, type SessionActionRecoveryAction, type SessionActionRecoveryPayload, type SessionActionRecoveryRecord, type SessionActionRecoverySnapshot, -} from "../session/prepared-actions.js"; -export { RefineSkippedError } from "../session/refinement/refinement.js"; -export type { AgentSessionEvent, AgentSessionEventListener } from "../session/turns/events.js"; -export type { TurnExecutionPolicy } from "../session/turns/turn-preparation.js"; -export type { SessionStats } from "./session-stats.js"; -export { type ParsedSkillBlock, parseSkillBlock } from "./skill-blocks.js"; - -export interface AgentSessionConfig { - agent: Agent; - sessionManager: SessionManager; - settingsManager: SettingsManager; - serviceTierPreference?: ServiceTier; - cwd: string; - agentDir?: string; - scopedModels?: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>; - resourceLoader: ResourceLoader; - customTools?: ToolDefinition[]; - modelRegistry: ModelRegistry; - initialActiveToolNames?: string[]; - allowedToolNames?: string[]; - /** - * Whether the built-in long-running goals feature is available: the bundled - * goal skill in the Python kernel, its goal.* host handlers, and /goal. - * Default: true. - */ - includeGoals?: boolean; - agentMessageController?: AgentSessionMessageController; - agentObserveController?: AgentObserveController; - /** - * Whether the bundled compact skill and its compact.* host handlers are - * available to the model. Default: the compaction.agentCallable setting. - */ - includeCompactSkill?: boolean; - /** - * Optional host-side controller for the bundled rlm-heartbeat Python skill. - * When omitted, rlm_heartbeat.* host requests are unavailable. - */ - rlmHeartbeatController?: AgentRlmHeartbeatController; - /** - * Optional MCP integration manager. When present, its mcp.* host requests - * (refresh, begin_login) are exposed to the kernel. - */ - mcpManager?: McpManager; - /** - * Override base tools (useful for custom runtimes). - * - * These are synthesized into minimal ToolDefinitions internally so AgentSession can keep - * a definition-first registry even when callers provide plain AgentTool instances. - */ - baseToolsOverride?: Record; - extensionRunnerRef?: { current?: ExtensionRunner }; - sessionStartEvent?: SessionStartEvent; - rlmDepth?: number; - rlmMaxDepth?: number; - rlmSessionDir?: string; - rlmParentNodeId?: string; - rlmParentAgent?: string; - semanticParentSessionId?: string; - semanticSpawnedByRequestId?: string; - subagentRuntimeHost?: SubagentRuntimeHost; - autonomous?: AgentAutonomousConfig; - prewarmIpythonKernel?: boolean; - autoRefineReviewer?: AutoRefineReviewer; - /** - * When true, auto-refine runs synchronously between turns at the - * shouldStopAfterTurn boundary instead of in the background after - * agent_end. Used for print/headless autonomous runs so refinement - * never overlaps the primary model request. Default: false. - */ - serializedRefine?: boolean; - /** - * Initial goal to seed at session creation. Only applied when rlmDepth - * is 0 and no persisted thread_goal_state entry exists in the branch. - */ - initialGoal?: { objective: string; tokenBudget?: number }; -} - -export type { ExtensionBindings } from "../session/extensions/extensions.js"; -export type { PromptOptions } from "../session/input/prompt-submission.js"; -export type { ModelCycleResult } from "../session/models/model-selection.js"; -export type { AutoRefineReviewer, AutoRefineReviewRequest } from "../session/refinement/refinement.js"; - -import type { RlmMaxDepthStatus, SetRlmMaxDepthResult } from "./rlm-max-depth.js"; - -export type { RlmMaxDepthSource, RlmMaxDepthStatus, SetRlmMaxDepthResult } from "./rlm-max-depth.js"; - -export class AgentSession { - private readonly _tools: SessionTools; - private readonly _extensions: SessionExtensions; - private readonly _kernel: SessionKernel; - private readonly _kernelEnvironment: KernelEnvironment; - private get _extensionRunner(): ExtensionRunner { - return this._extensions.runner; - } - private get _ipythonKernelProvisioner(): IpythonKernelProvisioner | undefined { - return this._kernel.provisioner; - } - private get _rlmSessionDir(): string | undefined { - return this._kernelEnvironment.sessionDir; - } - private get _allowedToolNames(): ReadonlySet | undefined { - return this._tools.allowedToolNames; - } - private get _customTools(): ToolDefinition[] { - return this._tools.customTools; - } - private get _toolRegistry(): ReadonlyMap { - return this._tools.registry; - } - private get _baseSystemPrompt(): string { - return this._tools.baseSystemPrompt; - } - private set _baseSystemPrompt(prompt: string) { - this._tools.baseSystemPrompt = prompt; - } - private get _baseSystemPromptOptions(): BuildSystemPromptOptions { - return this._tools.baseSystemPromptOptions; - } - private readonly _childState: SessionChildState; - - private readonly _childUsage = new SessionChildUsage({ - sessionManager: { - getEntries: () => this.sessionManager.getEntries(), - appendChildUsageAttribution: (...args) => this.sessionManager.appendChildUsageAttribution(...args), - }, - invalidateOwnUsage: () => this._invalidateOwnUsage(), - afterParentDrain: (flush) => { - this._events.enqueue(flush); - }, - }); - private readonly _children = new SessionChildren({ - isDisposed: () => this._disposed || this._disposing, - isInputSuspended: () => this._inputScheduler.suspended, - isStreaming: () => this.isStreaming, - isSessionActive: () => this.isSessionActive, - getMessageController: () => this._agentMessageController, - getDepth: () => this._childState.depth, - getMaxDepth: () => this._childState.maxDepth, - getParentNodeId: () => this._rlmParentNodeId, - getCwd: () => this._cwd, - getSessionId: () => this.sessionId, - getSessionName: () => this.sessionName, - getSessionFile: () => this.sessionFile, - getThinkingLevel: () => this.thinkingLevel, - getSemanticEdges: () => this._semanticEdges, - getChildOwner: (child) => child._children, - getParentReplyCount: (child) => child._childState.replyCount, - getChildSessionDir: (child) => child._rlmSessionDir, - listRlmSubagents: () => this.listRlmSubagents(), - deleteRlmSubagent: (target) => this.deleteRlmSubagent(target), - registerRlmChildSession: (id, child) => this.registerRlmChildSession(id, child), - resolveModel: (reference, target) => this._resolveRlmSubagentModel(reference, target), - createSessionDir: () => this._createChildRlmSessionDir(), - createRuntimeOptions: (request) => this._createRlmSubagentRuntimeOptions(request), - createRuntime: (options) => this._createRlmSubagentRuntime(options), - createUsageTracker: () => this._childUsage.createTracker(this._findLastAssistantMessage()), - hasDeferredTerminalNotices: () => this._hasDeferredRlmTerminalNotices(), - waitForHeadlessIdle: () => this.waitForHeadlessIdle(), - waitForActivityChange: (signal) => this._waitForSessionActivityChange(signal), - deliverTerminalNotice: (message) => this._deferRlmTerminalNotice(message), - emit: (event) => this._emit(event), - onSettled: () => this._maybeResumeGoalContinuationAfterRlmWork(), - }); - - private readonly _turnPolicy = new SessionTurnPolicy({ - steeringStopPending: () => this._steeringStopPending, - stopGoalForTerminalMessage: (message) => this._stopGoalContinuationForTerminalMessage(message), - getGoals: () => this._goals, - accountAssistantBudget: (message) => this._goalContinuation.accountAssistantBudget(message), - getRefinement: () => this._refinement, - getEventQueue: () => this._events.queue, - getCompaction: () => this._compaction, - getMessages: () => this.agent.state.messages, - getSettings: () => this.settingsManager, - getModel: () => this.model, - getStore: () => this.sessionManager, - queueThresholdGoal: (message) => this._queueGoalContinuationForThresholdCompaction(message), - queueThresholdAutonomous: (message) => this._queueAutonomousContinuationForThresholdCompaction(message), - getQueuedCount: () => this.queuedActionCount, - getArrivalEpoch: () => this._inputAdmission.arrivalEpoch, - getGoalMessages: (context, signal) => this._getGoalContinuationMessages(context, signal), - getAutonomous: () => this._autonomousContinuation, - snapshotAutonomous: () => this._snapshotAutonomousRuntimeState(), - restoreAutonomous: (snapshot) => this._restoreAutonomousRuntimeSnapshot(snapshot), - }); - private readonly _turnExecution = new SessionTurnExecution({ - getPreparer: () => this._turnPreparer, - getFence: () => this._commitFence, - acquireFence: () => this._acquireSessionActionCommitFence(), - isDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), - isStreaming: () => this.isStreaming, - getBasePrompt: () => this._baseSystemPrompt, - getBasePromptOptions: () => this._baseSystemPromptOptions, - refreshExtensionSystemPrompt: (prompt, snapshot) => this._refreshExtensionSystemPrompt(prompt, snapshot), - getExtensions: () => this._extensionRunner, - getAgent: () => this.agent, - takeNextTurnMessages: () => this._takePendingNextTurnMessages(), - restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), - consumePendingDigest: () => this._harnessContext.consumePendingDigest(), - rearmDigest: () => this._harnessContext.rearmDigest(), - getDigest: () => this._harnessDigest(), - getLatestDigest: () => this._latestContextHarnessDigest(), - suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), - runSuppressed: (run) => this._runWithAutonomousContinuationSuppressed(run), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - emitQueueUpdate: () => this._emitQueueUpdate(), - hasCancelledCapture: () => this._hasCancelledDispatchCapture(), - getEventQueue: () => this._events.queue, - waitForRetry: () => this.waitForRetry(), - forgetContinuations: (messages) => this._forgetConsumedPostCompactionContinuations(messages), - }); - /** Session-owned actions. Items are never fed into Agent.steer/followUp. */ - private readonly _actionStore = new ActionStore(); - private readonly _inputCheckpoints = new SessionInputCheckpoints(this._actionStore, { - getFence: () => this._commitFence, - getScheduler: () => this._inputScheduler, - getEventQueue: () => this._events.queue, - acquireFence: (signal) => this._acquireSessionActionCommitFence(signal), - getStore: () => this.sessionManager, - assertAdmissionAvailable: () => this._assertSessionActionAdmissionAvailable(), - getContinuation: () => this._continuation, - scheduleInput: () => this._scheduleSessionInputPump(), - getAgent: () => this.agent, - getUnfinishedCount: () => this.unfinishedActionCount, - waitForIdle: () => this.waitForIdle(), - }); - private readonly _promptSubmission = new SessionPromptSubmission(this._actionStore, { - promptInjectedMessage: (text, message, options) => this._promptInjectedMessage(text, message, options), - waitForActivityChange: (signal) => this._waitForSessionActivityChange(signal), - queueAgentMessagePrompt: (text, streamingBehavior, customMessage) => - this.queueAgentMessagePrompt(text, streamingBehavior, customMessage), - getScheduler: () => this._inputScheduler, - getFence: () => this._commitFence, - isStreaming: () => this.isStreaming, - isCompacting: () => this.isCompacting, - isRetrying: () => this.isRetrying, - isBashRunning: () => this.isBashRunning, - resumeAdmission: () => this._resumeSessionInputAdmission(), - assertAdmissionAvailable: () => this._assertSessionActionAdmissionAvailable(), - acquireAdmissionFence: (signal) => this._acquireDirectTurnAdmissionFence(signal), - normalize: (text, images, policy) => this._normalizeSubmission(text, images, policy), - settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), - canStartImmediately: () => this._canStartSessionActionImmediately(), - admit: (action, options) => this._admitSessionInput(action, options), - waitForInputIdle: () => this.waitForSessionInputIdle(), - isBusy: (point) => this._isBusyForSessionInput(point), - takeNextTurnMessages: () => this._takePendingNextTurnMessages(), - restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), - appendNextTurnMessage: (message) => this._pendingContext.appendMessages(message), - getActivity: () => this._runtimeActivity(), - suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), - observeDeferral: (action) => this._observeSessionActionDeferral(action), - rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), - cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), - emitQueueUpdate: () => this._emitQueueUpdate(), - getClearEpoch: () => this._actionQueue.clearEpoch, - queuePrompt: (schedule, text, images, options) => this._queuePreparedPrompt(schedule, text, images, options), - resetParentReply: () => { - this._childState.resetReply(); - }, - getAgent: () => this.agent, - getStore: () => this.sessionManager, - emit: (event) => this._emit(event), - }); - private readonly _inputAdmission = new SessionInputAdmission(this._actionStore, { - getScheduler: () => this._inputScheduler, - isDisposed: () => this._disposed, - isDisposing: () => this._disposing, - isStreaming: () => this.isStreaming, - rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), - emitQueueUpdate: () => this._emitQueueUpdate(), - resumeAdmission: () => this._resumeSessionInputAdmission(), - scheduleInput: () => this._scheduleSessionInputPump(), - suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), - }); - private readonly _pendingContext = new SessionPendingContext(this._actionStore, { - getScheduler: () => this._inputScheduler, - getFence: () => this._commitFence, - isDisposed: () => this._disposed, - isDisposing: () => this._disposing, - admit: (action, options) => this._admitSessionInput(action, options), - scheduleInput: () => this._scheduleSessionInputPump(), - addCheckpointWaiter: (waiter) => this._inputCheckpoints.add(waiter), - removeCheckpointWaiter: (waiter) => this._inputCheckpoints.remove(waiter), - acquireFence: (signal) => this._acquireSessionActionCommitFence(signal), - cancelActions: (predicate, error, candidates) => this._cancelSessionActions(predicate, error, candidates), - }); - private readonly _actionQueue = new SessionActionQueue(this._actionStore, { - formatLabel: (text) => compactRlmText(text), - getScheduler: () => this._inputScheduler, - getAgent: () => this.agent, - rearmDigest: () => this._harnessContext.rearmDigest(), - restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - emitQueueUpdate: () => this._emitQueueUpdate(), - settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), - rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), - admit: (action, options) => this._admitSessionInput(action, options), - queuePrompt: (schedule, text, images, options) => this._queuePreparedPrompt(schedule, text, images, options), - resumeQueuedWork: () => this.resumeQueuedWork(), - }); - private readonly _actionRecovery = new SessionActionRecovery(this._actionStore, { - isTerminalNoticeAction: (action) => this._isRlmTerminalNoticeAction(action), - retainTerminalNotice: (id) => this._pendingContext.retainTerminalNotice(id), - releaseTerminalNotice: (id) => this._pendingContext.releaseTerminalNotice(id), - admit: (action, options) => this._admitSessionInput(action, options), - }); - private readonly _commandExecution = new SessionCommandExecution(this._actionStore, { - getFence: () => this._commitFence, - acquireFence: () => this._acquireSessionActionCommitFence(), - getRefinement: () => this._refinement, - isDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), - getActivity: () => this._runtimeActivity(), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - emitQueueUpdate: () => this._emitQueueUpdate(), - settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), - rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), - compact: (instructions, options) => this.compact(instructions, options), - refine: (options, internal) => this.refine(options, internal), - handleGoalCommand: (text, images) => this._handleGoalSlashCommand(text, images), - handleAutonomousCommand: (text) => this._handleAutonomousSlashCommand(text), - getGoalState: () => this._goals.state, - getStore: () => this.sessionManager, - getAgent: () => this.agent, - emit: (event) => this._emit(event), - }); - private readonly _events = new SessionEvents(this._actionStore, { - getAgent: () => this.agent, - getStore: () => this.sessionManager, - getExtensions: () => this._extensionRunner, - getRetry: () => this._retry, - getCompaction: () => this._compaction, - getRefinement: () => this._refinement, - addAutonomousUsage: (usage) => this._autonomousContinuation.recordUsage(usage), - applyLateMessages: (message) => this._applyLateIpythonSentAgentMessages(message), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), - getSnapshot: () => this.getSessionActionSnapshot(), - accountAssistantBudget: (message) => this._goalContinuation.accountAssistantBudget(message), - finishGoal: (message) => this._finishGoalForTerminalAssistantMessage(message), - checkCompaction: (message) => this._checkCompaction(message), - }); - private readonly _messageDelivery = new SessionMessageDelivery(this._actionStore, { - isDisposed: () => this._disposed, - getMessages: () => this.agent.state.messages, - getStore: () => this.sessionManager, - enqueue: (work) => this._events.enqueue(work), - emit: (event) => this._emit(event), - promptUntilAccepted: (text, options) => this.promptUntilAccepted(text, options), - cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), - }); - private readonly _submissionNormalizer = new SubmissionNormalizer({ - getExtensions: () => this._extensionRunner, - getPrompts: () => this.promptTemplates, - getSkills: () => this.resourceLoader.getSkills().skills, - }); - private readonly _refinement: SessionRefinement; - readonly agent: Agent; - readonly sessionManager: SessionManager; - readonly settingsManager: SettingsManager; - private readonly _export: SessionExport; - private readonly _contextView: SessionContextView; - private readonly _modelSelection: SessionModelSelection; - private get _scopedModels() { - return this._modelSelection.scopedModels; - } - - private readonly _inputScheduler = new SessionInputScheduler({ - canSchedule: () => !this._disposed && !this._disposing && this._hasSelectableSessionInput(), - run: (epoch) => this._inputDispatcher.run(epoch), - }); - private readonly _inputDispatcher = new SessionInputDispatcher(this._actionStore, { - isDisposed: () => this._disposed || this._disposing, - getEpoch: () => this._inputScheduler.epoch, - getActivity: () => this._runtimeActivity(), - isBusy: () => this._isBusyForSessionInput("pump"), - isHandoffDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), - getDeliveryMode: (delivery) => (delivery === "next_turn_boundary" ? this.steeringMode : this.followUpMode), - waitForAgentIdle: () => this.agent.waitForIdle(), - hasCancelledDispatchCapture: () => this._hasCancelledDispatchCapture(), - getEventQueue: () => this._events.queue, - waitForRefinement: () => this._refinement._waitForRefineIdle(), - getTranscript: () => this.agent.state.messages, - startTurns: (actions, epoch) => this._startPreparedTurnActions(actions, epoch), - executeCommand: (action, epoch) => this._executeSelectedSessionCommand(action, epoch), - settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), - releaseTurn: (id) => { - this._pendingContext.releaseTerminalNotice(id); - }, - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - emitQueueUpdate: () => this._emitQueueUpdate(), - surfaceError: (error) => this._surfaceSessionInputError(error), - schedule: () => this._scheduleSessionInputPump(), - }); - private readonly _commitFence = new SessionCommitFence(); - private readonly _turnPreparer = new TurnPreparer({ - hasRefinement: () => this._refinement.isApplying, - waitForRefinement: () => this._refinement._waitForRefineIdle(), - flushPendingBash: () => this._flushPendingBashMessages(), - validate: () => this._validateCanStartAgentRun(), - compact: () => this._runPreTurnCompaction(), - pendingModelSelection: () => this._pendingModelSelectEmit(), - }); - // Checkpoint, handoff, and activity waiters share lifecycle-edge notifications to avoid polling. - - private readonly _autonomousContinuation: SessionAutonomousContinuation; - private readonly _goalContinuation: SessionGoalContinuation; - private get _goals(): GoalController { - return this._goalContinuation.controller; - } - - private readonly _compaction = new SessionCompaction({ - includesCompactSkill: () => this._includeCompactSkill, - getContextUsage: () => this.getContextUsage(), - getModel: () => this.model, - isStreaming: () => this.isStreaming, - getSettings: () => this.settingsManager.getCompactionSettings(), - runAutomatic: (reason, willRetry) => this._runAutoCompaction(reason, willRetry), - queueGoalContinuation: (message) => this._queueGoalContinuationForThresholdCompaction(message), - queueAutonomousContinuation: (message) => this._queueAutonomousContinuationForThresholdCompaction(message), - beginRefinementAbort: () => this._refinement.beginAbortedTurnCleanup(), - getRequiredAuth: (model) => this._getRequiredRequestAuth(model), - getAuth: (model) => this._modelRegistry.getApiKeyAndHeaders(model), - perform: (options) => this._performCompaction(options), - disconnect: () => this._disconnectFromAgent(), - reconnect: () => this._reconnectToAgent(), - abortSession: () => this.abort(), - getContinuationState: () => ({ - scheduled: this._continuation.isScheduled, - continueAfterSessionInput: this._continuation.current?.continueAfterSessionInput ?? false, - }), - afterManualCompaction: (signal, scheduled, continueAfterInput) => - this._afterManualCompaction(signal, scheduled, continueAfterInput), - getMessages: () => this.agent.state.messages, - replaceMessages: (messages) => { - this.agent.state.messages = messages; - }, - hasAgentQueuedMessages: () => this.agent.hasQueuedMessages(), - hasPendingSessionWork: () => this.hasPendingSessionWork, - scheduleContinuation: (continueAfterInput) => this._schedulePostCompactionContinue(continueAfterInput), - scheduleRefinement: (willContinue) => this._refinement._scheduleAutoRefineAfterCompaction(willContinue), - takeThresholdAutonomousMessages: () => this._autonomousContinuation.takePendingThresholdMessages(), - getThresholdGoalContinuation: () => this._goalContinuation.thresholdContinuation, - clearAutonomousContinuations: (shouldContinue, messages) => - this._clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction(shouldContinue, messages), - clearGoalContinuation: (message) => this._clearQueuedGoalContinuationAfterCancelledThresholdCompaction(message), - getSessionStore: () => this.sessionManager, - retainUnpersistedOutcome: (message) => { - this._harnessContext.retainOutcome(message); - }, - emit: (event) => this._emit(event), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - scheduleInput: () => this._scheduleSessionInputPump(), - }); - private readonly _compactionExecution: CompactionExecutionHost = { - getSessionStore: () => this.sessionManager, - getSettings: () => this.settingsManager.getCompactionSettings(), - getSemanticEdges: () => this._semanticEdges, - getExtensions: () => this._extensionRunner, - getThinkingLevel: () => this.thinkingLevel, - getRetryPolicy: () => providerRetryPolicy(this.settingsManager), - getSessionId: () => this.sessionId, - getHarnessDigest: () => this._harnessDigest(), - rebuildContext: () => { - this.agent.state.messages = this.sessionManager.buildSessionContext().messages; - this._mergeUnpersistedOutcomes(this.agent.state.messages); - this._restoreLateIpythonSentAgentMessages(); - }, - syncKernelState: () => this._syncKernelStateAfterCompaction(), - reapDeletedChildren: () => this._reapDeletedRlmSubagentRuntimesAfterCompaction(), - }; - - private get _branchSummaryOperation(): Promise | undefined { - return this._history.operation; - } - private readonly _history: SessionHistoryNavigation; - - private readonly _retry = new SessionRetry({ - getRetrySettings: () => this.settingsManager.getRetrySettings(), - getMaxRetryDelayMs: () => this.settingsManager.getProviderRetrySettings().maxRetryDelayMs, - getContextWindow: () => this.model?.contextWindow ?? 0, - getAuthSource: (provider) => this._modelRegistry.getCurrentProviderAuthSourceToken(provider), - markAuthSourceStale: (token) => this._modelRegistry.markProviderAuthSourceStale(token), - markAuthStale: (provider) => this._modelRegistry.markProviderAuthStale(provider), - hasPayloadHooks: () => this._extensionRunner.hasHandlers("before_provider_request"), - prepareTurnRetry: () => this._semanticEdges.prepareTurnRetry(), - clearTurnRetry: () => this._semanticEdges.clearTurnRetry(), - removeLastAssistant: () => { - const messages = this.agent.state.messages; - if (messages.length > 0 && messages[messages.length - 1].role === "assistant") { - this.agent.state.messages = messages.slice(0, -1); - } - }, - continue: () => this.agent.continue(), - waitForIdle: () => this.agent.waitForIdle(), - cancelCompaction: () => { - this._compaction.abortAutomatic(); - this._cancelPostCompactionContinue(); - }, - emit: (event) => this._emit(event), - onResolved: () => { - this._notifySessionInputCheckpointChange(); - this._scheduleSessionInputPump(); - }, - }); - /** Fresh/empty contexts defer digest injection to the first committed turn so untouched sessions stay empty. */ - private readonly _harnessContext: SessionHarnessContext; - - private readonly _bash = new SessionBash({ - getCwd: () => this.sessionManager.getCwd(), - getShellCommandPrefix: () => this.settingsManager.getShellCommandPrefix(), - getShellPath: () => this.settingsManager.getShellPath(), - isStreaming: () => this.isStreaming, - intercept: (event) => this._extensionRunner.emitUserBash(event), - emit: (event) => this._emit(event), - appendMessage: (message) => { - this.agent.state.messages.push(message); - this.sessionManager.appendMessage(message); - }, - onStateChange: () => this._notifySessionInputCheckpointChange(), - onUserBashEnd: () => this._drainQueuedMessagesAfterBash(), - executeBash: (command, onChunk, options) => this.executeBash(command, onChunk, options), - recordBashResult: (command, result, options) => this.recordBashResult(command, result, options), - }); - - private readonly _resourceLoader: ResourceLoader; - private readonly _cwd: string; - private readonly _agentDir?: string; - private readonly _initialActiveToolNames?: string[]; - private readonly _includeGoals: boolean; - private readonly _includeCompactSkill: boolean; - private _rlmHeartbeatController?: AgentRlmHeartbeatController; - private readonly _agentMessageController?: AgentSessionMessageController; - private readonly _agentObserveController?: AgentObserveController; - private readonly _mcpManager?: McpManager; - private _disposed = false; - private readonly _disposeCallbacks = new Set<() => void | Promise>(); - private _disposeCallbacksPromise?: Promise; - // Set at the start of async teardown so a child finishing mid-disposeAsync doesn't - // re-populate the retained map after it's been cleared. - private _disposing = false; - private _disposeAsyncPromise?: Promise; - private readonly _semanticEdges: SemanticEdgeRecorder; - private readonly _rlmParentNodeId?: string; - private readonly _rlmParentAgent?: string; - - private readonly _modelRegistry: ModelRegistry; - - private readonly _continuation = new SessionContinuation({ - waitForAgentIdle: () => this.agent.waitForIdle(), - waitForRetry: () => this.waitForRetry(), - waitForRefinement: () => this._refinement._waitForRefineIdle(), - queuedWorkPauseCount: () => this._inputScheduler.queuedWorkPauseCount, - addCheckpointWaiter: (waiter) => { - this._inputCheckpoints.add(waiter); - }, - removeCheckpointWaiter: (waiter) => { - this._inputCheckpoints.remove(waiter); - }, - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - compactionOperation: () => this._compaction.operation, - isRefinementApplying: () => this._refinement.isApplying, - acquireCommitFence: () => this._acquireSessionActionCommitFence(), - scheduleRefinement: () => this._refinement._scheduleAutoRefineAfterAgentEnd(), - unfinishedActionCount: () => this.unfinishedActionCount, - isInputRequested: () => this._inputScheduler.requested, - scheduleInput: () => this._scheduleSessionInputPump(), - continue: () => this.agent.continue(), - waitForIdleOrSettlement: (token) => this._waitForIdleOrSettlement(token), - removeQueuedMessages: (predicate) => this.agent.removeQueuedMessages(predicate), - followUp: (message) => this.agent.followUp(message), - onMessageConsumed: (message) => { - this._autonomousContinuation.forgetSnapshot(message); - }, - }); - - constructor(config: AgentSessionConfig) { - this.agent = config.agent; - this.sessionManager = config.sessionManager; - this.settingsManager = config.settingsManager; - this._harnessContext = new SessionHarnessContext({ - sessionManager: this.sessionManager, - getMessages: () => this.messages, - getActiveToolNames: () => this.getActiveToolNames(), - getVisibleSkills: () => this._modelVisibleSkills(), - loadHarnessState: () => this._refinement._loadMergedHarnessState(), - applyLateSentMessages: (message) => this._applyLateIpythonSentAgentMessages(message), - }); - this._export = new SessionExport({ - sessionManager: this.sessionManager, - getState: () => this.state, - getTheme: () => this.settingsManager.getTheme(), - getToolDefinition: (name) => this.getToolDefinition(name), - }); - this._contextView = new SessionContextView({ - getContextUsage: () => this.getContextUsage(), - sessionManager: this.sessionManager, - getMessages: () => this.messages, - getModel: () => this.model, - findModel: (provider, modelId) => this._modelRegistry.find(provider, modelId), - subtractUnindexedChildUsage: (ownUsage, entries) => this._childUsage.subtractUnindexed(ownUsage, entries), - getRlmSessionDir: () => this._rlmSessionDirForReading(), - getLiveChildren: () => this._contextViewChildren(), - }); - - this._history = new SessionHistoryNavigation({ - getSessionId: () => this.sessionId, - getRetryPolicy: () => providerRetryPolicy(this.settingsManager), - sessionManager: this.sessionManager, - settingsManager: this.settingsManager, - getModel: () => this.model, - getExtensions: () => this._extensionRunner, - getRequiredAuth: (model) => this._getRequiredRequestAuth(model), - acquireQueuedWorkPause: () => this.acquireQueuedWorkPause(), - acquireCommitFence: () => this._acquireSessionActionCommitFence(), - runWithCommitFence: (lease, run) => this._commitFence.run(lease, run), - waitForAgentIdle: () => this.agent.waitForIdle(), - getEventQueue: () => this._events.queue, - invalidateRefinement: () => this._refinement._invalidatePendingAutoRefineForBranchChange(), - rebuildBranchContext: () => { - this.agent.state.messages = this.sessionManager.buildSessionContext().messages; - this._mergeUnpersistedOutcomes(this.agent.state.messages); - this._restoreLateIpythonSentAgentMessages(); - this._ensureHarnessDigestContext(); - this._goals.reload(); - this._reloadRlmMaxDepthFromBranch(); - this._invalidateQueuedPromptPreparation(); - }, - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - }); - - this._refinement = new SessionRefinement( - { - sessionManager: this.sessionManager, - settingsManager: this.settingsManager, - getRetryPolicy: () => providerRetryPolicy(this.settingsManager), - getSessionId: () => this.sessionId, - isDisposed: () => this._disposed, - isDisposing: () => this._disposing, - isStreaming: () => this.isStreaming, - isCompacting: () => this.isCompacting, - getDepth: () => this._childState.depth, - getRlmSessionDir: () => this._rlmSessionDir, - getModel: () => this.model, - getThinkingLevel: () => this.thinkingLevel, - getMessages: () => this.agent.state.messages, - getRequiredRequestAuth: (model) => this._getRequiredRequestAuth(model), - getExtensionRunner: () => this._extensionRunner, - getEventQueue: () => this._events.queue, - getCompactionOperation: () => this._compaction.operation, - getBranchSummaryOperation: () => this._branchSummaryOperation, - waitForAgentIdle: () => this.agent.waitForIdle(), - dispatchRefine: (options, internal) => this.refine(options, internal), - disconnect: () => this._disconnectFromAgent(), - reconnect: () => this._reconnectToAgent(), - emit: (event) => this._emit(event), - retainUnpersistedOutcome: (message) => this._harnessContext.retainOutcome(message), - notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), - scheduleInputPump: () => this._scheduleSessionInputPump(), - isContinuationScheduled: () => this._continuation.isScheduled, - cancelContinuation: () => this._cancelPostCompactionContinue(), - }, - { - serializedRefine: config.serializedRefine, - autoRefineReviewer: config.autoRefineReviewer?.bind(this), - }, - ); - this._modelSelection = new SessionModelSelection( - { - getModel: () => this.model, - getState: () => this.agent.state, - setThinkingLevel: (level) => this.setThinkingLevel(level), - getAvailableThinkingLevels: () => this.getAvailableThinkingLevels(), - supportsThinking: () => this.supportsThinking(), - getRegistry: () => this._modelRegistry, - getExtensions: () => this._extensionRunner, - sessionManager: this.sessionManager, - settingsManager: this.settingsManager, - emit: (event) => this._emit(event), - }, - config.serviceTierPreference ?? config.agent.state.serviceTier, - config.scopedModels ?? [], - ); - this._resourceLoader = config.resourceLoader; - this._cwd = config.cwd; - this._agentDir = config.agentDir; - this._modelRegistry = config.modelRegistry; - this._initialActiveToolNames = config.initialActiveToolNames; - this._includeGoals = config.includeGoals ?? true; - this._includeCompactSkill = config.includeCompactSkill ?? this.settingsManager.getCompactionAgentCallable(); - this._rlmHeartbeatController = config.rlmHeartbeatController; - this._agentMessageController = config.agentMessageController; - this._agentObserveController = config.agentObserveController; - this._mcpManager = config.mcpManager; - this._childState = new SessionChildState( - { - sessionManager: this.sessionManager, - settingsManager: this.settingsManager, - getRlmMaxDepthStatus: () => this.getRlmMaxDepthStatus(), - refreshPrompt: (preserveExtensionPrompt) => { - const oldBase = this._baseSystemPrompt; - this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); - this.agent.state.systemPrompt = preserveExtensionPrompt - ? this._refreshExtensionSystemPrompt(this.agent.state.systemPrompt, oldBase) - : this._baseSystemPrompt; - }, - emitRecap: (recap) => this._emit({ type: "recap_update", recap }), - }, - config, - ); - - this._rlmParentNodeId = config.rlmParentNodeId; - this._rlmParentAgent = config.rlmParentAgent; - this._kernelEnvironment = new KernelEnvironment( - { - agentDir: this._agentDir, - authStorage: this._modelRegistry.authStorage, - resourceLoader: this._resourceLoader, - getDepth: () => this._childState.depth, - getMaxDepth: () => this._childState.maxDepth, - getArtifactDir: () => this.sessionManager.getSessionArtifactDir(), - getLocalHarnessStateDir: () => this._refinement._localHarnessStateDir(), - }, - config.rlmSessionDir, - ); - this._kernel = new SessionKernel( - { - cwd: this._cwd, - getArtifactDir: () => this.sessionManager.getSessionArtifactDir(), - getSessionId: () => this.sessionId, - getEnv: () => this._rlmKernelEnv(), - getShellCommandPrefix: () => this.settingsManager.getShellCommandPrefix(), - getShellPath: () => this.settingsManager.getShellPath(), - createHostHandlers: () => this._createKernelHostHandlers(), - recordLateSentAgentMessage: (id, message) => this._recordLateIpythonSentAgentMessage(id, message), - getMessages: () => this.agent.state.messages, - appendCustomMessageEntry: (...args) => this.sessionManager.appendCustomMessageEntry(...args), - emit: (event) => this._emit(event), - sendCustomMessage: (message, options) => this.sendCustomMessage(message, options), - }, - (config.prewarmIpythonKernel ?? false) && this._childState.depth === 0, - ); - this._tools = new SessionTools( - { - cwd: this._cwd, - resourceLoader: this._resourceLoader, - getExtensionRunner: () => this._extensionRunner, - getSessionFile: () => this.sessionManager.getSessionFile(), - getModelVisibleSkills: () => this._modelVisibleSkills(), - getDepth: () => this._childState.depth, - getMaxDepth: () => this._childState.maxDepth, - getParentAgent: () => this._rlmParentAgent, - getMcpManager: () => this._mcpManager, - getProvisioner: () => this._kernel.provisioner, - getActiveToolNames: () => this.getActiveToolNames(), - setActiveToolsByName: (names) => this.setActiveToolsByName(names), - getActiveTools: () => this.agent.state.tools, - setActiveTools: (tools) => { - this.agent.state.tools = tools; - }, - setSystemPrompt: (prompt) => { - this.agent.state.systemPrompt = prompt; - }, - isStreaming: () => this.isStreaming, - rebuildRuntime: (options) => this._buildRuntime(options), - acquireInputPause: () => this.acquireSessionInputPause(), - waitForAgentIdle: () => this.agent.waitForIdle(), - getEventQueue: () => this._events.queue, - }, - { - customTools: config.customTools, - allowedToolNames: config.allowedToolNames, - baseToolsOverride: config.baseToolsOverride, - }, - ); - this._extensions = new SessionExtensions( - { - cwd: this._cwd, - sessionManager: this.sessionManager, - resourceLoader: this._resourceLoader, - modelRegistry: this._modelRegistry, - getModelRegistry: () => this.modelRegistry, - getPromptTemplates: () => this.promptTemplates, - bindShutdownHandler: (handler) => handler?.bind(this), - getAgentMessageController: () => this._agentMessageController, - refreshCurrentModel: () => this._refreshCurrentModelFromRegistry(), - sendCustomMessage: (message, options) => this.sendCustomMessage(message, options), - sendUserMessage: (content, options) => this.sendUserMessage(content, options), - setSessionName: (name) => this.setSessionName(name), - getActiveToolNames: () => this.getActiveToolNames(), - getAllTools: () => this.getAllTools(), - setActiveToolsByName: (names) => this.setActiveToolsByName(names), - refreshTools: () => this._refreshToolRegistry(), - setModel: (model) => this.setModel(model), - getThinkingLevel: () => this.thinkingLevel, - setThinkingLevel: (level) => this.setThinkingLevel(level), - getModel: () => this.model, - isStreaming: () => this.isStreaming, - getSignal: () => this.agent.signal, - abort: () => this.abort(), - getQueuedActionCount: () => this.queuedActionCount, - getContextUsage: () => this.getContextUsage(), - compact: (instructions) => this.compact(instructions), - getSystemPrompt: () => this.systemPrompt, - rebuildSystemPrompt: () => { - this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); - this.agent.state.systemPrompt = this._baseSystemPrompt; - }, - reloadSettings: () => this.settingsManager.reload(), - getMcpManager: () => this._mcpManager, - rebuildRuntime: (options) => this._buildRuntime(options), - }, - config.sessionStartEvent ?? { type: "session_start", reason: "startup" }, - config.extensionRunnerRef, - ); - - this._semanticEdges = new SemanticEdgeRecorder({ - ledgerPath: semanticEdgeLedgerPath({ - rlmSessionDir: this._rlmSessionDir, - sessionArtifactDir: this.sessionManager.getSessionArtifactDir(), - }), - sessionId: this.sessionManager.getSessionId(), - parentSessionId: config.semanticParentSessionId, - spawnedByRequestId: config.semanticSpawnedByRequestId, - }); - this.agent.streamFn = wrapStreamFnWithSemanticEdges(this.agent.streamFn, this._semanticEdges); - this._childState.initializeParentReply(); - this._children.setRuntimeHost(config.subagentRuntimeHost); - this._autonomousContinuation = new SessionAutonomousContinuation(config.autonomous, { - getStatus: () => this.getAutonomousStatus(), - getCwd: () => this._cwd, - getAgent: () => this.agent, - getStore: () => this.sessionManager, - emit: (event) => this._emit(event), - getContinuation: () => this._continuation, - getArrivalEpoch: () => this._inputAdmission.arrivalEpoch, - admit: (action, options) => this._admitSessionInput(action, options), - cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), - emitQueueUpdate: () => this._emitQueueUpdate(), - getCompaction: () => this._compaction, - getUnfinishedActionCount: () => this.unfinishedActionCount, - cancelContinuation: () => this._cancelPostCompactionContinue(), - }); - const goalPersistence = createGoalPersistence(this.sessionManager); - this._goalContinuation = new SessionGoalContinuation( - new GoalController(goalPersistence, (goal) => this._emit({ type: "goal_update", goal })), - this._actionStore, - { - getGoalState: () => this.goalState, - queuePrompt: (schedule, text, images, options) => - this._queuePreparedPrompt(schedule, text, images, options), - getScheduler: () => this._inputScheduler, - isDisposed: () => this._disposed, - isDisposing: () => this._disposing, - hasUnsettledChildWork: () => this._hasUnsettledRlmQuiescenceWork(), - ensureRuntimeActive: (context) => this._ensureGoalRuntimeActive(context), - admit: (action, options) => this._admitSessionInput(action, options), - cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), - clearPendingGoalContexts: () => { - this._pendingContext.removeMessagesMatching( - (message) => message.customType === GOAL_CONTEXT_CUSTOM_TYPE, - ); - }, - emitQueueUpdate: () => this._emitQueueUpdate(), - emitGoalUpdate: () => this._emitGoalUpdate(), - validate: () => this._validateCanStartAgentRun(), - isStreaming: () => this.isStreaming, - includesGoals: () => this._includeGoals, - getAgent: () => this.agent, - }, - ); - // Seed initial goal from CLI --goal flag, but only for top-level sessions - // and only when the branch contains only bootstrap entry types (model_change, - // thinking_level_change, service_tier_change) and no persisted - // thread_goal_state. This prevents reseeding after clear/complete/error - // or restart/rehydration of a session that already has messages or a goal. - if (this._childState.depth === 0 && config.initialGoal && goalPersistence.canSeed()) { - this._startGoal(config.initialGoal.objective, config.initialGoal.tokenBudget); - // Goal context is the model's only source of goal visibility; action - // admission is unavailable mid-construction, so ride the next turn. - this._pendingContext.appendMessages(createGoalContextMessage(this._goals.state, "continuation")); - } - this._restoreLateIpythonSentAgentMessages(); - this._goals.restartAccounting(); - - this._events.reconnectToAgent(); - this._installAgentToolHooks(); - this._installAgentTurnHook(); - this._installAgentContinuationHook(); - - this._buildRuntime({ - activeToolNames: this._initialActiveToolNames, - includeAllExtensionTools: true, - }); - this._ensureHarnessDigestContext(); - } - - /** Refreshes MCP provider registrations without rebuilding the session runtime. */ - refreshMcpProviders(): void { - this._mcpManager?.refresh(); - } - - /** - * Set the RLM heartbeat controller after construction. Used by - * print/headless mode to attach an in-process heartbeat scheduler - * when the session is created outside the daemon. - */ - setRlmHeartbeatController(controller: AgentRlmHeartbeatController): void { - if (this._rlmHeartbeatController === controller) { - return; - } - this._rlmHeartbeatController = controller; - this._buildRuntime({ - activeToolNames: this.getActiveToolNames(), - includeAllExtensionTools: true, - }); - this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); - this.agent.state.systemPrompt = this._baseSystemPrompt; - } - - replaceAcpMcpServers(servers: readonly AcpMcpServerConfig[], ownerId: string): void { - this._tools.replaceAcpMcpServers(servers, ownerId); - } - - releaseAcpMcpServers(ownerId: string, serverNames: readonly string[]): Promise { - return this._tools.releaseAcpMcpServers(ownerId, serverNames); - } - - get modelRegistry(): ModelRegistry { - return this._modelRegistry; - } - - setSubagentRuntimeHost(host?: SubagentRuntimeHost): void { - this._children.setRuntimeHost(host); - } - - private _getRequiredRequestAuth( - ...args: Parameters - ): ReturnType { - return this._modelSelection.getRequiredRequestAuth(...args); - } - - /** - * Install tool hooks once on the Agent instance. - * - * The callbacks read `this._extensionRunner` at execution time, so extension reload swaps in the - * new runner without reinstalling hooks. Extension-specific tool wrappers are still used to adapt - * registered tool execution to the extension context. Tool call and tool result interception now - * happens here instead of in wrappers. - */ - private _installAgentToolHooks(): void { - installExtensionToolHooks( - this.agent, - () => this._extensionRunner, - () => this._events.queue, - ); - } - - private _installAgentContinuationHook(): void { - this.agent.getContinuationMessages = (context, signal) => this._getContinuationMessages(context, signal); - } - - private _installAgentTurnHook(): void { - this.agent.shouldStopBeforeTurn = () => this._shouldStopBeforeTurn(); - this.agent.shouldStopAfterTurn = (context) => this._shouldStopAfterTurn(context); - } - - private _emit(...args: Parameters): ReturnType { - return this._events.emit(...args); - } - - private _emitQueueUpdate( - ...args: Parameters - ): ReturnType { - return this._events.emitQueueUpdate(...args); - } - - private _restoreLateIpythonSentAgentMessages( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.restoreLateIpythonSentAgentMessages(...args); - } - - private _applyLateIpythonSentAgentMessages( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.applyLateIpythonSentAgentMessages(...args); - } - - private _recordLateIpythonSentAgentMessage( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.recordLateIpythonSentAgentMessage(...args); - } - - private _emitGoalUpdate(): void { - this._emit({ type: "goal_update", goal: this.goalState }); - } - - private _reloadRlmMaxDepthFromBranch(): void { - this._childState.reloadFromBranch(); - } - - private _cancelSessionActions( - ...args: Parameters - ): ReturnType { - return this._actionQueue.cancelSessionActions(...args); - } - - private _startGoal( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.startGoal(...args); - } - - private _finishGoalForTerminalAssistantMessage( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.finishGoalForTerminalAssistantMessage(...args); - } - - private _stopGoalContinuationForTerminalMessage( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.stopGoalContinuationForTerminalMessage(...args); - } - - private _handleAutonomousSlashCommand( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.handleAutonomousSlashCommand(...args); - } - - private _validateCanStartAgentRun( - ...args: Parameters - ): ReturnType { - return this._modelSelection.validateCanStartAgentRun(...args); - } - - /** - * Goals are pursued through the kernel goal skill, so the only tool the - * model needs is ipython. Force-activate it (including into a live - * continuation context) so the model can always reach `goal.complete()`. - */ - private _ensureGoalRuntimeActive(context?: AgentContext): void { - if (!this._includeGoals) { - throw new Error("Goals are disabled. Enable goals before using /goal."); - } - const ipythonTool = this._toolRegistry.get("ipython"); - if (!ipythonTool) { - throw new Error("Goals require the ipython tool, which is not available in this session."); - } - const activeToolNames = new Set(this.getActiveToolNames()); - if (!activeToolNames.has("ipython")) { - activeToolNames.add("ipython"); - this.setActiveToolsByName([...activeToolNames]); - } - if (context) { - const contextTools = [...(context.tools ?? [])]; - if (!contextTools.some((tool) => tool.name === "ipython")) { - contextTools.push(ipythonTool); - context.tools = contextTools; - } - } - } - - private _maybeResumeGoalContinuationAfterRlmWork( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.maybeResumeGoalContinuationAfterRlmWork(...args); - } - - private _handleGoalSlashCommand( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.handleGoalSlashCommand(...args); - } - - private get _steeringStopPending(): boolean { - return this._actionQueue.steeringStopPending; - } - - private _shouldStopBeforeTurn( - ...args: Parameters - ): ReturnType { - return this._turnPolicy.shouldStopBeforeTurn(...args); - } - - private _shouldStopAfterTurn( - ...args: Parameters - ): ReturnType { - return this._turnPolicy.shouldStopAfterTurn(...args); - } - - private _snapshotAutonomousRuntimeState( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.snapshotAutonomousRuntimeState(...args); - } - - private _restoreAutonomousRuntimeSnapshot( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.restoreAutonomousRuntimeSnapshot(...args); - } - - private _queueAutonomousContinuationForThresholdCompaction( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.queueAutonomousContinuationForThresholdCompaction(...args); - } - - // The role heuristic reads an assistant-last threshold stop as "task finished" and - // agent.continue() cannot resume from it, so the goal continuation is queued as a session input. - private _queueGoalContinuationForThresholdCompaction( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.queueGoalContinuationForThresholdCompaction(...args); - } - - // Withdraws a goal continuation queued for a threshold compaction the user cancelled, - // rolling back the continuationsUsed increment so the next natural stop re-queues it. - private _clearQueuedGoalContinuationAfterCancelledThresholdCompaction( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.clearQueuedGoalContinuationAfterCancelledThresholdCompaction(...args); - } - - private _clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction( - ...args: Parameters< - SessionAutonomousContinuation["clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction"] - > - ): ReturnType { - return this._autonomousContinuation.clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction(...args); - } - - /** - * Handle a goal.* request from the Python kernel host bridge (the bundled - * goal skill). All goal state stays host-side; the kernel only sees the - * serialized snake_case response. - */ - handleGoalHostRequest( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.handleGoalHostRequest(...args); - } - - /** - * Handle a compact.* request from the kernel host bridge. Compaction would - * abort the run executing the requesting cell, so compact.run only schedules - * it; _checkCompaction consumes the request at the turn boundary. - */ - handleCompactHostRequest( - ...args: Parameters - ): ReturnType { - return this._compaction.handleCompactHostRequest(...args); - } - - /** - * Handle a refine.* request from the kernel host bridge. Like compact, - * refinement waits for the current turn to become idle before applying - * changes, so refine.run only schedules it; _consumePendingRequestedRefine - * fires it at the turn boundary. This prevents a deadlock that would occur - * if refine() awaited agent idle from within the active tool call. - */ - handleRefineHostRequest(type: string, payload: Record = {}): Record { - return this._refinement.handleRefineHostRequest(type, payload); - } - - /** - * Handle an rlm_heartbeat.* request from the bundled rlm-heartbeat skill. - * These heartbeats are internal to this active session and never read or - * mutate the user-level /heartbeat. - */ - handleRlmHeartbeatHostRequest(type: string, payload: Record = {}): Record { - return handleRlmHeartbeatHostRequest(this._rlmHeartbeatController, type, payload); - } - - handleAgentMessageHostRequest( - type: string, - payload: Record = {}, - ): Promise { - return handleAgentMessageHostRequest(() => this._agentMessageController, type, payload); - } - - handleAgentObserveHostRequest( - type: string, - payload: Record = {}, - ): - | AgentObserveListResult - | AgentObserveAgentSnapshot - | AgentObserveRecentMessagesResult - | Promise { - return handleAgentObserveHostRequest(this._agentObserveController, type, payload); - } - - private _getGoalContinuationMessages( - ...args: Parameters - ): ReturnType { - return this._goalContinuation.getGoalContinuationMessages(...args); - } - - private _getContinuationMessages( - ...args: Parameters - ): ReturnType { - return this._turnPolicy.getContinuationMessages(...args); - } - - /** - * Register a delivery waiter before submitting the prompt. Delivery outcomes are not retained - * for late lookup, so callers that register after admission may wait for a future use of the id. - */ - waitForAgentMessagePromptDelivery( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.waitForAgentMessagePromptDelivery(...args); - } - - private _settleAgentMessage( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.settleAgentMessage(...args); - } - - private _rejectAgentMessage( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.rejectAgentMessage(...args); - } - - private _hasCancelledDispatchCapture( - ...args: Parameters - ): ReturnType { - return this._events.hasCancelledDispatchCapture(...args); - } - - private _findLastAssistantMessage(): AssistantMessage | undefined { - return this._events.findLastAssistantInMessages(this.agent.state.messages); - } - - /** - * Subscribe to agent events. - * Session persistence is handled internally (saves messages on message_end). - * Multiple listeners can be added. Returns unsubscribe function for this listener. - */ - subscribe(...args: Parameters): ReturnType { - return this._events.subscribe(...args); - } - - /** - * Temporarily disconnect from agent events. - * User listeners are preserved and will receive events again after resubscribe(). - * Used internally during operations that need to pause event processing. - */ - private _disconnectFromAgent( - ...args: Parameters - ): ReturnType { - return this._events.disconnectFromAgent(...args); - } - - /** - * Reconnect to agent events after _disconnectFromAgent(). - * Preserves all existing listeners. - */ - private _reconnectToAgent( - ...args: Parameters - ): ReturnType { - return this._events.reconnectToAgent(...args); - } - - /** - * Remove all listeners and disconnect from agent. - * Call this when completely done with the session. - */ - /** - * Async teardown for graceful quit/switch: await the Python kernel's dispose - * (which flushes a final namespace snapshot) before the synchronous dispose, so - * the latest state reaches disk instead of racing process exit. - */ - async disposeAsync(options?: { kernelSnapshot?: boolean }): Promise { - if (this._disposed) { - return this._disposeCallbacksPromise; - } - // Concurrent callers await the same in-flight teardown so none resolves before - // the kernel snapshot flush finishes. - if (this._disposeAsyncPromise) { - return this._disposeAsyncPromise; - } - const kernelSnapshot = options?.kernelSnapshot ?? true; - this._disposeAsyncPromise = (async () => { - // Drain before marking _disposing so a refine triggered at the final - // agent_end completes instead of being aborted by dispose(). - await this._refinement._drainPendingRefinementForDisposal(); - if (this._disposed) { - return this._disposeCallbacksPromise; - } - this._disposing = true; - this._commitFence.dispose(); - await this._disposeAsyncOnce(kernelSnapshot); - })(); - return this._disposeAsyncPromise; - } - - private _disposeAsyncOnce(kernelSnapshot: boolean): Promise { - // Flush kernels/traces for both still-running and retained children; the sync - // dispose() below only tears them down synchronously. - return this._children.disposeAsync(() => - this._kernel.dispose(kernelSnapshot, () => { - this.dispose(); - return this._disposeCallbacksPromise; - }), - ); - } - - private _startDisposeCallbacks(): Promise { - if (this._disposeCallbacksPromise) { - return this._disposeCallbacksPromise; - } - const pending: Promise[] = []; - for (const callback of this._disposeCallbacks) { - try { - const result = callback(); - if (result) { - pending.push(result.catch(() => undefined)); - } - } catch { - // Disposal remains best-effort; one owner must not block the rest. - } - } - this._disposeCallbacks.clear(); - this._disposeCallbacksPromise = Promise.all(pending).then(() => undefined); - return this._disposeCallbacksPromise; - } - - dispose(): void { - if (this._disposed) { - return; - } - this._disposed = true; - this._children.beginDisposal(); - this._commitFence.dispose(); - try { - // Invalidate scheduled timers and abort any in-flight review so a late - // resolution cannot write harness state or re-subscribe handlers. - this._refinement.dispose(); - this._children.dispose(); - this._pendingContext.dispose(); - const deliveryError = new Error("Session disposed before prompt delivery."); - const completionError = new Error("Session disposed before prompt completion."); - this._messageDelivery.dispose(deliveryError, completionError); - this._cancelSessionActions(() => true, deliveryError); - this.agent.clearAllQueues(); - this._extensionRunner.invalidate( - "This extension ctx is stale after session replacement or reload. Do not use a captured pi or command ctx after ctx.newSession(), ctx.fork(), ctx.switchSession(), or ctx.reload(). For newSession, fork, and switchSession, move post-replacement work into withSession and use the ctx passed to withSession. For reload, do not use the old ctx after await ctx.reload().", - ); - this._disconnectFromAgent(); - this._events.dispose(); - cleanupSessionResources(this.sessionId); - } finally { - void this._startDisposeCallbacks(); - } - } - - registerDisposeCallback(callback: () => void | Promise): void { - if (this._disposed) { - try { - const result = callback(); - if (result) void result.catch(() => undefined); - } catch { - // Late registration follows the same best-effort disposal contract. - } - return; - } - this._disposeCallbacks.add(callback); - } - - get state(): AgentState { - return this.agent.state; - } - - get model(): Model | undefined { - return this.agent.state.model; - } - - get thinkingLevel(): ThinkingLevel { - return this.agent.state.thinkingLevel; - } - - get serviceTier(): ServiceTier { - return this.agent.state.serviceTier; - } - - get isStreaming(): boolean { - return this.agent.state.isStreaming; - } - - get systemPrompt(): string { - return this.agent.state.systemPrompt; - } - - get retryAttempt(): number { - return this._retry.attempt; - } - - getActiveToolNames(): string[] { - return this._tools.getActiveToolNames(); - } - - getAllTools(): ToolInfo[] { - return this._tools.getAllTools(); - } - - getToolDefinition(name: string): ToolDefinition | undefined { - return this._tools.getToolDefinition(name); - } - - setActiveToolsByName(toolNames: string[]): void { - this._tools.setActiveToolsByName(toolNames); - } - - get isCompacting(): boolean { - return this._compaction.isRunning || this._history.isSummarizing; - } - - get messages(): AgentMessage[] { - return this.agent.state.messages; - } - - buildSessionContext( - ...args: Parameters - ): ReturnType { - return this._harnessContext.buildSessionContext(...args); - } - - private _mergeUnpersistedOutcomes( - ...args: Parameters - ): ReturnType { - return this._harnessContext.mergeUnpersistedOutcomes(...args); - } - - get steeringMode(): "all" | "one-at-a-time" { - return this.agent.steeringMode; - } - - get followUpMode(): "all" | "one-at-a-time" { - return this.agent.followUpMode; - } - - get sessionFile(): string | undefined { - return this.sessionManager.getSessionFile(); - } - - get sessionId(): string { - return this.sessionManager.getSessionId(); - } - - get rlmDepth(): number { - return this._childState.depth; - } - - get semanticEdges(): SemanticEdgeRecorder { - return this._semanticEdges; - } - - get rlmMaxDepth(): number { - return this._childState.maxDepth; - } - - get sessionName(): string | undefined { - return this.sessionManager.getSessionName(); - } - - get goalState(): GoalState { - return this._goals.current; - } - - getAutonomousStatus( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.getAutonomousStatus(...args); - } - - recordHostAutonomousContinuation( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.recordHostAutonomousContinuation(...args); - } - - refreshAutonomousGates( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.refreshAutonomousGates(...args); - } - - private _runWithAutonomousContinuationSuppressed(fn: () => Promise): Promise { - return this._autonomousContinuation.runWithAutonomousContinuationSuppressed(fn); - } - - private _markAutonomousContinuationSuppressed( - ...args: Parameters - ): ReturnType { - return this._autonomousContinuation.markAutonomousContinuationSuppressed(...args); - } - - get scopedModels(): ReadonlyArray<{ - model: Model; - thinkingLevel?: ThinkingLevel; - }> { - return this._scopedModels; - } - - setScopedModels(scopedModels: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>): void { - this._modelSelection.setScopedModels(scopedModels); - } - - get promptTemplates(): ReadonlyArray { - return this._resourceLoader.getPrompts().prompts; - } - - private _rebuildSystemPrompt(toolNames: string[]): string { - return this._tools.rebuildSystemPrompt(toolNames); - } - - private _refreshExtensionSystemPrompt( - ...args: Parameters - ): ReturnType { - return this._tools.refreshExtensionSystemPrompt(...args); - } - - private _normalizeSubmission( - ...args: Parameters - ): ReturnType { - return this._submissionNormalizer.normalizeSubmission(...args); - } - - private async _runPreTurnCompaction(): Promise { - const lastAssistant = this._findLastAssistantMessage(); - if (lastAssistant) await this._checkCompaction(lastAssistant, false, false); - } - - private _canStartSessionActionImmediately(): boolean { - return ( - !this.isStreaming && - !this.isCompacting && - !this.isRetrying && - !this.isBashRunning && - !this._inputScheduler.suspended && - this._inputScheduler.queuedWorkPauseCount === 0 && - !this._disposed && - !this._disposing - ); - } - - /** - * Send a prompt to the agent. - * - Handles extension commands (registered via pi.registerCommand) immediately, even during streaming - * - Expands file-based prompt templates by default - * - During streaming, queues via steer() or followUp() based on streamingBehavior option - * - Validates model and API key before sending (when not streaming) - * @throws Error if streaming and no streamingBehavior specified - * @throws Error if no model selected or no API key available (when not streaming) - */ - async prompt(text: string, options?: PromptOptions): Promise { - return this._prompt(text, options); - } - - async promptUntilAccepted(text: string, options?: PromptOptions): Promise { - return this._prompt(text, { ...options, returnAfterAccepted: true }); - } - - promptAndWait( - ...args: Parameters - ): ReturnType { - return this._messageDelivery.promptAndWait(...args); - } - - acceptAgentMessagePrompt( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.acceptAgentMessagePrompt(...args); - } - - queueAgentMessagePrompt( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.queueAgentMessagePrompt(...args); - } - - async promptHeartbeat(job: AgentCronJob, options?: PromptOptions): Promise { - const message = createHeartbeatPromptMessage(job); - await this._promptInjectedMessage(message.content, message, { - ...options, - followUpQueueKey: options?.followUpQueueKey ?? `heartbeat:${job.id}`, - resumeIfIdle: true, - }); - } - - private _isRlmTerminalNoticeAction( - ...args: Parameters - ): ReturnType { - return this._pendingContext.isRlmTerminalNoticeAction(...args); - } - - private _hasDeferredRlmTerminalNotices( - ...args: Parameters - ): ReturnType { - return this._pendingContext.hasDeferredRlmTerminalNotices(...args); - } - - private _flushDeferredRlmTerminalNotices( - ...args: Parameters - ): ReturnType { - return this._pendingContext.flushDeferredRlmTerminalNotices(...args); - } - - private _deferRlmTerminalNotice( - ...args: Parameters - ): ReturnType { - return this._pendingContext.deferRlmTerminalNotice(...args); - } - - private _demoteRlmTerminalNoticeActions( - ...args: Parameters - ): ReturnType { - return this._pendingContext.demoteRlmTerminalNoticeActions(...args); - } - - private _promptInjectedMessage( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.promptInjectedMessage(...args); - } - - private _prompt( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.prompt(...args); - } - - /** - * Queue a steering message while the agent is running. - * Delivered after the current assistant turn finishes executing its tool calls, - * before the next LLM call. - * Expands skill commands and prompt templates. Errors on extension commands. - * @param images Optional image attachments to include with the message - * @throws Error if text is an extension command - */ - steer(...args: Parameters): ReturnType { - return this._promptSubmission.steer(...args); - } - - /** - * Queue a follow-up message to be processed after the agent finishes. - * Delivered only when agent has no more tool calls or steering messages. - * Expands skill commands and prompt templates. Errors on extension commands. - * @param images Optional image attachments to include with the message - * @throws Error if text is an extension command - */ - followUp(...args: Parameters): ReturnType { - return this._promptSubmission.followUp(...args); - } - - restoreSessionActions( - ...args: Parameters - ): ReturnType { - return this._actionRecovery.restoreSessionActions(...args); - } - - restoreSteeringMessage( - ...args: Parameters - ): ReturnType { - return this._actionQueue.restoreSteeringMessage(...args); - } - - restoreFollowUpMessage( - ...args: Parameters - ): ReturnType { - return this._actionQueue.restoreFollowUpMessage(...args); - } - - private _takePendingNextTurnMessages( - ...args: Parameters - ): ReturnType { - return this._pendingContext.takePendingNextTurnMessages(...args); - } - - private _assertSessionActionAdmissionAvailable( - ...args: Parameters - ): ReturnType { - return this._inputAdmission.assertSessionActionAdmissionAvailable(...args); - } - - private _admitSessionInput( - ...args: Parameters - ): ReturnType { - return this._inputAdmission.admitSessionInput(...args); - } - - private _queuePreparedPrompt( - ...args: Parameters - ): ReturnType { - return this._inputAdmission.queuePreparedPrompt(...args); - } - - private _runtimeActivity(): RuntimeActivity { - return { - lowerAgentRun: this.isStreaming, - compaction: this.isCompacting, - retry: this.isRetrying, - bash: this.isBashRunning, - refinementApply: this._refinement.isApplying, - branchMutation: this._branchSummaryOperation !== undefined, - schedulerPauseCount: this._inputScheduler.queuedWorkPauseCount + (this._inputScheduler.suspended ? 1 : 0), - disposing: this._disposed || this._disposing, - }; - } - - private _hasSelectableSessionInput(): boolean { - return this._inputDispatcher.hasSelectableInput(); - } - - get hasPendingSessionWork(): boolean { - return this._actionQueue.hasPendingSessionWork; - } - - get hasPendingAdmissionWaiters(): boolean { - return this._commitFence.hasPendingWork || this._inputCheckpoints.hasWaiters; - } - - private _scheduleSessionInputPump(): void { - this._inputScheduler.schedule(); - } - - private _executeSelectedSessionCommand( - ...args: Parameters - ): ReturnType { - return this._commandExecution.executeSelectedSessionCommand(...args); - } - - private _isBusyForSessionInput(point: "preflight" | "pump"): boolean { - const externalBusy = this.isCompacting || this.isRetrying || this.isBashRunning; - if (point === "pump") { - return ( - externalBusy || - this._disposed || - this._disposing || - this._inputScheduler.suspended || - this._inputScheduler.queuedWorkPauseCount > 0 || - this._branchSummaryOperation !== undefined - ); - } - return externalBusy || this._actionStore.unfinishedActions().length > 0; - } - - private _isSessionInputHandoffDeferred(epoch: number): boolean { - return epoch !== this._inputScheduler.epoch || this._isBusyForSessionInput("pump"); - } - - private _asError(error: unknown): Error { - return error instanceof Error ? error : new Error(String(error)); - } - - private _surfaceSessionInputError(error: unknown): void { - const normalized = this._asError(error); - try { - this._extensionRunner.emitError({ - extensionPath: "", - event: "session_input", - error: normalized.message, - stack: normalized.stack, - }); - } catch { - // Best-effort: a throwing error listener must not break the pump's requeue path. - } - } - - private _startPreparedTurnActions( - ...args: Parameters - ): ReturnType { - return this._turnExecution.startPreparedTurnActions(...args); - } - - /** - * Send a custom message to the session. Creates a CustomMessageEntry. - * - * Handles three cases: - * - Streaming: queues message, processed when loop pulls from queue - * - Not streaming + triggerTurn: appends to state/session, starts new turn - * - Not streaming + no trigger: appends to state/session, no turn - * - * @param message Custom message with customType, content, display, details - * @param options.triggerTurn If true and not streaming, triggers a new LLM turn - * @param options.deliverAs Delivery mode: "steer", "followUp", or "nextTurn" - */ - sendCustomMessage( - message: Pick, "customType" | "content" | "display" | "details">, - options?: { - triggerTurn?: boolean; - deliverAs?: "steer" | "followUp" | "nextTurn"; - }, - ): Promise { - return this._promptSubmission.sendCustomMessage(message, options); - } - - /** - * Send a user message to the agent. Always triggers a turn. - * When the agent is streaming, use deliverAs to specify how to queue the message. - * - * @param content User message content (string or content array) - * @param options.deliverAs Delivery mode when streaming: "steer" or "followUp" - */ - sendUserMessage( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.sendUserMessage(...args); - } - - clearQueue(...args: Parameters): ReturnType { - return this._actionQueue.clearQueue(...args); - } - - private _invalidateQueuedPromptPreparation( - ...args: Parameters - ): ReturnType { - return this._actionQueue.invalidateQueuedPromptPreparation(...args); - } - - clearQueuedAgentMessages( - ...args: Parameters - ): ReturnType { - return this._actionQueue.clearQueuedAgentMessages(...args); - } - - clearQueuedUserMessagesMatching( - ...args: Parameters - ): ReturnType { - return this._actionQueue.clearQueuedUserMessagesMatching(...args); - } - - /** - * Mutate a single visible queued message, addressed by its position in the same - * projection the session-action snapshot publishes. expectedText must match the - * item's current preview so clients never edit a shifted queue by accident. - */ - mutateQueuedMessage( - ...args: Parameters - ): ReturnType { - return this._actionQueue.mutateQueuedMessage(...args); - } - - get queuedActionCount(): number { - return visibleSessionActionProjection(this._actionStore.queuedActions()).length; - } - - get unfinishedActionCount(): number { - return this._actionStore.unfinishedActions().length; - } - - get isQueuedWorkSuspended(): boolean { - return this._inputScheduler.suspended; - } - - get isSessionActive(): boolean { - return ( - this._ipythonKernelProvisioner?.manager?.hasBackgroundWork === true || - this.isStreaming || - this.isCompacting || - this.isRetrying || - this.isBashRunning || - this._refinement.isApplying || - this._branchSummaryOperation !== undefined || - this._continuation.current !== undefined || - this.unfinishedActionCount > 0 - ); - } - - getSessionActionSnapshot( - ...args: Parameters - ): ReturnType { - return this._actionQueue.getSessionActionSnapshot(...args); - } - - getSteeringMessages( - ...args: Parameters - ): ReturnType { - return this._actionQueue.getSteeringMessages(...args); - } - - getSteeringMessagePreviews( - ...args: Parameters - ): ReturnType { - return this._actionQueue.getSteeringMessagePreviews(...args); - } - - getFollowUpMessages( - ...args: Parameters - ): ReturnType { - return this._actionQueue.getFollowUpMessages(...args); - } - - getFollowUpMessagePreviews( - ...args: Parameters - ): ReturnType { - return this._actionQueue.getFollowUpMessagePreviews(...args); - } - - getSessionActionRecoverySnapshot( - ...args: Parameters - ): ReturnType { - return this._actionRecovery.getSessionActionRecoverySnapshot(...args); - } - - private _notifySessionInputCheckpointChange( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.notifySessionInputCheckpointChange(...args); - } - - private _waitForSessionActivityChange( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.waitForSessionActivityChange(...args); - } - - private _observeSessionActionDeferral( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.observeSessionActionDeferral(...args); - } - - waitForSessionInputCheckpoint( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.waitForSessionInputCheckpoint(...args); - } - - acquireSessionInputPause(): { release(): void } { - return this._inputScheduler.acquireAdmissionPause(() => { - this._notifySessionInputCheckpointChange(); - this._flushDeferredRlmTerminalNotices(); - this._maybeResumeGoalContinuationAfterRlmWork(); - this._scheduleSessionInputPump(); - }); - } - - acquireQueuedWorkPause(): { release(): void } { - return this._inputScheduler.acquireQueuedWorkPause(() => { - this._notifySessionInputCheckpointChange(); - this._flushDeferredRlmTerminalNotices(); - this._scheduleSessionInputPump(); - }); - } - - private _acquireDirectTurnAdmissionFence( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.acquireDirectTurnAdmissionFence(...args); - } - - private _acquireSessionActionCommitFence(signal?: AbortSignal): Promise { - return this._commitFence.acquire(signal); - } - - private _resumeSessionInputAdmission(): void { - if (!this._inputScheduler.resume()) return; - this._notifySessionInputCheckpointChange(); - this._flushDeferredRlmTerminalNotices(); - } - - /** Resume the scheduler after requestAbort/abortForUpdateRestart suspended it; owned pause leases are unaffected. */ - resumeQueuedWork(): boolean { - this._resumeSessionInputAdmission(); - this._maybeResumeGoalContinuationAfterRlmWork(); - this._scheduleSessionInputPump(); - return this._hasSelectableSessionInput(); - } - - waitForSessionInputIdle(): Promise { - return this._inputScheduler.waitForIdle(); - } - - async waitForIdle(): Promise { - await this._waitForIdleOrSettlement(); - } - - /** - * {@link waitForIdle} loop; with a settlement, returns once that settlement is - * superseded so a cancelled post-compaction runner cannot keep a checkpoint - * waiter registered (a leaked waiter holds hasPendingAdmissionWaiters true and - * blocks daemon passivation). - */ - private _waitForIdleOrSettlement( - ...args: Parameters - ): ReturnType { - return this._inputCheckpoints.waitForIdleOrSettlement(...args); - } - - /** Waits out any owned post-compaction continuation and rejects when one cannot start; {@link waitForIdle} never rejects. */ - waitForHeadlessIdle(): Promise { - return this._inputCheckpoints.waitForHeadlessIdle(); - } - - getPendingNextTurnMessageSnapshots( - ...args: Parameters - ): ReturnType { - return this._pendingContext.getPendingNextTurnMessageSnapshots(...args); - } - - restorePendingNextTurnMessages( - ...args: Parameters - ): ReturnType { - return this._pendingContext.restorePendingNextTurnMessages(...args); - } - - removeQueuedFollowUp( - ...args: Parameters - ): ReturnType { - return this._actionQueue.removeQueuedFollowUp(...args); - } - - get resourceLoader(): ResourceLoader { - return this._resourceLoader; - } - - requestAbort(): void { - this._children.requestAbort(); - this._inputScheduler.suspend("abort"); - this._demoteRlmTerminalNoticeActions(); - this._cancelSessionActions( - (action) => - action.payload.kind === "turn" && - !action.payload.queueVisible && - !this._pendingContext.isRetainedTerminalNotice(action.id), - new Error("Prompt aborted before delivery."), - ); - this._cancelPostCompactionContinue(); - this.abortRetry(); - this.abortCompaction(); - this.abortBranchSummary(); - this.abortBash(); - this._refinement.requestAbort(); - this.agent.abort(); - } - - async abort(): Promise { - const compactionOperation = this._compaction.operation; - const branchSummaryOperation = this._branchSummaryOperation; - this.requestAbort(); - this._cancelActiveRlmChildRuns("Parent session aborted"); - this._goalContinuation.beginAbort(); - try { - await Promise.allSettled([ - this.agent.waitForIdle(), - this._events.queue, - ...(compactionOperation ? [compactionOperation] : []), - ...(branchSummaryOperation ? [branchSummaryOperation] : []), - ]); - } finally { - this._goalContinuation.finishAbort(); - } - } - - abortForUpdateRestart(): void { - // Cancel scheduled pumps and suspend new ones: queued inputs must survive - // into the restart manifest instead of starting a turn during teardown. - this._inputScheduler.suspend("update-restart"); - this._cancelPostCompactionContinue(); - this.abortRetry(); - this._children.cancelQuiescenceWaits(); - this._cancelActiveRlmChildRuns("Parent session aborted for update restart"); - this._goalContinuation.beginAbort(); - this.agent.abort(); - if (this._goalContinuation.abortInProgress) { - void this.agent - .waitForIdle() - .then(() => this._events.queue) - .catch(() => undefined) - .finally(() => { - this._goalContinuation.finishAbort(); - }); - } - } - - setModel(...args: Parameters): ReturnType { - return this._modelSelection.setModel(...args); - } - - private _pendingModelSelectEmit( - ...args: Parameters - ): ReturnType { - return this._modelSelection.pendingModelSelectEmit(...args); - } - - cycleModel( - ...args: Parameters - ): ReturnType { - return this._modelSelection.cycleModel(...args); - } - - setThinkingLevel( - ...args: Parameters - ): ReturnType { - return this._modelSelection.setThinkingLevel(...args); - } - - setServiceTier( - ...args: Parameters - ): ReturnType { - return this._modelSelection.setServiceTier(...args); - } - - cycleThinkingLevel( - ...args: Parameters - ): ReturnType { - return this._modelSelection.cycleThinkingLevel(...args); - } - - getAvailableThinkingLevels( - ...args: Parameters - ): ReturnType { - return this._modelSelection.getAvailableThinkingLevels(...args); - } - - supportsThinking( - ...args: Parameters - ): ReturnType { - return this._modelSelection.supportsThinking(...args); - } - - private _syncKernelStateAfterCompaction(): Promise { - return this._kernel.syncAfterCompaction(); - } - - setSteeringMode(mode: "all" | "one-at-a-time"): void { - this.agent.steeringMode = mode; - this.settingsManager.setSteeringMode(mode); - } - - setFollowUpMode(mode: "all" | "one-at-a-time"): void { - this.agent.followUpMode = mode; - this.settingsManager.setFollowUpMode(mode); - } - - compact(customInstructions?: string, options: { skipAbort?: boolean } = {}): Promise { - return this._compaction.compact(customInstructions, options); - } - - private _afterManualCompaction( - signal: AbortSignal, - hadPostCompactionContinue: boolean, - continueAfterSessionInput: boolean, - ): void { - this._refinement._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); - if (this._goals.state.status === "active" && !signal.aborted) { - if (!this._goalContinuation.awaitsChildWork && !this.agent.hasQueuedMessages()) { - this._goalContinuation.deferUntilChildSettlement(); - } - this.resumeQueuedWork(); - if (this.agent.hasQueuedMessages()) this._schedulePostCompactionContinue(); - } - if (hadPostCompactionContinue) { - this._schedulePostCompactionContinue(continueAfterSessionInput); - } - // Queued agent or session-owned inputs resume the loop; defer refine - // behind them instead of interleaving it before their turns. - this._refinement._scheduleAutoRefineAfterCompaction( - this._goalContinuation.awaitsChildWork || - hadPostCompactionContinue || - this.agent.hasQueuedMessages() || - this.unfinishedActionCount > 0, - ); - } - - /** - * Shared compaction core behind /compact, auto-compaction, and the compact - * skill. Throws CompactionSkippedError when there is nothing to compact and - * Error("Compaction cancelled") on abort or extension cancel. - */ - private _performCompaction(options: CompactionExecutionOptions): Promise { - return performSessionCompaction(this._compactionExecution, options); - } - - private _reapDeletedRlmSubagentRuntimesAfterCompaction(): Promise { - return this._children.reapAfterCompaction(); - } - - abortCompaction(): void { - this._compaction.abort(); - } - - private _cancelPostCompactionContinue(): void { - this._continuation.cancel(); - } - - private _schedulePostCompactionContinue(continueAfterSessionInput = false): void { - this._continuation.schedule(continueAfterSessionInput); - } - - private _forgetConsumedPostCompactionContinuations(messages: AgentMessage[]): void { - this._continuation.forgetConsumed(messages); - } - - /** The compact harness digest delivered at cold context boundaries (session start, resume, compaction head). */ - private _harnessDigest( - ...args: Parameters - ): ReturnType { - return this._harnessContext.harnessDigest(...args); - } - - /** Cold-boundary digest delivery: empty contexts defer to the first committed turn (untouched sessions must stay empty); non-empty contexts append only when the newest in-context digest mismatches disk. */ - private _ensureHarnessDigestContext( - ...args: Parameters - ): ReturnType { - return this._harnessContext.ensureHarnessDigestContext(...args); - } - - private _latestContextHarnessDigest( - ...args: Parameters - ): ReturnType { - return this._harnessContext.latestContextHarnessDigest(...args); - } - - /** - * Refine editable continual harness state: prompt notes, memory, skills, and subagent specs. - * The base system prompt is intentionally not editable through this path. - * - * Planning runs in the background and does NOT block turn entry points - * (`_waitForRefineIdle` only waits for `_refineInFlight`). Only the fast - * application phase (disk I/O + in-memory mutation) blocks turn entry points. - */ - refine( - options: { instructions?: string; rollbackId?: string; global?: boolean } = {}, - internal: { skipAbort?: boolean; trigger?: "manual" | "auto"; source?: RefinementSource } = {}, - ): Promise { - return this._refinement.refine(options, internal); - } - - abortBranchSummary( - ...args: Parameters - ): ReturnType { - return this._history.abortBranchSummary(...args); - } - - /** - * Check if compaction is needed and run it. - * Called after agent_end and before prompt submission. - * - * Two cases: - * 1. Overflow: LLM returned context overflow error, remove error message from agent state, compact, auto-retry - * 2. Threshold: Context over threshold, compact, and continue only for stopped in-progress loops or queued messages - * - * @param assistantMessage The assistant message to check - * @param skipAbortedCheck If false, include aborted messages (for pre-prompt check). Default: true - */ - private _checkCompaction( - assistantMessage: AssistantMessage, - skipAbortedCheck = true, - queueAutonomousContinuation = true, - ): Promise { - return this._compaction.check(assistantMessage, skipAbortedCheck, queueAutonomousContinuation); - } - - /** - * Internal: Run automatic (threshold/overflow) or model-requested compaction - * with events. - */ - - private _runAutoCompaction(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise { - return this._compaction.runAutomatic(reason, willRetry); - } - - setAutoCompactionEnabled(enabled: boolean): void { - this.settingsManager.setCompactionEnabled(enabled); - } - - get autoCompactionEnabled(): boolean { - return this.settingsManager.getCompactionEnabled(); - } - - /** - * Set the provider for extra env vars merged over process.env in extension - * pi.exec() subprocesses. The function is read at exec time, so a host (e.g. - * the daemon) can update the underlying value per attach without rebinding. - */ - setExecEnvProvider(provider: (() => Record | undefined) | undefined): void { - this._extensions.setExecEnvProvider(provider); - } - - bindExtensions(bindings: ExtensionBindings): Promise { - return this._extensions.bindExtensions(bindings); - } - - refreshModelMetadata(): void { - this._modelSelection.refreshModelMetadata(); - } - - private _refreshCurrentModelFromRegistry( - ...args: Parameters - ): ReturnType { - return this._modelSelection.refreshCurrentModelFromRegistry(...args); - } - - private _refreshToolRegistry(options?: { activeToolNames?: string[]; includeAllExtensionTools?: boolean }): void { - this._tools.refreshToolRegistry(options); - } - - private _buildRuntime(options: { - activeToolNames?: string[]; - flagValues?: Map; - includeAllExtensionTools?: boolean; - }): void { - const pythonSkills = getPythonSkillRuntimeInfo(this._modelVisibleSkills()); - this._tools.setBaseDefinitions(this._tools.buildBaseOverrides() ?? this._kernel.build(pythonSkills)); - this._extensions.build(options.flagValues); - this._tools.updateAcpDefinitions(); - const baseActiveToolNames = [...(options.activeToolNames ?? this._tools.defaultActiveToolNames)]; - if (this._goals.state.status === "active" && this._includeGoals) baseActiveToolNames.push("ipython"); - this._refreshToolRegistry({ - activeToolNames: [...new Set(baseActiveToolNames)], - includeAllExtensionTools: options.includeAllExtensionTools, - }); - this._kernel.finishBuild(this.getActiveToolNames()); - } - - /** - * Skills exposed to the model (system prompt + kernel). The bundled goal - * and compact skills are withheld when disabled for this session. - */ - private _modelVisibleSkills(): Skill[] { - let skills = this._resourceLoader.getSkills().skills; - if (!this._includeGoals) { - skills = skills.filter((skill) => skill.name !== GOAL_SKILL_NAME); - } - if (!this._includeCompactSkill) { - skills = skills.filter((skill) => skill.name !== COMPACT_SKILL_NAME); - } - if (!this._refinement._autoRefineAllowedForSession()) { - skills = skills.filter((skill) => skill.name !== REFINE_SKILL_NAME); - } - if (!this._agentMessageController) { - skills = skills.filter((skill) => skill.name !== AGENT_MESSAGE_SKILL_NAME); - } - if (!this._agentObserveController) { - skills = skills.filter((skill) => skill.name !== AGENT_OBSERVE_SKILL_NAME); - } - if (!this._agentObserveController || !this._rlmHeartbeatController) { - skills = skills.filter((skill) => skill.name !== ORCHESTRATION_HEARTBEAT_SKILL_NAME); - } - return skills; - } - - private _createKernelHostHandlers(): HostRequestHandlers { - return createSessionKernelHostHandlers({ - runChild: (prompt, kwargs, code) => this.runRlmChild(prompt, kwargs, code), - createSession: (prompt, kwargs) => this.createRlmSession(prompt, kwargs), - findModels: (query, limit) => this.findRlmModels(query, limit), - listSubagents: () => this.listRlmSubagents(), - deleteSubagent: (target) => this.deleteRlmSubagent(target), - handleBashCompletion: (details) => this._handleKernelBashCompletion(details), - withdrawBashCompletion: (details) => this._actionQueue.withdrawAsyncBashCompletionNotice(details), - getModel: () => this.model, - includeGoals: this._includeGoals, - includeCompactSkill: this._includeCompactSkill, - isRefineAllowed: () => this._refinement._autoRefineAllowedForSession(), - hasHeartbeatController: () => !!this._rlmHeartbeatController, - getModelVisibleSkills: () => this._modelVisibleSkills(), - getAgentMessageController: () => this._agentMessageController, - hasObserveController: () => !!this._agentObserveController, - getMcpManager: () => this._mcpManager, - getDepth: () => this._childState.depth, - awaitChildPublication: (selector) => this._awaitPendingRlmChildPublication(selector), - recordParentReply: () => this._childState.recordReply(), - handleGoal: (type, payload) => this.handleGoalHostRequest(type, payload), - handleCompact: (type, payload) => this.handleCompactHostRequest(type, payload), - handleRefine: (type, payload) => this.handleRefineHostRequest(type, payload), - handleHeartbeat: (type, payload) => this.handleRlmHeartbeatHostRequest(type, payload), - handleMessage: (type, payload) => this.handleAgentMessageHostRequest(type, payload), - handleObserve: (type, payload) => this.handleAgentObserveHostRequest(type, payload), - }); - } - - private _handleKernelBashCompletion( - ...args: Parameters - ): ReturnType { - return this._promptSubmission.handleKernelBashCompletion(...args); - } - - reload(): Promise { - return this._extensions.reload(); - } - - // Undefined when there's no persistent artifact dir (e.g. the viewer client): - // don't mkdtemp here, since this runs on every kernel build but a viewer never - // does RLM work. The temp dir is created lazily in _createChildRlmSessionDir. - - private _createChildRlmSessionDir(): string { - return createChildSessionDir(() => this._ensureRlmSessionDir() ?? this._createEphemeralRlmSessionDir()); - } - - private _rlmKernelEnv(): Record { - return this._kernelEnvironment.buildEnv(); - } - private _ensureRlmSessionDir(): string | undefined { - return this._kernelEnvironment.ensureSessionDir(); - } - private _createEphemeralRlmSessionDir(): string { - return this._kernelEnvironment.createEphemeralSessionDir(); - } - - _contextTokensForCurrentMessages(): number | undefined { - const last = this._findLastAssistantMessage(); - return last ? calculateContextTokens(last.usage) : undefined; - } - - setCurrentRecap(recap: string | undefined): void { - this._childState.setCurrentRecap(recap); - } - - get repliedToParentSinceTask(): boolean | undefined { - return this._childState.repliedSinceTask; - } - - getCurrentRecap(): string | undefined { - return this._childState.getCurrentRecap(); - } - - private _createRlmSubagentRuntimeOptions(options: { - id: string; - prompt: string; - sessionName: string; - spawnCode?: string; - sessionDir: string; - model: Model; - thinkingLevel?: ThinkingLevel; - spawnedByRequestId?: string; - }): CreateRlmSubagentRuntimeOptions { - return { - parentSession: this, - id: options.id, - prompt: options.prompt, - sessionName: options.sessionName, - spawnCode: options.spawnCode, - sessionDir: options.sessionDir, - model: options.model, - thinkingLevel: - options.thinkingLevel ?? (clampThinkingLevel(options.model, this.thinkingLevel) as ThinkingLevel), - serviceTier: - this.serviceTier === "priority" && !supportsFastMode(options.model) ? "default" : this.serviceTier, - scopedModels: [...this._scopedModels], - activeToolNames: this.getActiveToolNames(), - allowedToolNames: this._allowedToolNames ? [...this._allowedToolNames] : undefined, - customTools: [...this._customTools], - includeGoals: this._includeGoals, - includeCompactSkill: this._includeCompactSkill, - rlmDepth: this._childState.depth + 1, - rlmMaxDepth: this._childState.maxDepth, - rlmParentNodeId: options.id, - spawnedByRequestId: options.spawnedByRequestId, - }; - } - - private async _createRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): Promise { - const host = this._children.getRuntimeHost(); - if (host) { - return await host.createRlmSubagentRuntime(options); - } - - return this._createInlineRlmSubagentRuntime(options); - } - - private _createInlineRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): RlmSubagentRuntime { - return createInlineChildRuntime( - { - cwd: this._cwd, - agentDir: this._agentDir, - agent: this.agent, - settingsManager: this.settingsManager, - resourceLoader: this._resourceLoader, - modelRegistry: this._modelRegistry, - }, - options, - ); - } - - private _cancelActiveRlmChildRuns(reason: string): void { - this._children.cancelActiveRuns(reason); - } - - getRlmChildRunStatus(childId: string): RlmChildAgentStatus | undefined { - return this._children.getRlmChildRunStatus(childId); - } - - private _awaitPendingRlmChildPublication(selector: string): Promise { - return this._children.awaitPublication(selector); - } - - listRlmSubagents(): Promise { - return this._children.listRlmSubagents(); - } - - deleteInactiveRlmSubagent( - childId: string, - isExternallyRunning: () => boolean = () => false, - ): Promise<"deleted" | "not_found" | "running"> { - return this._children.deleteInactiveRlmSubagent(childId, isExternallyRunning); - } - - deleteRlmSubagent(target: string): Promise { - return this._children.deleteRlmSubagent(target); - } - - /** - * Retain a finished child session for the parent lifetime so inspectors and - * daemon-hosted agent messaging can keep addressing it. Returns false (and disposes - * the child) when the parent is already tearing down, so the caller can drop the - * matching event forwarder too. - */ - registerRlmChildSession(childId: string, session: AgentSession, unsubscribe?: () => void): boolean { - return this._children.registerRlmChildSession(childId, session, unsubscribe); - } - - releaseRlmChildSession(childId: string, session: AgentSession): (() => void) | false { - return this._children.releaseRlmChildSession(childId, session); - } - - /** Live recursive child roster from lifecycle state, including nested work under retained parents. */ - getRlmChildSnapshots(): RlmChildAgentSnapshot[] { - return this._children.getRlmChildSnapshots(); - } - - /** True when any direct or nested subagent is still running or queued. */ - hasRunningRlmChildren(): boolean { - return this._children.hasRunningRlmChildren(); - } - - private _hasUnsettledRlmQuiescenceWork(): boolean { - return this._children.hasUnsettledWork(); - } - - /** - * Wait for every admitted descendant run to publish its terminal parent - * message and for the resulting parent turns to drain. Re-snapshotting after - * each drain includes descendants spawned while earlier results were consumed. - */ - waitForRlmQuiescence(externalSignal?: AbortSignal): Promise { - return this._children.waitForRlmQuiescence(externalSignal); - } - - // Inline (non-daemon) mode only; daemon clients attach to the child session directly. - getRlmChildSession(childId: string): AgentSession | undefined { - return this._children.getRlmChildSession(childId); - } - - /** - * Cancel a single RLM child run by id, searching nested child sessions. - * - * @returns true when a live run was cancelled or its unsettled terminal notice - * was suppressed; false when the id is unknown or the run already settled. - */ - cancelRlmChildRun(childId: string, reason = "Cancelled by user"): boolean { - return this._children.cancelRlmChildRun(childId, reason); - } - - /** Cancel every running or queued run in this session's subtree. */ - cancelRunningRlmDescendants(reason = "Cancelled by user"): boolean { - return this._children.cancelRunningRlmDescendants(reason); - } - - findRlmModels( - ...args: Parameters - ): ReturnType { - return this._modelSelection.findRlmModels(...args); - } - - private _resolveRlmSubagentModel( - ...args: Parameters - ): ReturnType { - return this._modelSelection.resolveRlmSubagentModel(...args); - } - - createRlmSession(prompt: string, kwargs: Record = {}): Promise { - return this._children.createRlmSession(prompt, kwargs); - } - - async runRlmChild( - prompt: string, - kwargs: Record = {}, - spawnCode?: string, - ): Promise { - return this._children.run(prompt, kwargs, spawnCode); - } - - abortRetry(): void { - this._retry.abortRetry(); - } - - private waitForRetry(): Promise { - return this._retry.waitForRetry(); - } - - get isRetrying(): boolean { - return this._retry.isRetrying; - } - - get hasAcceptedPromptInFlight(): boolean { - return this._actionQueue.hasAcceptedPromptInFlight; - } - - get autoRetryEnabled(): boolean { - return this.settingsManager.getRetryEnabled(); - } - - setAutoRetryEnabled(enabled: boolean): void { - this.settingsManager.setRetryEnabled(enabled); - } - - /** Execute a shell command and record its result unless transient. */ - executeBash(command: string, onChunk?: (chunk: string) => void, options?: ExecuteBashOptions): Promise { - return this._bash.executeBash(command, onChunk, options); - } - - /** Run ! / !! input with extension interception and bash lifecycle events. */ - runUserBash(command: string, options?: RunUserBashOptions): Promise { - return this._bash.runUserBash(command, options); - } - - private async _drainQueuedMessagesAfterBash(): Promise { - await this.agent.waitForIdle(); - this._scheduleSessionInputPump(); - } - - recordBashResult(command: string, result: BashResult, options?: { excludeFromContext?: boolean }): void { - this._bash.recordBashResult(command, result, options); - } - - /** Cancel every in-flight shell command, including pending extension dispatch. */ - abortBash(): void { - this._bash.abortBash(); - } - - get isBashRunning(): boolean { - return this._bash.isBashRunning; - } - - get hasPendingBashMessages(): boolean { - return this._bash.hasPendingBashMessages; - } - - private _flushPendingBashMessages(): void { - this._bash.flushPendingMessages(); - } - - getRlmMaxDepthStatus(): RlmMaxDepthStatus { - return this._childState.getRlmMaxDepthStatus(); - } - - setRlmMaxDepth(maxDepth: number, options: { global?: boolean } = {}): Promise { - return this._childState.setRlmMaxDepth(maxDepth, options); - } - - setSessionName(name: string): void { - this.sessionManager.appendSessionInfo(name); - this._emit({ - type: "session_info_changed", - name: this.sessionManager.getSessionName(), - }); - } - - /** - * Navigate to a different node in the session tree. - * Unlike fork() which creates a new session file, this stays in the same file. - * - * @param targetId The entry ID to navigate to - * @param options.summarize Whether user wants to summarize abandoned branch - * @param options.customInstructions Custom instructions for summarizer - * @param options.replaceInstructions If true, customInstructions replaces the default prompt - * @param options.label Label to attach to the branch summary entry - * @returns Result with editorText (if user message) and cancelled status - */ - - navigateTree( - ...args: Parameters - ): ReturnType { - return this._history.navigateTree(...args); - } - - getUserMessagesForForking( - ...args: Parameters - ): ReturnType { - return this._history.getUserMessagesForForking(...args); - } - - getSessionStats( - ...args: Parameters - ): ReturnType { - return this._contextView.getSessionStats(...args); - } - - getContextUsage( - ...args: Parameters - ): ReturnType { - return this._contextView.getContextUsage(...args); - } - - private _rlmSessionDirForReading(): string | undefined { - return this._rlmSessionDir ?? this.sessionManager.getSessionArtifactDir(); - } - - private *_contextViewChildren(): Generator { - for (const run of this._children.getActiveRuns()) { - yield { - id: run.id, - get label() { - return rlmChildLabel(run.prompt); - }, - get status() { - return run.status; - }, - sessionDir: run.sessionDir, - getContextTree: run.session ? () => run.session!.getContextTree() : undefined, - }; - } - } - - private _invalidateOwnUsage(): void { - this._contextView.invalidateOwnUsage(); - } - - // Whole-file own spend, identical to the catalog scan so rows never shift at passivation. - getOwnUsageSummary( - ...args: Parameters - ): ReturnType { - return this._contextView.getOwnUsageSummary(...args); - } - - /** - * Build the agent context overview for /context: this session as the root - * plus one node per RLM sub-agent, recursively. Running children are read - * from their live sessions; completed children from their persisted session - * dirs, so the tree survives child disposal and session resume. - */ - getContextTree( - ...args: Parameters - ): ReturnType { - return this._contextView.getContextTree(...args); - } - - /** - * Export session to HTML. - * @param outputPath Optional output path (defaults to session directory) - * @returns Path to exported file - */ - exportToHtml(...args: Parameters): ReturnType { - return this._export.exportToHtml(...args); - } - - /** - * Export the current session branch to a JSONL file. - * Writes the session header followed by all entries on the current branch path. - * @param outputPath Target file path. If omitted, generates a timestamped file in cwd. - * @returns The resolved output file path. - */ - exportToJsonl(...args: Parameters): ReturnType { - return this._export.exportToJsonl(...args); - } - - /** - * Get text content of last assistant message. - * Useful for /copy command. - * @returns Text content, or undefined if no assistant message exists - */ - getLastAssistantText( - ...args: Parameters - ): ReturnType { - return this._contextView.getLastAssistantText(...args); - } - - // ================================================================== // Extension System - // ================================================================== - createReplacedSessionContext(): ReplacedSessionContext { - const context = Object.defineProperties( - {}, - Object.getOwnPropertyDescriptors(this._extensionRunner.createCommandContext()), - ) as ReplacedSessionContext; - context.sendMessage = (message, options) => this.sendCustomMessage(message, options); - context.sendUserMessage = (content, options) => this.sendUserMessage(content, options); - return context; - } - - hasExtensionHandlers(eventType: string): boolean { - return this._extensionRunner.hasHandlers(eventType); - } - - get extensionRunner(): ExtensionRunner { - return this._extensionRunner; - } -} + type SessionStats, + type SetRlmMaxDepthResult, + type TurnExecutionPolicy, +} from "../session/agent-session.js"; diff --git a/packages/coding-agent/src/core/autonomous.ts b/packages/coding-agent/src/core/autonomous.ts index c5efd36063..75605adb24 100644 --- a/packages/coding-agent/src/core/autonomous.ts +++ b/packages/coding-agent/src/core/autonomous.ts @@ -1,627 +1,26 @@ -import { createHash } from "node:crypto"; -import { createReadStream } from "node:fs"; -import { lstat, readlink } from "node:fs/promises"; -import { resolve } from "node:path"; -import type { AssistantMessage, Usage, UserMessage } from "@earendil-works/pi-ai"; -import { spawnHidden, waitForChildProcess } from "../utils/child-process.js"; -import { killProcessTree, trackDetachedChildPid, untrackDetachedChildPid } from "../utils/shell.js"; - -export interface AgentAutonomousConfig { - enabled?: boolean; - maxContinuations?: number; - maxTurns?: number; - maxTokens?: number; - timeoutMs?: number; - continuationPrompt?: string; - gates?: AgentAutonomousGateConfig; -} - -export interface AgentAutonomousGateConfig { - commands?: string[]; - maxRetries?: number; - timeoutMs?: number; -} - -export interface AgentAutonomousGateFailure { - command: string; - attempt: number; - exitText: string; - output: string; -} - -export interface AgentAutonomousStatus { - enabled: boolean; - continuationsUsed: number; - turnsUsed: number; - tokensUsed: number; - startedAt?: number; - limits: Required>; - gates: Required; - gateAttempts: Record; - lastGateFailure?: AgentAutonomousGateFailure; -} - -export const DEFAULT_AUTONOMOUS_CONTINUATION_PROMPT = - "No human input is available in autonomous mode. Continue working until the host evaluator, verifier, or configured autonomous limits stop the run. If you were asking the user a question, make a reasonable assumption and verify it. If you believe you are blocked, prove it with host-observable evidence, preserve that evidence, and keep looking for safe progress while budget remains. Do not end the session yourself; the verifier/evaluator decides completion when configured gates pass."; - -export const DEFAULT_AUTONOMOUS_LIMITS: Required< - Omit -> = { - maxContinuations: 3, - maxTurns: 12, - maxTokens: 80_000, - timeoutMs: 30 * 60 * 1000, -}; - -export const DEFAULT_AUTONOMOUS_GATES: Required = { - commands: [], - maxRetries: 3, - timeoutMs: 5 * 60 * 1000, -}; - -/** - * JSON-safe sentinel meaning "no cap". Limit checks compare usage against the - * configured value, so this stays finite and serializes to JSON while no - * realistic run can ever reach it. - */ -export const UNLIMITED_AUTONOMOUS_LIMIT = Number.MAX_SAFE_INTEGER; - -export function isUnlimitedAutonomousLimit(value: number): boolean { - return value >= UNLIMITED_AUTONOMOUS_LIMIT; -} - -const MAX_GATE_OUTPUT_CHARS = 6000; -const MAX_CHILD_PROCESS_OUTPUT_CHARS = 1024 * 1024; - -export interface AutonomousRuntimeState { - enabled: boolean; - continuationsUsed: number; - turnsUsed: number; - tokensUsed: number; - startedAt?: number; - limits: Required>; - continuationPrompt: string; - gates: Required; - gateAttempts: Record; - lastGateFailure?: GateFailure; - lastGateFailureSnapshot?: GitWorktreeSnapshot; -} - -export type AutonomousLimitReason = "maxContinuations" | "maxTurns" | "maxTokens" | "timeoutMs"; -export type AutonomousGateResult = "passed" | "failed" | "retry_exhausted"; - -type AutonomousLimitState = Pick< - AgentAutonomousStatus, - "continuationsUsed" | "turnsUsed" | "tokensUsed" | "startedAt" | "limits" ->; - -export interface AutonomousDecision { - shouldContinue: boolean; - reason: "missing_terminal_evidence" | "gate_failed" | "not_needed" | "limit_reached"; -} - -interface GitWorktreeSnapshot { - status: string; - diff: string; - untrackedHash: string; -} - -interface AutonomousOperationOptions { - cwd?: string; - signal?: AbortSignal; -} - -type GateFailure = AgentAutonomousGateFailure; - -export function createAutonomousRuntimeState( - config?: AgentAutonomousConfig, - _options: { cwd?: string } = {}, -): AutonomousRuntimeState { - const enabled = config?.enabled === true; - return { - enabled, - continuationsUsed: 0, - turnsUsed: 0, - tokensUsed: 0, - startedAt: enabled ? Date.now() : undefined, - limits: { - maxContinuations: normalizeLimit(config?.maxContinuations, DEFAULT_AUTONOMOUS_LIMITS.maxContinuations), - maxTurns: normalizeLimit(config?.maxTurns, DEFAULT_AUTONOMOUS_LIMITS.maxTurns), - maxTokens: normalizeLimit(config?.maxTokens, DEFAULT_AUTONOMOUS_LIMITS.maxTokens), - timeoutMs: normalizeLimit(config?.timeoutMs, DEFAULT_AUTONOMOUS_LIMITS.timeoutMs), - }, - continuationPrompt: config?.continuationPrompt?.trim() || DEFAULT_AUTONOMOUS_CONTINUATION_PROMPT, - gates: { - commands: [...(config?.gates?.commands ?? DEFAULT_AUTONOMOUS_GATES.commands)], - maxRetries: normalizeLimit(config?.gates?.maxRetries, DEFAULT_AUTONOMOUS_GATES.maxRetries), - timeoutMs: normalizeLimit(config?.gates?.timeoutMs, DEFAULT_AUTONOMOUS_GATES.timeoutMs), - }, - gateAttempts: {}, - lastGateFailure: undefined, - lastGateFailureSnapshot: undefined, - }; -} - -export function setAutonomousEnabled( - state: AutonomousRuntimeState, - enabled: boolean, - _options: { cwd?: string } = {}, -): void { - state.enabled = enabled; - if (enabled) { - state.continuationsUsed = 0; - state.turnsUsed = 0; - state.tokensUsed = 0; - state.startedAt = Date.now(); - state.gateAttempts = {}; - state.lastGateFailure = undefined; - state.lastGateFailureSnapshot = undefined; - } else { - state.startedAt = undefined; - state.gateAttempts = {}; - state.lastGateFailure = undefined; - state.lastGateFailureSnapshot = undefined; - } -} - -/** - * Apply user-provided budget and gate options to a live runtime state. - * Only fields present in `config` change; unspecified fields keep the state's - * current values, which come from the session/CLI configuration or defaults. - */ -export function setAutonomousLimits(state: AutonomousRuntimeState, config?: AgentAutonomousConfig): void { - if (!config) { - return; - } - state.limits.maxContinuations = normalizeLimit(config.maxContinuations, state.limits.maxContinuations); - state.limits.maxTurns = normalizeLimit(config.maxTurns, state.limits.maxTurns); - state.limits.maxTokens = normalizeLimit(config.maxTokens, state.limits.maxTokens); - state.limits.timeoutMs = normalizeLimit(config.timeoutMs, state.limits.timeoutMs); - if (config.continuationPrompt?.trim()) { - state.continuationPrompt = config.continuationPrompt.trim(); - } - if (config.gates) { - if (config.gates.commands !== undefined) { - state.gates.commands = [...config.gates.commands]; - } - state.gates.maxRetries = normalizeLimit(config.gates.maxRetries, state.gates.maxRetries); - state.gates.timeoutMs = normalizeLimit(config.gates.timeoutMs, state.gates.timeoutMs); - } -} - -export function autonomousStatus(state: AutonomousRuntimeState): AgentAutonomousStatus { - return { - enabled: state.enabled, - continuationsUsed: state.continuationsUsed, - turnsUsed: state.turnsUsed, - tokensUsed: state.tokensUsed, - startedAt: state.startedAt, - limits: { ...state.limits }, - gates: { ...state.gates, commands: [...state.gates.commands] }, - gateAttempts: { ...state.gateAttempts }, - lastGateFailure: state.lastGateFailure ? { ...state.lastGateFailure } : undefined, - }; -} - -export function addAutonomousUsage(state: AutonomousRuntimeState, usage: Usage | undefined): void { - if (!state.enabled) { - return; - } - state.turnsUsed++; - state.tokensUsed += autonomousTokenDelta(usage); -} - -export function addAutonomousContinuation(state: AutonomousRuntimeState): void { - if (!state.enabled) { - return; - } - state.continuationsUsed++; -} - -function autonomousTokenDelta(usage: Usage | undefined): number { - if (!usage) { - return 0; - } - // Cache-read tokens are repeated context served from provider cache. Counting them - // cumulatively makes long autonomous verifier loops exhaust their host-side token - // budget far before the non-cached work reaches the configured cap. - return usage.input + usage.output + usage.cacheWrite; -} - -export async function nextAutonomousContinuation( - state: AutonomousRuntimeState, - message: AssistantMessage, - options: AutonomousOperationOptions = {}, - now = Date.now(), -): Promise { - options.signal?.throwIfAborted(); - if (!state.enabled) { - return undefined; - } - const decision = await shouldAutonomouslyContinue(state, message, options, now); - options.signal?.throwIfAborted(); - if (!decision.shouldContinue) { - return undefined; - } - state.continuationsUsed++; - const gateFailureText = decision.reason === "gate_failed" ? buildGateFailureContinuation(state, now) : undefined; - return { - role: "user", - content: [ - { - type: "text", - text: gateFailureText ?? `[autonomous-continuation]\n\n${state.continuationPrompt}`, - }, - ], - timestamp: now, - }; -} - -export async function shouldAutonomouslyContinue( - state: AutonomousRuntimeState, - message: AssistantMessage, - options: AutonomousOperationOptions = {}, - now = Date.now(), -): Promise { - options.signal?.throwIfAborted(); - if (!state.enabled || message.stopReason === "error" || message.stopReason === "aborted") { - return { shouldContinue: false, reason: "not_needed" }; - } - const gateResult = await refreshAutonomousQualityGates(state, options); - options.signal?.throwIfAborted(); - if (gateResult) { - if (gateResult === "passed") { - return { shouldContinue: false, reason: "not_needed" }; - } - if (gateResult === "retry_exhausted" || autonomousLimitReason(state, now)) { - return { shouldContinue: false, reason: "limit_reached" }; - } - return { shouldContinue: true, reason: "gate_failed" }; - } - if (autonomousLimitReason(state, now)) { - return { shouldContinue: false, reason: "limit_reached" }; - } - return { shouldContinue: true, reason: "missing_terminal_evidence" }; -} - -export function autonomousLimitReason( - state: AutonomousLimitState, - now = Date.now(), -): AutonomousLimitReason | undefined { - if (state.continuationsUsed >= state.limits.maxContinuations) { - return "maxContinuations"; - } - if (state.turnsUsed >= state.limits.maxTurns) { - return "maxTurns"; - } - if (state.tokensUsed >= state.limits.maxTokens) { - return "maxTokens"; - } - if (state.startedAt !== undefined && now - state.startedAt >= state.limits.timeoutMs) { - return "timeoutMs"; - } - return undefined; -} - -export async function refreshAutonomousQualityGates( - state: AutonomousRuntimeState, - options: AutonomousOperationOptions = {}, -): Promise { - options.signal?.throwIfAborted(); - if (!state.enabled || state.gates.commands.length === 0) { - return undefined; - } - return await runAutonomousQualityGates(state, options.cwd, options.signal); -} - -async function runAutonomousQualityGates( - state: AutonomousRuntimeState, - cwd: string | undefined, - signal: AbortSignal | undefined, -): Promise { - signal?.throwIfAborted(); - if (!cwd) { - return "failed"; - } - for (const command of state.gates.commands) { - const currentSnapshot = await captureGitWorktreeSnapshot(cwd, signal); - signal?.throwIfAborted(); - if ( - state.lastGateFailure?.command === command && - state.lastGateFailureSnapshot && - gitWorktreeSnapshotsEqual(currentSnapshot, state.lastGateFailureSnapshot) - ) { - const attempt = (state.gateAttempts[command] ?? state.lastGateFailure.attempt) + 1; - state.gateAttempts[command] = attempt; - state.lastGateFailure = { - ...state.lastGateFailure, - attempt, - exitText: "not rerun: workspace unchanged since previous failed gate", - output: - "The autonomous gate was not rerun because the workspace has not changed since this failure. Edit source files, tests, or a blocker artifact before attempting to finish again.", - }; - return attempt > state.gates.maxRetries ? "retry_exhausted" : "failed"; - } - const result = await runChildProcess(command, [], { - cwd, - shell: true, - timeoutMs: state.gates.timeoutMs, - maxOutputChars: MAX_GATE_OUTPUT_CHARS, - signal, - }); - signal?.throwIfAborted(); - const postRunSnapshot = await captureGitWorktreeSnapshot(cwd, signal); - signal?.throwIfAborted(); - if (result.status === 0 && !result.error && !result.timedOut) { - state.gateAttempts[command] = 0; - if (state.lastGateFailure?.command === command) { - state.lastGateFailure = undefined; - state.lastGateFailureSnapshot = undefined; - } - continue; - } - const attempt = (state.gateAttempts[command] ?? 0) + 1; - state.gateAttempts[command] = attempt; - const exitText = formatProcessExit(result); - state.lastGateFailure = { - command, - attempt, - exitText, - output: truncateGateOutput( - [result.stdout, result.stderr].filter(Boolean).join("\n").trim(), - result.outputTruncated, - ), - }; - state.lastGateFailureSnapshot = postRunSnapshot; - return attempt > state.gates.maxRetries ? "retry_exhausted" : "failed"; - } - state.lastGateFailure = undefined; - state.lastGateFailureSnapshot = undefined; - return "passed"; -} - -export function buildAutonomousGateFailureContinuation( - failure: AgentAutonomousGateFailure, - maxRetries: number, - timestamp = Date.now(), -): string { - return ( - `[autonomous-continuation: gate-failed]\n\n` + - `Autonomous quality gate failed (attempt ${failure.attempt}/${maxRetries}): \`${failure.command}\` ${failure.exitText}.\n` + - (failure.output ? `\nOutput:\n${failure.output}\n` : "\n") + - `\nContinue working. Fix the failure, then produce terminal evidence. Timestamp: ${new Date(timestamp).toISOString()}.` - ); -} - -function buildGateFailureContinuation(state: AutonomousRuntimeState, timestamp: number): string | undefined { - const failure = state.lastGateFailure; - if (!failure) { - return undefined; - } - return buildAutonomousGateFailureContinuation(failure, state.gates.maxRetries, timestamp); -} - -function gitWorktreeSnapshotsEqual(a: GitWorktreeSnapshot | undefined, b: GitWorktreeSnapshot | undefined): boolean { - return !!a && !!b && a.status === b.status && a.diff === b.diff && a.untrackedHash === b.untrackedHash; -} - -async function captureGitWorktreeSnapshot( - cwd: string | undefined, - signal?: AbortSignal, -): Promise { - signal?.throwIfAborted(); - if (!cwd) { - return undefined; - } - const pathspec = [ - "--", - ".", - ":(exclude)verification", - ":(exclude)target", - ":(exclude).vf-prime-agent", - ":(exclude)Cargo.lock", - ":(exclude)submission.tar.gz", - ":(exclude)runner_args.log", - ]; - const status = await runChildProcess( - "git", - ["--no-optional-locks", "status", "--porcelain=v1", "-z", "-uall", "--no-renames", ...pathspec], - { - cwd, - timeoutMs: 10_000, - signal, - }, - ); - signal?.throwIfAborted(); - if (status.status !== 0 || status.error || status.timedOut || status.outputTruncated) { - return undefined; - } - const diff = await runChildProcess( - "git", - ["--no-optional-locks", "diff", "--no-ext-diff", "--binary", "HEAD", ...pathspec], - { - cwd, - timeoutMs: 10_000, - signal, - }, - ); - signal?.throwIfAborted(); - if (diff.status !== 0 || diff.error || diff.timedOut || diff.outputTruncated) { - return undefined; - } - return { - status: status.stdout, - diff: diff.stdout, - untrackedHash: await hashUntrackedFiles(cwd, status.stdout, signal), - }; -} - -function untrackedPathsFromStatus(status: string): string[] { - return status - .split("\0") - .filter((entry) => entry.startsWith("?? ")) - .map((entry) => entry.slice(3)) - .sort(); -} - -async function hashUntrackedFiles(cwd: string, status: string, signal?: AbortSignal): Promise { - const aggregate = createHash("sha256"); - for (const path of untrackedPathsFromStatus(status)) { - signal?.throwIfAborted(); - aggregate.update(path); - aggregate.update("\0"); - aggregate.update(await hashUntrackedPath(resolve(cwd, path), signal)); - aggregate.update("\0"); - } - signal?.throwIfAborted(); - return aggregate.digest("hex"); -} - -async function hashUntrackedPath(path: string, signal?: AbortSignal): Promise { - try { - signal?.throwIfAborted(); - const stat = await lstat(path); - signal?.throwIfAborted(); - if (stat.isSymbolicLink()) { - const target = await readlink(path); - signal?.throwIfAborted(); - return `symlink:${target}`; - } - if (!stat.isFile()) { - return `other:${stat.mode}:${stat.size}:${stat.mtimeMs}`; - } - const hash = createHash("sha256"); - for await (const chunk of createReadStream(path, { signal })) { - hash.update(chunk); - } - signal?.throwIfAborted(); - return `file:${hash.digest("hex")}`; - } catch (error) { - signal?.throwIfAborted(); - return `error:${error instanceof Error ? error.message : String(error)}`; - } -} - -interface ChildProcessResult { - status: number | null; - signal: NodeJS.Signals | null; - stdout: string; - stderr: string; - error?: Error; - timedOut?: boolean; - outputTruncated: boolean; -} - -function runChildProcess( - command: string, - args: string[], - options: { - cwd?: string; - shell?: boolean; - timeoutMs?: number; - maxOutputChars?: number; - signal?: AbortSignal; - } = {}, -): Promise { - options.signal?.throwIfAborted(); - return new Promise((resolve) => { - const child = spawnHidden(command, args, { - cwd: options.cwd, - detached: process.platform !== "win32", - shell: options.shell === true, - stdio: ["ignore", "pipe", "pipe"], - }); - if (child.pid) { - trackDetachedChildPid(child.pid); - } - let stdout = ""; - let stderr = ""; - let error: Error | undefined; - let timedOut = false; - let outputTruncated = false; - let settled = false; - const maxOutputChars = options.maxOutputChars ?? MAX_CHILD_PROCESS_OUTPUT_CHARS; - const finish = (result: Pick) => { - if (settled) { - return; - } - settled = true; - if (timer) { - clearTimeout(timer); - } - options.signal?.removeEventListener("abort", abort); - if (child.pid) { - untrackDetachedChildPid(child.pid); - } - resolve({ ...result, stdout, stderr, error, timedOut, outputTruncated }); - }; - const timer = options.timeoutMs - ? setTimeout(() => { - timedOut = true; - if (child.pid) { - killProcessTree(child.pid); - } else { - child.kill("SIGKILL"); - } - }, options.timeoutMs) - : undefined; - const abort = () => { - if (child.pid) { - killProcessTree(child.pid); - } else { - child.kill("SIGKILL"); - } - }; - options.signal?.addEventListener("abort", abort, { once: true }); - if (options.signal?.aborted) { - abort(); - } - child.stdout?.setEncoding("utf8"); - child.stderr?.setEncoding("utf8"); - child.stdout?.on("data", (chunk: string) => { - const remaining = maxOutputChars - stdout.length; - if (remaining > 0) { - stdout += chunk.slice(0, remaining); - } - outputTruncated ||= chunk.length > remaining; - }); - child.stderr?.on("data", (chunk: string) => { - const remaining = maxOutputChars - stderr.length; - if (remaining > 0) { - stderr += chunk.slice(0, remaining); - } - outputTruncated ||= chunk.length > remaining; - }); - void waitForChildProcess(child).then( - (status) => finish({ status, signal: child.signalCode }), - (err: Error) => { - error = err; - finish({ status: child.exitCode, signal: child.signalCode }); - }, - ); - }); -} - -function formatProcessExit(result: ChildProcessResult): string { - if (result.timedOut) { - return "timed out"; - } - if (result.error) { - return result.error.message; - } - return result.signal ? `terminated by ${result.signal}` : `exited ${result.status ?? "unknown"}`; -} - -function truncateGateOutput(output: string, outputAlreadyTruncated = false, maxChars = MAX_GATE_OUTPUT_CHARS): string { - if (output.length <= maxChars && !outputAlreadyTruncated) { - return output; - } - return `${output.slice(0, maxChars)}\n... [truncated]`; -} - -function normalizeLimit(value: number | undefined, fallback: number): number { - if (!Number.isFinite(value) || value === undefined || value <= 0) { - return fallback; - } - return Math.trunc(value); -} +export { + type AgentAutonomousConfig, + type AgentAutonomousGateConfig, + type AgentAutonomousGateFailure, + type AgentAutonomousStatus, + type AutonomousDecision, + type AutonomousGateResult, + type AutonomousLimitReason, + type AutonomousRuntimeState, + addAutonomousContinuation, + addAutonomousUsage, + autonomousLimitReason, + autonomousStatus, + buildAutonomousGateFailureContinuation, + createAutonomousRuntimeState, + DEFAULT_AUTONOMOUS_CONTINUATION_PROMPT, + DEFAULT_AUTONOMOUS_GATES, + DEFAULT_AUTONOMOUS_LIMITS, + isUnlimitedAutonomousLimit, + nextAutonomousContinuation, + refreshAutonomousQualityGates, + setAutonomousEnabled, + setAutonomousLimits, + shouldAutonomouslyContinue, + UNLIMITED_AUTONOMOUS_LIMIT, +} from "../session/autonomy/autonomous.js"; diff --git a/packages/coding-agent/src/core/compaction/branch-summarization.ts b/packages/coding-agent/src/core/compaction/branch-summarization.ts index af8ee00615..d52f5ad329 100644 --- a/packages/coding-agent/src/core/compaction/branch-summarization.ts +++ b/packages/coding-agent/src/core/compaction/branch-summarization.ts @@ -1,330 +1,12 @@ -/** - * Branch summarization for tree navigation. - * - * When navigating to a different point in the session tree, this generates - * a summary of the branch being left so context isn't lost. - */ - -import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { Model, Usage } from "@earendil-works/pi-ai"; -import { completeSimple } from "@earendil-works/pi-ai"; -import { - convertToLlm, - createBranchSummaryMessage, - createCompactionSummaryMessage, - createCustomMessage, - HARNESS_DIGEST_CUSTOM_TYPE, -} from "../messages.js"; -import { completeWithProviderRetry, type ProviderRetryPolicy } from "../provider-retry.js"; -import type { ReadonlySessionManager, SessionEntry } from "../session-manager.js"; -import { estimateTokens } from "./compaction.js"; -import { - computeFileLists, - createFileOps, - extractFileOpsFromMessage, +// Compatibility exports; implementation lives with its session owner. +export { + type BranchPreparation, + type BranchSummaryDetails, + type BranchSummaryResult, + type CollectEntriesResult, + collectEntriesForBranchSummary, type FileOperations, - formatFileOperations, - SUMMARIZATION_SYSTEM_PROMPT, - serializeConversation, -} from "./utils.js"; -export interface BranchSummaryResult { - summary?: string; - readFiles?: string[]; - modifiedFiles?: string[]; - aborted?: boolean; - error?: string; - usage?: Usage; -} - -/** Details stored in BranchSummaryEntry.details for file tracking */ -export interface BranchSummaryDetails { - readFiles: string[]; - modifiedFiles: string[]; -} - -export type { FileOperations } from "./utils.js"; - -export interface BranchPreparation { - /** Messages extracted for summarization, in chronological order */ - messages: AgentMessage[]; - /** File operations extracted from tool calls */ - fileOps: FileOperations; - /** Total estimated tokens in messages */ - totalTokens: number; -} - -export interface CollectEntriesResult { - /** Entries to summarize, in chronological order */ - entries: SessionEntry[]; - /** Common ancestor between old and new position, if any */ - commonAncestorId: string | null; -} - -export interface GenerateBranchSummaryOptions { - /** Model to use for summarization */ - model: Model; - /** API key for the model */ - apiKey: string; - /** Request headers for the model */ - headers?: Record; - /** Owning conversation identity for provider routing and caching. */ - sessionId?: string; - /** Abort signal for cancellation */ - signal: AbortSignal; - /** Optional custom instructions for summarization */ - customInstructions?: string; - /** If true, customInstructions replaces the default prompt instead of being appended */ - replaceInstructions?: boolean; - retry?: ProviderRetryPolicy; - /** Tokens reserved for prompt + LLM response (default 16384) */ - reserveTokens?: number; -} -/** - * Collect entries that should be summarized when navigating from one position to another. - * - * Walks from oldLeafId back to the common ancestor with targetId, collecting entries - * along the way. Does NOT stop at compaction boundaries - those are included and their - * summaries become context. - * - * @param session - Session manager (read-only access) - * @param oldLeafId - Current position (where we're navigating from) - * @param targetId - Target position (where we're navigating to) - * @returns Entries to summarize and the common ancestor - */ -export function collectEntriesForBranchSummary( - session: ReadonlySessionManager, - oldLeafId: string | null, - targetId: string, -): CollectEntriesResult { - if (!oldLeafId) { - return { entries: [], commonAncestorId: null }; - } - const oldPath = new Set(session.getBranch(oldLeafId).map((e) => e.id)); - const targetPath = session.getBranch(targetId); - let commonAncestorId: string | null = null; - for (let i = targetPath.length - 1; i >= 0; i--) { - if (oldPath.has(targetPath[i].id)) { - commonAncestorId = targetPath[i].id; - break; - } - } - const entries: SessionEntry[] = []; - let current: string | null = oldLeafId; - - while (current && current !== commonAncestorId) { - const entry = session.getEntry(current); - if (!entry) break; - entries.push(entry); - current = entry.parentId; - } - entries.reverse(); - - return { entries, commonAncestorId }; -} -/** - * Extract AgentMessage from a session entry. - * Similar to getMessageFromEntry in compaction.ts but also handles compaction entries. - */ -function getMessageFromEntry(entry: SessionEntry): AgentMessage | undefined { - switch (entry.type) { - case "message": - // Tool-result context remains attached to its assistant tool call. - if (entry.message.role === "toolResult") return undefined; - return entry.message; - - case "custom_message": - // Harness digests are regenerated at cold boundaries; never summarizer input. - if (entry.customType === HARNESS_DIGEST_CUSTOM_TYPE) return undefined; - return createCustomMessage(entry.customType, entry.content, entry.display, entry.details, entry.timestamp); - - case "branch_summary": - return createBranchSummaryMessage(entry.summary, entry.fromId, entry.timestamp); - - case "compaction": - return createCompactionSummaryMessage( - entry.summary, - entry.tokensBefore, - entry.timestamp, - entry.customInstructions, - ); - case "thinking_level_change": - case "model_change": - case "custom": - case "label": - case "session_info": - return undefined; - } -} - -/** - * Prepare entries for summarization with token budget. - * - * Walks entries from NEWEST to OLDEST, adding messages until we hit the token budget. - * This ensures we keep the most recent context when the branch is too long. - * - * Also collects file operations from: - * - Tool calls in assistant messages - * - Existing branch_summary entries' details (for cumulative tracking) - * - * @param entries - Entries in chronological order - * @param tokenBudget - Maximum tokens to include (0 = no limit) - */ -export function prepareBranchEntries(entries: SessionEntry[], tokenBudget: number = 0): BranchPreparation { - const messages: AgentMessage[] = []; - const fileOps = createFileOps(); - let totalTokens = 0; - - // First pass: collect file ops from ALL entries (even if they don't fit in token budget) - // This ensures we capture cumulative file tracking from nested branch summaries - // Only extract from pi-generated summaries (fromHook !== true), not extension-generated ones - for (const entry of entries) { - if (entry.type === "branch_summary" && !entry.fromHook && entry.details) { - const details = entry.details as BranchSummaryDetails; - if (Array.isArray(details.readFiles)) { - for (const f of details.readFiles) fileOps.read.add(f); - } - if (Array.isArray(details.modifiedFiles)) { - for (const f of details.modifiedFiles) { - fileOps.edited.add(f); - } - } - } - } - for (let i = entries.length - 1; i >= 0; i--) { - const entry = entries[i]; - const message = getMessageFromEntry(entry); - if (!message) continue; - extractFileOpsFromMessage(message, fileOps); - - const tokens = estimateTokens(message); - if (tokenBudget > 0 && totalTokens + tokens > tokenBudget) { - if (entry.type === "compaction" || entry.type === "branch_summary") { - if (totalTokens < tokenBudget * 0.9) { - messages.unshift(message); - totalTokens += tokens; - } - } - break; - } - - messages.unshift(message); - totalTokens += tokens; - } - - return { messages, fileOps, totalTokens }; -} -const BRANCH_SUMMARY_PREAMBLE = `The user explored a different conversation branch before returning here. -Summary of that exploration: - -`; - -const BRANCH_SUMMARY_PROMPT = `Create a structured summary of this conversation branch for context when returning later. - -Use this EXACT format: - -## Goal -[What was the user trying to accomplish in this branch?] - -## Constraints & Preferences -- [Any constraints, preferences, or requirements mentioned] -- [Or "(none)" if none were mentioned] - -## Progress -### Done -- [x] [Completed tasks/changes] - -### In Progress -- [ ] [Work that was started but not finished] - -### Blocked -- [Issues preventing progress, if any] - -## Key Decisions -- **[Decision]**: [Brief rationale] - -## Next Steps -1. [What should happen next to continue this work] - -Keep each section concise. Preserve exact file paths, function names, and error messages.`; - -/** - * Generate a summary of abandoned branch entries. - * - * @param entries - Session entries to summarize (chronological order) - * @param options - Generation options - */ -export async function generateBranchSummary( - entries: SessionEntry[], - options: GenerateBranchSummaryOptions, -): Promise { - const { - model, - apiKey, - headers, - sessionId, - signal, - customInstructions, - replaceInstructions, - retry, - reserveTokens = 16384, - } = options; - const contextWindow = model.contextWindow || 128000; - const tokenBudget = contextWindow - reserveTokens; - - const { messages, fileOps } = prepareBranchEntries(entries, tokenBudget); - - // Nothing model-visible remains after filtering. - if (messages.length === 0) { - return { summary: "No content to summarize" }; - } - // Serialize before the LLM call so it summarizes rather than continues this branch. - const llmMessages = convertToLlm(messages); - const conversationText = serializeConversation(llmMessages); - let instructions: string; - if (replaceInstructions && customInstructions) { - instructions = customInstructions; - } else if (customInstructions) { - instructions = `${BRANCH_SUMMARY_PROMPT}\n\nAdditional focus: ${customInstructions}`; - } else { - instructions = BRANCH_SUMMARY_PROMPT; - } - const promptText = `\n${conversationText}\n\n\n${instructions}`; - - const summarizationMessages = [ - { - role: "user" as const, - content: [{ type: "text" as const, text: promptText }], - timestamp: Date.now(), - }, - ]; - const response = await completeWithProviderRetry( - () => - completeSimple( - model, - { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, - { apiKey, headers, sessionId, signal, maxTokens: 2048 }, - ), - { policy: retry, signal }, - ); - if (response.stopReason === "aborted") { - return { aborted: true }; - } - if (response.stopReason === "error") { - return { error: response.errorMessage || "Summarization failed" }; - } - - let summary = response.content - .filter((c): c is { type: "text"; text: string } => c.type === "text") - .map((c) => c.text) - .join("\n"); - summary = BRANCH_SUMMARY_PREAMBLE + summary; - const { readFiles, modifiedFiles } = computeFileLists(fileOps); - summary += formatFileOperations(readFiles, modifiedFiles); - - return { - summary: summary || "No summary generated", - readFiles, - modifiedFiles, - usage: response.usage, - }; -} + type GenerateBranchSummaryOptions, + generateBranchSummary, + prepareBranchEntries, +} from "../../session/context/branch-summary.js"; diff --git a/packages/coding-agent/src/core/compaction/compaction.ts b/packages/coding-agent/src/core/compaction/compaction.ts index c3a0acafed..11d42d08ed 100644 --- a/packages/coding-agent/src/core/compaction/compaction.ts +++ b/packages/coding-agent/src/core/compaction/compaction.ts @@ -1,851 +1,29 @@ -/** - * Context compaction for long sessions. - * - * Pure functions for compaction logic. The session manager handles I/O, - * and after compaction the session is reloaded. - */ - -import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { AssistantMessage, Model, Usage } from "@earendil-works/pi-ai"; -import { completeSimple } from "@earendil-works/pi-ai"; -import { - convertToLlm, - createBranchSummaryMessage, - createCompactionSummaryMessage, - createCustomMessage, - HARNESS_DIGEST_CUSTOM_TYPE, -} from "../messages.js"; -import { completeWithProviderRetry, type ProviderRetryPolicy } from "../provider-retry.js"; -import { buildSessionContext, type CompactionEntry, type SessionEntry } from "../session-manager.js"; -import { addAssistantUsage, emptyUsage } from "../usage.js"; -import { - computeFileLists, - createFileOps, - extractFileOpsFromMessage, - type FileOperations, - formatFileOperations, - SUMMARIZATION_SYSTEM_PROMPT, - serializeConversation, -} from "./utils.js"; -/** Details stored in CompactionEntry.details for file tracking */ -export interface CompactionDetails { - readFiles: string[]; - modifiedFiles: string[]; -} - -export interface SummarySlice { - summary: string; - usage?: Usage; -} - -/** - * Extract file operations from messages and previous compaction entries. - */ -/** Preserve file operations recorded by prior compactions and current tool calls. */ -function extractFileOperations( - messages: AgentMessage[], - entries: SessionEntry[], - prevCompactionIndex: number, -): FileOperations { - const fileOps = createFileOps(); - if (prevCompactionIndex >= 0) { - const prevCompaction = entries[prevCompactionIndex] as CompactionEntry; - if (!prevCompaction.fromHook && prevCompaction.details) { - // fromHook field kept for session file compatibility - const details = prevCompaction.details as CompactionDetails; - if (Array.isArray(details.readFiles)) { - for (const f of details.readFiles) fileOps.read.add(f); - } - if (Array.isArray(details.modifiedFiles)) { - for (const f of details.modifiedFiles) fileOps.edited.add(f); - } - } - } - for (const msg of messages) { - extractFileOpsFromMessage(msg, fileOps); - } - - return fileOps; -} -/** - * Extract AgentMessage from an entry if it produces one. - * Returns undefined for entries that don't contribute to LLM context. - */ -function getMessageFromEntry(entry: SessionEntry): AgentMessage | undefined { - if (entry.type === "message") { - return entry.message; - } - if (entry.type === "custom_message") { - return createCustomMessage(entry.customType, entry.content, entry.display, entry.details, entry.timestamp); - } - if (entry.type === "branch_summary") { - return createBranchSummaryMessage(entry.summary, entry.fromId, entry.timestamp); - } - if (entry.type === "compaction") { - return createCompactionSummaryMessage( - entry.summary, - entry.tokensBefore, - entry.timestamp, - entry.customInstructions, - ); - } - return undefined; -} - -function getMessageFromEntryForCompaction(entry: SessionEntry): AgentMessage | undefined { - if (entry.type === "compaction") { - return undefined; - } - // Harness digests are regenerated on the new compaction head; never summarizer input. - if (entry.type === "custom_message" && entry.customType === HARNESS_DIGEST_CUSTOM_TYPE) { - return undefined; - } - return getMessageFromEntry(entry); -} - -/** Result from compact() - SessionManager adds uuid/parentUuid when saving */ -export interface CompactionResult { - summary: string; - firstKeptEntryId: string; - tokensBefore: number; - /** Extension-specific data (e.g., ArtifactIndex, version markers for structured compaction) */ - details?: T; - /** What the summarization call(s) billed; persisted on the compaction entry. */ - usage?: Usage; -} -export const COMPACT_SKILL_NAME = "compact"; - -export interface CompactionSettings { - enabled: boolean; - reserveTokens: number; - keepRecentTokens: number; -} - -export const DEFAULT_COMPACTION_SETTINGS: CompactionSettings = { - enabled: true, - reserveTokens: 16384, - keepRecentTokens: 20000, -}; -/** - * Calculate total context tokens from usage. - * Uses the native totalTokens field when available, falls back to computing from components. - * - * Includes output: the assistant's response becomes part of the prompt on the next - * request, so it counts toward the context the next turn will send. - */ -export function calculateContextTokens(usage: Usage): number { - return usage.totalTokens || usage.input + usage.output + usage.cacheRead + usage.cacheWrite; -} - -/** - * Get usage from an assistant message if available. - * Skips aborted and error messages as they don't have valid usage data. - */ -function getAssistantUsage(msg: AgentMessage): Usage | undefined { - if (msg.role === "assistant" && "usage" in msg) { - const assistantMsg = msg as AssistantMessage; - if (assistantMsg.stopReason !== "aborted" && assistantMsg.stopReason !== "error" && assistantMsg.usage) { - return assistantMsg.usage; - } - } - return undefined; -} - -/** - * Find the last non-aborted assistant message usage from session entries. - */ -export function getLastAssistantUsage(entries: SessionEntry[]): Usage | undefined { - for (let i = entries.length - 1; i >= 0; i--) { - const entry = entries[i]; - if (entry.type === "message") { - const usage = getAssistantUsage(entry.message); - if (usage) return usage; - } - } - return undefined; -} - -export interface ContextUsageEstimate { - tokens: number; - usageTokens: number; - trailingTokens: number; - lastUsageIndex: number | null; -} - -function getLastAssistantUsageInfo(messages: AgentMessage[]): { usage: Usage; index: number } | undefined { - for (let i = messages.length - 1; i >= 0; i--) { - const usage = getAssistantUsage(messages[i]); - if (usage) return { usage, index: i }; - } - return undefined; -} - -/** - * Estimate context tokens from messages, using the last assistant usage when available. - * If there are messages after the last usage, estimate their tokens with estimateTokens. - */ -export function estimateContextTokens(messages: AgentMessage[]): ContextUsageEstimate { - const usageInfo = getLastAssistantUsageInfo(messages); - - if (!usageInfo) { - let estimated = 0; - for (const message of messages) { - estimated += estimateTokens(message); - } - return { - tokens: estimated, - usageTokens: 0, - trailingTokens: estimated, - lastUsageIndex: null, - }; - } - - const usageTokens = calculateContextTokens(usageInfo.usage); - let trailingTokens = 0; - for (let i = usageInfo.index + 1; i < messages.length; i++) { - trailingTokens += estimateTokens(messages[i]); - } - - return { - tokens: usageTokens + trailingTokens, - usageTokens, - trailingTokens, - lastUsageIndex: usageInfo.index, - }; -} - -/** - * Check if compaction should trigger based on context usage. - */ -export function shouldCompact(contextTokens: number, contextWindow: number, settings: CompactionSettings): boolean { - if (!settings.enabled) return false; - if (contextWindow <= 0) return false; - return contextTokens > contextWindow - settings.reserveTokens; -} -/** - * Estimate token count for a message using chars/4 heuristic. - * This is conservative (overestimates tokens). - */ -export function estimateTokens(message: AgentMessage): number { - let chars = 0; - - switch (message.role) { - case "user": { - const content = (message as { content: string | Array<{ type: string; text?: string }> }).content; - if (typeof content === "string") { - chars = content.length; - } else if (Array.isArray(content)) { - for (const block of content) { - if (block.type === "text" && block.text) { - chars += block.text.length; - } - } - } - return Math.ceil(chars / 4); - } - case "assistant": { - const assistant = message as AssistantMessage; - for (const block of assistant.content) { - if (block.type === "text") { - chars += block.text.length; - } else if (block.type === "thinking") { - chars += block.thinking.length; - } else if (block.type === "toolCall") { - chars += block.name.length + JSON.stringify(block.arguments).length; - } - } - return Math.ceil(chars / 4); - } - case "custom": - case "toolResult": { - if (typeof message.content === "string") { - chars = message.content.length; - } else { - for (const block of message.content) { - if (block.type === "text" && block.text) { - chars += block.text.length; - } - if (block.type === "image") { - chars += 4800; // Estimate images as 4000 chars, or 1200 tokens - } - } - } - return Math.ceil(chars / 4); - } - case "bashExecution": { - chars = message.command.length + message.output.length; - return Math.ceil(chars / 4); - } - case "branchSummary": - case "compactionSummary": { - chars = message.summary.length; - return Math.ceil(chars / 4); - } - } - - return 0; -} - -/** - * Find valid cut points: indices of user, assistant, custom, or bashExecution messages. - * Never cut at tool results (they must follow their tool call). - * When we cut at an assistant message with tool calls, its tool results follow it - * and will be kept. - * BashExecutionMessage is treated like a user message (user-initiated context). - */ -function findValidCutPoints(entries: SessionEntry[], startIndex: number, endIndex: number): number[] { - const cutPoints: number[] = []; - for (let i = startIndex; i < endIndex; i++) { - const entry = entries[i]; - switch (entry.type) { - case "message": { - const role = entry.message.role; - switch (role) { - case "bashExecution": - case "custom": - case "branchSummary": - case "compactionSummary": - case "user": - case "assistant": - cutPoints.push(i); - break; - case "toolResult": - break; - } - break; - } - case "thinking_level_change": - case "model_change": - case "compaction": - case "branch_summary": - case "custom": - case "custom_message": - case "label": - case "session_info": - break; - } - // Branch summaries and custom messages are user-role turn boundaries. - if (entry.type === "branch_summary" || entry.type === "custom_message") { - cutPoints.push(i); - } - } - return cutPoints; -} - -/** - * Find the user message (or bashExecution) that starts the turn containing the given entry index. - * Returns -1 if no turn start found before the index. - * BashExecutionMessage is treated like a user message for turn boundaries. - */ -export function findTurnStartIndex(entries: SessionEntry[], entryIndex: number, startIndex: number): number { - for (let i = entryIndex; i >= startIndex; i--) { - const entry = entries[i]; - if (entry.type === "branch_summary" || entry.type === "custom_message") { - return i; - } - if (entry.type === "message") { - const role = entry.message.role; - if (role === "user" || role === "bashExecution") { - return i; - } - } - } - return -1; -} - -export interface CutPointResult { - /** Index of first entry to keep */ - firstKeptEntryIndex: number; - /** Index of user message that starts the turn being split, or -1 if not splitting */ - turnStartIndex: number; - /** Whether this cut splits a turn (cut point is not a user message) */ - isSplitTurn: boolean; -} - -/** - * Find the cut point in session entries that keeps approximately `keepRecentTokens`. - * - * Algorithm: Walk backwards from newest, accumulating estimated message sizes. - * Stop when we've accumulated >= keepRecentTokens. Cut at that point. - * - * Can cut at user OR assistant messages (never tool results). When cutting at an - * assistant message with tool calls, its tool results come after and will be kept. - * - * Returns CutPointResult with: - * - firstKeptEntryIndex: the entry index to start keeping from - * - turnStartIndex: if cutting mid-turn, the user message that started that turn - * - isSplitTurn: whether we're cutting in the middle of a turn - * - * Only considers entries between `startIndex` and `endIndex` (exclusive). - */ -export function findCutPoint( - entries: SessionEntry[], - startIndex: number, - endIndex: number, - keepRecentTokens: number, -): CutPointResult { - const cutPoints = findValidCutPoints(entries, startIndex, endIndex); - - if (cutPoints.length === 0) { - return { firstKeptEntryIndex: startIndex, turnStartIndex: -1, isSplitTurn: false }; - } - let accumulatedTokens = 0; - let cutIndex = cutPoints[0]; // Default: keep from first message (not header) - - for (let i = endIndex - 1; i >= startIndex; i--) { - const entry = entries[i]; - if (entry.type !== "message") continue; - const messageTokens = estimateTokens(entry.message); - accumulatedTokens += messageTokens; - if (accumulatedTokens >= keepRecentTokens) { - // No cut point at/after i (trailing tool results): keep only the final turn, not everything. - cutIndex = cutPoints[cutPoints.length - 1]; - for (let c = 0; c < cutPoints.length; c++) { - if (cutPoints[c] >= i) { - cutIndex = cutPoints[c]; - break; - } - } - break; - } - } - while (cutIndex > startIndex) { - const prevEntry = entries[cutIndex - 1]; - if (prevEntry.type === "compaction") { - break; - } - if (prevEntry.type === "message") { - break; - } - cutIndex--; - } - const cutEntry = entries[cutIndex]; - const isUserMessage = cutEntry.type === "message" && cutEntry.message.role === "user"; - // A cut in a non-user turn requires a prefix summary. - const turnStartIndex = isUserMessage ? -1 : findTurnStartIndex(entries, cutIndex, startIndex); - - return { - firstKeptEntryIndex: cutIndex, - turnStartIndex, - isSplitTurn: !isUserMessage && turnStartIndex !== -1, - }; -} -const SUMMARIZATION_PROMPT = `The messages above are a conversation to summarize. Create a structured context checkpoint summary that another LLM will use to continue the work. - -Use this EXACT format: - -## Goal -[What is the user trying to accomplish? Can be multiple items if the session covers different tasks.] - -## Constraints & Preferences -- [Any constraints, preferences, or requirements mentioned by user] -- [Or "(none)" if none were mentioned] - -## Progress -### Done -- [x] [Completed tasks/changes] - -### In Progress -- [ ] [Current work] - -### Blocked -- [Issues preventing progress, if any] - -## Key Decisions -- **[Decision]**: [Brief rationale] - -## Next Steps -1. [Ordered list of what should happen next] - -## Critical Context -- [Any data, examples, or references needed to continue] -- [Or "(none)" if not applicable] - -Keep each section concise. Preserve exact file paths, function names, and error messages.`; - -const KERNEL_PERSIST_SUMMARY_NOTE = - "Note: the Python kernel keeps running after this summary — every Python variable, import, and helper you defined stays available. The cells that defined them won't appear above, so record in the summary any names worth remembering so you reuse them instead of redefining them."; - -const UPDATE_SUMMARIZATION_PROMPT = `The messages above are NEW conversation messages to incorporate into the existing summary provided in tags. - -Update the existing structured summary with new information. RULES: -- PRESERVE all existing information from the previous summary -- ADD new progress, decisions, and context from the new messages -- UPDATE the Progress section: move items from "In Progress" to "Done" when completed -- UPDATE "Next Steps" based on what was accomplished -- PRESERVE exact file paths, function names, and error messages -- If something is no longer relevant, you may remove it - -Use this EXACT format: - -## Goal -[Preserve existing goals, add new ones if the task expanded] - -## Constraints & Preferences -- [Preserve existing, add new ones discovered] - -## Progress -### Done -- [x] [Include previously done items AND newly completed items] - -### In Progress -- [ ] [Current work - update based on progress] - -### Blocked -- [Current blockers - remove if resolved] - -## Key Decisions -- **[Decision]**: [Brief rationale] (preserve all previous, add new) - -## Next Steps -1. [Update based on current state] - -## Critical Context -- [Preserve important context, add new if needed] - -Keep each section concise. Preserve exact file paths, function names, and error messages.`; - -/** - * Build the instruction portion of the summarization prompt: the initial or - * update template, optional user instructions, and the kernel persistence note. - */ -export function buildSummarizationPrompt(customInstructions?: string, previousSummary?: string): string { - let basePrompt = previousSummary ? UPDATE_SUMMARIZATION_PROMPT : SUMMARIZATION_PROMPT; - if (customInstructions) { - basePrompt += `\n\n\nThe user provided these instructions for this summary. Follow them with high priority while keeping the section format above: emphasize what they ask to focus on, and preserve verbatim anything they ask to remember.\n${customInstructions}\n`; - } - return `${basePrompt}\n\n${KERNEL_PERSIST_SUMMARY_NOTE}`; -} - -/** - * Generate a summary of the conversation using the LLM. - * If previousSummary is provided, uses the update prompt to merge. - */ -export async function generateSummary( - currentMessages: AgentMessage[], - model: Model, - reserveTokens: number, - apiKey: string, - headers?: Record, - signal?: AbortSignal, - customInstructions?: string, - previousSummary?: string, - thinkingLevel?: ThinkingLevel, - retry?: ProviderRetryPolicy, - sessionId?: string, -): Promise { - const maxTokens = Math.floor(0.8 * reserveTokens); - - const basePrompt = buildSummarizationPrompt(customInstructions, previousSummary); - // Serialize before the LLM call so it summarizes rather than continues this conversation. - const llmMessages = convertToLlm(currentMessages); - const conversationText = serializeConversation(llmMessages); - let promptText = `\n${conversationText}\n\n\n`; - if (previousSummary) { - promptText += `\n${previousSummary}\n\n\n`; - } - promptText += basePrompt; - - const summarizationMessages = [ - { - role: "user" as const, - content: [{ type: "text" as const, text: promptText }], - timestamp: Date.now(), - }, - ]; - - const completionOptions = - model.reasoning && thinkingLevel && thinkingLevel !== "off" - ? { maxTokens, signal, apiKey, headers, sessionId, reasoning: thinkingLevel } - : { maxTokens, signal, apiKey, headers, sessionId }; - - const response = await completeWithProviderRetry( - () => - completeSimple( - model, - { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, - completionOptions, - ), - { policy: retry, signal }, - ); - - if (response.stopReason === "error") { - throw new Error(`Summarization failed: ${response.errorMessage || "Unknown error"}`); - } - - const textContent = response.content - .filter((c): c is { type: "text"; text: string } => c.type === "text") - .map((c) => c.text) - .join("\n"); - - return { summary: textContent, usage: response.usage }; -} -export interface CompactionPreparation { - /** UUID of first entry to keep */ - firstKeptEntryId: string; - /** Messages that will be summarized and discarded */ - messagesToSummarize: AgentMessage[]; - /** Messages that will be turned into turn prefix summary (if splitting) */ - turnPrefixMessages: AgentMessage[]; - /** Whether this is a split turn (cut point in middle of turn) */ - isSplitTurn: boolean; - tokensBefore: number; - /** Summary from previous compaction, for iterative update */ - previousSummary?: string; - /** File operations extracted from messagesToSummarize */ - fileOps: FileOperations; - /** Compaction settions from settings.jsonl */ - settings: CompactionSettings; -} - -export function prepareCompaction( - pathEntries: SessionEntry[], - settings: CompactionSettings, -): CompactionPreparation | undefined { - if (pathEntries.length > 0 && pathEntries[pathEntries.length - 1].type === "compaction") { - return undefined; - } - - let prevCompactionIndex = -1; - for (let i = pathEntries.length - 1; i >= 0; i--) { - if (pathEntries[i].type === "compaction") { - prevCompactionIndex = i; - break; - } - } - - let previousSummary: string | undefined; - let boundaryStart = 0; - if (prevCompactionIndex >= 0) { - const prevCompaction = pathEntries[prevCompactionIndex] as CompactionEntry; - previousSummary = prevCompaction.summary; - const firstKeptEntryIndex = pathEntries.findIndex((entry) => entry.id === prevCompaction.firstKeptEntryId); - boundaryStart = firstKeptEntryIndex >= 0 ? firstKeptEntryIndex : prevCompactionIndex + 1; - } - const boundaryEnd = pathEntries.length; - - const tokensBefore = estimateContextTokens(buildSessionContext(pathEntries).messages).tokens; - - const cutPoint = findCutPoint(pathEntries, boundaryStart, boundaryEnd, settings.keepRecentTokens); - const firstKeptEntry = pathEntries[cutPoint.firstKeptEntryIndex]; - if (!firstKeptEntry?.id) { - return undefined; // Session needs migration - } - const firstKeptEntryId = firstKeptEntry.id; - - const historyEnd = cutPoint.isSplitTurn ? cutPoint.turnStartIndex : cutPoint.firstKeptEntryIndex; - const messagesToSummarize: AgentMessage[] = []; - for (let i = boundaryStart; i < historyEnd; i++) { - const msg = getMessageFromEntryForCompaction(pathEntries[i]); - if (msg) messagesToSummarize.push(msg); - } - const turnPrefixMessages: AgentMessage[] = []; - if (cutPoint.isSplitTurn) { - for (let i = cutPoint.turnStartIndex; i < cutPoint.firstKeptEntryIndex; i++) { - const msg = getMessageFromEntryForCompaction(pathEntries[i]); - if (msg) turnPrefixMessages.push(msg); - } - } - - // Avoid a compaction that would summarize no history. - if (messagesToSummarize.length === 0 && turnPrefixMessages.length === 0 && !previousSummary) { - return undefined; - } - const fileOps = extractFileOperations(messagesToSummarize, pathEntries, prevCompactionIndex); - // Split turns retain their suffix, but their prefix file operations still belong in the summary. - if (cutPoint.isSplitTurn) { - for (const msg of turnPrefixMessages) { - extractFileOpsFromMessage(msg, fileOps); - } - } - - return { - firstKeptEntryId, - messagesToSummarize, - turnPrefixMessages, - isSplitTurn: cutPoint.isSplitTurn, - tokensBefore, - previousSummary, - fileOps, - settings, - }; -} -const TURN_PREFIX_SUMMARIZATION_PROMPT = `This is the PREFIX of a turn that was too large to keep. The SUFFIX (recent work) is retained. - -Summarize the prefix to provide context for the retained suffix: - -## Original Request -[What did the user ask for in this turn?] - -## Early Progress -- [Key decisions and work done in the prefix] - -## Context for Suffix -- [Information needed to understand the retained recent work] - -Be concise. Focus on what's needed to understand the kept suffix.`; - -/** - * Generate summaries for compaction using prepared data. - * Returns CompactionResult - SessionManager adds uuid/parentUuid when saving. - * - * @param preparation - Pre-calculated preparation from prepareCompaction() - * @param customInstructions - Optional custom focus for the summary - */ -/** Runs one summary wire call; hosts decorate each call with its own request identity. */ -export type SummaryCallRunner = ( - call: (callHeaders: Record | undefined) => Promise, -) => Promise; - -export async function compact( - preparation: CompactionPreparation, - model: Model, - apiKey: string, - headers?: Record, - customInstructions?: string, - signal?: AbortSignal, - thinkingLevel?: ThinkingLevel, - summaryCall: SummaryCallRunner = (call) => call(headers), - retry?: ProviderRetryPolicy, - sessionId?: string, -): Promise { - const { - firstKeptEntryId, - messagesToSummarize, - turnPrefixMessages, - isSplitTurn, - tokensBefore, - previousSummary, - fileOps, - settings, - } = preparation; - let summary: string; - const slices: SummarySlice[] = []; - - if (isSplitTurn && turnPrefixMessages.length > 0) { - // Split turns make two wire calls with different bodies; each needs its own identity. - const [historyResult, turnPrefixResult] = await Promise.all([ - messagesToSummarize.length > 0 - ? summaryCall((callHeaders) => - generateSummary( - messagesToSummarize, - model, - settings.reserveTokens, - apiKey, - callHeaders, - signal, - customInstructions, - previousSummary, - thinkingLevel, - retry, - sessionId, - ), - ) - : Promise.resolve({ summary: "No prior history." }), - summaryCall((callHeaders) => - generateTurnPrefixSummary( - turnPrefixMessages, - model, - settings.reserveTokens, - apiKey, - callHeaders, - signal, - thinkingLevel, - retry, - sessionId, - ), - ), - ]); - slices.push(historyResult, turnPrefixResult); - summary = `${historyResult.summary}\n\n---\n\n**Turn Context (split turn):**\n\n${turnPrefixResult.summary}`; - } else { - const result = await summaryCall((callHeaders) => - generateSummary( - messagesToSummarize, - model, - settings.reserveTokens, - apiKey, - callHeaders, - signal, - customInstructions, - previousSummary, - thinkingLevel, - retry, - sessionId, - ), - ); - slices.push(result); - summary = result.summary; - } - const { readFiles, modifiedFiles } = computeFileLists(fileOps); - summary += formatFileOperations(readFiles, modifiedFiles); - - if (!firstKeptEntryId) { - throw new Error("First kept entry has no UUID - session may need migration"); - } - - let usage: Usage | undefined; - for (const slice of slices) { - if (!slice.usage) continue; - usage ??= emptyUsage(); - addAssistantUsage(usage, slice.usage); - } - return { - summary, - firstKeptEntryId, - tokensBefore, - details: { readFiles, modifiedFiles } as CompactionDetails, - usage, - }; -} - -/** - * Generate a summary for a turn prefix (when splitting a turn). - */ -async function generateTurnPrefixSummary( - messages: AgentMessage[], - model: Model, - reserveTokens: number, - apiKey: string, - headers?: Record, - signal?: AbortSignal, - thinkingLevel?: ThinkingLevel, - retry?: ProviderRetryPolicy, - sessionId?: string, -): Promise { - const maxTokens = Math.floor(0.5 * reserveTokens); // Smaller budget for turn prefix - const llmMessages = convertToLlm(messages); - const conversationText = serializeConversation(llmMessages); - const promptText = `\n${conversationText}\n\n\n${TURN_PREFIX_SUMMARIZATION_PROMPT}`; - const summarizationMessages = [ - { - role: "user" as const, - content: [{ type: "text" as const, text: promptText }], - timestamp: Date.now(), - }, - ]; - - const response = await completeWithProviderRetry( - () => - completeSimple( - model, - { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, - model.reasoning && thinkingLevel && thinkingLevel !== "off" - ? { maxTokens, signal, apiKey, headers, sessionId, reasoning: thinkingLevel } - : { maxTokens, signal, apiKey, headers, sessionId }, - ), - { policy: retry, signal }, - ); - - if (response.stopReason === "error") { - throw new Error(`Turn prefix summarization failed: ${response.errorMessage || "Unknown error"}`); - } - - return { - summary: response.content - .filter((c): c is { type: "text"; text: string } => c.type === "text") - .map((c) => c.text) - .join("\n"), - usage: response.usage, - }; -} +// Compatibility exports; implementation lives with its session owner. + +export { + buildSummarizationPrompt, + compact, + findCutPoint, + findTurnStartIndex, + generateSummary, + prepareCompaction, + shouldCompact, +} from "../../session/compaction/summary.js"; +export { + COMPACT_SKILL_NAME, + type CompactionDetails, + type CompactionPreparation, + type CompactionResult, + type CompactionSettings, + type CutPointResult, + DEFAULT_COMPACTION_SETTINGS, + type SummaryCallRunner, + type SummarySlice, +} from "../../session/compaction/types.js"; +export { + type ContextUsageEstimate, + calculateContextTokens, + estimateContextTokens, + estimateTokens, + getLastAssistantUsage, +} from "../../session/context/token-estimate.js"; diff --git a/packages/coding-agent/src/core/compaction/index.ts b/packages/coding-agent/src/core/compaction/index.ts index d8c92a67b0..9e94c29cdb 100644 --- a/packages/coding-agent/src/core/compaction/index.ts +++ b/packages/coding-agent/src/core/compaction/index.ts @@ -1,7 +1,47 @@ -/** - * Compaction and summarization utilities. - */ +// Compatibility exports; implementation lives with its session owner. -export * from "./branch-summarization.js"; -export * from "./compaction.js"; -export * from "./utils.js"; +export { + buildSummarizationPrompt, + compact, + findCutPoint, + findTurnStartIndex, + generateSummary, + prepareCompaction, + shouldCompact, +} from "../../session/compaction/summary.js"; +export { + COMPACT_SKILL_NAME, + type CompactionDetails, + type CompactionPreparation, + type CompactionResult, + type CompactionSettings, + type CutPointResult, + DEFAULT_COMPACTION_SETTINGS, + type SummaryCallRunner, + type SummarySlice, +} from "../../session/compaction/types.js"; +export { + type BranchPreparation, + type BranchSummaryDetails, + type BranchSummaryResult, + type CollectEntriesResult, + collectEntriesForBranchSummary, + type GenerateBranchSummaryOptions, + generateBranchSummary, + prepareBranchEntries, +} from "../../session/context/branch-summary.js"; +export { SUMMARIZATION_SYSTEM_PROMPT, serializeConversation } from "../../session/context/conversation-text.js"; +export { + computeFileLists, + createFileOps, + extractFileOpsFromMessage, + type FileOperations, + formatFileOperations, +} from "../../session/context/file-tracking.js"; +export { + type ContextUsageEstimate, + calculateContextTokens, + estimateContextTokens, + estimateTokens, + getLastAssistantUsage, +} from "../../session/context/token-estimate.js"; diff --git a/packages/coding-agent/src/core/compaction/utils.ts b/packages/coding-agent/src/core/compaction/utils.ts index 72a764566d..13fac0a0f6 100644 --- a/packages/coding-agent/src/core/compaction/utils.ts +++ b/packages/coding-agent/src/core/compaction/utils.ts @@ -1,149 +1,10 @@ -/** - * Shared utilities for compaction and branch summarization. - */ - -import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { Message } from "@earendil-works/pi-ai"; -export interface FileOperations { - read: Set; - written: Set; - edited: Set; -} - -export function createFileOps(): FileOperations { - return { - read: new Set(), - written: new Set(), - edited: new Set(), - }; -} - -/** - * Extract file operations from tool calls in an assistant message. - */ -export function extractFileOpsFromMessage(message: AgentMessage, fileOps: FileOperations): void { - if (message.role !== "assistant") return; - if (!("content" in message) || !Array.isArray(message.content)) return; - - for (const block of message.content) { - if (typeof block !== "object" || block === null) continue; - if (!("type" in block) || block.type !== "toolCall") continue; - if (!("arguments" in block) || !("name" in block)) continue; - - const args = block.arguments as Record | undefined; - if (!args) continue; - - const path = typeof args.path === "string" ? args.path : undefined; - if (!path) continue; - - switch (block.name) { - case "edit": - fileOps.edited.add(path); - break; - } - } -} - -/** - * Compute final file lists from file operations. - * Returns readFiles (files only read, not modified) and modifiedFiles. - */ -export function computeFileLists(fileOps: FileOperations): { readFiles: string[]; modifiedFiles: string[] } { - const modified = new Set([...fileOps.edited, ...fileOps.written]); - const readOnly = [...fileOps.read].filter((f) => !modified.has(f)).sort(); - const modifiedFiles = [...modified].sort(); - return { readFiles: readOnly, modifiedFiles }; -} - -/** - * Format file operations as XML tags for summary. - */ -export function formatFileOperations(readFiles: string[], modifiedFiles: string[]): string { - const sections: string[] = []; - if (readFiles.length > 0) { - sections.push(`\n${readFiles.join("\n")}\n`); - } - if (modifiedFiles.length > 0) { - sections.push(`\n${modifiedFiles.join("\n")}\n`); - } - if (sections.length === 0) return ""; - return `\n\n${sections.join("\n\n")}`; -} -/** Maximum characters for a tool result in serialized summaries. */ -const TOOL_RESULT_MAX_CHARS = 2000; - -/** - * Truncate text to a maximum character length for summarization. - * Keeps the beginning and appends a truncation marker. - */ -function truncateForSummary(text: string, maxChars: number): string { - if (text.length <= maxChars) return text; - const truncatedChars = text.length - maxChars; - return `${text.slice(0, maxChars)}\n\n[... ${truncatedChars} more characters truncated]`; -} - -/** - * Serialize LLM messages to text for summarization. - * This prevents the model from treating it as a conversation to continue. - * Call convertToLlm() first to handle custom message types. - * - * Tool results are truncated to keep the summarization request within - * reasonable token budgets. Full content is not needed for summarization. - */ -export function serializeConversation(messages: Message[]): string { - const parts: string[] = []; - - for (const msg of messages) { - if (msg.role === "user") { - const content = - typeof msg.content === "string" - ? msg.content - : msg.content - .filter((c): c is { type: "text"; text: string } => c.type === "text") - .map((c) => c.text) - .join(""); - if (content) parts.push(`[User]: ${content}`); - } else if (msg.role === "assistant") { - const textParts: string[] = []; - const thinkingParts: string[] = []; - const toolCalls: string[] = []; - - for (const block of msg.content) { - if (block.type === "text") { - textParts.push(block.text); - } else if (block.type === "thinking") { - thinkingParts.push(block.thinking); - } else if (block.type === "toolCall") { - const args = block.arguments as Record; - const argsStr = Object.entries(args) - .map(([k, v]) => `${k}=${JSON.stringify(v)}`) - .join(", "); - toolCalls.push(`${block.name}(${argsStr})`); - } - } - - if (thinkingParts.length > 0) { - parts.push(`[Assistant thinking]: ${thinkingParts.join("\n")}`); - } - if (textParts.length > 0) { - parts.push(`[Assistant]: ${textParts.join("\n")}`); - } - if (toolCalls.length > 0) { - parts.push(`[Assistant tool calls]: ${toolCalls.join("; ")}`); - } - } else if (msg.role === "toolResult") { - const content = msg.content - .filter((c): c is { type: "text"; text: string } => c.type === "text") - .map((c) => c.text) - .join(""); - if (content) { - parts.push(`[Tool result]: ${truncateForSummary(content, TOOL_RESULT_MAX_CHARS)}`); - } - } - } - - return parts.join("\n\n"); -} -export const SUMMARIZATION_SYSTEM_PROMPT = `You are a context summarization assistant. Your task is to read a conversation between a user and an AI coding assistant, then produce a structured summary following the exact format specified. - -Do NOT continue the conversation. Do NOT respond to any questions in the conversation. ONLY output the structured summary.`; +// Compatibility exports; implementation lives with its session owner. + +export { SUMMARIZATION_SYSTEM_PROMPT, serializeConversation } from "../../session/context/conversation-text.js"; +export { + computeFileLists, + createFileOps, + extractFileOpsFromMessage, + type FileOperations, + formatFileOperations, +} from "../../session/context/file-tracking.js"; diff --git a/packages/coding-agent/src/core/context-tree.ts b/packages/coding-agent/src/core/context-tree.ts index 7546f8679c..1422ac9490 100644 --- a/packages/coding-agent/src/core/context-tree.ts +++ b/packages/coding-agent/src/core/context-tree.ts @@ -1,321 +1,8 @@ -import { existsSync, readdirSync, statSync } from "node:fs"; -import { basename, join } from "node:path"; -import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; -import type { RlmChildAgentStatus } from "./agent-session.js"; -import { calculateContextTokens, estimateContextTokens } from "./compaction/index.js"; -import type { ContextUsage } from "./extensions/index.js"; -import { buildSessionContext, type FileEntry, loadEntriesFromFile, type SessionEntry } from "./session-manager.js"; -import { addAssistantUsage, cloneUsage, emptyUsage, subtractAssistantUsage } from "./usage.js"; - -/** Resolves a model's context window so disk-only nodes can report utilization. */ -export type ContextWindowResolver = (provider: string, modelId: string) => number | undefined; - -/** - * One agent in the context overview: the main session or an RLM (sub-)agent. - * `ownUsage` excludes descendants; `totalUsage` includes completed descendants, matching /usage. - */ -export interface ContextTreeNode { - /** "root" for the session itself; sub-xxxx for an RLM child. */ - id: string; - label: string; - status: "active" | RlmChildAgentStatus; - model?: { provider: string; id: string }; - ownUsage: Usage; - totalUsage: Usage; - contextUsage?: ContextUsage; - children: ContextTreeNode[]; -} - -function isAssistantEntry(entry: SessionEntry): entry is SessionEntry & { - type: "message"; - message: AssistantMessage; -} { - return entry.type === "message" && entry.message.role === "assistant"; -} - -function readUserMessageText(content: unknown): string { - if (typeof content === "string") { - return content; - } - if (!Array.isArray(content)) { - return ""; - } - return content - .filter( - (block): block is { type: "text"; text: string } => - typeof block === "object" && - block !== null && - (block as { type?: unknown }).type === "text" && - typeof (block as { text?: unknown }).text === "string", - ) - .map((block) => block.text) - .join("\n"); -} - -function compactLabel(text: string, maxLength = 80): string { - const compact = text.replace(/\s+/g, " ").trim(); - if (compact.length <= maxLength) { - return compact; - } - return `${compact.slice(0, Math.max(0, maxLength - 3)).trimEnd()}...`; -} - -/** - * Usage totals for one agent: `totalUsage` sums the branch's assistant usage - * (attributed aggregates, so descendants are included), `ownUsage` removes the - * attributions targeting those assistants. Attribution entries are matched by - * target across ALL entries, not just the branch: attributions rewrite the - * target assistant's usage no matter which branch they were appended on, so a - * fork that keeps the assistant but drops the attribution entry must still - * subtract it. - * - * Totals are deliberately cumulative across compactions: compaction shrinks - * the model-facing context, not what the session has spent, so assistants - * dropped from the resolved context still count here. - */ -export function computeOwnAndTotalUsage( - branch: SessionEntry[], - allEntries: SessionEntry[], -): { ownUsage: Usage; totalUsage: Usage } { - const totalUsage = emptyUsage(); - const branchAssistantIds = new Set(); - for (const entry of branch) { - if (isAssistantEntry(entry)) { - branchAssistantIds.add(entry.id); - addAssistantUsage(totalUsage, entry.message.usage); - } else if ((entry.type === "compaction" || entry.type === "branch_summary") && entry.usage) { - addAssistantUsage(totalUsage, entry.usage); - } - } - const ownUsage = cloneUsage(totalUsage); - for (const entry of allEntries) { - if (entry.type === "child_usage_attributed" && branchAssistantIds.has(entry.targetId)) { - subtractAssistantUsage(ownUsage, entry.childUsage); - } - } - return { ownUsage, totalUsage }; -} - -/** - * Current context utilization from persisted entries, mirroring - * AgentSession.getContextUsage(): unknown right after a compaction until the - * next assistant response, otherwise the last assistant usage plus an - * estimate for trailing messages (tool results, queued user input) that have - * not hit the model yet. - */ -function computeContextUsageFromEntries( - allEntries: SessionEntry[], - branch: SessionEntry[], - contextWindow: number | undefined, -): ContextUsage | undefined { - if (!contextWindow || contextWindow <= 0) { - return undefined; - } - - let latestCompactionIndex = -1; - for (let i = branch.length - 1; i >= 0; i--) { - if (branch[i].type === "compaction") { - latestCompactionIndex = i; - break; - } - } - - if (latestCompactionIndex >= 0) { - let hasPostCompactionUsage = false; - for (let i = branch.length - 1; i > latestCompactionIndex; i--) { - const entry = branch[i]; - if (!isAssistantEntry(entry)) { - continue; - } - const assistant = entry.message; - if (assistant.stopReason === "aborted" || assistant.stopReason === "error") { - continue; - } - if (calculateContextTokens(assistant.usage) > 0) { - hasPostCompactionUsage = true; - } - break; - } - if (!hasPostCompactionUsage) { - return { tokens: null, contextWindow, percent: null }; - } - } - - const estimate = estimateContextTokens(buildSessionContext(allEntries).messages); - if (estimate.tokens <= 0) { - return undefined; - } - return { tokens: estimate.tokens, contextWindow, percent: (estimate.tokens / contextWindow) * 100 }; -} - -function sessionEntriesFromFile(file: string): SessionEntry[] { - return loadEntriesFromFile(file).filter((entry: FileEntry): entry is SessionEntry => entry.type !== "session"); -} - -/** - * Entries on the current branch, root to leaf, mirroring - * SessionManager.getBranch(): the leaf is the last appended entry and the - * branch is its parentId chain. Keeps forked/abandoned paths out of usage - * sums so disk nodes match what a live session would report. - */ -function branchEntries(entries: SessionEntry[]): SessionEntry[] { - if (entries.length === 0) { - return []; - } - const byId = new Map(entries.map((entry) => [entry.id, entry])); - const branch: SessionEntry[] = []; - const seen = new Set(); - let current: SessionEntry | undefined = entries[entries.length - 1]; - while (current && !seen.has(current.id)) { - seen.add(current.id); - branch.push(current); - current = current.parentId ? byId.get(current.parentId) : undefined; - } - return branch.reverse(); -} - -/** - * Terminal status for a persisted child, inferred from how its last assistant - * turn ended: errored and aborted runs should not render as successful. - */ -function statusFromBranch(entries: SessionEntry[]): "done" | "error" | "cancelled" { - for (let i = entries.length - 1; i >= 0; i--) { - const entry = entries[i]; - if (!isAssistantEntry(entry)) { - continue; - } - if (entry.message.stopReason === "error") { - return "error"; - } - if (entry.message.stopReason === "aborted") { - return "cancelled"; - } - return "done"; - } - return "done"; -} - -function findSessionFile(dir: string): string | undefined { - let newest: { path: string; mtime: number } | undefined; - for (const name of readdirSync(dir)) { - if (!name.endsWith(".jsonl")) { - continue; - } - const path = join(dir, name); - try { - const mtime = statSync(path).mtime.getTime(); - if (!newest || mtime > newest.mtime) { - newest = { path, mtime }; - } - } catch { - // Skip unreadable files. - } - } - return newest?.path; -} - -function listChildSessionDirs(rlmSessionDir: string): string[] { - let names: string[]; - try { - names = readdirSync(rlmSessionDir); - } catch { - return []; - } - return names - .filter((name) => name.startsWith("sub-")) - .map((name) => join(rlmSessionDir, name)) - .filter((path) => { - try { - return statSync(path).isDirectory(); - } catch { - return false; - } - }) - .sort((a, b) => { - try { - return statSync(a).mtime.getTime() - statSync(b).mtime.getTime(); - } catch { - return 0; - } - }); -} - -/** - * Build a context node for a completed RLM child from its persisted session - * dir (sub-xxxx/). Children that already attributed grandchild usage carry the - * aggregate on their assistant messages (applyChildUsageAttributions), so own - * usage is recovered by subtracting the attribution entries. Returns undefined - * when the dir holds no readable session. - */ -export function loadContextTreeChildFromDisk( - childSessionDir: string, - resolveContextWindow: ContextWindowResolver, -): ContextTreeNode | undefined { - const sessionFile = findSessionFile(childSessionDir); - if (!sessionFile) { - return undefined; - } - const allEntries = sessionEntriesFromFile(sessionFile); - const branch = branchEntries(allEntries); - if (branch.length === 0) { - return undefined; - } - - const { ownUsage, totalUsage } = computeOwnAndTotalUsage(branch, allEntries); - - let model: { provider: string; id: string } | undefined; - for (const entry of branch) { - if (entry.type === "model_change") { - model = { provider: entry.provider, id: entry.modelId }; - } - } - - let label = ""; - for (const entry of branch) { - if (entry.type === "message" && entry.message.role === "user") { - label = compactLabel(readUserMessageText(entry.message.content)); - if (label) { - break; - } - } - } - - const contextWindow = model ? resolveContextWindow(model.provider, model.id) : undefined; - - return { - id: basename(childSessionDir), - label: label || "child agent", - status: statusFromBranch(branch), - model, - ownUsage, - totalUsage, - contextUsage: computeContextUsageFromEntries(allEntries, branch, contextWindow), - children: loadContextTreeChildrenFromDisk(childSessionDir, resolveContextWindow), - }; -} - -/** - * Build context nodes for all persisted RLM children under an RLM session - * dir, recursing into nested sub-* dirs for grandchildren. `skipIds` - * excludes children that are already represented live. - */ -export function loadContextTreeChildrenFromDisk( - rlmSessionDir: string | undefined, - resolveContextWindow: ContextWindowResolver, - skipIds?: ReadonlySet, -): ContextTreeNode[] { - if (!rlmSessionDir || !existsSync(rlmSessionDir)) { - return []; - } - const nodes: ContextTreeNode[] = []; - for (const childDir of listChildSessionDirs(rlmSessionDir)) { - if (skipIds?.has(basename(childDir))) { - continue; - } - const node = loadContextTreeChildFromDisk(childDir, resolveContextWindow); - if (node) { - nodes.push(node); - } - } - return nodes; -} +// Compatibility exports; implementation lives with its session owner. +export { + type ContextTreeNode, + type ContextWindowResolver, + computeOwnAndTotalUsage, + loadContextTreeChildFromDisk, + loadContextTreeChildrenFromDisk, +} from "../session/context/context-tree.js"; diff --git a/packages/coding-agent/src/core/extensions/runner.ts b/packages/coding-agent/src/core/extensions/runner.ts index e27b5c9111..c927a93d61 100644 --- a/packages/coding-agent/src/core/extensions/runner.ts +++ b/packages/coding-agent/src/core/extensions/runner.ts @@ -6,11 +6,11 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; import type { ImageContent, Model } from "@earendil-works/pi-ai"; import type { KeyId } from "@earendil-works/pi-tui"; import { type Theme, theme } from "../../modes/interactive/theme/theme.js"; +import type { BuildSystemPromptOptions } from "../../session/context/system-prompt.js"; import type { ResourceDiagnostic } from "../diagnostics.js"; import type { KeybindingsConfig } from "../keybindings.js"; import type { ModelRegistry } from "../model-registry.js"; import type { SessionManager } from "../session-manager.js"; -import type { BuildSystemPromptOptions } from "../system-prompt.js"; import type { BeforeAgentStartEvent, BeforeAgentStartEventResult, diff --git a/packages/coding-agent/src/core/extensions/types.ts b/packages/coding-agent/src/core/extensions/types.ts index 21abe7a16a..0c741831f7 100644 --- a/packages/coding-agent/src/core/extensions/types.ts +++ b/packages/coding-agent/src/core/extensions/types.ts @@ -41,15 +41,16 @@ import type { } from "@earendil-works/pi-tui"; import type { Static, TSchema } from "typebox"; import type { Theme } from "../../modes/interactive/theme/theme.js"; +import type { CompactionPreparation, CompactionResult } from "../../session/compaction/types.js"; +import type { CustomMessage } from "../../session/context/messages.js"; +import type { BuildSystemPromptOptions } from "../../session/context/system-prompt.js"; +import type { HarnessState, RefinementProposal, RefinementResult } from "../../session/refinement/types.js"; import type { BashResult } from "../bash-executor.js"; -import type { CompactionPreparation, CompactionResult } from "../compaction/index.js"; import type { EventBus } from "../event-bus.js"; import type { ExecOptions, ExecResult } from "../exec.js"; import type { ReadonlyFooterDataProvider } from "../footer-data-provider.js"; import type { KeybindingsManager } from "../keybindings.js"; -import type { CustomMessage } from "../messages.js"; import type { ModelRegistry } from "../model-registry.js"; -import type { HarnessState, RefinementProposal, RefinementResult } from "../refinement/index.js"; import type { BranchSummaryEntry, CompactionEntry, @@ -59,7 +60,6 @@ import type { } from "../session-manager.js"; import type { SlashCommandInfo } from "../slash-commands.js"; import type { SourceInfo } from "../source-info.js"; -import type { BuildSystemPromptOptions } from "../system-prompt.js"; import type { BashOperations } from "../tools/bash.js"; import type { EditToolDetails } from "../tools/edit.js"; import type { @@ -70,9 +70,9 @@ import type { IpythonToolInput, } from "../tools/index.js"; +export type { BuildSystemPromptOptions } from "../../session/context/system-prompt.js"; export type { ExecOptions, ExecResult } from "../exec.js"; export type { AppKeybinding, KeybindingsManager } from "../keybindings.js"; -export type { BuildSystemPromptOptions } from "../system-prompt.js"; export type { AgentToolResult, AgentToolUpdateCallback, ToolExecutionMode }; /** Options for extension UI dialogs. */ export interface ExtensionUIDialogOptions { diff --git a/packages/coding-agent/src/core/index.ts b/packages/coding-agent/src/core/index.ts index 587b0f5178..bdf0f8cd9e 100644 --- a/packages/coding-agent/src/core/index.ts +++ b/packages/coding-agent/src/core/index.ts @@ -9,7 +9,15 @@ export { type AgentSessionEventListener, type ModelCycleResult, type PromptOptions, -} from "./agent-session.js"; +} from "../session/agent-session.js"; +export type { + CreateRlmSubagentRuntimeOptions, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "../session/children/runtime-contracts.js"; +export type { CompactionResult } from "../session/compaction/types.js"; +export type { SessionStats } from "../session/context/session-stats.js"; +export type { RefinementResult } from "../session/refinement/types.js"; export type { AgentSessionRuntimeConfig } from "./agent-session-config.js"; export { AgentSessionRuntime, @@ -29,7 +37,6 @@ export { createAgentSessionServices, } from "./agent-session-services.js"; export { type BashExecutorOptions, type BashResult, executeBashWithOperations } from "./bash-executor.js"; -export type { CompactionResult } from "./compaction/index.js"; export { createEventBus, type EventBus, type EventBusController } from "./event-bus.js"; // Extensions system export { @@ -77,8 +84,5 @@ export { type TurnStartEvent, type WorkingIndicatorOptions, } from "./extensions/index.js"; -export type { RefinementResult } from "./refinement/index.js"; -export type { CreateRlmSubagentRuntimeOptions, RlmSubagentRuntime, SubagentRuntimeHost } from "./rlm-runtime.js"; export { SessionImportFileNotFoundError } from "./session-import-errors.js"; -export type { SessionStats } from "./session-stats.js"; export { createSyntheticSourceInfo } from "./source-info.js"; diff --git a/packages/coding-agent/src/core/messages.ts b/packages/coding-agent/src/core/messages.ts index 09f3c5ceae..d279b3c1c5 100644 --- a/packages/coding-agent/src/core/messages.ts +++ b/packages/coding-agent/src/core/messages.ts @@ -1,670 +1,68 @@ -/** - * Custom message types and transformers for the coding agent. - * - * Extends the base AgentMessage type with coding-agent specific message types, - * and provides a transformer to convert them to LLM-compatible messages. - */ - -import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { ImageContent, Message, TextContent } from "@earendil-works/pi-ai"; -import type { AgentCronJob } from "./cron-jobs.js"; -import { - type AppliedRefinementEdit, - formatRefinementNoticeBody, - type HarnessScope, - type RefinementResult, -} from "./refinement/refinement.js"; -import { isSessionSlashCommandName, parseSessionSlashCommand, type SessionSlashCommand } from "./slash-commands.js"; - -export const COMPACTION_SUMMARY_PREFIX = `[compaction-summary] - -The conversation history before this point was compacted into the following summary: - - -`; - -export const COMPACTION_SUMMARY_SUFFIX = ` -`; - -export const BRANCH_SUMMARY_PREFIX = `[branch-summary] - -The following is a summary of a branch that this conversation came back from: - - -`; - -export const BRANCH_SUMMARY_SUFFIX = ``; - -export const HEARTBEAT_PROMPT_CUSTOM_TYPE = "heartbeat_prompt"; -export const HEARTBEAT_PROMPT_PREVIEW_LABEL = "Heartbeat prompt"; -export const IPYTHON_STATE_RESTORED_CUSTOM_TYPE = "ipython_state_restored"; -export const SESSION_SLASH_COMMAND_CUSTOM_TYPE = "session_slash_command"; -export const SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE = "session_slash_command_result"; -export const COMPACTION_OUTCOME_CUSTOM_TYPE = "compaction_outcome"; -export const REFINEMENT_OUTCOME_CUSTOM_TYPE = "refinement_outcome"; -export const REFINEMENT_NOTICE_CUSTOM_TYPE = "refinement_notice"; -export const HARNESS_DIGEST_CUSTOM_TYPE = "harness_digest"; -export const RLM_CHILD_FAILURE_CUSTOM_TYPE = "rlm_child_failure"; -export const RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE = "rlm_child_terminal_notice"; -export const ASYNC_BASH_COMPLETION_CUSTOM_TYPE = "async_bash_completion"; -export const ASYNC_BASH_COMPLETION_PREVIEW_LABEL = "Background command finished"; - -/** - * Names and other metadata interpolated into a `[ ...]` header line must not - * carry the characters that delimit the header itself (brackets, newlines, commas, - * or the relationship separator ":"). - */ -export function sanitizeMessageHeaderValue(value: string): string { - return value.replace(/[\s,:[\]]+/g, " ").trim(); -} - -export interface SessionSlashCommandDetails { - command: SessionSlashCommand; - commandEntryId?: string; -} - -export interface SessionSlashCommandResultDetails { - command: SessionSlashCommand; - success: boolean; - severity: "info" | "warning" | "error"; - error?: string; - commandEntryId?: string; -} - -export interface SessionSlashCommandMessage extends CustomMessage { - customType: typeof SESSION_SLASH_COMMAND_CUSTOM_TYPE; - content: string; - details: SessionSlashCommandDetails; -} - -export interface SessionSlashCommandResultMessage extends CustomMessage { - customType: typeof SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE; - content: string; - details: SessionSlashCommandResultDetails; -} - -export type CompactionOutcomeReason = "threshold" | "overflow" | "requested"; -export type CompactionOutcome = "skipped" | "cancelled" | "failed"; - -export interface CompactionOutcomeDetails { - reason: CompactionOutcomeReason; - outcome: CompactionOutcome; -} - -export interface CompactionOutcomeMessage extends CustomMessage { - customType: typeof COMPACTION_OUTCOME_CUSTOM_TYPE; - content: string; - details: CompactionOutcomeDetails; -} - -export interface RefinementOutcomeDetails { - refinementId: string; - summary: string; - scope: HarnessScope; - rollbackOf?: string; - edits: AppliedRefinementEdit[]; -} - -export interface RefinementOutcomeMessage extends CustomMessage { - customType: typeof REFINEMENT_OUTCOME_CUSTOM_TYPE; - content: string; - details: RefinementOutcomeDetails; -} - -/** How a refinement was initiated: reviewer-triggered auto-refine, the /refine slash command, or the model's own refine.run(). */ -export type RefinementSource = "auto" | "user" | "self"; - -export interface RefinementNoticeDetails extends RefinementOutcomeDetails { - source: RefinementSource; -} - -export interface RefinementNoticeMessage extends CustomMessage { - customType: typeof REFINEMENT_NOTICE_CUSTOM_TYPE; - content: string; - details: RefinementNoticeDetails; -} - -export interface HarnessDigestDetails { - digest: string; -} - -export const HARNESS_DIGEST_PREFIX = `[harness-digest] - -The persistent memories produced across this session so far: - - -`; - -export const HARNESS_DIGEST_SUFFIX = ` -`; - -export function createHarnessDigestMessage( - digest: string, - timestamp = Date.now(), -): CustomMessage { - return { - role: "custom", - customType: HARNESS_DIGEST_CUSTOM_TYPE, - content: HARNESS_DIGEST_PREFIX + digest + HARNESS_DIGEST_SUFFIX, - display: false, - details: { digest }, - timestamp, - }; -} - -export interface RlmChildFailureDetails { - childId: string; - sessionName: string; - error: string; -} - -export type RlmChildTerminalNoticeDetails = - | { - kind: "cancelled"; - childId: string; - sessionName: string; - reason?: string; - } - | { - kind: "completed_without_reply"; - childId: string; - sessionName: string; - lastAssistantTextPreview?: string; - }; - -export interface AsyncBashCompletionDetails { - pid: number; - command: string; - exitCode: number; -} - -interface AsyncBashCompletionMessage extends CustomMessage { - customType: typeof ASYNC_BASH_COMPLETION_CUSTOM_TYPE; - content: string; -} - -export function createAsyncBashCompletionMessage( - details: AsyncBashCompletionDetails, - timestamp = Date.now(), -): AsyncBashCompletionMessage { - return { - role: "custom", - customType: ASYNC_BASH_COMPLETION_CUSTOM_TYPE, - content: `[bash-done pid:${details.pid} exit:${details.exitCode}] - -Command: ${JSON.stringify(details.command)}`, - display: true, - details, - timestamp, - }; -} - -export function createRlmChildFailureMessage( - details: RlmChildFailureDetails, - timestamp = Date.now(), -): CustomMessage { - return { - role: "custom", - customType: RLM_CHILD_FAILURE_CUSTOM_TYPE, - content: `[child-failed child:${sanitizeMessageHeaderValue(details.sessionName)}] - -${details.error}`, - display: true, - details, - timestamp, - }; -} - -export function createRlmChildTerminalNoticeMessage( - details: RlmChildTerminalNoticeDetails, - timestamp = Date.now(), -): CustomMessage { - const childName = sanitizeMessageHeaderValue(details.sessionName); - const content = - details.kind === "cancelled" - ? `[child-exited: cancelled child:${childName}]${details.reason ? `\n\n${details.reason}` : ""}` - : `[child-exited: no-reply child:${childName}]${details.lastAssistantTextPreview ? `\n\nLast assistant text: ${details.lastAssistantTextPreview}` : ""}`; - return { - role: "custom", - customType: RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, - content, - display: true, - details, - timestamp, - }; -} - -/** - * Message type for bash executions via the ! command. - */ -export interface BashExecutionMessage { - role: "bashExecution"; - command: string; - output: string; - exitCode: number | undefined; - cancelled: boolean; - truncated: boolean; - fullOutputPath?: string; - timestamp: number; - /** If true, this message is excluded from LLM context (!! prefix) */ - excludeFromContext?: boolean; -} - -/** - * Message type for extension-injected messages via sendMessage(). - * These are custom messages that extensions can inject into the conversation. - */ -export interface CustomMessage { - role: "custom"; - customType: string; - content: string | (TextContent | ImageContent)[]; - display: boolean; - details?: T; - timestamp: number; -} - -export interface HeartbeatPromptDetails { - jobId: string; - schedule: string; - status: AgentCronJob["status"]; - runCount: number; - nextRunAt?: string; - lastRunAt?: string; -} - -export interface IpythonStateRestoredDetails { - restored: boolean; -} - -export interface BranchSummaryMessage { - role: "branchSummary"; - summary: string; - fromId: string; - timestamp: number; -} - -export interface CompactionSummaryMessage { - role: "compactionSummary"; - summary: string; - tokensBefore: number; - /** Number of retained messages that precede this summary in transcript presentation. */ - retainedMessageCount?: number; - /** User instructions that guided the summary (from `/compact `) */ - customInstructions?: string; - /** Harness digest snapshot rendered before the summary in LLM context. Attached mechanically at compaction, never summarized. */ - harnessDigest?: string; - timestamp: number; -} - -declare module "@earendil-works/pi-agent-core" { - interface CustomAgentMessages { - bashExecution: BashExecutionMessage; - custom: CustomMessage; - branchSummary: BranchSummaryMessage; - compactionSummary: CompactionSummaryMessage; - } -} - -/** - * Format bash output for LLM context. The fence must be longer than any - * backtick run in the output so command output cannot terminate it early. - */ -export function bashOutputToText( - msg: Pick, -): string { - let text = ""; - if (msg.output) { - let longestBacktickRun = 0; - for (const match of msg.output.matchAll(/`+/g)) { - longestBacktickRun = Math.max(longestBacktickRun, match[0].length); - } - const fence = "`".repeat(Math.max(3, longestBacktickRun + 1)); - text += `${fence}\n${msg.output}\n${fence}`; - } else { - text += "(no output)"; - } - if (msg.cancelled) { - text += "\n\n(command cancelled)"; - } else if (msg.exitCode !== null && msg.exitCode !== undefined && msg.exitCode !== 0) { - text += `\n\nCommand exited with code ${msg.exitCode}`; - } - if (msg.truncated) { - text += msg.fullOutputPath - ? `\n\n[Output truncated. Full output: ${msg.fullOutputPath}]` - : "\n\n[Output truncated.]"; - } - return text; -} - -/** - * Convert a BashExecutionMessage to user message text for LLM context. - */ -export function bashExecutionToText(msg: BashExecutionMessage): string { - return `Ran \`${msg.command}\`\n${bashOutputToText(msg)}`; -} - -export function createBranchSummaryMessage(summary: string, fromId: string, timestamp: string): BranchSummaryMessage { - return { - role: "branchSummary", - summary, - fromId, - timestamp: new Date(timestamp).getTime(), - }; -} - -export function createCompactionSummaryMessage( - summary: string, - tokensBefore: number, - timestamp: string, - customInstructions?: string, - retainedMessageCount?: number, - harnessDigest?: string, -): CompactionSummaryMessage { - return { - role: "compactionSummary", - summary, - tokensBefore, - retainedMessageCount, - customInstructions, - harnessDigest, - timestamp: new Date(timestamp).getTime(), - }; -} - -/** Convert CustomMessageEntry to AgentMessage format */ -export function createCustomMessage( - customType: string, - content: string | (TextContent | ImageContent)[], - display: boolean, - details: unknown | undefined, - timestamp: string, -): CustomMessage { - return { - role: "custom", - customType, - content, - display, - details, - timestamp: new Date(timestamp).getTime(), - }; -} - -export function createSessionSlashCommandMessage( - command: SessionSlashCommand, - details: Omit = {}, - display = true, - timestamp = Date.now(), -): SessionSlashCommandMessage { - return { - role: "custom", - customType: SESSION_SLASH_COMMAND_CUSTOM_TYPE, - content: command.text, - display, - details: { ...details, command: { ...command } }, - timestamp, - }; -} - -export function createSessionSlashCommandResultMessage( - content: string, - details: SessionSlashCommandResultDetails, - display = true, - timestamp = Date.now(), -): SessionSlashCommandResultMessage { - return { - role: "custom", - customType: SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, - content, - display, - details: { ...details, command: { ...details.command } }, - timestamp, - }; -} - -export function createCompactionOutcomeMessage( - content: string, - details: CompactionOutcomeDetails, - display = true, - timestamp = Date.now(), -): CompactionOutcomeMessage { - return { - role: "custom", - customType: COMPACTION_OUTCOME_CUSTOM_TYPE, - content, - display, - details: { ...details }, - timestamp, - }; -} - -export function createRefinementOutcomeMessage( - result: RefinementResult, - display = true, - timestamp = Date.now(), -): RefinementOutcomeMessage { - return { - role: "custom", - customType: REFINEMENT_OUTCOME_CUSTOM_TYPE, - content: `Refinement complete: ${result.summary}`, - display, - details: { - refinementId: result.id, - summary: result.summary, - scope: result.scope ?? "local", - ...(result.rollbackOf ? { rollbackOf: result.rollbackOf } : {}), - edits: result.appliedEdits, - }, - timestamp, - }; -} - -/** Model-facing refinement notice: passes convertToLlm (unlike the refinement_outcome audit entry); display false because the TUI renders the outcome message. */ -export function createRefinementNoticeMessage( - result: RefinementResult, - source: RefinementSource, - timestamp = Date.now(), -): RefinementNoticeMessage { - return { - role: "custom", - customType: REFINEMENT_NOTICE_CUSTOM_TYPE, - content: `[${source}-refinement]\n\n${formatRefinementNoticeBody(result)}`, - display: false, - details: { - refinementId: result.id, - summary: result.summary, - scope: result.scope ?? "local", - ...(result.rollbackOf ? { rollbackOf: result.rollbackOf } : {}), - edits: result.appliedEdits, - source, - }, - timestamp, - }; -} - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null; -} - -function hasValidCustomMessageEnvelope(message: Record, customType: string): boolean { - return ( - message.role === "custom" && - message.customType === customType && - typeof message.content === "string" && - typeof message.display === "boolean" && - typeof message.timestamp === "number" && - Number.isFinite(message.timestamp) - ); -} - -export function isSessionSlashCommand(value: unknown): value is SessionSlashCommand { - if ( - !isRecord(value) || - !isSessionSlashCommandName(value.name) || - typeof value.args !== "string" || - typeof value.text !== "string" - ) { - return false; - } - const parsed = parseSessionSlashCommand(value.text); - return ( - parsed !== undefined && parsed.name === value.name && parsed.args === value.args && parsed.text === value.text - ); -} - -function isValidCommandEntryId(value: unknown): value is string | undefined { - return value === undefined || (typeof value === "string" && value.length > 0); -} - -export function isSessionSlashCommandMessage(message: unknown): message is SessionSlashCommandMessage { - if ( - !isRecord(message) || - !hasValidCustomMessageEnvelope(message, SESSION_SLASH_COMMAND_CUSTOM_TYPE) || - typeof message.content !== "string" - ) { - return false; - } - if (!isRecord(message.details) || !isSessionSlashCommand(message.details.command)) return false; - return message.content === message.details.command.text && isValidCommandEntryId(message.details.commandEntryId); -} - -export function isSessionSlashCommandResultMessage(message: unknown): message is SessionSlashCommandResultMessage { - if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE)) - return false; - if (!isRecord(message.details) || !isSessionSlashCommand(message.details.command)) return false; - return ( - typeof message.details.success === "boolean" && - (message.details.severity === "info" || - message.details.severity === "warning" || - message.details.severity === "error") && - (message.details.error === undefined || typeof message.details.error === "string") && - isValidCommandEntryId(message.details.commandEntryId) - ); -} - -export function isCompactionOutcomeMessage(message: unknown): message is CompactionOutcomeMessage { - if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, COMPACTION_OUTCOME_CUSTOM_TYPE)) return false; - if (!isRecord(message.details)) return false; - return ( - (message.details.reason === "threshold" || - message.details.reason === "overflow" || - message.details.reason === "requested") && - (message.details.outcome === "skipped" || - message.details.outcome === "cancelled" || - message.details.outcome === "failed") - ); -} - -function isAppliedRefinementEdit(value: unknown): value is AppliedRefinementEdit { - return ( - isRecord(value) && - (value.action === "create" || value.action === "update" || value.action === "delete") && - typeof value.kind === "string" && - typeof value.id === "string" && - typeof value.applied === "boolean" - ); -} - -export function isRefinementOutcomeMessage(message: unknown): message is RefinementOutcomeMessage { - if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, REFINEMENT_OUTCOME_CUSTOM_TYPE)) return false; - if (!isRecord(message.details)) return false; - return ( - typeof message.details.summary === "string" && - (message.details.scope === "local" || message.details.scope === "global") && - Array.isArray(message.details.edits) && - message.details.edits.every(isAppliedRefinementEdit) - ); -} - -export interface HeartbeatPromptMessage extends CustomMessage { - customType: typeof HEARTBEAT_PROMPT_CUSTOM_TYPE; - content: string; -} - -export function createHeartbeatPromptMessage(job: AgentCronJob, timestamp = Date.now()): HeartbeatPromptMessage { - return { - role: "custom", - customType: HEARTBEAT_PROMPT_CUSTOM_TYPE, - content: `[heartbeat: ${sanitizeMessageHeaderValue(job.schedule.expression)} run#${job.runCount}]\n\n${job.prompt}`, - display: true, - details: { - jobId: job.id, - schedule: job.schedule.expression, - status: job.status, - runCount: job.runCount, - nextRunAt: job.nextRunAt, - lastRunAt: job.lastRunAt, - }, - timestamp, - }; -} - -/** - * Transform AgentMessages (including custom types) to LLM-compatible Messages. - * - * This is used by: - * - Agent's transormToLlm option (for prompt calls and queued messages) - * - Compaction's generateSummary (for summarization) - * - Custom extensions and tools - */ -export function convertToLlm(messages: AgentMessage[]): Message[] { - return messages - .map((m): Message | undefined => { - switch (m.role) { - case "bashExecution": - if (m.excludeFromContext) { - return undefined; - } - return { - role: "user", - content: [{ type: "text", text: bashExecutionToText(m) }], - timestamp: m.timestamp, - }; - case "custom": { - if ( - m.customType === SESSION_SLASH_COMMAND_CUSTOM_TYPE || - m.customType === SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE || - m.customType === COMPACTION_OUTCOME_CUSTOM_TYPE || - m.customType === REFINEMENT_OUTCOME_CUSTOM_TYPE - ) { - return undefined; - } - const content = typeof m.content === "string" ? [{ type: "text" as const, text: m.content }] : m.content; - return { - role: "user", - content, - timestamp: m.timestamp, - }; - } - case "branchSummary": - return { - role: "user", - content: [{ type: "text" as const, text: BRANCH_SUMMARY_PREFIX + m.summary + BRANCH_SUMMARY_SUFFIX }], - timestamp: m.timestamp, - }; - case "compactionSummary": { - const digestBlock = m.harnessDigest - ? `${HARNESS_DIGEST_PREFIX}${m.harnessDigest}${HARNESS_DIGEST_SUFFIX}\n\n` - : ""; - return { - role: "user", - content: [ - { - type: "text" as const, - text: digestBlock + COMPACTION_SUMMARY_PREFIX + m.summary + COMPACTION_SUMMARY_SUFFIX, - }, - ], - timestamp: m.timestamp, - }; - } - case "user": - case "assistant": - case "toolResult": - return m; - default: - // biome-ignore lint/correctness/noSwitchDeclarations: fine - const _exhaustiveCheck: never = m; - return undefined; - } - }) - .filter((m) => m !== undefined); -} +// Compatibility exports; implementation lives with its session owner. +export { + ASYNC_BASH_COMPLETION_CUSTOM_TYPE, + ASYNC_BASH_COMPLETION_PREVIEW_LABEL, + type AsyncBashCompletionDetails, + type BashExecutionMessage, + BRANCH_SUMMARY_PREFIX, + BRANCH_SUMMARY_SUFFIX, + type BranchSummaryMessage, + bashExecutionToText, + bashOutputToText, + COMPACTION_OUTCOME_CUSTOM_TYPE, + COMPACTION_SUMMARY_PREFIX, + COMPACTION_SUMMARY_SUFFIX, + type CompactionOutcome, + type CompactionOutcomeDetails, + type CompactionOutcomeMessage, + type CompactionOutcomeReason, + type CompactionSummaryMessage, + type CustomMessage, + convertToLlm, + createAsyncBashCompletionMessage, + createBranchSummaryMessage, + createCompactionOutcomeMessage, + createCompactionSummaryMessage, + createCustomMessage, + createHarnessDigestMessage, + createHeartbeatPromptMessage, + createRefinementNoticeMessage, + createRefinementOutcomeMessage, + createRlmChildFailureMessage, + createRlmChildTerminalNoticeMessage, + createSessionSlashCommandMessage, + createSessionSlashCommandResultMessage, + HARNESS_DIGEST_CUSTOM_TYPE, + HARNESS_DIGEST_PREFIX, + HARNESS_DIGEST_SUFFIX, + type HarnessDigestDetails, + HEARTBEAT_PROMPT_CUSTOM_TYPE, + HEARTBEAT_PROMPT_PREVIEW_LABEL, + type HeartbeatPromptDetails, + type HeartbeatPromptMessage, + IPYTHON_STATE_RESTORED_CUSTOM_TYPE, + type IpythonStateRestoredDetails, + isCompactionOutcomeMessage, + isRefinementOutcomeMessage, + isSessionSlashCommand, + isSessionSlashCommandMessage, + isSessionSlashCommandResultMessage, + REFINEMENT_NOTICE_CUSTOM_TYPE, + REFINEMENT_OUTCOME_CUSTOM_TYPE, + type RefinementNoticeDetails, + type RefinementNoticeMessage, + type RefinementOutcomeDetails, + type RefinementOutcomeMessage, + type RefinementSource, + RLM_CHILD_FAILURE_CUSTOM_TYPE, + RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, + type RlmChildFailureDetails, + type RlmChildTerminalNoticeDetails, + SESSION_SLASH_COMMAND_CUSTOM_TYPE, + SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, + type SessionSlashCommandDetails, + type SessionSlashCommandMessage, + type SessionSlashCommandResultDetails, + type SessionSlashCommandResultMessage, + sanitizeMessageHeaderValue, +} from "../session/context/messages.js"; diff --git a/packages/coding-agent/src/core/prompt-admission.ts b/packages/coding-agent/src/core/prompt-admission.ts index 7b7908ba78..451002e657 100644 --- a/packages/coding-agent/src/core/prompt-admission.ts +++ b/packages/coding-agent/src/core/prompt-admission.ts @@ -1,42 +1,5 @@ -export class PromptAdmissionCancelledError extends Error { - constructor() { - super("Prompt admission was cancelled."); - this.name = "PromptAdmissionCancelledError"; - } -} - -export function throwIfPromptAdmissionCancelled(signal: AbortSignal | undefined): void { - if (signal?.aborted) throw new PromptAdmissionCancelledError(); -} - -/** - * Await `promise` unless `signal` aborts first. Always observes the supplied - * work's rejection so a cancelled admission never leaks an unhandled rejection. - */ -export function waitForPromptAdmission(promise: Promise, signal: AbortSignal | undefined): Promise { - if (!signal) return promise; - if (signal.aborted) { - void promise.catch(() => {}); - return Promise.reject(new PromptAdmissionCancelledError()); - } - return new Promise((resolve, reject) => { - const cleanup = () => signal.removeEventListener("abort", onAbort); - const onAbort = () => { - cleanup(); - reject(new PromptAdmissionCancelledError()); - }; - signal.addEventListener("abort", onAbort, { once: true }); - // Close the listener-registration race before observing the awaited work. - if (signal.aborted) return onAbort(); - promise.then( - (value) => { - cleanup(); - resolve(value); - }, - (error: unknown) => { - cleanup(); - reject(error); - }, - ); - }); -} +export { + PromptAdmissionCancelledError, + throwIfPromptAdmissionCancelled, + waitForPromptAdmission, +} from "../session/input/prompt-admission.js"; diff --git a/packages/coding-agent/src/core/prompts/index.ts b/packages/coding-agent/src/core/prompts/index.ts index ecf670098f..2114832a43 100644 --- a/packages/coding-agent/src/core/prompts/index.ts +++ b/packages/coding-agent/src/core/prompts/index.ts @@ -1,7 +1,8 @@ +// Compatibility exports; implementation lives with its session owner. export { buildChildAgentDoctrine, buildRlmPrompt, buildSubagentGuidance, type ChildAgentDoctrineOptions, type RlmPromptOptions, -} from "./rlm.js"; +} from "../../session/context/prompts/index.js"; diff --git a/packages/coding-agent/src/core/prompts/rlm.ts b/packages/coding-agent/src/core/prompts/rlm.ts index c8b932cc8b..ffe8ba51e0 100644 --- a/packages/coding-agent/src/core/prompts/rlm.ts +++ b/packages/coding-agent/src/core/prompts/rlm.ts @@ -1,232 +1,8 @@ -import { DEFAULT_RLM_EXTRA_IMPORT_LABELS } from "../kernel/bootstrap.js"; - -export interface RlmPromptOptions { - cwd: string; - skillsDir?: string; - installedSkills?: string[]; - messagesPath: string; - allowRecursion?: boolean; - depth?: number; - parentAgent?: string; - activeTools?: string[]; -} - -const LONG_RUNNING_WORK_PROMPT = [ - "For slow or independently completing work, use a nonblocking control loop: start the work, record its handle or output location, then end your turn. A `bash()` handle left running beyond its creating cell sends a completion follow-up; when it arrives, inspect the saved handle and continue. Reading a finished handle's result first cancels that follow-up.", - "When delegation is available and useful, assign independent substantive tasks to separate workers. Start independent workers without waiting for each one sequentially, and let them run in parallel.", - "Do not keep the turn open by polling with `time.sleep()` or shell `sleep`, and do not replace polling with a long blocking `await`. Await only the short operation needed to start work or inspect a result that is already available; otherwise end the turn.", -].join("\n"); - -const USER_PROGRESS_PROMPT = - "As the user-facing root agent, when work follows a plan, uses many subagents, or spans multiple turns, proactively give regular concise progress updates so the user does not have to ask. State the current plan, what has completed, any blockers, the proposed fixes, and the next actions. Lead with user-visible outcomes rather than internal process or gate names. Mention internal details only when they explain a blocker or decision. Send an update at meaningful milestones and before ending a turn while work is still running. Do not repeat unchanged status or interrupt short work with unnecessary updates."; - -const SIMPLIFIED_TECHNICAL_ENGLISH_PROMPT = [ - "Use simplified technical English by default for user-facing prose.", - "Prefer short sentences, common words, and concrete verbs. State one main action or fact per sentence when practical. Use lists for steps or conditions.", - "Keep necessary technical terms, names, commands, code, paths, and exact quoted text unchanged. State uncertainty directly.", - "Treat this as clarity guidance, not a claim of formal ASD-STE100 compliance. Preserve a user-requested format, tone, terminology, and necessary precision.", -].join("\n"); - -const REPL_CONTROL_PROMPT = [ - "The `ipython` tool is a persistent Python REPL — the agent's long-lived control environment for reasoning, context management, state, tool orchestration, and recursive subcalls. Top-level `await` works directly. Use it to keep intermediate variables, inspect and transform outputs, and write small helper functions. Compaction removes individual variables whose serialized form exceeds 16 MiB; keep large source data on disk and reload it when needed.", - "", - "Python is the orchestration language: use Python for loops, conditionals, parsing, and state. Use `bash()` to invoke programs, not to write shell programs — no shell loops or heredocs; do those in Python.", - "", - "Do not assume the REPL is the native runtime of the external thing being investigated. A repository, package, service, dataset, paper, website, benchmark, or API may have its own environment and normal interface. Evaluate external systems through their own interface, then use the REPL to coordinate the process and analyze what comes back.", - "", - "`bash(command)` starts a shell command in the background and returns a handle immediately: `h = bash('npm test')`. Use `h.pid` / `h.running` for liveness, `h.tail(n)` / `h.output()` for combined stdout+stderr so far, `h.poll()` for a non-blocking result, `h.kill()` to terminate (SIGTERM, escalating to SIGKILL; on Windows kill() uses taskkill /T and detached or reparented descendants may survive), and `await h` (or `await bash('cmd')`) for the completed result with exit_code, output, and duration. Prefer bash() for long-running commands so the turn keeps working. Run shell commands with `bash()`, not `subprocess`/`os.system`: subprocess calls block the kernel, show the user nothing while they run, and spawn processes the harness cannot see or stop.", - "", - "Important: do not install dependencies into the kernel just to make an external project import or run there. If a project import, test, script, CLI, or dependency check is needed, run it through that project's own environment and normal command interface. For example, in a Python repo use its documented commands, `uv run ...`, `.venv/bin/python ...`, or the active project interpreter from the repo root. Treat failures from that native environment as the relevant result.", - "", - "Use Python for reading, searching, and editing files — it gives you reusable variables you can slice, filter, and act on without re-reading. Always assign read/search results to named variables so you can revisit them later.", - "", - "Each `bash()` call is its own process, so shell state does not persist between calls; use `os.chdir(...)` for the working directory and `os.environ[...]` for environment variables — both persist in the REPL and apply to later `bash()` calls.", - "", - "Python state in the kernel persists across cells: named variables, helper functions, classes, imports, notes, parsed outputs, and helper data structures all remain available in every later turn. Tool calls are themselves Python `await` expressions, so their return values can be bound to variables and composed into program logic just like any other call.", - "", - "Continual harness state is available as `rlm.harness` and `rlm.get_harness_state()`. CRUD calls are local to this Prime Agent session by default: `rlm.harness.create_memory(...)`, `rlm.harness.update_memory(...)`, `rlm.harness.delete_memory(...)`, `rlm.harness.create_skill(...)`, `rlm.harness.update_skill(...)`, `rlm.harness.delete_skill(...)`, `rlm.harness.create_subagent(...)`, `rlm.harness.update_subagent(...)`, `rlm.harness.delete_subagent(...)`, `rlm.harness.create_prompt_note(...)`, `rlm.harness.update_prompt_note(...)`, `rlm.harness.delete_prompt_note(...)`, plus `rlm.harness.record_refinement(...)` and `rlm.harness.overview()`. Use `global_=True` only for stable cross-session lessons; Python reserves `global`, so literal `global=True` is invalid syntax.", - "", - "Terminology: continual harness names the persisted prompt, memory, skill, and subagent layer; RLM names the runtime, Python REPL kernel, and native call interface exposed to the model.", - "", - "RLM-native call contract: installed Python skills are pre-imported modules. Read the matching SKILL.md and call its documented function, such as `await .(...)`; when a CLI exists, use ` ...` from shell. Continual harness skill entries are Python REPL skills with an explicit Python `reference` and `arguments` contract. Spawn a reusable delegation spec with `await rlm.spawn('sub-task', name='worker')`; admission returns a child handle immediately. Results arrive only through an available messaging capability or files, never as an `rlm.spawn()` return value. Do not invent non-native wrappers such as `call_skill(...)` or `run_subagent(...)`.", -].join("\n"); - -export interface ChildAgentDoctrineOptions { - depth?: number; - parentAgent?: string; - installedSkills?: string[]; - activeTools?: string[]; -} - -export function buildChildAgentDoctrine(options: ChildAgentDoctrineOptions): string | undefined { - const depth = options.depth ?? 0; - const hasIpython = options.activeTools === undefined || options.activeTools.includes("ipython"); - const hasAgentMessage = options.installedSkills?.includes("agent_message") ?? false; - if (depth <= 0) return undefined; - - const lines = [ - `You are a child agent spawned by ${options.parentAgent ?? "your parent agent"}. Task prompts are labeled \`[task from parent]\`.`, - ]; - if (hasAgentMessage && hasIpython) { - lines.push( - 'When a task calls for an answer, reply explicitly with `await agent_message.send(message, receiver_role="parent")`. Not every message or task needs a reply; continue cleanup after sending and go idle normally.', - ); - } - return lines.join("\n"); -} - -export function buildRlmPrompt(options: RlmPromptOptions): string { - const { cwd, skillsDir, messagesPath } = options; - const installedSkills = options.installedSkills ?? []; - const hasAgentMessage = installedSkills.includes("agent_message"); - const hasAgentObserve = installedSkills.includes("agent_observe"); - const allowRecursion = options.allowRecursion ?? true; - const depth = options.depth ?? 0; - const activeTools = options.activeTools ?? []; - const hasIpython = options.activeTools === undefined ? true : activeTools.includes("ipython"); - const canRunShellSkills = hasIpython || activeTools.includes("bash"); - const parts = [ - "You are a general purpose agent that uses code to solve tasks.", - "You solve tasks by breaking down problems into sub-tasks, writing and executing code, observing results, and iterating one step at a time.", - "When you are done, stop calling tools and state your final answer.", - "", - LONG_RUNNING_WORK_PROMPT, - "", - ...(depth === 0 ? [USER_PROGRESS_PROMPT, ""] : []), - SIMPLIFIED_TECHNICAL_ENGLISH_PROMPT, - "", - `Working directory: ${cwd}`, - `Conversation log: ${messagesPath}`, - `Recursive agent depth: ${depth}`, - `Pre-installed Python packages: ${DEFAULT_RLM_EXTRA_IMPORT_LABELS.join(", ")}.`, - "Install additional packages with `uv pip install ` (this is a uv-managed venv with no pip module).", - ]; - - const childDoctrine = buildChildAgentDoctrine(options); - if (childDoctrine) { - parts.push("", childDoctrine); - } - - const skillLines: string[] = []; - if (skillsDir) { - skillLines.push(`Local skills live under ${skillsDir}. Read their SKILL.md files when helpful.`); - } - if (installedSkills.length > 0) { - const installed = installedSkills.map((skill) => `\`${skill}\``).join(", "); - if (hasIpython) { - skillLines.push(`Installed Python skill modules (pre-imported): ${installed}.`); - skillLines.push( - "Read each skill's SKILL.md for its API. Inspect a module with `help()` or `dir()`, then inspect a documented callable with `inspect.signature(.)`.", - ); - } else if (canRunShellSkills) { - skillLines.push(`Installed skills available as shell commands: ${installed}.`); - } - if (canRunShellSkills) { - skillLines.push( - "Each skill is also available as a shell command by the same name: ` ...`. Discover its CLI usage with ` --help`.", - ); - } - if (hasIpython && installedSkills.includes("edit")) { - skillLines.push( - "For targeted existing-file edits, prefer the pre-imported async `edit` skill from the REPL: `old = '''...'''; new = '''...'''; await edit(path=\"pkg/file.py\", old_str=old, new_str=new)`. Use exact old/new strings; if the text contains triple double quotes, use triple single-quoted variables or build `old`/`new` from inspected file slices.", - ); - } - } - if (skillLines.length > 0) { - parts.push("", ...skillLines); - } - if (hasAgentMessage) { - parts.push( - "Agent messaging is restricted to your parent, siblings, and direct children; roots are siblings, and deeper communication relays through the intermediate child.", - ); - } - if (hasAgentObserve) { - parts.push( - "Agent observation is restricted to your parent, siblings, and direct children; roots are siblings, and deeper inspection relays through the intermediate child.", - ); - } - - if (depth === 0 && hasIpython) { - parts.push( - "", - "From a daemon-backed depth-0 session, use `await rlm.create_session('task', name='researcher')` to start a separate top-level session. The call returns after the daemon creates the session and accepts its first prompt. Inline and nested sessions cannot use it. `rlm.spawn(...)` still creates a child.", - ); - } - - if (allowRecursion && hasIpython) { - parts.push( - "", - "An `rlm` object is already in your global namespace. `await rlm.spawn('sub-task', name='api-reviewer')` spawns a child and returns immediately after task admission with `rlm_child_id`, `name`, `session_dir`, and `model`; it never waits for or returns the child's answer.", - "`name` is required: choose a stable child name that is unique among siblings.", - "A child inherits your model. If a different model is explicitly requested, use `await rlm.find_models(...)` and an exact returned selector. An unavailable requested model fails spawn; decide whether to retry or omit `model`. Children also inherit your thinking level; the `thinking` option overrides it with any level the resolved child model supports, and an unsupported level fails spawn.", - ); - parts.push( - hasAgentObserve - ? "Use `await agent_observe.list_agents()` to discover family, including inactive members, and `await rlm.list_subagents()` to recover direct child handles." - : "Use `await rlm.list_subagents()` to recover direct child handles after admission.", - ); - if (hasAgentMessage) { - parts.push( - "Children reply explicitly with `await agent_message.send(message, receiver_role='parent')` when an answer is needed. Replies and follow-ups arrive as ordinary agent messages; not every task requires a reply.", - "Use `agent_message.send(..., receiver_role='child', receiver_name=child.name)` for follow-ups.", - ); - } - if (hasAgentObserve) { - parts.push( - "Use `agent_observe` to inspect a child's rollout. Observation is restricted to your parent, siblings, and direct children; relay through the intermediate child for deeper descendants.", - ); - } else { - parts.push("Inspect files a child wrote when you need to collect its work without an observation capability."); - } - parts.push( - "Spawn independent children in separate calls and end your turn instead of awaiting completion. Multiple replies may arrive over multiple turns. Delete a direct child explicitly with `await rlm.delete_subagent(child)` when it is no longer needed.", - ); - } - - if (hasIpython) { - parts.push("", REPL_CONTROL_PROMPT); - if (installedSkills.includes("refine")) { - parts.push( - "", - "Treat continual harness refinement as a small, evidence-backed update after observing a repeated failure or reusable tactic: diagnose the issue, update the smallest relevant continual harness component, validate on the next action, then record the outcome. Use `await refine.run()` to turn repeated delegation patterns into reusable subagent specs, repeated procedures into skills, durable facts/preferences into memories, and narrow behavioral policies into prompt addendums. It returns immediately and runs when the current turn ends, so continue working normally after calling it. Do not rewrite the whole continual harness when a focused memory, skill, prompt note, or subagent spec is enough.", - ); - } - } - - return parts.join("\n"); -} - -/** - * Supplemental sub-agent delegation guidance, appended after the base RLM - * prompt (see system-prompt.ts). The recursion block covers the mechanics - * (`rlm.spawn(...)` admission and handle management); this block adds the - * when and why in the same When -> Why -> menu order Claude Code's Agent tool - * uses. The subagent-spec menu itself renders just after this, inside the - * harness-state block. - */ -export function buildSubagentGuidance( - options: { includeRefineExamples?: boolean; hasAgentMessage?: boolean; hasAgentObserve?: boolean } = {}, -): string { - const lines = [ - "# Delegating to sub-agents", - "", - "Spawn independent, self-contained work with `handle = await rlm.spawn('task', name='worker')`. This returns at admission, not completion; keep the handle to stop or inspect the child later.", - ]; - if (options.hasAgentMessage) { - lines.push( - "Ask for an explicit reply when needed. A child replies with `await agent_message.send(message, receiver_role='parent')`; parent follow-ups use `receiver_role='child'` plus the child's name or id. Not every message needs a reply.", - ); - } - lines.push("Use `await rlm.list_subagents()` after kernel restart or compaction."); - if (options.hasAgentObserve) { - lines.push("Use `agent_observe` for bounded transcript inspection."); - } - lines.push( - "Have children write files and read those files for fan-in.", - "Delegate parallel context-heavy research or independent implementation; do a single known lookup, edit, or command inline.", - ); - if (options.includeRefineExamples ?? true) { - lines.push("Persist genuinely reusable delegation patterns with `await refine.run()`."); - } - return lines.join("\n"); -} +// Compatibility exports; implementation lives with its session owner. +export { + buildChildAgentDoctrine, + buildRlmPrompt, + buildSubagentGuidance, + type ChildAgentDoctrineOptions, + type RlmPromptOptions, +} from "../../session/context/prompts/rlm.js"; diff --git a/packages/coding-agent/src/core/refinement/index.ts b/packages/coding-agent/src/core/refinement/index.ts index 975275f0c1..8cf8bab924 100644 --- a/packages/coding-agent/src/core/refinement/index.ts +++ b/packages/coding-agent/src/core/refinement/index.ts @@ -1 +1,40 @@ -export * from "./refinement.js"; +// Compatibility exports; implementation lives with its session owner. + +export { formatHarnessStateForPrompt, formatRefinementNoticeBody } from "../../session/refinement/format.js"; +export { + appendGlobalRefinement, + applyRefinementProposal, + generateRefinementId, + getGlobalHarnessStateDir, + getHarnessStatePath, + getLocalHarnessStateDir, + getRefinementHistory, + getRefinementHistoryPath, + inferRefinementResultScope, + loadGlobalRefinementHistory, + loadHarnessState, + mergeHarnessStates, + mergeRefinementHistory, + normalizeRefinementProposal, + saveHarnessState, +} from "../../session/refinement/harness-state.js"; +export { planRefinement, refineHarness, reviewAutoRefine } from "../../session/refinement/planning.js"; +export { + type AppliedRefinementEdit, + type AutoRefineReason, + type AutoRefineReview, + type AutoRefineReviewContext, + type HarnessEntry, + type HarnessRefinementEvent, + type HarnessScope, + type HarnessState, + REFINE_SKILL_NAME, + REFINEMENT_CUSTOM_TYPE, + type RefinementAction, + type RefinementEdit, + type RefinementKind, + type RefinementPlan, + type RefinementProposal, + type RefinementResult, + type RefineOptions, +} from "../../session/refinement/types.js"; diff --git a/packages/coding-agent/src/core/refinement/refinement.ts b/packages/coding-agent/src/core/refinement/refinement.ts index 5b1cb44205..8cf8bab924 100644 --- a/packages/coding-agent/src/core/refinement/refinement.ts +++ b/packages/coding-agent/src/core/refinement/refinement.ts @@ -1,1122 +1,40 @@ -import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync } from "node:fs"; -import { join } from "node:path"; -import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { Api, Model } from "@earendil-works/pi-ai"; -import { completeSimple } from "@earendil-works/pi-ai"; -import { getAgentDir } from "../../config.js"; -import { realpathIfPresentSync, writeFileAtomicSync } from "../../utils/atomic-file.js"; -import { serializeConversation } from "../compaction/utils.js"; -import { convertToLlm } from "../messages.js"; -import { completeWithProviderRetry, type ProviderRetryPolicy } from "../provider-retry.js"; -import type { CustomEntry } from "../session-manager.js"; -import { getAuxiliaryThinkingLevel } from "../thinking-levels.js"; - -export const REFINEMENT_CUSTOM_TYPE = "prime-agent.refinement"; - -export const REFINE_SKILL_NAME = "refine"; -const HARNESS_STATE_DIR_NAME = "harness"; -const REFINEMENT_HISTORY_FILE_NAME = "refinements.jsonl"; -const DEFAULT_OVERVIEW_ENTRY_LIMIT = 6; -const DEFAULT_OVERVIEW_REFINEMENT_LIMIT = 5; -const DEFAULT_OVERVIEW_CONTENT_LIMIT = 180; - -export type RefinementKind = "prompt" | "memory" | "skill" | "subagent"; -export type RefinementAction = "create" | "update" | "delete"; -export type HarnessScope = "local" | "global"; - -export interface HarnessEntry { - id: string; - kind: RefinementKind; - title: string; - content: string; - path: string; - scope?: HarnessScope; - reference: Record; - arguments: Record; - metadata: Record; - source: string; - created_at: string; - updated_at: string; - version: number; -} - -export interface HarnessRefinementEvent { - id: string; - trigger: string; - changes: string[]; - evidence: string; - outcome: string; - created_at: string; -} - -export interface HarnessState { - schema: number; - entries: Record>; - refinements: HarnessRefinementEvent[]; -} - -export interface RefinementEdit { - action: RefinementAction; - kind: RefinementKind; - id?: string; - title?: string; - content?: string; - path?: string; - reference?: Record; - arguments?: Record; - metadata?: Record; - reason?: string; -} - -export interface RefinementProposal { - summary: string; - rationale: string; - edits: RefinementEdit[]; - expectedOutcome: string; -} - -export interface AppliedRefinementEdit extends RefinementEdit { - id: string; - before?: HarnessEntry; - after?: HarnessEntry; - applied: boolean; - error?: string; -} - -export interface RefinementResult { - id: string; - summary: string; - rationale: string; - expectedOutcome: string; - appliedEdits: AppliedRefinementEdit[]; - harnessStatePath: string; - rollbackOf?: string; - scope?: HarnessScope; -} - -export interface RefineOptions { - instructions?: string; - rollbackId?: string; - global?: boolean; - retry?: ProviderRetryPolicy; -} - -export type AutoRefineReason = "turn_interval" | "compact"; - -export interface AutoRefineReviewContext { - reason: AutoRefineReason; - turnsSinceLastReview: number; -} - -export interface AutoRefineReview { - shouldRefine: boolean; - rationale: string; - instructions?: string; -} - -const REFINEMENT_SYSTEM_PROMPT = `You are Prime Agent's /refine continual harness subsystem. - -Your job is to improve the editable continual harness state from the current trajectory. -This is similar in spirit to context compaction, but instead of summarizing the -conversation you emit precise Create, Update, or Delete edits to reusable state. -The continual harness is the persistent, editable set of prompt notes, memories, -skills, and subagent specs that lets Prime Agent improve reusable behavior -outside the token history. -Use "continual harness" for that persistent artifact layer; keep "RLM" for the -runtime, Python REPL kernel, and native call interface that executes those artifacts. - -Continual harness components: -- prompt: supplemental prompt notes only. The base system prompt is immutable and MUST NOT be rewritten. -- memory: durable facts, decisions, failures, preferences, and outcomes. -- skill: installed Python REPL skill. Skill create/update edits MUST include a \`reference\` object with \`{"type":"python"}\`, a Python import, and a callable or call pattern; they also MUST include an \`arguments\` object describing accepted inputs, required fields, defaults, and constraints. Use \`{}\` for \`arguments\` only when the Python callable truly needs no external inputs. Include the RLM-native call form \`await (...)\`. -- subagent: reusable delegation specs, including purpose, instructions, and when to invoke. Include the RLM-native call form: compose a concise task prompt and spawn with \`handle = await rlm.spawn("sub-task", name="worker")\`; admission returns immediately with \`rlm_child_id\`, \`name\`, \`session_dir\`, and \`model\`, never the child's answer. Results arrive only through explicit \`agent_message\` replies or files; children reply with \`await agent_message.send(message, receiver_role="parent")\`. Use \`await rlm.list_subagents()\` to recover direct child handles and \`await agent_message.send(..., receiver_role="child", receiver_name=handle.name)\` for follow-ups. Do not invent wrappers like \`run_subagent(...)\`. - -Scope and persistence policy: -- The default editable continual harness store is local to the current Prime Agent session. Use it for session-specific progress, active task state, current-run coordination notes, temporary blockers, and project facts that should not affect other sessions. -- A caller may explicitly request global refinement. Global edits must be stable cross-session lessons, durable user preferences, reusable skills/subagents, or tool/environment facts that should affect future sessions. -- Entry ids in the harness overview may carry a display-only \`local:\` or \`global:\` prefix. Always use the bare id (no prefix) in edits. -- All edits in one refinement apply only to the requested scope's store. During a local refinement, global entries are read-only context: never propose update or delete edits for them; create a local entry instead when a session-specific override is genuinely needed. -- Project/workspace-specific lessons may be persisted globally only when the title, path, or content explicitly names the project/workspace and the lesson is likely to be reused in future sessions for that project. Prefer local edits when the lesson only belongs in the current conversation. -- Use memory for declarative facts and preferences, skill for repeatable procedures exposed as Python calls, prompt for narrow behavioral policy addendums, and subagent for reusable delegation roles. -- Create or update the smallest relevant component: repeated delegation roles should become subagent specs, repeated procedures should become skills, durable facts/preferences should become memories, and narrow behavioral policies should become prompt addendums. -- When an edit is persisted, include metadata such as \`{"scope":"local"}\` or \`{"scope":"global"}\` when that helps future review understand the intended blast radius. - -Use the trajectory, current continual harness state, and prior refinement history. Prefer -small evidence-backed edits. If prior refinements caused issues, rollback or -replace the faulty editable entries. Never edit source files directly. Output -JSON only with this exact shape: - -{ - "summary": "one sentence", - "rationale": "why these edits are justified by trajectory evidence", - "expectedOutcome": "what should improve and how to validate it", - "edits": [ - { - "action": "create|update|delete", - "kind": "prompt|memory|skill|subagent", - "id": "stable id for update/delete, optional for create", - "title": "required for create/update except delete", - "content": "required for create/update except delete", - "path": "optional grouping path", - "reference": {"type": "python", "import": "package.module", "callable": "function_name", "call_pattern": "await function_name(...)"}, - "arguments": {"name": {"type": "string", "required": true, "description": "accepted input"}}, - "metadata": {}, - "reason": "why this edit is useful" - } - ] -}`; - -const AUTO_REFINE_REVIEW_SYSTEM_PROMPT = `You are Prime Agent's automatic /refine review gate. - -Decide whether this checkpoint should run /refine. Auto /refine writes local continual harness state by default, so approve when the trajectory contains evidence useful to this session's future turns. -Reject one-off noise, unsupported hypotheses, and transient tool outputs. Ask for global refinement only for durable cross-session lessons or explicitly project-qualified lessons likely to be reused in future sessions. - -Return JSON only: -{ - "shouldRefine": true|false, - "rationale": "short reason", - "instructions": "optional concise instructions for /refine if shouldRefine is true" -}`; - -// These caps apply only with reasoning off; thinking and JSON otherwise share the model's output budget. -const REFINEMENT_MAX_OUTPUT_TOKENS = 32_000; -const AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS = 4_096; -const REFINEMENT_CONTEXT_OVERHEAD_TOKENS = 1_024; - -const TRUNCATED_JSON_ERROR = - "the model stopped before completing its JSON object. This usually means the output budget was exhausted; retry with a smaller request."; - -function refinementInputTokenBound(text: string): number { - // One token per UTF-8 byte bounds byte-based tokenizers, including dense or unusual text. - return Buffer.byteLength(text, "utf8"); -} - -function refinementRequest( - model: Model, - systemPrompt: string, - conversationText: string, - buildPrompt: (conversation: string) => string, - outputReserve: number, -): { model: Model; userPrompt: string } { - const systemReserve = refinementInputTokenBound(systemPrompt) + REFINEMENT_CONTEXT_OVERHEAD_TOKENS; - const inputBudget = - model.contextWindow - Math.min(model.maxTokens, outputReserve, Math.floor(model.contextWindow / 2)); - let userPrompt = buildPrompt(conversationText); - if (systemReserve + refinementInputTokenBound(userPrompt) > inputBudget && conversationText.length > 0) { - const promptForLength = (length: number): string => { - let start = conversationText.length - length; - const first = conversationText.charCodeAt(start); - if (first >= 0xdc00 && first <= 0xdfff) start++; - return buildPrompt( - `[Earlier conversation omitted to fit the model context.]\n${conversationText.slice(start)}`, - ); - }; - let low = 0; - let high = conversationText.length; - while (low < high) { - const length = Math.ceil((low + high) / 2); - if (systemReserve + refinementInputTokenBound(promptForLength(length)) <= inputBudget) low = length; - else high = length - 1; - } - userPrompt = promptForLength(low); - } - const maxTokens = Math.min( - model.maxTokens, - model.contextWindow - systemReserve - refinementInputTokenBound(userPrompt), - ); - if (maxTokens <= 0) { - throw new Error( - "Refinement prompt leaves no room for output in the model's context window; retry with a smaller request.", - ); - } - // Bound the request's model ceiling too: some adapters add thinking tokens before clamping to it. - return { model: { ...model, maxTokens }, userPrompt }; -} - -function now(): string { - return new Date().toISOString(); -} - -function emptyHarnessState(): HarnessState { - return { - schema: 1, - entries: { - prompt: {}, - memory: {}, - skill: {}, - subagent: {}, - }, - refinements: [], - }; -} - -function slug(raw: string, fallback: string): string { - const normalized = raw - .trim() - .toLowerCase() - .replace(/[^a-z0-9]+/g, "_") - .replace(/^_+|_+$/g, "") - .slice(0, 80); - return normalized || fallback; -} - -function cloneEntry(entry: HarnessEntry | undefined): HarnessEntry | undefined { - return entry ? JSON.parse(JSON.stringify(entry)) : undefined; -} - -function objectRecord(value: unknown): Record | undefined { - if (typeof value !== "object" || value === null || Array.isArray(value)) { - return undefined; - } - return value as Record; -} - -function normalizeHarnessScope(value: unknown, fallback: HarnessScope): HarnessScope { - return value === "global" || value === "local" ? value : fallback; -} - -export function inferRefinementResultScope(result: RefinementResult): HarnessScope | undefined { - if (result.scope) { - return result.scope; - } - - const scopes = new Set(); - for (const edit of result.appliedEdits) { - const scope = edit.after?.scope ?? edit.before?.scope; - if (scope) { - scopes.add(scope); - } - } - return scopes.size === 1 ? [...scopes][0] : undefined; -} - -function withDefaultRefinementScope(result: RefinementResult, scope: HarnessScope): RefinementResult { - const inferred = inferRefinementResultScope(result); - return { ...result, scope: inferred ?? scope }; -} - -export function getGlobalHarnessStateDir(agentDir: string = getAgentDir()): string { - return join(agentDir, HARNESS_STATE_DIR_NAME); -} - -export function getLocalHarnessStateDir(sessionArtifactDir: string | undefined): string | undefined { - return sessionArtifactDir ? join(sessionArtifactDir, HARNESS_STATE_DIR_NAME) : undefined; -} - -export function getHarnessStatePath(harnessStateDir: string = getGlobalHarnessStateDir()): string { - return join(harnessStateDir, "harness_state.json"); -} - -export function loadHarnessState( - harnessStateDir: string = getGlobalHarnessStateDir(), - scope: HarnessScope = "global", -): HarnessState { - const statePath = getHarnessStatePath(harnessStateDir); - if (!existsSync(statePath)) { - return emptyHarnessState(); - } - let parsed: Partial; - try { - const raw = JSON.parse(readFileSync(statePath, "utf8")); - // loadHarnessState runs on every system-prompt build and before each /refine, so - // a corrupt or unreadable (or non-object) state file must degrade to empty rather - // than throw and break the session. The next saveHarnessState rewrites it cleanly. - if (typeof raw !== "object" || raw === null || Array.isArray(raw)) { - return emptyHarnessState(); - } - parsed = raw as Partial; - } catch { - return emptyHarnessState(); - } - const state = emptyHarnessState(); - state.schema = typeof parsed.schema === "number" ? parsed.schema : 1; - for (const kind of Object.keys(state.entries) as RefinementKind[]) { - const records = parsed.entries?.[kind]; - if (records && typeof records === "object") { - for (const [id, rawEntry] of Object.entries(records)) { - const entry = objectRecord(rawEntry); - if (!entry) continue; - state.entries[kind][id] = { - ...(entry as unknown as HarnessEntry), - scope: normalizeHarnessScope(entry.scope, scope), - reference: objectRecord(entry.reference) ?? {}, - arguments: objectRecord(entry.arguments) ?? {}, - metadata: objectRecord(entry.metadata) ?? {}, - }; - } - } - } - if (Array.isArray(parsed.refinements)) { - state.refinements = parsed.refinements; - } - return state; -} - -export function mergeHarnessStates(globalState: HarnessState, localState?: HarnessState): HarnessState { - const merged = emptyHarnessState(); - merged.schema = Math.max(globalState.schema, localState?.schema ?? 1); - for (const kind of Object.keys(merged.entries) as RefinementKind[]) { - for (const [id, entry] of Object.entries(globalState.entries[kind])) { - const cloned = cloneEntry(entry)!; - merged.entries[kind][id] = { ...cloned, scope: normalizeHarnessScope(cloned.scope, "global") }; - } - for (const [id, entry] of Object.entries(localState?.entries[kind] ?? {})) { - const cloned = cloneEntry(entry)!; - const scopedEntry = { ...cloned, scope: normalizeHarnessScope(cloned.scope, "local") }; - const mergedId = merged.entries[kind][id] ? `${scopedEntry.scope}:${id}` : id; - merged.entries[kind][mergedId] = scopedEntry; - } - } - merged.refinements = [...globalState.refinements, ...(localState?.refinements ?? [])]; - return merged; -} - -export function saveHarnessState(harnessStateDir: string, state: HarnessState): string { - const statePath = getHarnessStatePath(harnessStateDir); - mkdirSync(harnessStateDir, { recursive: true }); - const targetPath = realpathIfPresentSync(statePath); - const mode = existsSync(targetPath) ? statSync(targetPath).mode & 0o777 : 0o600; - writeFileAtomicSync(targetPath, `${JSON.stringify(state, null, 2)}\n`, { mode }); - return statePath; -} - -export function getRefinementHistoryPath(harnessStateDir: string = getGlobalHarnessStateDir()): string { - return join(harnessStateDir, REFINEMENT_HISTORY_FILE_NAME); -} - -function isRefinementResult(data: unknown): data is RefinementResult { - return typeof data === "object" && data !== null && "id" in data && "appliedEdits" in data; -} - -/** - * Append a global-scope refinement to the cross-session history log so it can be - * rolled back from any session. Local-scope refinements are recorded only in the - * session JSONL and roll back via their recorded harnessStatePath. - */ -export function appendGlobalRefinement(harnessStateDir: string, result: RefinementResult): string { - const historyPath = getRefinementHistoryPath(harnessStateDir); - mkdirSync(harnessStateDir, { recursive: true }); - appendFileSync(historyPath, `${JSON.stringify(result)}\n`, "utf8"); - return historyPath; -} - -export function loadGlobalRefinementHistory(harnessStateDir: string = getGlobalHarnessStateDir()): RefinementResult[] { - const historyPath = getRefinementHistoryPath(harnessStateDir); - if (!existsSync(historyPath)) { - return []; - } - const results: RefinementResult[] = []; - for (const line of readFileSync(historyPath, "utf8").split("\n")) { - const trimmed = line.trim(); - if (!trimmed) continue; - try { - const parsed = JSON.parse(trimmed); - if (isRefinementResult(parsed)) { - results.push(withDefaultRefinementScope(parsed, "global")); - } - } catch { - // Skip malformed lines so a single bad append cannot break rollback. - } - } - return results; -} - -/** - * Merge global and session refinement history, de-duplicating by id. Session entries - * win on conflict so a session that is mid-flight still resolves its own latest result. - */ -export function mergeRefinementHistory( - global: readonly RefinementResult[], - session: readonly RefinementResult[], -): RefinementResult[] { - const byId = new Map(); - for (const result of global) { - byId.set(result.id, result); - } - for (const result of session) { - const existing = byId.get(result.id); - byId.set(result.id, result.scope || !existing?.scope ? result : { ...result, scope: existing.scope }); - } - return [...byId.values()]; -} - -function compactText(text: string, maxLength: number): string { - const normalized = text.replace(/\s+/g, " ").trim(); - if (normalized.length <= maxLength) { - return normalized; - } - return `${normalized.slice(0, Math.max(0, maxLength - 3))}...`; -} - -/** Notice body in digest notation: trigger line plus applied edits as `action kind [scope:id] title: content`; rollbacks print via their rollback summaries. */ -export function formatRefinementNoticeBody(result: RefinementResult): string { - const lines = [compactText(result.summary, DEFAULT_OVERVIEW_CONTENT_LIMIT)]; - for (const edit of result.appliedEdits) { - if (!edit.applied) continue; - const entry = edit.after ?? edit.before; - const scope = entry?.scope ?? result.scope ?? "local"; - lines.push( - `- ${edit.action} ${edit.kind} [${scope}:${edit.id}] ${entry?.title ?? edit.id}: ${compactText( - entry?.content ?? "", - DEFAULT_OVERVIEW_CONTENT_LIMIT, - )}`, - ); - } - return lines.join("\n"); -} - -export function formatHarnessStateForPrompt( - state: HarnessState, - options: { - maxEntriesPerKind?: number; - maxRefinements?: number; - maxContentLength?: number; - includeIpythonExamples?: boolean; - includeShellExamples?: boolean; - includeRefineExamples?: boolean; - } = {}, -): string { - const maxEntriesPerKind = options.maxEntriesPerKind ?? DEFAULT_OVERVIEW_ENTRY_LIMIT; - const maxRefinements = options.maxRefinements ?? DEFAULT_OVERVIEW_REFINEMENT_LIMIT; - const maxContentLength = options.maxContentLength ?? DEFAULT_OVERVIEW_CONTENT_LIMIT; - const includeIpythonExamples = options.includeIpythonExamples ?? true; - const includeRefineExamples = options.includeRefineExamples ?? includeIpythonExamples; - const lines = [ - "# Continual Harness State", - "", - "Local continual harness entries belong to this Prime Agent session. Global continual harness entries persist across Prime Agent sessions.", - "The continual harness entries below are compact summaries, not full descriptions. Use them as routing/context hints; inspect or refine the underlying continual harness entry only when detail matters.", - "Default to local continual harness refinement for current task progress, temporary blockers, and session coordination. Use global continual harness refinement only for stable cross-session lessons, durable user preferences, reusable skills/subagents, or explicitly project-qualified facts.", - "Use these continual harness prompt notes, memories, skills, and subagent specs when they are relevant. The base system prompt is immutable; prompt entries below are supplemental notes only.", - "", - includeRefineExamples - ? "When to call `await refine.run()`: after a repeated failure, a reusable tactic emerges, a repeated delegation role should become a subagent spec, a repeated procedure should become a skill, a durable fact/preference should become a memory, a narrow behavioral policy should become a prompt addendum, a user corrects behavior that should persist locally or globally, validation shows a continual harness entry is wrong, or a skill/subagent/memory/prompt note should be created, updated, deleted, or rolled back. Keep `await refine.run()` continual harness edits small and evidence-backed." - : "When to refine the continual harness: after a repeated failure, a reusable tactic emerges, a repeated delegation role should become a subagent spec, a repeated procedure should become a skill, a durable fact/preference should become a memory, a narrow behavioral policy should become a prompt addendum, a user corrects behavior that should persist locally or globally, validation shows a continual harness entry is wrong, or a skill/subagent/memory/prompt note should be created, updated, deleted, or rolled back. Keep continual harness edits small and evidence-backed.", - "", - includeIpythonExamples - ? "Call contract: read each installed Python skill's SKILL.md and call its documented module function in the Python REPL; do not assume a `.run` entrypoint. Use ` ...` in shell when a CLI exists. Continual harness skill entries are Python REPL skills with an explicit Python `reference` and `arguments` contract. Spawn a continual harness subagent spec by composing a concise task prompt and calling `handle = await rlm.spawn('sub-task', name='worker')`; admission returns immediately with `rlm_child_id`, `name`, `session_dir`, and `model`, never the child's answer. Results arrive only through explicit `agent_message` replies or files; children reply with `await agent_message.send(message, receiver_role='parent')`. Use `await rlm.list_subagents()` to recover direct child handles and `await agent_message.send(..., receiver_role='child', receiver_name=handle.name)` for follow-ups. Do not invent wrappers such as `call_skill(...)`, `run_subagent(...)`, or named subagent registries." - : options.includeShellExamples - ? "Call contract: use installed skills as shell commands when available (for example ` ...`). Continual harness entries are routing/context hints only in sessions without the Python REPL; do not use Python `await`, `asyncio`, or `rlm` examples unless the prompt also documents a Python kernel." - : "Call contract: continual harness entries are routing/context hints only in sessions without the Python REPL or shell access; do not use Python `await`, `asyncio`, `rlm`, or shell skill commands unless the prompt also documents those interfaces.", - "", - ]; - - let totalEntries = 0; - for (const kind of Object.keys(state.entries) as RefinementKind[]) { - const entries = Object.values(state.entries[kind]).sort((a, b) => - [a.path, a.title, a.id].join("\0").localeCompare([b.path, b.title, b.id].join("\0")), - ); - totalEntries += entries.length; - // Render subagent specs as a task-shaped roster the model can match against — the - // analogue of Claude Code's agent-type menu — rather than a bare count. In - // REPL sessions, include the native `rlm` invocation hint. - if (kind === "subagent" && entries.length > 0 && includeIpythonExamples) { - lines.push( - `${kind}: ${entries.length} (invoke a spec by turning it into a concise task prompt and spawning with \`await rlm.spawn('', name='')\`; admission returns a child handle, never the answer)`, - ); - } else { - lines.push(`${kind}: ${entries.length}`); - } - for (const entry of entries.slice(0, maxEntriesPerKind)) { - const argumentsText = - entry.kind === "skill" && Object.keys(entry.arguments).length > 0 - ? ` args=${compactText(JSON.stringify(entry.arguments), maxContentLength)}` - : ""; - const referenceText = - entry.kind === "skill" && Object.keys(entry.reference).length > 0 - ? ` ref=${compactText(JSON.stringify(entry.reference), maxContentLength)}` - : ""; - lines.push( - `- [${entry.scope ?? "global"}:${entry.id}] ${entry.title} (${entry.path}, v${entry.version})${referenceText}${argumentsText}: ${compactText( - entry.content, - maxContentLength, - )}`, - ); - } - const overflow = entries.length - Math.min(entries.length, maxEntriesPerKind); - if (overflow > 0) { - lines.push(`- +${overflow} more ${kind} entries`); - } - lines.push(""); - } - - if (totalEntries === 0) { - lines.push("No saved harness entries yet.", ""); - } - - lines.push(`recent refinements: ${state.refinements.length}`); - for (const event of state.refinements.slice(-maxRefinements)) { - const changes = event.changes.length > 0 ? event.changes.join(", ") : "no applied edits"; - const outcome = event.outcome ? `; outcome: ${compactText(event.outcome, maxContentLength)}` : ""; - lines.push(`- [${event.id}] ${compactText(event.trigger, maxContentLength)}: ${changes}${outcome}`); - } - const refinementOverflow = state.refinements.length - Math.min(state.refinements.length, maxRefinements); - if (refinementOverflow > 0) { - lines.push(`- +${refinementOverflow} older refinement events`); - } - - return lines.join("\n").trim(); -} - -function overviewForPrompt(state: HarnessState): string { - const lines: string[] = []; - for (const kind of Object.keys(state.entries) as RefinementKind[]) { - const entries = Object.values(state.entries[kind]); - lines.push(`${kind}: ${entries.length}`); - for (const entry of entries.slice(0, 40)) { - const content = entry.content.replace(/\s+/g, " ").slice(0, 240); - const argumentsText = - entry.kind === "skill" && Object.keys(entry.arguments).length > 0 - ? ` args=${JSON.stringify(entry.arguments).slice(0, 240)}` - : ""; - const referenceText = - entry.kind === "skill" && Object.keys(entry.reference).length > 0 - ? ` ref=${JSON.stringify(entry.reference).slice(0, 240)}` - : ""; - lines.push( - `- [${entry.scope ?? "global"}:${entry.id}] ${entry.title} (${entry.path}, v${entry.version})${referenceText}${argumentsText}: ${content}`, - ); - } - if (entries.length > 40) { - lines.push(`- +${entries.length - 40} more ${kind} entries`); - } - } - return lines.join("\n"); -} - -function historyForPrompt(history: RefinementResult[]): string { - if (history.length === 0) { - return "No prior refinement history."; - } - return history - .slice(-20) - .map((item) => { - const edits = item.appliedEdits - .map((edit) => `${edit.applied ? "applied" : "failed"} ${edit.action} ${edit.kind}:${edit.id}`) - .join(", "); - const rollback = item.rollbackOf ? ` rollbackOf=${item.rollbackOf}` : ""; - return `[${item.id}]${rollback} ${item.summary}\n${edits}\nExpected outcome: ${item.expectedOutcome}`; - }) - .join("\n\n"); -} - -/** - * Whether a JSON candidate ends mid-value: an unterminated string, or unclosed - * objects/arrays. A reply cut off by an exhausted output budget is incomplete in - * this sense, while a complete-but-malformed reply is balanced. Brace slicing can - * also produce a balanced fragment, so callers treat "balanced" as malformed. - */ -function isIncompleteJson(candidate: string): boolean { - let depth = 0; - let inString = false; - let escaped = false; - for (const char of candidate) { - if (escaped) { - escaped = false; - continue; - } - if (inString) { - if (char === "\\") escaped = true; - else if (char === '"') inString = false; - continue; - } - if (char === '"') inString = true; - else if (char === "{" || char === "[") depth++; - else if (char === "}" || char === "]") depth--; - } - return inString || depth > 0; -} - -function parseJsonCandidate(candidate: string): unknown { - try { - return JSON.parse(candidate); - } catch (error) { - // A truncated reply and a malformed one both fail here, and JSON.parse - // describes the fragment rather than the cause. Name the cause instead. - if (isIncompleteJson(candidate)) { - throw new Error(TRUNCATED_JSON_ERROR); - } - throw new Error(`the model did not return valid JSON: ${error instanceof Error ? error.message : String(error)}`); - } -} - -function extractJsonObject(text: string): unknown { - const trimmed = text.trim(); - if (trimmed.startsWith("{") && trimmed.endsWith("}")) { - // A reply truncated after a nested closing brace still looks well-formed - // here, so this path needs the same diagnosis as the slicing fallback. - return parseJsonCandidate(trimmed); - } - const fenced = trimmed.match(/```(?:json)?\s*([\s\S]*?)```/); - if (fenced) { - return parseJsonCandidate(fenced[1].trim()); - } - // Brace slicing recovers JSON wrapped in prose. On a reply truncated inside the - // edits array it slices to an earlier edit's closing brace, so a failure here - // is diagnosed against the original text rather than the balanced fragment. - const start = trimmed.indexOf("{"); - const end = trimmed.lastIndexOf("}"); - if (start !== -1 && end > start) { - try { - return JSON.parse(trimmed.slice(start, end + 1)); - } catch { - return parseJsonCandidate(trimmed.slice(start)); - } - } - if (isIncompleteJson(trimmed)) { - throw new Error(TRUNCATED_JSON_ERROR); - } - throw new Error("Refiner did not return a JSON object"); -} - -/** - * Normalizes an untrusted refinement proposal while preserving invalid edit - * fields for apply-time validation. - */ -export function normalizeRefinementProposal(value: unknown): RefinementProposal { - const record = - typeof value === "object" && value !== null && !Array.isArray(value) ? (value as Record) : {}; - const edits = Array.isArray(record.edits) ? record.edits : []; - return { - summary: typeof record.summary === "string" ? record.summary : "Refined continual harness state", - rationale: typeof record.rationale === "string" ? record.rationale : "", - expectedOutcome: typeof record.expectedOutcome === "string" ? record.expectedOutcome : "", - edits: edits - .filter((edit): edit is Record => typeof edit === "object" && edit !== null) - .map((edit) => ({ - action: edit.action as RefinementAction, - kind: edit.kind as RefinementKind, - id: typeof edit.id === "string" ? edit.id : undefined, - title: typeof edit.title === "string" ? edit.title : undefined, - content: typeof edit.content === "string" ? edit.content : undefined, - path: typeof edit.path === "string" ? edit.path : undefined, - reference: objectRecord(edit.reference), - arguments: objectRecord(edit.arguments), - metadata: - typeof edit.metadata === "object" && edit.metadata !== null && !Array.isArray(edit.metadata) - ? (edit.metadata as Record) - : undefined, - reason: typeof edit.reason === "string" ? edit.reason : undefined, - })), - }; -} - -function parseProposal(text: string): RefinementProposal { - const value = extractJsonObject(text); - if (typeof value !== "object" || value === null || Array.isArray(value)) { - throw new Error("Refiner JSON must be an object"); - } - return normalizeRefinementProposal(value); -} - -function validateEdit(edit: RefinementEdit, computedId?: string): string | undefined { - if (!["create", "update", "delete"].includes(edit.action)) { - return `unsupported action ${String(edit.action)}`; - } - if (!["prompt", "memory", "skill", "subagent"].includes(edit.kind)) { - return `unsupported kind ${String(edit.kind)}`; - } - if (edit.kind === "prompt" && (edit.id === "base_system_prompt" || computedId === "base_system_prompt")) { - return "base system prompt is not editable"; - } - if (edit.action !== "create" && !edit.id) { - return `${edit.action} requires id`; - } - if (edit.action !== "delete" && (!edit.title || !edit.content)) { - return `${edit.action} requires title and content`; - } - if (edit.action !== "delete" && edit.kind === "skill" && edit.arguments === undefined) { - return `${edit.action} skill requires arguments`; - } - if (edit.action !== "delete" && edit.kind === "skill") { - const reference = edit.reference; - if (!reference) { - return `${edit.action} skill requires python reference`; - } - if (reference.type !== "python") { - return `${edit.action} skill reference.type must be python`; - } - const hasImport = - (typeof reference.import === "string" && reference.import.length > 0) || - (typeof reference.python_import === "string" && reference.python_import.length > 0); - const hasCallable = - (typeof reference.callable === "string" && reference.callable.length > 0) || - (typeof reference.call_pattern === "string" && reference.call_pattern.length > 0); - if (!hasImport) { - return `${edit.action} skill requires python import`; - } - if (!hasCallable) { - return `${edit.action} skill requires callable or call_pattern`; - } - } - return undefined; -} - -export function applyRefinementProposal( - state: HarnessState, - proposal: RefinementProposal, - options: { id: string; rollbackOf?: string; scope?: HarnessScope; baselineState?: HarnessState }, -): RefinementResult { - const appliedEdits: AppliedRefinementEdit[] = []; - const proposalModifiedKeys = new Set(); - for (const edit of proposal.edits) { - const computedId = edit.id ?? (edit.action === "create" ? slug(edit.title ?? edit.kind, edit.kind) : undefined); - const id = computedId ?? ""; - const validationError = validateEdit(edit, id); - if (validationError) { - appliedEdits.push({ ...edit, id, applied: false, error: validationError }); - continue; - } - - const records = state.entries[edit.kind]; - const before = cloneEntry(records[id]); - const entryKey = `${edit.kind}:${id}`; - const baseline = cloneEntry(options.baselineState?.entries[edit.kind][id]); - if ( - options.baselineState && - !proposalModifiedKeys.has(entryKey) && - JSON.stringify(before) !== JSON.stringify(baseline) - ) { - appliedEdits.push({ - ...edit, - id, - before, - applied: false, - error: "entry changed during refinement planning", - }); - continue; - } - if (edit.action === "delete") { - if (!before) { - appliedEdits.push({ ...edit, id, applied: false, error: "entry not found" }); - continue; - } - delete records[id]; - proposalModifiedKeys.add(entryKey); - appliedEdits.push({ ...edit, id, before, applied: true }); - continue; - } - if (edit.action === "create" && before) { - appliedEdits.push({ ...edit, id, before, applied: false, error: "entry already exists" }); - continue; - } - if (edit.action === "update" && !before) { - appliedEdits.push({ ...edit, id, applied: false, error: "entry not found" }); - continue; - } - - const createdAt = before?.created_at ?? now(); - const version = before ? before.version + 1 : 1; - const after: HarnessEntry = { - id, - kind: edit.kind, - title: edit.title ?? before?.title ?? id, - content: edit.content ?? before?.content ?? "", - path: edit.path ?? before?.path ?? "general", - scope: before?.scope ?? options.scope ?? "local", - reference: edit.reference ?? before?.reference ?? {}, - arguments: edit.arguments ?? before?.arguments ?? {}, - metadata: edit.metadata ?? before?.metadata ?? {}, - source: "refine", - created_at: createdAt, - updated_at: now(), - version, - }; - records[id] = after; - proposalModifiedKeys.add(entryKey); - appliedEdits.push({ ...edit, id, before, after: cloneEntry(after), applied: true }); - } - - const changes = appliedEdits.filter((edit) => edit.applied).map((edit) => `${edit.action} ${edit.kind}:${edit.id}`); - state.refinements.push({ - id: options.id, - trigger: proposal.summary, - changes, - evidence: proposal.rationale, - outcome: proposal.expectedOutcome, - created_at: now(), - }); - - return { - id: options.id, - summary: proposal.summary, - rationale: proposal.rationale, - expectedOutcome: proposal.expectedOutcome, - appliedEdits, - harnessStatePath: "", - rollbackOf: options.rollbackOf, - scope: options.scope, - }; -} - -function rollbackProposal(target: RefinementResult): RefinementProposal { - const edits: RefinementEdit[] = []; - for (const edit of [...target.appliedEdits].reverse()) { - if (!edit.applied) continue; - if (edit.before) { - edits.push({ - action: edit.after ? "update" : "create", - kind: edit.kind, - id: edit.id, - title: edit.before.title, - content: edit.before.content, - path: edit.before.path, - reference: edit.before.reference, - arguments: edit.before.arguments, - metadata: edit.before.metadata, - reason: `Rollback ${target.id}`, - }); - } else if (edit.after) { - edits.push({ - action: "delete", - kind: edit.kind, - id: edit.id, - reason: `Rollback ${target.id}`, - }); - } - } - return { - summary: `Rollback refinement ${target.id}`, - rationale: `Restores continual harness state snapshots from refinement ${target.id}.`, - expectedOutcome: "Faulty refinement edits are reverted.", - edits, - }; -} - -export function getRefinementHistory(entries: readonly CustomEntry[]): RefinementResult[] { - return entries - .filter((entry) => entry.customType === REFINEMENT_CUSTOM_TYPE) - .map((entry) => entry.data) - .filter((data): data is RefinementResult => { - return typeof data === "object" && data !== null && "id" in data && "appliedEdits" in data; - }); -} - -export interface RefinementPlan { - proposal: RefinementProposal; - id: string; - rollbackOf?: string; - rollbackScope?: HarnessScope; - /** Target-scope state captured before planning, used to reject conflicting edits at apply time. */ - baselineState?: HarnessState; -} - -/** - * Produce a refinement proposal (the LLM pass, or a rollback proposal) without - * mutating any harness state. Separated from {@link applyRefinementProposal} so - * callers can re-read the harness file immediately before applying — the LLM call - * here can take many seconds, during which the kernel or another session may write - * the shared `harness_state.json`. - */ -/** Mint a refinement id in the canonical `refine_` format. */ -export function generateRefinementId(): string { - return `refine_${new Date() - .toISOString() - .replace(/[^0-9]/g, "") - .slice(0, 17)}`; -} - -export async function planRefinement( - messages: AgentMessage[], - state: HarnessState, - history: RefinementResult[], - model: Model, - apiKey: string, - options: RefineOptions = {}, - headers?: Record, - signal?: AbortSignal, - thinkingLevel?: ThinkingLevel, - sessionId?: string, -): Promise { - const id = generateRefinementId(); - if (options.rollbackId) { - const target = history.find((item) => item.id === options.rollbackId); - if (!target) { - throw new Error(`Refinement ${options.rollbackId} not found`); - } - const fallbackScope: HarnessScope = options.global ? "global" : "local"; - return { - proposal: rollbackProposal(target), - id, - rollbackOf: target.id, - rollbackScope: inferRefinementResultScope(target) ?? fallbackScope, - }; - } - - const conversationText = serializeConversation(convertToLlm(messages)).slice(-80_000); - const scopeInstruction = options.global - ? "Requested refinement scope: global. Only propose stable cross-session continual harness edits, durable user preferences, reusable skills/subagents, or explicitly project-qualified facts that should affect future Prime Agent sessions. Do not persist session-only progress, temporary blockers, or current-run coordination globally." - : "Requested refinement scope: local. Prefer local continual harness edits for current task progress, temporary blockers, current-run coordination, and project facts that are not clearly reusable across Prime Agent sessions. Global entries in the overview are read-only context: do not propose update or delete edits for them; create a local entry instead if an override is needed."; - const buildPrompt = (conversation: string): string => - [ - `\n${overviewForPrompt(state)}\n`, - `\n${historyForPrompt(history)}\n`, - `\n${conversation}\n`, - `\n${scopeInstruction}\n`, - options.instructions ? `\n${options.instructions}\n` : "", - "Return only JSON edits. If no useful edit is justified, return an empty edits array with a rationale.", - ] - .filter(Boolean) - .join("\n\n"); - const reasoning = getAuxiliaryThinkingLevel(model, thinkingLevel); - const { model: requestModel, userPrompt } = refinementRequest( - model, - REFINEMENT_SYSTEM_PROMPT, - conversationText, - buildPrompt, - reasoning === "off" ? REFINEMENT_MAX_OUTPUT_TOKENS : model.maxTokens, - ); - const maxTokens = - reasoning === "off" ? Math.min(requestModel.maxTokens, REFINEMENT_MAX_OUTPUT_TOKENS) : requestModel.maxTokens; - - const response = await completeWithProviderRetry( - () => - completeSimple( - requestModel, - { - systemPrompt: REFINEMENT_SYSTEM_PROMPT, - messages: [{ role: "user", content: [{ type: "text", text: userPrompt }], timestamp: Date.now() }], - }, - { - reasoning, - maxTokens, - signal, - apiKey, - headers, - sessionId, - }, - ), - { policy: options.retry, signal }, - ); - - if (response.stopReason === "error") { - throw new Error(`Refinement failed: ${response.errorMessage || "Unknown error"}`); - } - if (response.stopReason === "length") { - throw new Error(`Refinement failed: ${TRUNCATED_JSON_ERROR}`); - } - - const text = response.content - .filter((content): content is { type: "text"; text: string } => content.type === "text") - .map((content) => content.text) - .join("\n"); - return { proposal: parseProposal(text), id }; -} - -function parseAutoRefineReview(text: string): AutoRefineReview { - const value = extractJsonObject(text); - if (typeof value !== "object" || value === null || Array.isArray(value)) { - throw new Error("Auto-refine review JSON must be an object"); - } - const record = value as Record; - return { - shouldRefine: record.shouldRefine === true, - rationale: typeof record.rationale === "string" ? record.rationale : "No rationale provided.", - instructions: typeof record.instructions === "string" ? record.instructions : undefined, - }; -} - -export async function reviewAutoRefine( - messages: AgentMessage[], - state: HarnessState, - history: RefinementResult[], - model: Model, - apiKey: string, - context: AutoRefineReviewContext, - headers?: Record, - signal?: AbortSignal, - thinkingLevel?: ThinkingLevel, - retry?: ProviderRetryPolicy, - sessionId?: string, -): Promise { - const conversationText = serializeConversation(convertToLlm(messages)).slice(-40_000); - const buildPrompt = (conversation: string): string => - [ - ` -${context.reason}; ${context.turnsSinceLastReview} assistant turns since last auto-refine review -`, - ` -${overviewForPrompt(state)} -`, - ` -${historyForPrompt(history)} -`, - ` -${conversation} -`, - "Return shouldRefine=true when the trajectory contains evidence useful to this session's future turns. Prefer local harness edits for current task progress, temporary blockers, and current-run coordination. Ask for global refinement only for durable cross-session lessons or explicitly project-qualified facts likely to be reused in future sessions.", - ].join("\n\n"); - const reasoning = getAuxiliaryThinkingLevel(model, thinkingLevel); - const { model: requestModel, userPrompt } = refinementRequest( - model, - AUTO_REFINE_REVIEW_SYSTEM_PROMPT, - conversationText, - buildPrompt, - reasoning === "off" ? AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS : model.maxTokens, - ); - const maxTokens = - reasoning === "off" - ? Math.min(requestModel.maxTokens, AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS) - : requestModel.maxTokens; - const response = await completeWithProviderRetry( - () => - completeSimple( - requestModel, - { - systemPrompt: AUTO_REFINE_REVIEW_SYSTEM_PROMPT, - messages: [{ role: "user", content: [{ type: "text", text: userPrompt }], timestamp: Date.now() }], - }, - { - reasoning, - maxTokens, - signal, - apiKey, - headers, - sessionId, - }, - ), - { policy: retry, signal }, - ); - if (response.stopReason === "error") { - throw new Error(`Auto-refine review failed: ${response.errorMessage || "Unknown error"}`); - } - if (response.stopReason === "length") { - throw new Error(`Auto-refine review failed: ${TRUNCATED_JSON_ERROR}`); - } - const text = response.content - .filter((content): content is { type: "text"; text: string } => content.type === "text") - .map((content) => content.text) - .join("\n"); - return parseAutoRefineReview(text); -} - -export async function refineHarness( - messages: AgentMessage[], - state: HarnessState, - history: RefinementResult[], - model: Model, - apiKey: string, - options: RefineOptions = {}, - headers?: Record, - signal?: AbortSignal, - thinkingLevel?: ThinkingLevel, - sessionId?: string, -): Promise { - const plan = await planRefinement( - messages, - state, - history, - model, - apiKey, - options, - headers, - signal, - thinkingLevel, - sessionId, - ); - return applyRefinementProposal(state, plan.proposal, { - id: plan.id, - rollbackOf: plan.rollbackOf, - scope: plan.rollbackScope ?? (options.global ? "global" : "local"), - }); -} +// Compatibility exports; implementation lives with its session owner. + +export { formatHarnessStateForPrompt, formatRefinementNoticeBody } from "../../session/refinement/format.js"; +export { + appendGlobalRefinement, + applyRefinementProposal, + generateRefinementId, + getGlobalHarnessStateDir, + getHarnessStatePath, + getLocalHarnessStateDir, + getRefinementHistory, + getRefinementHistoryPath, + inferRefinementResultScope, + loadGlobalRefinementHistory, + loadHarnessState, + mergeHarnessStates, + mergeRefinementHistory, + normalizeRefinementProposal, + saveHarnessState, +} from "../../session/refinement/harness-state.js"; +export { planRefinement, refineHarness, reviewAutoRefine } from "../../session/refinement/planning.js"; +export { + type AppliedRefinementEdit, + type AutoRefineReason, + type AutoRefineReview, + type AutoRefineReviewContext, + type HarnessEntry, + type HarnessRefinementEvent, + type HarnessScope, + type HarnessState, + REFINE_SKILL_NAME, + REFINEMENT_CUSTOM_TYPE, + type RefinementAction, + type RefinementEdit, + type RefinementKind, + type RefinementPlan, + type RefinementProposal, + type RefinementResult, + type RefineOptions, +} from "../../session/refinement/types.js"; diff --git a/packages/coding-agent/src/core/rlm-max-depth.ts b/packages/coding-agent/src/core/rlm-max-depth.ts index e51e6f65b9..583fab941e 100644 --- a/packages/coding-agent/src/core/rlm-max-depth.ts +++ b/packages/coding-agent/src/core/rlm-max-depth.ts @@ -1,13 +1 @@ -/** Wire-safe types for the immediate /rlm-max-depth state APIs. */ - -export type RlmMaxDepthSource = "default" | "env" | "global" | "inherited" | "chat"; - -export interface RlmMaxDepthStatus { - maxDepth: number; - source: RlmMaxDepthSource; -} - -export interface SetRlmMaxDepthResult extends RlmMaxDepthStatus { - globalSaved: boolean; - globalError?: string; -} +export type { RlmMaxDepthSource, RlmMaxDepthStatus, SetRlmMaxDepthResult } from "../session/children/max-depth.js"; diff --git a/packages/coding-agent/src/core/rlm-runtime.ts b/packages/coding-agent/src/core/rlm-runtime.ts index fdef11f9c2..d96375d8fb 100644 --- a/packages/coding-agent/src/core/rlm-runtime.ts +++ b/packages/coding-agent/src/core/rlm-runtime.ts @@ -1,342 +1,41 @@ -import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { Api, Model, ServiceTier } from "@earendil-works/pi-ai"; -import type { AgentSession } from "./agent-session.js"; -import type { ToolDefinition } from "./extensions/index.js"; -import type { HostRequestHandler } from "./kernel/index.js"; -import { THINKING_LEVELS } from "./thinking-levels.js"; - -/** Request emitted by `rlm.spawn`; cellSourceCode preserves the spawning cell for display. */ -export interface RlmRunRequest { - prompt: string; - kwargs: Record; - cellSourceCode?: string; -} - -interface RlmCreateSessionRequest { - prompt: string; - kwargs: Record; -} - -export interface RlmCreateSessionResult { - active_session_id: string; - session_id: string; - name: string; - session_file: string; - model: string; -} - -export interface RlmSpawnHandle { - rlm_child_id: string; - name: string; - session_dir: string; - model: string; -} - -export type RlmSubagentRegistryStatus = "running" | "completed" | "error"; - -export interface RlmSubagentRegistryEntry { - rlm_child_id: string; - active_session_id: string | null; - session_id: string | null; - session_name: string; - session_dir: string; - status: RlmSubagentRegistryStatus; -} - -export interface RlmListSubagentsResult { - subagents: RlmSubagentRegistryEntry[]; -} - -export interface RlmDeleteSubagentResult { - subagent: RlmSubagentRegistryEntry; - outcome?: "deleted" | "skipped_running"; -} - -export interface RlmModelMatch { - provider: string; - id: string; - name: string; - selector: string; -} - -export interface RlmFindModelsResult { - models: RlmModelMatch[]; -} - -export type RlmRunHandler = (request: RlmRunRequest) => Promise>; -type RlmCreateSessionHandler = (request: RlmCreateSessionRequest) => Promise; - -interface AsyncBashCompletionRequest { - pid: number; - command: string; - exitCode: number; -} - -type AsyncBashCompletionHandler = (request: AsyncBashCompletionRequest) => void | Promise; - -interface AsyncBashConsumedRequest { - pid: number; - command: string; -} - -type AsyncBashConsumedHandler = (request: AsyncBashConsumedRequest) => void | Promise; -export type RlmListSubagentsHandler = () => RlmListSubagentsResult | Promise; -export type RlmDeleteSubagentHandler = (target: string) => Promise; -export type RlmFindModelsHandler = (query: string, limit: number) => RlmFindModelsResult | Promise; - -const RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH = 64; -export const DEFAULT_RLM_MODEL_SEARCH_LIMIT = 8; -export const MAX_RLM_MODEL_SEARCH_LIMIT = 20; - -export function normalizeRequestedRlmSubagentSessionName(value: unknown, operation = "rlm.spawn"): string | undefined { - if (value === undefined) { - return undefined; - } - if (typeof value !== "string") { - throw new Error(`${operation} name must be a string`); - } - const name = value.trim(); - if (!name) { - throw new Error(`${operation} name must not be empty`); - } - if (name.length > RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH) { - throw new Error(`${operation} name must be at most ${RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH} characters`); - } - return name; -} - -export function normalizeRequestedRlmSubagentThinkingLevel( - value: unknown, - operation = "rlm.spawn", -): ThinkingLevel | undefined { - if (value === undefined) { - return undefined; - } - if (typeof value !== "string") { - throw new Error(`${operation} thinking must be a string`); - } - const level = value.trim().toLowerCase(); - if (!THINKING_LEVELS.includes(level as ThinkingLevel)) { - throw new Error(`${operation} thinking must be one of: ${THINKING_LEVELS.join(", ")}`); - } - return level as ThinkingLevel; -} - -export function normalizeRequestedRlmSubagentModel(value: unknown, operation = "rlm.spawn"): string | undefined { - if (value === undefined) { - return undefined; - } - if (typeof value !== "string") { - throw new Error(`${operation} model must be a string`); - } - const model = value.trim(); - if (!model) { - throw new Error(`${operation} model must not be empty`); - } - return model; -} - -/** Create a readable, collision-resistant default name usable as an agent-message selector. */ -export function createDefaultRlmSubagentSessionName(prompt: string, childId: string): string { - const promptSlug = prompt - .normalize("NFKD") - .replace(/[\u0300-\u036f]/g, "") - .toLowerCase() - .replace(/[^a-z0-9]+/g, "-") - .replace(/^-+|-+$/g, ""); - const idSuffix = - childId - .replace(/^sub-/, "") - .replace(/[^A-Za-z0-9]+/g, "") - .slice(-8) || "child"; - const fixedLength = "subagent--".length + idSuffix.length; - const promptPart = (promptSlug || "worker") - .slice(0, Math.max(1, RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH - fixedLength)) - .replace(/-+$/g, ""); - return `subagent-${promptPart || "worker"}-${idSuffix}`; -} - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null && !Array.isArray(value); -} - -function normalizeModelSearchText(value: string): string { - return value.toLowerCase().replace(/[^a-z0-9]+/g, ""); -} - -export function findRlmModelMatches(query: string, models: Model[], limit: number): RlmModelMatch[] { - const normalizedQuery = normalizeModelSearchText(query.trim()); - return models - .map((model) => { - const selector = `${model.provider}/${model.id}`; - const fields = [selector, model.id, model.name || model.id]; - const normalizedFields = fields.map(normalizeModelSearchText); - let score = normalizedQuery ? Number.POSITIVE_INFINITY : 0; - if (normalizedQuery) { - const exactIndex = normalizedFields.indexOf(normalizedQuery); - const prefixIndex = normalizedFields.findIndex((field) => field.startsWith(normalizedQuery)); - const partialIndex = normalizedFields.findIndex((field) => field.includes(normalizedQuery)); - if (exactIndex >= 0) score = exactIndex; - else if (prefixIndex >= 0) score = 3 + prefixIndex; - else if (partialIndex >= 0) score = 6 + partialIndex; - } - return { model, selector, score }; - }) - .filter((candidate) => Number.isFinite(candidate.score)) - .sort((a, b) => a.score - b.score || a.selector.localeCompare(b.selector)) - .slice(0, limit) - .map(({ model, selector }) => ({ - provider: model.provider, - id: model.id, - name: model.name || model.id, - selector, - })); -} - -export function createRlmCreateSessionHostHandler(handler: RlmCreateSessionHandler): HostRequestHandler { - return async (payload) => { - if (typeof payload.prompt !== "string") { - throw new Error("rlm.create_session prompt must be a string"); - } - const kwargs = isRecord(payload.kwargs) ? payload.kwargs : {}; - const result = await handler({ prompt: payload.prompt, kwargs }); - return result as unknown as Record; - }; -} - -/** Adapt an RlmRunHandler into the typed `rlm.run` kernel host handler. */ -export function createRlmRunHostHandler(handler: RlmRunHandler): HostRequestHandler { - return async (payload) => { - if (typeof payload.prompt !== "string") { - throw new Error("rlm.spawn prompt must be a string"); - } - const kwargs = isRecord(payload.kwargs) ? payload.kwargs : {}; - const cellSourceCode = typeof payload.cellSourceCode === "string" ? payload.cellSourceCode : undefined; - const result = await handler({ - prompt: payload.prompt, - kwargs, - cellSourceCode, - }); - return result as unknown as Record; - }; -} - -/** Adapt detached kernel bash completions into a validated host notification. */ -export function createAsyncBashCompletionHostHandler(handler: AsyncBashCompletionHandler): HostRequestHandler { - return async (payload) => { - const { pid, command, exitCode } = payload; - if (typeof pid !== "number" || !Number.isInteger(pid) || pid <= 0) { - throw new Error("bash.completed pid must be a positive integer"); - } - if (typeof command !== "string" || !command) { - throw new Error("bash.completed command must be a non-empty string"); - } - if (typeof exitCode !== "number" || !Number.isInteger(exitCode)) { - throw new Error("bash.completed exitCode must be an integer"); - } - await handler({ pid, command, exitCode }); - return {}; - }; -} - -/** The kernel read a finished command's result, so its completion notice is stale. */ -export function createAsyncBashConsumedHostHandler(handler: AsyncBashConsumedHandler): HostRequestHandler { - return async (payload) => { - const { pid, command } = payload; - if (typeof pid !== "number" || !Number.isInteger(pid) || pid <= 0) { - throw new Error("bash.consumed pid must be a positive integer"); - } - if (typeof command !== "string" || !command) { - throw new Error("bash.consumed command must be a non-empty string"); - } - await handler({ pid, command }); - return {}; - }; -} - -/** Search a bounded authenticated model catalog without adding it to the system prompt. */ -export function createRlmFindModelsHostHandler(handler: RlmFindModelsHandler): HostRequestHandler { - return async (payload) => { - if (typeof payload.query !== "string") { - throw new Error("rlm.find_models query must be a string"); - } - const limit = payload.limit === undefined ? DEFAULT_RLM_MODEL_SEARCH_LIMIT : payload.limit; - if (!Number.isInteger(limit) || (limit as number) < 1 || (limit as number) > MAX_RLM_MODEL_SEARCH_LIMIT) { - throw new Error(`rlm.find_models limit must be an integer from 1 to ${MAX_RLM_MODEL_SEARCH_LIMIT}`); - } - return { models: (await handler(payload.query, limit as number)).models }; - }; -} - -/** Expose the current parent session's direct RLM child registry to its kernel. */ -export function createRlmListSubagentsHostHandler(handler: RlmListSubagentsHandler): HostRequestHandler { - return async () => { - const { subagents } = await handler(); - return { subagents }; - }; -} - -/** Delete one direct child selected from the current parent session's registry. */ -export function createRlmDeleteSubagentHostHandler(handler: RlmDeleteSubagentHandler): HostRequestHandler { - return async (payload) => { - if (typeof payload.target !== "string" || !payload.target.trim()) { - throw new Error("rlm.delete_subagent target must be a non-empty string"); - } - const { subagent, outcome } = await handler(payload.target.trim()); - return outcome === undefined ? { subagent } : { subagent, outcome }; - }; -} - -export interface RlmSubagentRuntime { - session: AgentSession; -} - -export interface CreateRlmSubagentRuntimeOptions { - parentSession: AgentSession; - id: string; - prompt: string; - sessionName: string; - sessionDir: string; - model: Model; - thinkingLevel: ThinkingLevel; - serviceTier: ServiceTier; - scopedModels: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>; - activeToolNames: string[]; - allowedToolNames?: string[]; - customTools: ToolDefinition[]; - includeGoals: boolean; - includeCompactSkill: boolean; - rlmDepth: number; - rlmMaxDepth: number; - rlmParentNodeId: string; - /** Request ID of the parent model call whose tool call caused this spawn. */ - spawnedByRequestId?: string; - /** Source of the Python cell that spawned this subagent, for display. */ - spawnCode?: string; - /** Publish the session to the parent before a host makes the runtime addressable. */ - onSessionPublished?: (session: AgentSession) => void; -} - -export interface CreateRlmRootSessionOptions { - prompt: string; - sessionName?: string; - cwd: string; - model: Model; - thinkingLevel: ThinkingLevel; -} - -export interface SubagentRuntimeHost { - createRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): Promise; - createRlmRootSession?(options: CreateRlmRootSessionOptions): Promise; - /** Persist host-owned completion before the child becomes passivation-eligible. */ - completeRlmSubagentRuntime?(childId: string, session: AgentSession): boolean; - /** Release a host-owned child after its detached initial task settles. */ - releaseRlmSubagentRuntime?: ( - runtime: RlmSubagentRuntime, - options: CreateRlmSubagentRuntimeOptions, - status: "done" | "error" | "cancelled", - ) => Promise; - /** Close or remove the host-owned child; session is absent when a persisted child is still passive. */ - deleteRlmSubagentRuntime(childId: string, session?: AgentSession): Promise; - disposeRlmSubagentRuntimes?(): Promise; -} +export { + createRlmCreateSessionHostHandler, + createRlmDeleteSubagentHostHandler, + createRlmListSubagentsHostHandler, + createRlmRunHostHandler, +} from "../session/children/host-requests.js"; +export type { + CreateRlmRootSessionOptions, + CreateRlmSubagentRuntimeOptions, + RlmCreateSessionResult, + RlmDeleteSubagentHandler, + RlmDeleteSubagentResult, + RlmListSubagentsHandler, + RlmListSubagentsResult, + RlmRunHandler, + RlmRunRequest, + RlmSpawnHandle, + RlmSubagentRegistryEntry, + RlmSubagentRegistryStatus, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "../session/children/runtime-contracts.js"; +export { + createDefaultRlmSubagentSessionName, + normalizeRequestedRlmSubagentModel, + normalizeRequestedRlmSubagentSessionName, + normalizeRequestedRlmSubagentThinkingLevel, +} from "../session/children/spawn-options.js"; +export { + createAsyncBashCompletionHostHandler, + createAsyncBashConsumedHostHandler, +} from "../session/input/bash-host-requests.js"; +export { + createRlmFindModelsHostHandler, + DEFAULT_RLM_MODEL_SEARCH_LIMIT, + findRlmModelMatches, + MAX_RLM_MODEL_SEARCH_LIMIT, + type RlmFindModelsHandler, + type RlmFindModelsResult, + type RlmModelMatch, +} from "../session/models/model-search.js"; diff --git a/packages/coding-agent/src/core/sdk.ts b/packages/coding-agent/src/core/sdk.ts index 601e001433..be1294e876 100644 --- a/packages/coding-agent/src/core/sdk.ts +++ b/packages/coding-agent/src/core/sdk.ts @@ -2,15 +2,15 @@ import { join } from "node:path"; import { Agent, type AgentMessage, type ThinkingLevel } from "@earendil-works/pi-agent-core"; import { clampThinkingLevel, type Message, type Model, streamSimple, supportsFastMode } from "@earendil-works/pi-ai"; import { getAgentDir } from "../config.js"; -import { AgentSession } from "./agent-session.js"; +import { AgentSession } from "../session/agent-session.js"; +import type { AgentAutonomousConfig } from "../session/autonomy/autonomous.js"; +import { convertToLlm } from "../session/context/messages.js"; import type { AgentSessionCreationOptions } from "./agent-session-services.js"; import { formatNoModelsAvailableMessage } from "./auth-guidance.js"; import { AuthStorage } from "./auth-storage.js"; -import type { AgentAutonomousConfig } from "./autonomous.js"; import { DEFAULT_THINKING_LEVEL } from "./defaults.js"; import type { ExtensionRunner, LoadExtensionsResult, SessionStartEvent, ToolDefinition } from "./extensions/index.js"; import { McpManager } from "./mcp/mcp-manager.js"; -import { convertToLlm } from "./messages.js"; import { ModelRegistry } from "./model-registry.js"; import { findInitialModel } from "./model-resolver.js"; import type { ResourceLoader } from "./resource-loader.js"; @@ -84,6 +84,11 @@ export interface CreateAgentSessionResult { modelFallbackMessage?: string; } +export type { + CreateRlmSubagentRuntimeOptions, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "../session/children/runtime-contracts.js"; export type { AgentSessionRuntimeConfig } from "./agent-session-config.js"; export * from "./agent-session-runtime.js"; export type { AgentSessionCreationOptions } from "./agent-session-services.js"; @@ -97,7 +102,6 @@ export type { ToolDefinition, } from "./extensions/index.js"; export type { PromptTemplate } from "./prompt-templates.js"; -export type { CreateRlmSubagentRuntimeOptions, RlmSubagentRuntime, SubagentRuntimeHost } from "./rlm-runtime.js"; export type { Skill } from "./skills.js"; export type { Tool } from "./tools/index.js"; diff --git a/packages/coding-agent/src/core/session-action-store.ts b/packages/coding-agent/src/core/session-action-store.ts index 937eea09a1..749f59f8de 100644 --- a/packages/coding-agent/src/core/session-action-store.ts +++ b/packages/coding-agent/src/core/session-action-store.ts @@ -1,435 +1,33 @@ -import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { ImageContent, UserMessage } from "@earendil-works/pi-ai"; -import type { InputSource } from "./extensions/index.js"; -import type { CustomMessage } from "./messages.js"; -import type { SessionSlashCommand } from "./slash-commands.js"; - -export type DeliveryPolicy = "next_turn_boundary" | "when_run_idle"; -export type WakePolicy = "immediate" | "on_lower_boundary" | "external_resume"; - -export type QueuedMessageLane = "steering" | "followUp"; - -export function queuedMessageLaneDeliveryPolicy(lane: QueuedMessageLane): DeliveryPolicy { - return lane === "steering" ? "next_turn_boundary" : "when_run_idle"; -} - -export type QueuedMessageMutation = - | { type: "delete" } - | { type: "move"; direction: -1 | 1 } - | { type: "replace"; text: string; images?: ImageContent[]; lane: QueuedMessageLane }; -export type QueuedMessageMutationStatus = "applied" | "rejected" | "invalid"; - -export interface SessionActionSnapshot { - queuedCount: number; - steering: readonly string[]; - followUps: readonly string[]; - active?: { - kind: "turn" | "session_command"; - phase: "preparing" | "committing" | "running"; - label?: string; - }; -} - -export interface DeliveryRecord { - id: string; - role: "primary" | "prefix" | "next_turn"; - message: UserMessage | CustomMessage; - started: boolean; - durable: boolean; - ownerActionId: string; -} - -export interface SessionTurnPayload { - kind: "turn"; - records: DeliveryRecord[]; - text: string; - preview?: string; -} - -export interface SessionCommandPayload { - kind: "session_command"; - command: SessionSlashCommand; - text: string; -} - -export type SessionActionPayload = SessionTurnPayload | SessionCommandPayload; - -export type ActionLifecycle = - | { state: "queued" } - | { state: "selected" } - | { state: "preparing"; preparation?: object } - | { state: "committing" } - | { state: "running"; execution: "agent_turn" | "session_command" } - | { state: "completed" } - | { state: "failed"; error: Error } - | { state: "cancelled" }; - -export interface SessionAction { - id: string; - source: InputSource | "internal"; - delivery: DeliveryPolicy; - wake: WakePolicy; - payload: TPayload; - lifecycle: ActionLifecycle; - queueKey?: string; - agentMessageId?: string; - suppressAutonomousContinuation?: boolean; -} - -export interface RollbackProof { - dispatchSettled: true; - transcript: readonly AgentMessage[]; -} - -const TERMINAL_STATES = new Set(["completed", "failed", "cancelled"]); -const ACTIVE_STATES = new Set(["selected", "preparing", "committing", "running"]); -const CLEARABLE_STATES = new Set(["queued", "selected", "preparing"]); - -function isClearable(action: SessionAction): boolean { - return CLEARABLE_STATES.has(action.lifecycle.state); -} - -const LEGAL_TRANSITIONS: Readonly>> = { - queued: new Set(["selected", "failed", "cancelled"]), - selected: new Set(["queued", "preparing", "running", "failed", "cancelled"]), - preparing: new Set(["queued", "committing", "failed", "cancelled"]), - committing: new Set(["queued", "running", "failed", "cancelled"]), - running: new Set(["completed", "failed", "cancelled"]), - completed: new Set(), - failed: new Set(), - cancelled: new Set(), -}; - -function primaryRecords(action: SessionAction): readonly DeliveryRecord[] { - return action.payload.kind === "turn" ? action.payload.records.filter((record) => record.role === "primary") : []; -} - -export function transitionSessionAction( - action: SessionAction, - next: ActionLifecycle, - options: { rollbackProof?: RollbackProof } = {}, -): void { - const previous = action.lifecycle.state; - if (!LEGAL_TRANSITIONS[previous].has(next.state)) { - throw new Error(`Illegal session action lifecycle transition: ${previous} -> ${next.state}`); - } - if (previous === "committing" && next.state === "queued") { - const proof = options.rollbackProof; - if (!proof?.dispatchSettled) { - throw new Error("Committing session action rollback requires a settled dispatch and transcript proof"); - } - const transcript = new Set(proof.transcript); - if (primaryRecords(action).some((record) => transcript.has(record.message))) { - throw new Error("Cannot roll back a session action whose primary message is durable in the transcript"); - } - } - action.lifecycle = next; -} - -export type AdmissionDisposition = "starts_when_admitted" | "queued"; -export type SubmissionOutcome = - | { status: "accepted"; actionId: string; disposition: AdmissionDisposition } - | { status: "coalesced"; existingActionId: string } - | { status: "handled_without_turn" } - | { status: "extension_command"; completion: Promise }; -export type DeliveryOutcome = { status: "delivered" } | { status: "not_applicable" }; - -export interface ActionTicket { - id: string; - accepted: Promise; - delivered: Promise; - completed: Promise; -} - -interface Deferred { - promise: Promise; - settle(value: T): boolean; - reject(error: Error): boolean; -} - -function createDeferred(): Deferred { - let settled = false; - let resolvePromise!: (value: T) => void; - let rejectPromise!: (error: Error) => void; - const promise = new Promise((resolve, reject) => { - resolvePromise = resolve; - rejectPromise = reject; - }); - void promise.catch(() => undefined); - return { - promise, - settle: (value) => { - if (settled) return false; - settled = true; - resolvePromise(value); - return true; - }, - reject: (error) => { - if (settled) return false; - settled = true; - rejectPromise(error); - return true; - }, - }; -} - -export class ActionTicketController { - readonly ticket: ActionTicket; - private readonly accepted = createDeferred(); - private readonly delivered = createDeferred(); - private readonly completed = createDeferred(); - - constructor(id: string) { - this.ticket = { - id, - accepted: this.accepted.promise, - delivered: this.delivered.promise, - completed: this.completed.promise, - }; - } - - settleAccepted(outcome: SubmissionOutcome): boolean { - return this.accepted.settle(outcome); - } - - settleDelivered(outcome: DeliveryOutcome): boolean { - return this.delivered.settle(outcome); - } - - rejectDelivered(error: Error): boolean { - return this.delivered.reject(error); - } - - settleCompleted(error?: Error): boolean { - return error ? this.completed.reject(error) : this.completed.settle(); - } -} - -export class ActionStore { - private readonly nextTurnBoundary: TAction[] = []; - private readonly whenRunIdle: TAction[] = []; - private readonly tickets = new Map(); - - enqueue(action: TAction): void { - this.assertNewAction(action); - this.list(action.delivery).push(action); - this.tickets.set(action.id, new ActionTicketController(action.id)); - } - - enqueueFront(action: TAction): void { - this.assertNewAction(action); - const list = this.list(action.delivery); - const firstQueued = list.findIndex((item) => item.lifecycle.state === "queued"); - list.splice(firstQueued < 0 ? list.length : firstQueued, 0, action); - this.tickets.set(action.id, new ActionTicketController(action.id)); - } - - selectFirst(): TAction | undefined { - const action = - this.nextTurnBoundary.find((item) => item.lifecycle.state === "queued") ?? - this.whenRunIdle.find((item) => item.lifecycle.state === "queued"); - if (action) transitionSessionAction(action, { state: "selected" }); - return action; - } - - remove(predicate: (action: TAction) => boolean, candidates = this.clearableActions()): TAction[] { - const removed = candidates.filter(predicate); - for (const action of removed) transitionSessionAction(action, { state: "cancelled" }); - return removed; - } - - rollback(action: TAction, proof?: RollbackProof): void { - transitionSessionAction(action, { state: "queued" }, { rollbackProof: proof }); - } - - swapQueued(left: TAction, right: TAction): void { - if (left.lifecycle.state !== "queued" || right.lifecycle.state !== "queued" || left.delivery !== right.delivery) { - throw new Error("Only queued actions in the same lane can be swapped"); - } - const list = this.list(left.delivery); - const leftIndex = list.indexOf(left); - const rightIndex = list.indexOf(right); - if (leftIndex < 0 || rightIndex < 0) throw new Error("Queued action is not owned by this store"); - [list[leftIndex], list[rightIndex]] = [right, left]; - } - - moveQueued(action: TAction, delivery: DeliveryPolicy, index: number): void { - if (action.lifecycle.state !== "queued") throw new Error("Only queued actions can be moved"); - const source = this.list(action.delivery); - const sourceIndex = source.indexOf(action); - if (sourceIndex < 0) throw new Error(`Session action ${action.id} is not owned by this store`); - source.splice(sourceIndex, 1); - action.delivery = delivery; - const target = this.list(delivery); - const queued = target.filter((item) => item.lifecycle.state === "queued"); - const before = queued[Math.max(0, Math.min(index, queued.length))]; - target.splice(before ? target.indexOf(before) : target.length, 0, action); - } - - queuedActions(policy?: DeliveryPolicy): readonly TAction[] { - return this.actions(policy).filter((action) => action.lifecycle.state === "queued"); - } - - clearableActions(policy?: DeliveryPolicy): readonly TAction[] { - return this.actions(policy).filter(isClearable); - } - - snapshotActions(): readonly TAction[] { - return this.queuedActions(); - } - - unfinishedActions(policy?: DeliveryPolicy): readonly TAction[] { - return this.actions(policy).filter((action) => !TERMINAL_STATES.has(action.lifecycle.state)); - } - - activeActions(policy?: DeliveryPolicy): readonly TAction[] { - return this.actions(policy).filter((action) => ACTIVE_STATES.has(action.lifecycle.state)); - } - - queuePreview(policy: DeliveryPolicy): readonly string[] { - return this.queuedActions(policy).map((action) => - action.payload.kind === "turn" ? (action.payload.preview ?? action.payload.text) : action.payload.text, - ); - } - - ticketFor(action: TAction): ActionTicketController { - const ticket = this.tickets.get(action.id); - if (!ticket) throw new Error(`Session action ${action.id} is not owned by this store`); - return ticket; - } - - ownedActions(): readonly TAction[] { - return this.actions(); - } - - actionsForMessage(message: UserMessage | CustomMessage): readonly TAction[] { - return this.actions().filter( - (action) => - action.payload.kind === "turn" && action.payload.records.some((record) => record.message === message), - ); - } - - releaseTerminal(action: TAction): void { - if (!TERMINAL_STATES.has(action.lifecycle.state)) { - throw new Error(`Cannot release nonterminal session action ${action.id}`); - } - const list = this.list(action.delivery); - const index = list.indexOf(action); - if (index >= 0) list.splice(index, 1); - this.tickets.delete(action.id); - } - - private actions(policy?: DeliveryPolicy): readonly TAction[] { - if (policy) return this.list(policy); - return [...this.nextTurnBoundary, ...this.whenRunIdle]; - } - - private list(policy: DeliveryPolicy): TAction[] { - return policy === "next_turn_boundary" ? this.nextTurnBoundary : this.whenRunIdle; - } - - private assertNewAction(action: TAction): void { - if (action.lifecycle.state !== "queued") throw new Error("Only queued session actions can be enqueued"); - if (this.tickets.has(action.id)) throw new Error(`Duplicate session action id: ${action.id}`); - } -} - -export interface RuntimeActivity { - lowerAgentRun: boolean; - compaction: boolean; - retry: boolean; - bash: boolean; - refinementApply: boolean; - branchMutation: boolean; - schedulerPauseCount: number; - disposing: boolean; -} - -export type IdleEvictionMinutes = number | "off"; - -export interface SessionEvictionSnapshot { - isSessionActive: boolean; - attachedClients: number; - hasRegisteredCronJob: boolean; - lastActivityAt: number; -} - -export interface SessionPassivationSnapshot extends SessionEvictionSnapshot { - hasParent: boolean; - hasNonPassiveDescendants: boolean; - isHydrating: boolean; -} - -export interface WorkerEvictionSnapshot { - lifecycle: "starting" | "ready" | "recovering" | "stopping" | "failed"; - isConnected: boolean; - isStopping: boolean; - hasOwnerClient: boolean; - isPreparingUpdateRestart: boolean; - hasWakeBlindSchedule: boolean; - sessions: readonly SessionEvictionSnapshot[]; -} - -function isIdleEvictionThresholdMet( - session: SessionEvictionSnapshot, - idleEvictionMinutes: IdleEvictionMinutes, - now: number, -): boolean { - if (idleEvictionMinutes === "off" || !Number.isFinite(idleEvictionMinutes) || idleEvictionMinutes <= 0) { - return false; - } - return ( - !session.isSessionActive && - session.attachedClients === 0 && - !session.hasRegisteredCronJob && - Number.isFinite(session.lastActivityAt) && - now - session.lastActivityAt >= idleEvictionMinutes * 60_000 - ); -} - -/** Pure per-node residency policy. Roots remain owned by whole-worker eviction. */ -export function canPassivateSession( - session: SessionPassivationSnapshot, - idleEvictionMinutes: IdleEvictionMinutes, - now = Date.now(), -): boolean { - return ( - session.hasParent && - !session.hasNonPassiveDescendants && - !session.isHydrating && - isIdleEvictionThresholdMet(session, idleEvictionMinutes, now) - ); -} - -/** Pure whole-tree residency policy. Callers must supply supervisor-owned attachment state. */ -export function canEvictWorker( - worker: WorkerEvictionSnapshot, - idleEvictionMinutes: IdleEvictionMinutes, - now = Date.now(), -): boolean { - if ( - worker.lifecycle !== "ready" || - !worker.isConnected || - worker.isStopping || - worker.hasOwnerClient || - worker.isPreparingUpdateRestart || - worker.hasWakeBlindSchedule || - worker.sessions.length === 0 - ) { - return false; - } - return worker.sessions.every((session) => isIdleEvictionThresholdMet(session, idleEvictionMinutes, now)); -} - -export function canSelectSessionAction(activity: RuntimeActivity): boolean { - return ( - !activity.lowerAgentRun && - !activity.compaction && - !activity.retry && - !activity.bash && - !activity.refinementApply && - !activity.branchMutation && - activity.schedulerPauseCount === 0 && - !activity.disposing - ); -} +export { + canEvictWorker, + canPassivateSession, + type IdleEvictionMinutes, + type SessionEvictionSnapshot, + type SessionPassivationSnapshot, + type WorkerEvictionSnapshot, +} from "../modes/daemon/workers/residency-policy.js"; +export { + type ActionLifecycle, + ActionStore, + type ActionTicket, + ActionTicketController, + type AdmissionDisposition, + canSelectSessionAction, + type DeliveryOutcome, + type DeliveryPolicy, + type DeliveryRecord, + type QueuedMessageLane, + type QueuedMessageMutation, + type QueuedMessageMutationStatus, + queuedMessageLaneDeliveryPolicy, + type RollbackProof, + type RuntimeActivity, + type SessionAction, + type SessionActionPayload, + type SessionActionSnapshot, + type SessionCommandPayload, + type SessionTurnPayload, + type SubmissionOutcome, + transitionSessionAction, + type WakePolicy, +} from "../session/input/action-store.js"; diff --git a/packages/coding-agent/src/core/session-manager.ts b/packages/coding-agent/src/core/session-manager.ts index d918a1e068..63bccb3971 100644 --- a/packages/coding-agent/src/core/session-manager.ts +++ b/packages/coding-agent/src/core/session-manager.ts @@ -18,16 +18,13 @@ import { readdir, readFile, stat } from "fs/promises"; import { basename, dirname, join, resolve } from "path"; import { v7 as uuidv7 } from "uuid"; import { getAgentDir as getDefaultAgentDir, getSessionsDir } from "../config.js"; -import { realpathIfPresentSync, writeFileAtomicSync } from "../utils/atomic-file.js"; -import { readBytesSync, readFirstLineSync, readLinesAsBuffers } from "../utils/file-lines.js"; -import { captureGitContext, type GitContext, gitContextsEqual } from "../utils/git.js"; import { type BashExecutionMessage, type CustomMessage, createBranchSummaryMessage, createCompactionSummaryMessage, createCustomMessage, -} from "./messages.js"; +} from "../session/context/messages.js"; import { addAssistantUsage, cloneUsage, @@ -35,7 +32,10 @@ import { type SessionUsageSummary, sessionUsageSummaryFrom, subtractAssistantUsage, -} from "./usage.js"; +} from "../session/context/usage.js"; +import { realpathIfPresentSync, writeFileAtomicSync } from "../utils/atomic-file.js"; +import { readBytesSync, readFirstLineSync, readLinesAsBuffers } from "../utils/file-lines.js"; +import { captureGitContext, type GitContext, gitContextsEqual } from "../utils/git.js"; export const CURRENT_SESSION_VERSION = 3; const SESSION_LIST_SEARCH_TEXT_MAX_CHARS = 64 * 1024; diff --git a/packages/coding-agent/src/core/session-stats.ts b/packages/coding-agent/src/core/session-stats.ts index bff8c91922..d50d817b27 100644 --- a/packages/coding-agent/src/core/session-stats.ts +++ b/packages/coding-agent/src/core/session-stats.ts @@ -1,20 +1,2 @@ -import type { ContextUsage } from "./extensions/index.js"; - -export interface SessionStats { - sessionFile: string | undefined; - sessionId: string; - userMessages: number; - assistantMessages: number; - toolCalls: number; - toolResults: number; - totalMessages: number; - tokens: { - input: number; - output: number; - cacheRead: number; - cacheWrite: number; - total: number; - }; - cost: number; - contextUsage?: ContextUsage; -} +// Compatibility exports; implementation lives with its session owner. +export type { SessionStats } from "../session/context/session-stats.js"; diff --git a/packages/coding-agent/src/core/system-prompt.ts b/packages/coding-agent/src/core/system-prompt.ts index 474a2c619b..1411f5ca20 100644 --- a/packages/coding-agent/src/core/system-prompt.ts +++ b/packages/coding-agent/src/core/system-prompt.ts @@ -1,197 +1,2 @@ -/** - * System prompt construction and project context loading - */ - -import { buildChildAgentDoctrine, buildRlmPrompt, buildSubagentGuidance } from "./prompts/index.js"; -import { REFINE_SKILL_NAME } from "./refinement/index.js"; -import { formatSkillsForPrompt, getPythonSkillRuntimeInfo, type Skill } from "./skills.js"; - -export interface BuildSystemPromptOptions { - /** Custom system prompt (replaces default). */ - customPrompt?: string; - /** Active tools. Tool schemas carry tool descriptions outside the prompt body. */ - selectedTools?: string[]; - /** Optional one-line tool snippets keyed by tool name. Used only for custom prompts. */ - toolSnippets?: Record; - /** Additional guideline bullets appended to the system prompt. */ - promptGuidelines?: string[]; - /** Text to append to system prompt. */ - appendSystemPrompt?: string; - /** Working directory. */ - cwd: string; - /** Conversation log path. */ - messagesPath?: string; - /** Pre-loaded context files. */ - contextFiles?: Array<{ path: string; content: string }>; - /** Pre-loaded skills. */ - skills?: Skill[]; - /** Whether to include the model-facing rlm recursion guidance. */ - allowRecursion?: boolean; - /** Fixed recursive-agent depth for this session. */ - rlmDepth?: number; - /** Human-readable parent name or id for child communication doctrine. */ - rlmParentAgent?: string; - /** Enabled user-configured servers available through the generic kernel MCP API. */ - genericMcpServers?: string[]; -} - -/** Build the system prompt with tools, guidelines, and context */ -export function buildSystemPrompt(options: BuildSystemPromptOptions): string { - const { - customPrompt, - selectedTools, - promptGuidelines, - appendSystemPrompt, - cwd, - messagesPath, - contextFiles: providedContextFiles, - skills: providedSkills, - allowRecursion, - } = options; - const promptCwd = cwd.replace(/\\/g, "/"); - const promptMessagesPath = (messagesPath ?? "not persisted").replace(/\\/g, "/"); - - const now = new Date(); - const year = now.getFullYear(); - const month = String(now.getMonth() + 1).padStart(2, "0"); - const day = String(now.getDate()).padStart(2, "0"); - const date = `${year}-${month}-${day}`; - - const appendSection = appendSystemPrompt ? `\n\n${appendSystemPrompt}` : ""; - - const contextFiles = providedContextFiles ?? []; - const skills = providedSkills ?? []; - const tools = selectedTools ?? ["ipython"]; - const hasIpython = tools.includes("ipython"); - const visibleSkills = skills.filter((skill) => !skill.disableModelInvocation); - const visiblePythonSkillImportNames = getPythonSkillRuntimeInfo(visibleSkills).map((skill) => skill.importName); - const hasRefineSkill = visibleSkills.some((skill) => skill.name === REFINE_SKILL_NAME); - const genericMcpSection = hasIpython ? formatGenericMcpGuidance(options.genericMcpServers) : ""; - - if (customPrompt) { - let prompt = customPrompt; - - // Append project context files - if (contextFiles.length > 0) { - prompt += "\n\n# Project Context\n\n"; - prompt += "Project-specific instructions and guidelines:\n\n"; - for (const { path: filePath, content } of contextFiles) { - prompt += `## ${filePath}\n\n${content}\n\n`; - } - } - - // Append skills section only when the model has a way to inspect skill files. - const customPromptHasFileAccess = - !selectedTools || selectedTools.includes("ipython") || selectedTools.includes("bash"); - if (customPromptHasFileAccess && skills.length > 0) { - prompt += formatSkillsForPrompt(skills); - } - - // Add date and working directory last - prompt += `\nCurrent date: ${date}`; - prompt += `\nCurrent working directory: ${promptCwd}`; - - const childDoctrine = buildChildAgentDoctrine({ - depth: options.rlmDepth, - parentAgent: options.rlmParentAgent, - installedSkills: visiblePythonSkillImportNames, - activeTools: tools, - }); - if (childDoctrine) { - prompt += `\n\n${childDoctrine}`; - } - - if (genericMcpSection) { - prompt += `\n\n${genericMcpSection}`; - } - - if (appendSection) { - prompt += appendSection; - } - - return prompt; - } - - let prompt = buildRlmPrompt({ - cwd: promptCwd, - messagesPath: promptMessagesPath, - installedSkills: visiblePythonSkillImportNames, - activeTools: tools.filter((name) => name === "ipython" || name === "bash" || name === "edit"), - allowRecursion, - depth: options.rlmDepth, - parentAgent: options.rlmParentAgent, - }); - - // Appended AFTER the trained buildRlmPrompt prefix: delegation doctrine precedes the subagent specs delivered via the harness digest. - if ((allowRecursion ?? true) && hasIpython) { - const visiblePythonSkillNames = new Set( - getPythonSkillRuntimeInfo(visibleSkills).map((skill) => skill.importName), - ); - prompt += `\n\n${buildSubagentGuidance({ - includeRefineExamples: hasRefineSkill, - hasAgentMessage: visiblePythonSkillNames.has("agent_message"), - hasAgentObserve: visiblePythonSkillNames.has("agent_observe"), - })}`; - } - - if (genericMcpSection) { - prompt += `\n\n${genericMcpSection}`; - } - - const guidelines = formatPromptGuidelines(promptGuidelines); - if (guidelines) { - prompt += `\n\n# Additional Guidance\n\n${guidelines}`; - } - - // Append project context files - if (contextFiles.length > 0) { - prompt += "\n\n# Project Context\n\n"; - prompt += "Project-specific instructions and guidelines:\n\n"; - for (const { path: filePath, content } of contextFiles) { - prompt += `## ${filePath}\n\n${content}\n\n`; - } - } - - // Append skills section only when the model has a way to inspect skill files. - const hasFileAccess = tools.includes("ipython") || tools.includes("bash"); - if (hasFileAccess && skills.length > 0) { - prompt += formatSkillsForPrompt(skills); - } - - if (appendSection) { - prompt += appendSection; - } - - return prompt; -} - -function formatGenericMcpGuidance(servers: string[] | undefined): string { - const enabledServers = [...new Set(servers ?? [])].sort((left, right) => left.localeCompare(right)); - if (enabledServers.length === 0) return ""; - - return [ - "# Generic MCP Connections", - "", - "Generic MCP connections are accessed through the pre-imported Python `mcp` object in the Python REPL, not as top-level native tool namespaces or installed Python skills.", - `Enabled generic MCP servers: ${enabledServers.map((server) => `\`${server}\``).join(", ")}.`, - ...enabledServers.map( - (server) => - `For \`${server}\`, first discover its tools with \`await mcp.list_tools("${server}")\`, then call one with \`await mcp.call_tool("${server}", "", arguments)\`.`, - ), - ].join("\n"); -} - -function formatPromptGuidelines(promptGuidelines: string[] | undefined): string { - const guidelinesList: string[] = []; - const guidelinesSet = new Set(); - - for (const guideline of promptGuidelines ?? []) { - const normalized = guideline.trim(); - if (normalized.length > 0 && !guidelinesSet.has(normalized)) { - guidelinesSet.add(normalized); - guidelinesList.push(normalized); - } - } - - return guidelinesList.map((guideline) => `- ${guideline}`).join("\n"); -} +// Compatibility exports; implementation lives with its session owner. +export { type BuildSystemPromptOptions, buildSystemPrompt } from "../session/context/system-prompt.js"; diff --git a/packages/coding-agent/src/core/telemetry.ts b/packages/coding-agent/src/core/telemetry.ts index 15dbcb5fd1..e3466d1237 100644 --- a/packages/coding-agent/src/core/telemetry.ts +++ b/packages/coding-agent/src/core/telemetry.ts @@ -4,8 +4,8 @@ import { arch, platform } from "node:os"; import { join } from "node:path"; import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; import { detectInstallMethod, VERSION } from "../config.js"; +import type { AgentSession, AgentSessionEvent } from "../session/agent-session.js"; import { writeFileAtomicSync } from "../utils/atomic-file.js"; -import type { AgentSession, AgentSessionEvent } from "./agent-session.js"; import type { AgentExecutionMode } from "./agent-session-config.js"; import type { AuthCredential, AuthStatus } from "./auth-storage.js"; import type { SettingsManager } from "./settings-manager.js"; diff --git a/packages/coding-agent/src/core/usage.ts b/packages/coding-agent/src/core/usage.ts index ae45dbc923..d93ecff7e4 100644 --- a/packages/coding-agent/src/core/usage.ts +++ b/packages/coding-agent/src/core/usage.ts @@ -1,76 +1,9 @@ -import type { Usage } from "@earendil-works/pi-ai"; - -export interface SessionUsageSummary { - inputTokens: number; - outputTokens: number; - cost: number; -} - -export function sessionUsageSummaryFrom(usage: Usage): SessionUsageSummary | undefined { - const inputTokens = usage.input + usage.cacheRead + usage.cacheWrite; - if (inputTokens === 0 && usage.output === 0 && usage.cost.total === 0) { - return undefined; - } - return { inputTokens, outputTokens: usage.output, cost: usage.cost.total }; -} - -export function emptyUsage(): Usage { - return { - input: 0, - output: 0, - cacheRead: 0, - cacheWrite: 0, - totalTokens: 0, - cost: { - input: 0, - output: 0, - cacheRead: 0, - cacheWrite: 0, - total: 0, - }, - }; -} - -export function addAssistantUsage(total: Usage, usage: Usage): void { - total.input += usage.input; - total.output += usage.output; - total.cacheRead += usage.cacheRead; - total.cacheWrite += usage.cacheWrite; - total.totalTokens += usage.totalTokens; - total.cost.input += usage.cost.input; - total.cost.output += usage.cost.output; - total.cost.cacheRead += usage.cost.cacheRead; - total.cost.cacheWrite += usage.cost.cacheWrite; - total.cost.total += usage.cost.total; -} - -/** Remove a previously added usage, clamping at zero to absorb attribution drift. */ -export function subtractAssistantUsage(total: Usage, usage: Usage): void { - total.input = Math.max(0, total.input - usage.input); - total.output = Math.max(0, total.output - usage.output); - total.cacheRead = Math.max(0, total.cacheRead - usage.cacheRead); - total.cacheWrite = Math.max(0, total.cacheWrite - usage.cacheWrite); - total.totalTokens = Math.max(0, total.totalTokens - usage.totalTokens); - total.cost.input = Math.max(0, total.cost.input - usage.cost.input); - total.cost.output = Math.max(0, total.cost.output - usage.cost.output); - total.cost.cacheRead = Math.max(0, total.cost.cacheRead - usage.cost.cacheRead); - total.cost.cacheWrite = Math.max(0, total.cost.cacheWrite - usage.cost.cacheWrite); - total.cost.total = Math.max(0, total.cost.total - usage.cost.total); -} - -export function cloneUsage(usage: Usage): Usage { - return { - input: usage.input, - output: usage.output, - cacheRead: usage.cacheRead, - cacheWrite: usage.cacheWrite, - totalTokens: usage.totalTokens, - cost: { - input: usage.cost.input, - output: usage.cost.output, - cacheRead: usage.cost.cacheRead, - cacheWrite: usage.cost.cacheWrite, - total: usage.cost.total, - }, - }; -} +// Compatibility exports; implementation lives with its session owner. +export { + addAssistantUsage, + cloneUsage, + emptyUsage, + type SessionUsageSummary, + sessionUsageSummaryFrom, + subtractAssistantUsage, +} from "../session/context/usage.js"; diff --git a/packages/coding-agent/src/index.ts b/packages/coding-agent/src/index.ts index 1eaa7a0927..545034dd50 100644 --- a/packages/coding-agent/src/index.ts +++ b/packages/coding-agent/src/index.ts @@ -2,14 +2,6 @@ // Config paths export { getAgentDir, VERSION } from "./config.js"; -export { - AgentSession, - type AgentSessionConfig, - type AgentSessionEvent, - type AgentSessionEventListener, - type ModelCycleResult, - type PromptOptions, -} from "./core/agent-session.js"; // Auth and model registry export { type ApiKeyCredential, @@ -21,29 +13,6 @@ export { InMemoryAuthStorageBackend, type OAuthCredential, } from "./core/auth-storage.js"; -// Compaction -export { - type BranchPreparation, - type BranchSummaryResult, - type CollectEntriesResult, - type CompactionResult, - type CutPointResult, - calculateContextTokens, - collectEntriesForBranchSummary, - compact, - DEFAULT_COMPACTION_SETTINGS, - estimateTokens, - type FileOperations, - findCutPoint, - findTurnStartIndex, - type GenerateBranchSummaryOptions, - generateBranchSummary, - generateSummary, - getLastAssistantUsage, - prepareBranchEntries, - serializeConversation, - shouldCompact, -} from "./core/compaction/index.js"; export { createEventBus, type EventBus, type EventBusController } from "./core/event-bus.js"; // Extension system export type { @@ -140,7 +109,6 @@ export { } from "./core/extensions/index.js"; // Footer data provider (git branch + extension statuses - data not otherwise available to extensions) export type { ReadonlyFooterDataProvider } from "./core/footer-data-provider.js"; -export { convertToLlm } from "./core/messages.js"; export { ModelRegistry } from "./core/model-registry.js"; export type { PackageManager, @@ -151,12 +119,6 @@ export type { ResolvedResource, } from "./core/package-manager.js"; export { DefaultPackageManager } from "./core/package-manager.js"; -export type { - HarnessState, - RefinementEdit, - RefinementProposal, - RefinementResult, -} from "./core/refinement/index.js"; export type { ResourceCollision, ResourceDiagnostic, ResourceLoader } from "./core/resource-loader.js"; export { DefaultResourceLoader, loadProjectContextFiles } from "./core/resource-loader.js"; // SDK for programmatic usage @@ -188,7 +150,6 @@ export { type RlmSubagentRuntime, type SubagentRuntimeHost, } from "./core/sdk.js"; -export type { SessionActionSnapshot } from "./core/session-action-store.js"; export { SessionImportFileNotFoundError } from "./core/session-import-errors.js"; export { type BranchSummaryEntry, @@ -216,7 +177,6 @@ export { type SessionStateStatus, type ThinkingLevelChangeEntry, } from "./core/session-manager.js"; -export type { SessionStats } from "./core/session-stats.js"; export { type CompactionSettings, type ImageSettings, @@ -399,6 +359,39 @@ export { Theme, type ThemeColor, } from "./modes/interactive/theme/theme.js"; +export { + AgentSession, + type AgentSessionConfig, + type AgentSessionEvent, + type AgentSessionEventListener, + type ModelCycleResult, + type PromptOptions, +} from "./session/agent-session.js"; +export { + compact, + findCutPoint, + findTurnStartIndex, + generateSummary, + shouldCompact, +} from "./session/compaction/summary.js"; +export { type CompactionResult, type CutPointResult, DEFAULT_COMPACTION_SETTINGS } from "./session/compaction/types.js"; +// Compaction +export { + type BranchPreparation, + type BranchSummaryResult, + type CollectEntriesResult, + collectEntriesForBranchSummary, + type GenerateBranchSummaryOptions, + generateBranchSummary, + prepareBranchEntries, +} from "./session/context/branch-summary.js"; +export { serializeConversation } from "./session/context/conversation-text.js"; +export type { FileOperations } from "./session/context/file-tracking.js"; +export { convertToLlm } from "./session/context/messages.js"; +export type { SessionStats } from "./session/context/session-stats.js"; +export { calculateContextTokens, estimateTokens, getLastAssistantUsage } from "./session/context/token-estimate.js"; +export type { SessionActionSnapshot } from "./session/input/action-store.js"; +export type { HarnessState, RefinementEdit, RefinementProposal, RefinementResult } from "./session/refinement/types.js"; // Clipboard utilities export { copyToClipboard } from "./utils/clipboard.js"; export { parseFrontmatter, stripFrontmatter } from "./utils/frontmatter.js"; diff --git a/packages/coding-agent/src/modes/acp/acp-mode.ts b/packages/coding-agent/src/modes/acp/acp-mode.ts index db1087f76c..d338d9e9b0 100644 --- a/packages/coding-agent/src/modes/acp/acp-mode.ts +++ b/packages/coding-agent/src/modes/acp/acp-mode.ts @@ -7,8 +7,8 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; import type { ImageContent } from "@earendil-works/pi-ai"; import { VERSION } from "../../config.js"; import type { AgentSessionRuntime } from "../../core/agent-session-runtime.js"; -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; import { takeOverStdout, writeRawStdout } from "../../core/output-guard.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; import { InProcessAgentConnection } from "../agent-connection/in-process-agent-connection.js"; import type { AgentConnection, diff --git a/packages/coding-agent/src/modes/acp/acp-stop-reason.ts b/packages/coding-agent/src/modes/acp/acp-stop-reason.ts index aa78be0aa6..da39c03ea2 100644 --- a/packages/coding-agent/src/modes/acp/acp-stop-reason.ts +++ b/packages/coding-agent/src/modes/acp/acp-stop-reason.ts @@ -1,5 +1,5 @@ -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; -import { autonomousLimitReason } from "../../core/autonomous.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; +import { autonomousLimitReason } from "../../session/autonomy/autonomous.js"; /** * ACP stop reasons. `session/prompt` resolves with one of these after the agent diff --git a/packages/coding-agent/src/modes/agent-connection/daemon-agent-connection.ts b/packages/coding-agent/src/modes/agent-connection/daemon-agent-connection.ts index cba3b6d22d..5916cac550 100644 --- a/packages/coding-agent/src/modes/agent-connection/daemon-agent-connection.ts +++ b/packages/coding-agent/src/modes/agent-connection/daemon-agent-connection.ts @@ -3,12 +3,8 @@ import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core" import type { ImageContent, ServiceTier, Transport } from "@earendil-works/pi-ai"; import { appendRotatingLog, getAgentLogPath, getDaemonLogPath } from "../../config.js"; import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus } from "../../core/agent-messages.js"; -import type { AgentSessionEvent } from "../../core/agent-session.js"; import type { AgentSessionRuntimeConfig } from "../../core/agent-session-config.js"; -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; import type { BashResult } from "../../core/bash-executor.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; -import type { ContextTreeNode } from "../../core/context-tree.js"; import type { AgentCronJob, AgentHeartbeatDeliveryMode, @@ -16,10 +12,14 @@ import type { AgentHeartbeatUpdateAction, } from "../../core/cron-jobs.js"; import type { AcpMcpServerConfig } from "../../core/mcp/acp-mcp-types.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; import type { DeleteSessionFileResult } from "../../core/session-file-actions.js"; import { SessionAlreadyActiveError } from "../../core/session-lease.js"; -import type { SessionStats } from "../../core/session-stats.js"; +import type { AgentSessionEvent } from "../../session/agent-session.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; +import type { CompactionResult } from "../../session/compaction/types.js"; +import type { ContextTreeNode } from "../../session/context/context-tree.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; +import type { RefinementResult } from "../../session/refinement/types.js"; import { AgentsViewRosterStore, STALE_ROSTER_DAEMON_MESSAGE } from "../agents-view/roster-store.js"; import { DaemonCapabilityUnavailableError, diff --git a/packages/coding-agent/src/modes/agent-connection/in-process-agent-connection.ts b/packages/coding-agent/src/modes/agent-connection/in-process-agent-connection.ts index fa6ded0202..5e8701862c 100644 --- a/packages/coding-agent/src/modes/agent-connection/in-process-agent-connection.ts +++ b/packages/coding-agent/src/modes/agent-connection/in-process-agent-connection.ts @@ -3,10 +3,7 @@ import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core" import type { ImageContent, ServiceTier, Transport } from "@earendil-works/pi-ai"; import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus } from "../../core/agent-messages.js"; import type { AgentSessionRuntime } from "../../core/agent-session-runtime.js"; -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; import type { BashResult } from "../../core/bash-executor.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; -import type { ContextTreeNode } from "../../core/context-tree.js"; import type { AgentCronJob, AgentHeartbeatDeliveryMode, @@ -16,11 +13,14 @@ import type { import type { ExtensionUIContext } from "../../core/extensions/types.js"; import type { AcpMcpServerConfig } from "../../core/mcp/acp-mcp-types.js"; import { providerRetryPolicy } from "../../core/provider-retry.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; import { type DeleteSessionFileResult, deleteSessionFile } from "../../core/session-file-actions.js"; import { SessionManager } from "../../core/session-manager.js"; -import type { SessionStats } from "../../core/session-stats.js"; import { type SideQuestionRun, startSideQuestion } from "../../core/side-question.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; +import type { CompactionResult } from "../../session/compaction/types.js"; +import type { ContextTreeNode } from "../../session/context/context-tree.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; +import type { RefinementResult } from "../../session/refinement/types.js"; import { waitForHeadlessCompletion } from "../headless-completion.js"; import { createAgentConnectionCommands, diff --git a/packages/coding-agent/src/modes/agent-connection/snapshot.ts b/packages/coding-agent/src/modes/agent-connection/snapshot.ts index 3b08cc4cda..aa6d0bd911 100644 --- a/packages/coding-agent/src/modes/agent-connection/snapshot.ts +++ b/packages/coding-agent/src/modes/agent-connection/snapshot.ts @@ -1,8 +1,8 @@ import { createHash } from "node:crypto"; import { basename, isAbsolute, relative, resolve, sep } from "node:path"; import type { Api, Model } from "@earendil-works/pi-ai"; -import type { AgentSession } from "../../core/agent-session.js"; import type { AgentSessionRuntime } from "../../core/agent-session-runtime.js"; +import type { AgentSession } from "../../session/agent-session.js"; import type { AgentConnectionArtifactReference, AgentConnectionArtifactType, diff --git a/packages/coding-agent/src/modes/agent-connection/types.ts b/packages/coding-agent/src/modes/agent-connection/types.ts index f075c28419..fc0958e0d0 100644 --- a/packages/coding-agent/src/modes/agent-connection/types.ts +++ b/packages/coding-agent/src/modes/agent-connection/types.ts @@ -2,10 +2,7 @@ import type { AgentEvent, AgentMessage, ThinkingLevel } from "@earendil-works/pi import type { Api, ImageContent, Model, ServiceTier, TextContent, Transport, Usage } from "@earendil-works/pi-ai"; import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus } from "../../core/agent-messages.js"; import type { AuthSourceToken } from "../../core/auth-storage.js"; -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; import type { BashResult } from "../../core/bash-executor.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; -import type { ContextTreeNode } from "../../core/context-tree.js"; import type { AgentCronJob, AgentHeartbeatDeliveryMode, @@ -16,18 +13,21 @@ import type { ReplayBuiltInToolName } from "../../core/extensions/index.js"; import type { InputSource } from "../../core/extensions/types.js"; import type { KernelSentAgentMessage } from "../../core/kernel/index.js"; import type { AcpMcpServerConfig } from "../../core/mcp/acp-mcp-types.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; -import type { RlmMaxDepthStatus, SetRlmMaxDepthResult } from "../../core/rlm-max-depth.js"; +import type { DeleteSessionFileResult } from "../../core/session-file-actions.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; +import type { RlmMaxDepthStatus, SetRlmMaxDepthResult } from "../../session/children/max-depth.js"; +import type { CompactionResult } from "../../session/compaction/types.js"; +import type { ContextTreeNode } from "../../session/context/context-tree.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; +import type { SessionUsageSummary } from "../../session/context/usage.js"; +import type { GoalState } from "../../session/goals/contracts.js"; import type { QueuedMessageLane, QueuedMessageMutation, QueuedMessageMutationStatus, SessionActionSnapshot, -} from "../../core/session-action-store.js"; -import type { DeleteSessionFileResult } from "../../core/session-file-actions.js"; -import type { SessionStats } from "../../core/session-stats.js"; -import type { SessionUsageSummary } from "../../core/usage.js"; -import type { GoalState } from "../../session/goals/contracts.js"; +} from "../../session/input/action-store.js"; +import type { RefinementResult } from "../../session/refinement/types.js"; import type { SessionSummary } from "../daemon/daemon-session-list.js"; /** diff --git a/packages/coding-agent/src/modes/daemon/daemon-extension-binding.ts b/packages/coding-agent/src/modes/daemon/daemon-extension-binding.ts index cecd046186..e00ebae1bb 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-extension-binding.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-extension-binding.ts @@ -6,7 +6,7 @@ import type { ExtensionWidgetOptions, WorkingIndicatorOptions, } from "../../core/extensions/index.js"; -import type { SubagentRuntimeHost } from "../../core/rlm-runtime.js"; +import type { SubagentRuntimeHost } from "../../session/children/runtime-contracts.js"; import { createAgentConnectionState } from "../agent-connection/snapshot.js"; import type { AgentConnectionState } from "../agent-connection/types.js"; import { type Theme, theme } from "../interactive/theme/theme.js"; diff --git a/packages/coding-agent/src/modes/daemon/daemon-mode.ts b/packages/coding-agent/src/modes/daemon/daemon-mode.ts index e50d7c5213..4f6ef4391e 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-mode.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-mode.ts @@ -64,7 +64,6 @@ import { normalizeObserveLimit, normalizeObserveMaxChars, } from "../../core/agent-observe.js"; -import { type PromptOptions, rlmChildLabel } from "../../core/agent-session.js"; import { type AgentSessionRuntimeConfig, mergeAgentSessionRuntimeConfig } from "../../core/agent-session-config.js"; import { type AgentSessionRuntime, @@ -88,19 +87,7 @@ import { shouldDeferHeartbeatCronJob, } from "../../core/cron-jobs.js"; import { ORPHAN_PROCESS_JOURNAL_ENV } from "../../core/orphan-process-journal.js"; -import { PromptAdmissionCancelledError, waitForPromptAdmission } from "../../core/prompt-admission.js"; import { providerRetryPolicy } from "../../core/provider-retry.js"; -import type { - CreateRlmRootSessionOptions, - CreateRlmSubagentRuntimeOptions, - RlmCreateSessionResult, - SubagentRuntimeHost, -} from "../../core/rlm-runtime.js"; -import { - canPassivateSession, - type IdleEvictionMinutes, - type SessionPassivationSnapshot, -} from "../../core/session-action-store.js"; import { deleteSessionArtifacts, deleteSessionFile } from "../../core/session-file-actions.js"; import { acquireSessionLease, canonicalSessionPath, type SessionLease } from "../../core/session-lease.js"; import { @@ -111,9 +98,17 @@ import { SessionManager, } from "../../core/session-manager.js"; import { resolveSessionPath } from "../../core/session-resolver.js"; -import type { SessionStats } from "../../core/session-stats.js"; import { SettingsManager } from "../../core/settings-manager.js"; import { type SideQuestionRun, startSideQuestion } from "../../core/side-question.js"; +import { type PromptOptions, rlmChildLabel } from "../../session/agent-session.js"; +import type { + CreateRlmRootSessionOptions, + CreateRlmSubagentRuntimeOptions, + RlmCreateSessionResult, + SubagentRuntimeHost, +} from "../../session/children/runtime-contracts.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; +import { PromptAdmissionCancelledError, waitForPromptAdmission } from "../../session/input/prompt-admission.js"; import { isProcessAlive, spawnHidden, waitForChildProcess } from "../../utils/child-process.js"; import { tryAcquireDirLock } from "../../utils/dir-lock.js"; import { killTrackedDetachedChildren } from "../../utils/shell.js"; @@ -239,6 +234,11 @@ import { type SnapshotTranscriptChunkSource, } from "./snapshot-transcript-cache.js"; import { WorkerRecoveryJournal } from "./worker-recovery-journal.js"; +import { + canPassivateSession, + type IdleEvictionMinutes, + type SessionPassivationSnapshot, +} from "./workers/residency-policy.js"; export interface DaemonModeOptions { socketPath?: string; diff --git a/packages/coding-agent/src/modes/daemon/daemon-protocol.ts b/packages/coding-agent/src/modes/daemon/daemon-protocol.ts index b864623536..c17bad104a 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-protocol.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-protocol.ts @@ -5,10 +5,8 @@ import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus, } from "../../core/agent-messages.js"; -import type { SessionActionRecoverySnapshot } from "../../core/agent-session.js"; import type { AgentSessionRuntimeConfig } from "../../core/agent-session-config.js"; import type { AgentSessionRuntimeMetadata } from "../../core/agent-session-runtime.js"; -import type { AgentAutonomousStatus } from "../../core/autonomous.js"; import type { BashResult } from "../../core/bash-executor.js"; import type { AgentCronJob, @@ -18,11 +16,13 @@ import type { } from "../../core/cron-jobs.js"; import type { InputSource } from "../../core/extensions/types.js"; import type { AcpMcpServerConfig } from "../../core/mcp/acp-mcp-types.js"; -import type { CustomMessage } from "../../core/messages.js"; -import type { QueuedMessageLane, QueuedMessageMutation } from "../../core/session-action-store.js"; import type { SessionCwdIssue } from "../../core/session-cwd.js"; import type { DeleteSessionFileResult } from "../../core/session-file-actions.js"; -import type { SessionUsageSummary } from "../../core/usage.js"; +import type { SessionActionRecoverySnapshot } from "../../session/agent-session.js"; +import type { AgentAutonomousStatus } from "../../session/autonomy/autonomous.js"; +import type { CustomMessage } from "../../session/context/messages.js"; +import type { SessionUsageSummary } from "../../session/context/usage.js"; +import type { QueuedMessageLane, QueuedMessageMutation } from "../../session/input/action-store.js"; import type { AgentConnectionAgentStatus, AgentConnectionHeartbeat, diff --git a/packages/coding-agent/src/modes/daemon/daemon-session-list.ts b/packages/coding-agent/src/modes/daemon/daemon-session-list.ts index 21e0794e32..ea1a77d19c 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-session-list.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-session-list.ts @@ -2,12 +2,12 @@ import { statSync } from "node:fs"; import { resolve } from "node:path"; import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; import type { Api, Model } from "@earendil-works/pi-ai"; -import { compactRlmText } from "../../core/agent-session.js"; import type { AgentSessionRuntimeDiagnostic } from "../../core/agent-session-services.js"; import { type AgentCronJob, isHeartbeatCronJob } from "../../core/cron-jobs.js"; -import type { SessionActionSnapshot } from "../../core/session-action-store.js"; import type { AgentTaskState, SessionInfo } from "../../core/session-manager.js"; -import type { SessionUsageSummary } from "../../core/usage.js"; +import { compactRlmText } from "../../session/agent-session.js"; +import type { SessionUsageSummary } from "../../session/context/usage.js"; +import type { SessionActionSnapshot } from "../../session/input/action-store.js"; import type { AgentConnectionRlmChildAgentSnapshot } from "../agent-connection/types.js"; import type { ActiveSessionState } from "./active-session-state.js"; diff --git a/packages/coding-agent/src/modes/daemon/daemon-supervisor.ts b/packages/coding-agent/src/modes/daemon/daemon-supervisor.ts index 5a2782c777..5aa27f6e94 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-supervisor.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-supervisor.ts @@ -42,16 +42,11 @@ import { readActiveOrphanProcesses, shouldReapOrphanProcess, } from "../../core/orphan-process-journal.js"; -import { PromptAdmissionCancelledError, waitForPromptAdmission } from "../../core/prompt-admission.js"; -import { - canEvictWorker, - type IdleEvictionMinutes, - type WorkerEvictionSnapshot, -} from "../../core/session-action-store.js"; import { canonicalSessionPath, getProcessStartId, SessionAlreadyActiveError } from "../../core/session-lease.js"; import { getSessionArtifactPathForFile, readSessionInfo, type SessionInfo } from "../../core/session-manager.js"; import { looksLikeSessionPath } from "../../core/session-resolver.js"; import { SettingsManager } from "../../core/settings-manager.js"; +import { PromptAdmissionCancelledError, waitForPromptAdmission } from "../../session/input/prompt-admission.js"; import { writeFileAtomicSync } from "../../utils/atomic-file.js"; import { isProcessAlive, @@ -168,6 +163,7 @@ import { import { serializeSavedSessionInfo } from "./saved-session-info.js"; import { SNAPSHOT_TARGET_CHUNK_BYTES, SnapshotTranscriptCache } from "./snapshot-transcript-cache.js"; import { WorkerRecoveryJournal } from "./worker-recovery-journal.js"; +import { canEvictWorker, type IdleEvictionMinutes, type WorkerEvictionSnapshot } from "./workers/residency-policy.js"; type DistributiveOmit = T extends unknown ? Omit : never; type DaemonCommandBody = DistributiveOmit; diff --git a/packages/coding-agent/src/modes/daemon/daemon-worker-protocol.ts b/packages/coding-agent/src/modes/daemon/daemon-worker-protocol.ts index 3b14dc7e1e..a986bd6ddb 100644 --- a/packages/coding-agent/src/modes/daemon/daemon-worker-protocol.ts +++ b/packages/coding-agent/src/modes/daemon/daemon-worker-protocol.ts @@ -1,6 +1,6 @@ import { closeSync, readFileSync } from "node:fs"; import type { AgentSessionMessageDeliveryMode, AgentSessionMessageSender } from "../../core/agent-messages.js"; -import type { IdleEvictionMinutes } from "../../core/session-action-store.js"; +import type { IdleEvictionMinutes } from "./workers/residency-policy.js"; export { SESSION_LEASE_OWNER_ID_ENV, SESSION_LEASES_ENABLED_ENV } from "../../core/session-lease.js"; diff --git a/packages/coding-agent/src/modes/daemon/workers/residency-policy.ts b/packages/coding-agent/src/modes/daemon/workers/residency-policy.ts new file mode 100644 index 0000000000..e279e20302 --- /dev/null +++ b/packages/coding-agent/src/modes/daemon/workers/residency-policy.ts @@ -0,0 +1,75 @@ +export type IdleEvictionMinutes = number | "off"; + +export interface SessionEvictionSnapshot { + isSessionActive: boolean; + attachedClients: number; + hasRegisteredCronJob: boolean; + lastActivityAt: number; +} + +export interface SessionPassivationSnapshot extends SessionEvictionSnapshot { + hasParent: boolean; + hasNonPassiveDescendants: boolean; + isHydrating: boolean; +} + +export interface WorkerEvictionSnapshot { + lifecycle: "starting" | "ready" | "recovering" | "stopping" | "failed"; + isConnected: boolean; + isStopping: boolean; + hasOwnerClient: boolean; + isPreparingUpdateRestart: boolean; + hasWakeBlindSchedule: boolean; + sessions: readonly SessionEvictionSnapshot[]; +} + +function isIdleEvictionThresholdMet( + session: SessionEvictionSnapshot, + idleEvictionMinutes: IdleEvictionMinutes, + now: number, +): boolean { + if (idleEvictionMinutes === "off" || !Number.isFinite(idleEvictionMinutes) || idleEvictionMinutes <= 0) { + return false; + } + return ( + !session.isSessionActive && + session.attachedClients === 0 && + !session.hasRegisteredCronJob && + Number.isFinite(session.lastActivityAt) && + now - session.lastActivityAt >= idleEvictionMinutes * 60_000 + ); +} + +/** Pure per-node residency policy. Roots remain owned by whole-worker eviction. */ +export function canPassivateSession( + session: SessionPassivationSnapshot, + idleEvictionMinutes: IdleEvictionMinutes, + now = Date.now(), +): boolean { + return ( + session.hasParent && + !session.hasNonPassiveDescendants && + !session.isHydrating && + isIdleEvictionThresholdMet(session, idleEvictionMinutes, now) + ); +} + +/** Pure whole-tree residency policy. Callers must supply supervisor-owned attachment state. */ +export function canEvictWorker( + worker: WorkerEvictionSnapshot, + idleEvictionMinutes: IdleEvictionMinutes, + now = Date.now(), +): boolean { + if ( + worker.lifecycle !== "ready" || + !worker.isConnected || + worker.isStopping || + worker.hasOwnerClient || + worker.isPreparingUpdateRestart || + worker.hasWakeBlindSchedule || + worker.sessions.length === 0 + ) { + return false; + } + return worker.sessions.every((session) => isIdleEvictionThresholdMet(session, idleEvictionMinutes, now)); +} diff --git a/packages/coding-agent/src/modes/headless-completion.ts b/packages/coding-agent/src/modes/headless-completion.ts index ee257b8462..f5776c922f 100644 --- a/packages/coding-agent/src/modes/headless-completion.ts +++ b/packages/coding-agent/src/modes/headless-completion.ts @@ -1,11 +1,11 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; import type { AssistantMessage } from "@earendil-works/pi-ai"; -import type { AgentSession } from "../core/agent-session.js"; +import type { AgentSession } from "../session/agent-session.js"; import { type AgentAutonomousStatus, autonomousLimitReason, buildAutonomousGateFailureContinuation, -} from "../core/autonomous.js"; +} from "../session/autonomy/autonomous.js"; import { COMPACTION_OUTCOME_CUSTOM_TYPE, type CompactionOutcomeMessage, @@ -15,7 +15,7 @@ import { REFINEMENT_NOTICE_CUSTOM_TYPE, REFINEMENT_OUTCOME_CUSTOM_TYPE, type SessionSlashCommandResultMessage, -} from "../core/messages.js"; +} from "../session/context/messages.js"; export function latestAutonomousGateAttempt(status: AgentAutonomousStatus): number { return Math.max(status.lastGateFailure?.attempt ?? 0, 0, ...Object.values(status.gateAttempts)); diff --git a/packages/coding-agent/src/modes/interactive/components/branch-summary-message.ts b/packages/coding-agent/src/modes/interactive/components/branch-summary-message.ts index 1aa49bb7a8..bf7f488f0e 100644 --- a/packages/coding-agent/src/modes/interactive/components/branch-summary-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/branch-summary-message.ts @@ -1,5 +1,5 @@ import { Box, Markdown, type MarkdownTheme, Spacer, Text } from "@earendil-works/pi-tui"; -import type { BranchSummaryMessage } from "../../../core/messages.js"; +import type { BranchSummaryMessage } from "../../../session/context/messages.js"; import { getMarkdownTheme, theme } from "../theme/theme.js"; import { expandCollapseHint } from "./keybinding-hints.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/compaction-outcome-message.ts b/packages/coding-agent/src/modes/interactive/components/compaction-outcome-message.ts index 1f2885df80..e12d26adad 100644 --- a/packages/coding-agent/src/modes/interactive/components/compaction-outcome-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/compaction-outcome-message.ts @@ -1,5 +1,5 @@ import { Container, Spacer, Text } from "@earendil-works/pi-tui"; -import type { CompactionOutcomeMessage } from "../../../core/messages.js"; +import type { CompactionOutcomeMessage } from "../../../session/context/messages.js"; import { theme } from "../theme/theme.js"; /** Renders a durable unsuccessful automatic-compaction outcome. */ diff --git a/packages/coding-agent/src/modes/interactive/components/compaction-summary-message.ts b/packages/coding-agent/src/modes/interactive/components/compaction-summary-message.ts index cdeb3bf9d0..6537fe09d9 100644 --- a/packages/coding-agent/src/modes/interactive/components/compaction-summary-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/compaction-summary-message.ts @@ -1,5 +1,5 @@ import { Markdown, type MarkdownTheme, Spacer, Text } from "@earendil-works/pi-tui"; -import type { CompactionSummaryMessage } from "../../../core/messages.js"; +import type { CompactionSummaryMessage } from "../../../session/context/messages.js"; import { getMarkdownTheme, theme } from "../theme/theme.js"; import { ExpandableEventMessage } from "./expandable-event-message.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/context-tree-format.ts b/packages/coding-agent/src/modes/interactive/components/context-tree-format.ts index d7c32794d9..0940e87f55 100644 --- a/packages/coding-agent/src/modes/interactive/components/context-tree-format.ts +++ b/packages/coding-agent/src/modes/interactive/components/context-tree-format.ts @@ -1,8 +1,8 @@ import type { Usage } from "@earendil-works/pi-ai"; import { truncateToWidth, visibleWidth } from "@earendil-works/pi-tui"; -import type { ContextTreeNode } from "../../../core/context-tree.js"; import type { ContextUsage } from "../../../core/extensions/index.js"; -import { addAssistantUsage, emptyUsage } from "../../../core/usage.js"; +import type { ContextTreeNode } from "../../../session/context/context-tree.js"; +import { addAssistantUsage, emptyUsage } from "../../../session/context/usage.js"; import { formatTokenCount } from "../agent-activity.js"; import { theme } from "../theme/theme.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/conversation-components.ts b/packages/coding-agent/src/modes/interactive/components/conversation-components.ts index d2d7026f09..d7e6a03a78 100644 --- a/packages/coding-agent/src/modes/interactive/components/conversation-components.ts +++ b/packages/coding-agent/src/modes/interactive/components/conversation-components.ts @@ -12,7 +12,7 @@ import { REFINEMENT_OUTCOME_CUSTOM_TYPE, SESSION_SLASH_COMMAND_CUSTOM_TYPE, SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, -} from "../../../core/messages.js"; +} from "../../../session/context/messages.js"; import { AgentMessageComponent } from "./agent-message.js"; import { AssistantMessageComponent } from "./assistant-message.js"; import { BashExecutionComponent } from "./bash-execution.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/custom-message.ts b/packages/coding-agent/src/modes/interactive/components/custom-message.ts index a6840c347b..b8e5fb6614 100644 --- a/packages/coding-agent/src/modes/interactive/components/custom-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/custom-message.ts @@ -2,7 +2,7 @@ import type { TextContent } from "@earendil-works/pi-ai"; import type { Component } from "@earendil-works/pi-tui"; import { Box, Container, Markdown, type MarkdownTheme, Spacer, Text } from "@earendil-works/pi-tui"; import type { MessageRenderer } from "../../../core/extensions/types.js"; -import type { CustomMessage } from "../../../core/messages.js"; +import type { CustomMessage } from "../../../session/context/messages.js"; import { getMarkdownTheme, theme } from "../theme/theme.js"; export class CustomMessageComponent extends Container { diff --git a/packages/coding-agent/src/modes/interactive/components/injected-prompt-message.ts b/packages/coding-agent/src/modes/interactive/components/injected-prompt-message.ts index f1f96d1ae6..fca20dbafc 100644 --- a/packages/coding-agent/src/modes/interactive/components/injected-prompt-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/injected-prompt-message.ts @@ -20,7 +20,7 @@ import { RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, type RlmChildFailureDetails, type RlmChildTerminalNoticeDetails, -} from "../../../core/messages.js"; +} from "../../../session/context/messages.js"; import { GOAL_CONTEXT_CUSTOM_TYPE, type GoalContextDetails } from "../../../session/goals/contracts.js"; import { getMarkdownTheme, theme } from "../theme/theme.js"; import { expandCollapseHint } from "./keybinding-hints.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/refinement-outcome-message.ts b/packages/coding-agent/src/modes/interactive/components/refinement-outcome-message.ts index 350df9b0f1..98510597d9 100644 --- a/packages/coding-agent/src/modes/interactive/components/refinement-outcome-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/refinement-outcome-message.ts @@ -1,7 +1,7 @@ import { type Component, Spacer, Text } from "@earendil-works/pi-tui"; -import type { RefinementOutcomeMessage } from "../../../core/messages.js"; -import type { AppliedRefinementEdit, HarnessEntry } from "../../../core/refinement/refinement.js"; import { generateDiffString } from "../../../core/tools/edit-diff.js"; +import type { RefinementOutcomeMessage } from "../../../session/context/messages.js"; +import type { AppliedRefinementEdit, HarnessEntry } from "../../../session/refinement/types.js"; import { theme } from "../theme/theme.js"; import { renderRichDiff } from "./diff.js"; import { ExpandableEventMessage } from "./expandable-event-message.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/settings-selector.ts b/packages/coding-agent/src/modes/interactive/components/settings-selector.ts index 2820ba1597..f85295428f 100644 --- a/packages/coding-agent/src/modes/interactive/components/settings-selector.ts +++ b/packages/coding-agent/src/modes/interactive/components/settings-selector.ts @@ -10,8 +10,8 @@ import { Spacer, Text, } from "@earendil-works/pi-tui"; -import type { IdleEvictionMinutes } from "../../../core/session-action-store.js"; import type { MermaidRenderingMode, WarningSettings } from "../../../core/settings-manager.js"; +import type { IdleEvictionMinutes } from "../../daemon/workers/residency-policy.js"; import { getSelectListTheme, getSettingsListTheme, theme } from "../theme/theme.js"; import { DynamicBorder } from "./dynamic-border.js"; diff --git a/packages/coding-agent/src/modes/interactive/components/slash-command-result-message.ts b/packages/coding-agent/src/modes/interactive/components/slash-command-result-message.ts index 9b91bc4a14..553c737c4d 100644 --- a/packages/coding-agent/src/modes/interactive/components/slash-command-result-message.ts +++ b/packages/coding-agent/src/modes/interactive/components/slash-command-result-message.ts @@ -1,5 +1,5 @@ import { Box, Container, Text } from "@earendil-works/pi-tui"; -import type { SessionSlashCommandResultMessage } from "../../../core/messages.js"; +import type { SessionSlashCommandResultMessage } from "../../../session/context/messages.js"; import { theme } from "../theme/theme.js"; /** Renders a durable session-command outcome with user-message spacing. */ diff --git a/packages/coding-agent/src/modes/interactive/interactive-mode-services.ts b/packages/coding-agent/src/modes/interactive/interactive-mode-services.ts index 8d79acd828..d90f9dae62 100644 --- a/packages/coding-agent/src/modes/interactive/interactive-mode-services.ts +++ b/packages/coding-agent/src/modes/interactive/interactive-mode-services.ts @@ -1,10 +1,10 @@ -import type { AgentSession, ExtensionBindings } from "../../core/agent-session.js"; import type { AgentSessionRuntime } from "../../core/agent-session-runtime.js"; import type { AgentSessionServices } from "../../core/agent-session-services.js"; import type { ExtensionCommandContext, ExtensionRunner, ToolDefinition } from "../../core/extensions/index.js"; import type { ModelRegistry } from "../../core/model-registry.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; +import type { AgentSession, ExtensionBindings } from "../../session/agent-session.js"; import type { Theme } from "./theme/theme.js"; /** diff --git a/packages/coding-agent/src/modes/interactive/interactive-mode.ts b/packages/coding-agent/src/modes/interactive/interactive-mode.ts index 86652ea322..ae648c8324 100644 --- a/packages/coding-agent/src/modes/interactive/interactive-mode.ts +++ b/packages/coding-agent/src/modes/interactive/interactive-mode.ts @@ -102,21 +102,6 @@ import { FooterDataProvider, type ReadonlyFooterDataProvider } from "../../core/ import type { KernelSentAgentMessage } from "../../core/kernel/index.js"; import { type AppKeybinding, KeybindingsManager } from "../../core/keybindings.js"; import { runMcpManagementCommand } from "../../core/mcp/mcp-command.js"; -import { - ASYNC_BASH_COMPLETION_PREVIEW_LABEL, - bashOutputToText, - COMPACTION_OUTCOME_CUSTOM_TYPE, - type CustomMessage, - createHeartbeatPromptMessage, - HEARTBEAT_PROMPT_PREVIEW_LABEL, - isCompactionOutcomeMessage, - isRefinementOutcomeMessage, - isSessionSlashCommandMessage, - isSessionSlashCommandResultMessage, - REFINEMENT_OUTCOME_CUSTOM_TYPE, - SESSION_SLASH_COMMAND_CUSTOM_TYPE, - SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, -} from "../../core/messages.js"; import { findExactModelReferenceMatch, resolveModelScopeFromModels } from "../../core/model-resolver.js"; import { parseNewSessionCommand } from "../../core/new-session-command.js"; import { resolvePrimeAgentTracesBaseUrl } from "../../core/prime-inference-auth.js"; @@ -139,6 +124,21 @@ import { type TelemetryOnboardingOutcome, } from "../../core/telemetry.js"; import { type TruncationResult, truncateTail } from "../../core/tools/truncate.js"; +import { + ASYNC_BASH_COMPLETION_PREVIEW_LABEL, + bashOutputToText, + COMPACTION_OUTCOME_CUSTOM_TYPE, + type CustomMessage, + createHeartbeatPromptMessage, + HEARTBEAT_PROMPT_PREVIEW_LABEL, + isCompactionOutcomeMessage, + isRefinementOutcomeMessage, + isSessionSlashCommandMessage, + isSessionSlashCommandResultMessage, + REFINEMENT_OUTCOME_CUSTOM_TYPE, + SESSION_SLASH_COMMAND_CUSTOM_TYPE, + SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, +} from "../../session/context/messages.js"; import { emptyGoalState, formatGoalUsage, diff --git a/packages/coding-agent/src/modes/interactive/resume-hint.ts b/packages/coding-agent/src/modes/interactive/resume-hint.ts index a3918fdda1..c00b05449c 100644 --- a/packages/coding-agent/src/modes/interactive/resume-hint.ts +++ b/packages/coding-agent/src/modes/interactive/resume-hint.ts @@ -1,7 +1,7 @@ import { existsSync } from "node:fs"; import chalk from "chalk"; import { APP_NAME } from "../../config.js"; -import type { SessionStats } from "../../core/session-stats.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; export type ResumeHintStats = Pick; diff --git a/packages/coding-agent/src/modes/print-mode.ts b/packages/coding-agent/src/modes/print-mode.ts index c586de9cb0..08b0b9fdaa 100644 --- a/packages/coding-agent/src/modes/print-mode.ts +++ b/packages/coding-agent/src/modes/print-mode.ts @@ -8,8 +8,12 @@ import type { ImageContent } from "@earendil-works/pi-ai"; import type { AgentSessionRuntime } from "../core/agent-session-runtime.js"; -import { type AgentAutonomousStatus, type AutonomousLimitReason, autonomousLimitReason } from "../core/autonomous.js"; import { flushRawStdout, writeRawStdout } from "../core/output-guard.js"; +import { + type AgentAutonomousStatus, + type AutonomousLimitReason, + autonomousLimitReason, +} from "../session/autonomy/autonomous.js"; import { killTrackedDetachedChildren } from "../utils/shell.js"; import { InProcessAgentConnection } from "./agent-connection/in-process-agent-connection.js"; import type { AgentConnection } from "./agent-connection/types.js"; diff --git a/packages/coding-agent/src/modes/rpc/rpc-client.ts b/packages/coding-agent/src/modes/rpc/rpc-client.ts index f718809688..bef89ff53d 100644 --- a/packages/coding-agent/src/modes/rpc/rpc-client.ts +++ b/packages/coding-agent/src/modes/rpc/rpc-client.ts @@ -9,15 +9,15 @@ import type { AgentEvent, AgentMessage, ThinkingLevel } from "@earendil-works/pi import type { ImageContent } from "@earendil-works/pi-ai"; import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus } from "../../core/agent-messages.js"; import type { BashResult } from "../../core/bash-executor.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; import type { AgentCronJob, AgentHeartbeatDeliveryMode, AgentHeartbeatManagementAction, AgentHeartbeatUpdateAction, } from "../../core/cron-jobs.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; -import type { SessionStats } from "../../core/session-stats.js"; +import type { CompactionResult } from "../../session/compaction/types.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; +import type { RefinementResult } from "../../session/refinement/types.js"; import { spawnHidden } from "../../utils/child-process.js"; import type { AgentConnectionHeartbeat } from "../agent-connection/types.js"; import { attachJsonlLineReader, serializeJsonLine } from "./jsonl.js"; diff --git a/packages/coding-agent/src/modes/rpc/rpc-types.ts b/packages/coding-agent/src/modes/rpc/rpc-types.ts index f7119fc508..2d2205780d 100644 --- a/packages/coding-agent/src/modes/rpc/rpc-types.ts +++ b/packages/coding-agent/src/modes/rpc/rpc-types.ts @@ -9,17 +9,17 @@ import type { AgentEvent, AgentMessage, ThinkingLevel } from "@earendil-works/pi import type { ImageContent, Model } from "@earendil-works/pi-ai"; import type { AgentSessionMessageReceipt, AgentSessionMessageSafetyStatus } from "../../core/agent-messages.js"; import type { BashResult } from "../../core/bash-executor.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; import type { AgentCronJob, AgentHeartbeatDeliveryMode, AgentHeartbeatManagementAction, AgentHeartbeatUpdateAction, } from "../../core/cron-jobs.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; -import type { SessionActionSnapshot } from "../../core/session-action-store.js"; -import type { SessionStats } from "../../core/session-stats.js"; +import type { CompactionResult } from "../../session/compaction/types.js"; +import type { SessionStats } from "../../session/context/session-stats.js"; import type { GoalState } from "../../session/goals/contracts.js"; +import type { SessionActionSnapshot } from "../../session/input/action-store.js"; +import type { RefinementResult } from "../../session/refinement/types.js"; import type { AgentConnectionHeartbeat, AgentConnectionSourceInfo } from "../agent-connection/types.js"; // ============================================================================ diff --git a/packages/coding-agent/src/package-manager-cli.ts b/packages/coding-agent/src/package-manager-cli.ts index d85251aba9..c2b12dd10f 100644 --- a/packages/coding-agent/src/package-manager-cli.ts +++ b/packages/coding-agent/src/package-manager-cli.ts @@ -44,10 +44,7 @@ import { type SelfUpdateCommand, VERSION, } from "./config.js"; -import type { SessionActionRecoverySnapshot } from "./core/agent-session.js"; -import { SESSION_ACTION_RECOVERY_FORMAT_VERSION } from "./core/agent-session.js"; import type { AgentSessionRuntimeMetadata } from "./core/agent-session-runtime.js"; -import { type CustomMessage, isSessionSlashCommand } from "./core/messages.js"; import { DefaultPackageManager } from "./core/package-manager.js"; import { SettingsManager } from "./core/settings-manager.js"; import { DaemonClient, type DaemonHello } from "./modes/daemon/daemon-client.js"; @@ -69,6 +66,9 @@ import { DAEMON_WORKER_ACTIVE_SESSION_ID_ENV, DAEMON_WORKER_SUPERVISOR_SOCKET_ENV, } from "./modes/daemon/daemon-worker-protocol.js"; +import type { SessionActionRecoverySnapshot } from "./session/agent-session.js"; +import { SESSION_ACTION_RECOVERY_FORMAT_VERSION } from "./session/agent-session.js"; +import { type CustomMessage, isSessionSlashCommand } from "./session/context/messages.js"; import { shouldUseWindowsShell } from "./utils/child-process.js"; import { getLatestPiRelease, isNewerPackageVersion } from "./utils/version-check.js"; diff --git a/packages/coding-agent/src/session/agent-session.ts b/packages/coding-agent/src/session/agent-session.ts new file mode 100644 index 0000000000..b175e9ee80 --- /dev/null +++ b/packages/coding-agent/src/session/agent-session.ts @@ -0,0 +1,2934 @@ +import type { + Agent, + AgentContext, + AgentMessage, + AgentState, + AgentTool, + ThinkingLevel, +} from "@earendil-works/pi-agent-core"; +import type { AssistantMessage, Model, ServiceTier } from "@earendil-works/pi-ai"; +import { clampThinkingLevel, cleanupSessionResources, supportsFastMode } from "@earendil-works/pi-ai"; +import { + AGENT_MESSAGE_SKILL_NAME, + type AgentSessionMessageController, + type AgentSessionMessageReceipt, +} from "../core/agent-messages.js"; +import { + AGENT_OBSERVE_SKILL_NAME, + type AgentObserveAgentSnapshot, + type AgentObserveController, + type AgentObserveListResult, + type AgentObserveRecentMessagesResult, + ORCHESTRATION_HEARTBEAT_SKILL_NAME, +} from "../core/agent-observe.js"; +import type { BashResult } from "../core/bash-executor.js"; +import type { AgentCronJob, AgentRlmHeartbeatController } from "../core/cron-jobs.js"; +import type { + ExtensionRunner, + ReplacedSessionContext, + SessionStartEvent, + ToolDefinition, + ToolInfo, +} from "../core/extensions/index.js"; +import type { HostRequestHandlers } from "../core/kernel/index.js"; +import type { AcpMcpServerConfig } from "../core/mcp/acp-mcp-types.js"; +import type { McpManager } from "../core/mcp/mcp-manager.js"; +import type { ModelRegistry } from "../core/model-registry.js"; +import type { PromptTemplate } from "../core/prompt-templates.js"; +import { providerRetryPolicy } from "../core/provider-retry.js"; +import type { ResourceLoader } from "../core/resource-loader.js"; +import { SemanticEdgeRecorder, semanticEdgeLedgerPath, wrapStreamFnWithSemanticEdges } from "../core/semantic-edges.js"; +import type { SessionManager } from "../core/session-manager.js"; +import type { SettingsManager } from "../core/settings-manager.js"; +import { getPythonSkillRuntimeInfo, type Skill } from "../core/skills.js"; +import type { IpythonKernelProvisioner } from "../core/tools/ipython.js"; +import type { AgentAutonomousConfig } from "./autonomy/autonomous.js"; +import { SessionAutonomousContinuation } from "./autonomy/continuation.js"; +import { createChildSessionDir, createInlineChildRuntime } from "./children/child-runtime.js"; +import { SessionChildState } from "./children/child-state.js"; +import { + compactRlmText, + type RlmChildAgentSnapshot, + type RlmChildAgentStatus, + rlmChildLabel, +} from "./children/child-types.js"; +import { SessionChildUsage } from "./children/child-usage.js"; +import { SessionChildren } from "./children/children.js"; +import type { + CreateRlmSubagentRuntimeOptions, + RlmCreateSessionResult, + RlmDeleteSubagentResult, + RlmListSubagentsResult, + RlmSpawnHandle, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "./children/runtime-contracts.js"; +import { SessionCompaction } from "./compaction/controller.js"; +import { + type CompactionExecutionHost, + type CompactionExecutionOptions, + performSessionCompaction, +} from "./compaction/execution.js"; +import { COMPACT_SKILL_NAME, type CompactionResult } from "./compaction/types.js"; +import { type ContextViewChild, SessionContextView } from "./context/context-view.js"; +import { SessionExport } from "./context/export.js"; +import { SessionHarnessContext } from "./context/harness-context.js"; +import { SessionHistoryNavigation } from "./context/history-navigation.js"; +import { type CustomMessage, createHeartbeatPromptMessage, type RefinementSource } from "./context/messages.js"; +import { SessionPendingContext } from "./context/pending-context.js"; +import type { BuildSystemPromptOptions } from "./context/system-prompt.js"; +import { calculateContextTokens } from "./context/token-estimate.js"; +import { type ExtensionBindings, installExtensionToolHooks, SessionExtensions } from "./extensions/extensions.js"; +import { SessionGoalContinuation } from "./goals/continuation.js"; +import { + createGoalContextMessage, + GOAL_CONTEXT_CUSTOM_TYPE, + GOAL_SKILL_NAME, + type GoalState, +} from "./goals/contracts.js"; +import { GoalController } from "./goals/controller.js"; +import { createGoalPersistence } from "./goals/persistence.js"; +import { SessionActionQueue } from "./input/action-queue.js"; +import { SessionActionRecovery } from "./input/action-recovery.js"; +import { ActionStore, type RuntimeActivity } from "./input/action-store.js"; +import { SessionCommitFence, type SessionCommitLease } from "./input/commit-fence.js"; +import { SessionInputAdmission } from "./input/input-admission.js"; +import { SessionInputCheckpoints } from "./input/input-checkpoints.js"; +import { SessionInputDispatcher } from "./input/input-dispatcher.js"; +import { SessionInputScheduler } from "./input/input-scheduler.js"; +import { SessionMessageDelivery } from "./input/message-delivery.js"; +import { type QueuedSessionAction, visibleSessionActionProjection } from "./input/prepared-actions.js"; +import { type PromptOptions, SessionPromptSubmission } from "./input/prompt-submission.js"; +import { SubmissionNormalizer } from "./input/submission-normalization.js"; +import { handleRlmHeartbeatHostRequest } from "./kernel/heartbeat-host-requests.js"; +import { SessionKernel } from "./kernel/kernel.js"; +import { KernelEnvironment } from "./kernel/kernel-environment.js"; +import { createSessionKernelHostHandlers } from "./kernel/kernel-host-handlers.js"; +import { handleAgentMessageHostRequest } from "./kernel/message-host-requests.js"; +import { handleAgentObserveHostRequest } from "./kernel/observe-host-requests.js"; +import { SessionModelSelection } from "./models/model-selection.js"; +import { type AutoRefineReviewer, SessionRefinement } from "./refinement/controller.js"; +import { REFINE_SKILL_NAME, type RefinementResult } from "./refinement/types.js"; +import { type ExecuteBashOptions, type RunUserBashOptions, SessionBash } from "./tools/bash.js"; +import { SessionTools } from "./tools/tools.js"; +import { SessionCommandExecution } from "./turns/command-execution.js"; +import { SessionContinuation } from "./turns/continuation.js"; +import { SessionEvents } from "./turns/events.js"; +import { SessionRetry } from "./turns/retry.js"; +import { SessionTurnExecution } from "./turns/turn-execution.js"; +import { SessionTurnPolicy } from "./turns/turn-policy.js"; +import { TurnPreparer } from "./turns/turn-preparation.js"; + +export { type ParsedSkillBlock, parseSkillBlock } from "../core/skill-blocks.js"; +export type { + RlmChildAgentActivity, + RlmChildAgentSnapshot, + RlmChildAgentStatus, +} from "./children/child-types.js"; +export { compactRlmText, rlmChildLabel } from "./children/child-types.js"; +export type { CompactionReason } from "./compaction/controller.js"; +export { CompactionSkippedError } from "./compaction/execution.js"; +export type { SessionStats } from "./context/session-stats.js"; +export type { GoalState, GoalStatus } from "./goals/contracts.js"; +export { + SESSION_ACTION_RECOVERY_FORMAT_VERSION, + type SessionActionRecoveryAction, + type SessionActionRecoveryPayload, + type SessionActionRecoveryRecord, + type SessionActionRecoverySnapshot, +} from "./input/prepared-actions.js"; +export { RefineSkippedError } from "./refinement/controller.js"; +export type { AgentSessionEvent, AgentSessionEventListener } from "./turns/events.js"; +export type { TurnExecutionPolicy } from "./turns/turn-preparation.js"; + +export interface AgentSessionConfig { + agent: Agent; + sessionManager: SessionManager; + settingsManager: SettingsManager; + serviceTierPreference?: ServiceTier; + cwd: string; + agentDir?: string; + scopedModels?: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>; + resourceLoader: ResourceLoader; + customTools?: ToolDefinition[]; + modelRegistry: ModelRegistry; + initialActiveToolNames?: string[]; + allowedToolNames?: string[]; + /** + * Whether the built-in long-running goals feature is available: the bundled + * goal skill in the Python kernel, its goal.* host handlers, and /goal. + * Default: true. + */ + includeGoals?: boolean; + agentMessageController?: AgentSessionMessageController; + agentObserveController?: AgentObserveController; + /** + * Whether the bundled compact skill and its compact.* host handlers are + * available to the model. Default: the compaction.agentCallable setting. + */ + includeCompactSkill?: boolean; + /** + * Optional host-side controller for the bundled rlm-heartbeat Python skill. + * When omitted, rlm_heartbeat.* host requests are unavailable. + */ + rlmHeartbeatController?: AgentRlmHeartbeatController; + /** + * Optional MCP integration manager. When present, its mcp.* host requests + * (refresh, begin_login) are exposed to the kernel. + */ + mcpManager?: McpManager; + /** + * Override base tools (useful for custom runtimes). + * + * These are synthesized into minimal ToolDefinitions internally so AgentSession can keep + * a definition-first registry even when callers provide plain AgentTool instances. + */ + baseToolsOverride?: Record; + extensionRunnerRef?: { current?: ExtensionRunner }; + sessionStartEvent?: SessionStartEvent; + rlmDepth?: number; + rlmMaxDepth?: number; + rlmSessionDir?: string; + rlmParentNodeId?: string; + rlmParentAgent?: string; + semanticParentSessionId?: string; + semanticSpawnedByRequestId?: string; + subagentRuntimeHost?: SubagentRuntimeHost; + autonomous?: AgentAutonomousConfig; + prewarmIpythonKernel?: boolean; + autoRefineReviewer?: AutoRefineReviewer; + /** + * When true, auto-refine runs synchronously between turns at the + * shouldStopAfterTurn boundary instead of in the background after + * agent_end. Used for print/headless autonomous runs so refinement + * never overlaps the primary model request. Default: false. + */ + serializedRefine?: boolean; + /** + * Initial goal to seed at session creation. Only applied when rlmDepth + * is 0 and no persisted thread_goal_state entry exists in the branch. + */ + initialGoal?: { objective: string; tokenBudget?: number }; +} + +export type { ExtensionBindings } from "./extensions/extensions.js"; +export type { PromptOptions } from "./input/prompt-submission.js"; +export type { ModelCycleResult } from "./models/model-selection.js"; +export type { AutoRefineReviewer, AutoRefineReviewRequest } from "./refinement/controller.js"; + +import type { RlmMaxDepthStatus, SetRlmMaxDepthResult } from "./children/max-depth.js"; + +export type { RlmMaxDepthSource, RlmMaxDepthStatus, SetRlmMaxDepthResult } from "./children/max-depth.js"; + +export class AgentSession { + private readonly _tools: SessionTools; + private readonly _extensions: SessionExtensions; + private readonly _kernel: SessionKernel; + private readonly _kernelEnvironment: KernelEnvironment; + private get _extensionRunner(): ExtensionRunner { + return this._extensions.runner; + } + private get _ipythonKernelProvisioner(): IpythonKernelProvisioner | undefined { + return this._kernel.provisioner; + } + private get _rlmSessionDir(): string | undefined { + return this._kernelEnvironment.sessionDir; + } + private get _allowedToolNames(): ReadonlySet | undefined { + return this._tools.allowedToolNames; + } + private get _customTools(): ToolDefinition[] { + return this._tools.customTools; + } + private get _toolRegistry(): ReadonlyMap { + return this._tools.registry; + } + private get _baseSystemPrompt(): string { + return this._tools.baseSystemPrompt; + } + private set _baseSystemPrompt(prompt: string) { + this._tools.baseSystemPrompt = prompt; + } + private get _baseSystemPromptOptions(): BuildSystemPromptOptions { + return this._tools.baseSystemPromptOptions; + } + private readonly _childState: SessionChildState; + + private readonly _childUsage = new SessionChildUsage({ + sessionManager: { + getEntries: () => this.sessionManager.getEntries(), + appendChildUsageAttribution: (...args) => this.sessionManager.appendChildUsageAttribution(...args), + }, + invalidateOwnUsage: () => this._invalidateOwnUsage(), + afterParentDrain: (flush) => { + this._events.enqueue(flush); + }, + }); + private readonly _children = new SessionChildren({ + isDisposed: () => this._disposed || this._disposing, + isInputSuspended: () => this._inputScheduler.suspended, + isStreaming: () => this.isStreaming, + isSessionActive: () => this.isSessionActive, + getMessageController: () => this._agentMessageController, + getDepth: () => this._childState.depth, + getMaxDepth: () => this._childState.maxDepth, + getParentNodeId: () => this._rlmParentNodeId, + getCwd: () => this._cwd, + getSessionId: () => this.sessionId, + getSessionName: () => this.sessionName, + getSessionFile: () => this.sessionFile, + getThinkingLevel: () => this.thinkingLevel, + getSemanticEdges: () => this._semanticEdges, + getChildOwner: (child) => child._children, + getParentReplyCount: (child) => child._childState.replyCount, + getChildSessionDir: (child) => child._rlmSessionDir, + listRlmSubagents: () => this.listRlmSubagents(), + deleteRlmSubagent: (target) => this.deleteRlmSubagent(target), + registerRlmChildSession: (id, child) => this.registerRlmChildSession(id, child), + resolveModel: (reference, target) => this._resolveRlmSubagentModel(reference, target), + createSessionDir: () => this._createChildRlmSessionDir(), + createRuntimeOptions: (request) => this._createRlmSubagentRuntimeOptions(request), + createRuntime: (options) => this._createRlmSubagentRuntime(options), + createUsageTracker: () => this._childUsage.createTracker(this._findLastAssistantMessage()), + hasDeferredTerminalNotices: () => this._hasDeferredRlmTerminalNotices(), + waitForHeadlessIdle: () => this.waitForHeadlessIdle(), + waitForActivityChange: (signal) => this._waitForSessionActivityChange(signal), + deliverTerminalNotice: (message) => this._deferRlmTerminalNotice(message), + emit: (event) => this._emit(event), + onSettled: () => this._maybeResumeGoalContinuationAfterRlmWork(), + }); + + private readonly _turnPolicy = new SessionTurnPolicy({ + steeringStopPending: () => this._steeringStopPending, + stopGoalForTerminalMessage: (message) => this._stopGoalContinuationForTerminalMessage(message), + getGoals: () => this._goals, + accountAssistantBudget: (message) => this._goalContinuation.accountAssistantBudget(message), + getRefinement: () => this._refinement, + getEventQueue: () => this._events.queue, + getCompaction: () => this._compaction, + getMessages: () => this.agent.state.messages, + getSettings: () => this.settingsManager, + getModel: () => this.model, + getStore: () => this.sessionManager, + queueThresholdGoal: (message) => this._queueGoalContinuationForThresholdCompaction(message), + queueThresholdAutonomous: (message) => this._queueAutonomousContinuationForThresholdCompaction(message), + getQueuedCount: () => this.queuedActionCount, + getArrivalEpoch: () => this._inputAdmission.arrivalEpoch, + getGoalMessages: (context, signal) => this._getGoalContinuationMessages(context, signal), + getAutonomous: () => this._autonomousContinuation, + snapshotAutonomous: () => this._snapshotAutonomousRuntimeState(), + restoreAutonomous: (snapshot) => this._restoreAutonomousRuntimeSnapshot(snapshot), + }); + private readonly _turnExecution = new SessionTurnExecution({ + getPreparer: () => this._turnPreparer, + getFence: () => this._commitFence, + acquireFence: () => this._acquireSessionActionCommitFence(), + isDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), + isStreaming: () => this.isStreaming, + getBasePrompt: () => this._baseSystemPrompt, + getBasePromptOptions: () => this._baseSystemPromptOptions, + refreshExtensionSystemPrompt: (prompt, snapshot) => this._refreshExtensionSystemPrompt(prompt, snapshot), + getExtensions: () => this._extensionRunner, + getAgent: () => this.agent, + takeNextTurnMessages: () => this._takePendingNextTurnMessages(), + restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), + consumePendingDigest: () => this._harnessContext.consumePendingDigest(), + rearmDigest: () => this._harnessContext.rearmDigest(), + getDigest: () => this._harnessDigest(), + getLatestDigest: () => this._latestContextHarnessDigest(), + suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), + runSuppressed: (run) => this._runWithAutonomousContinuationSuppressed(run), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + emitQueueUpdate: () => this._emitQueueUpdate(), + hasCancelledCapture: () => this._hasCancelledDispatchCapture(), + getEventQueue: () => this._events.queue, + waitForRetry: () => this.waitForRetry(), + forgetContinuations: (messages) => this._forgetConsumedPostCompactionContinuations(messages), + }); + /** Session-owned actions. Items are never fed into Agent.steer/followUp. */ + private readonly _actionStore = new ActionStore(); + private readonly _inputCheckpoints = new SessionInputCheckpoints(this._actionStore, { + getFence: () => this._commitFence, + getScheduler: () => this._inputScheduler, + isBusyForInputPump: () => this._isBusyForSessionInput("pump"), + getEventQueue: () => this._events.queue, + acquireFence: (signal) => this._acquireSessionActionCommitFence(signal), + getStore: () => this.sessionManager, + assertAdmissionAvailable: () => this._assertSessionActionAdmissionAvailable(), + getContinuation: () => this._continuation, + scheduleInput: () => this._scheduleSessionInputPump(), + getAgent: () => this.agent, + getUnfinishedCount: () => this.unfinishedActionCount, + waitForIdle: () => this.waitForIdle(), + }); + private readonly _promptSubmission = new SessionPromptSubmission(this._actionStore, { + promptInjectedMessage: (text, message, options) => this._promptInjectedMessage(text, message, options), + waitForActivityChange: (signal) => this._waitForSessionActivityChange(signal), + queueAgentMessagePrompt: (text, streamingBehavior, customMessage) => + this.queueAgentMessagePrompt(text, streamingBehavior, customMessage), + getScheduler: () => this._inputScheduler, + getFence: () => this._commitFence, + isStreaming: () => this.isStreaming, + isCompacting: () => this.isCompacting, + isRetrying: () => this.isRetrying, + isBashRunning: () => this.isBashRunning, + resumeAdmission: () => this._resumeSessionInputAdmission(), + assertAdmissionAvailable: () => this._assertSessionActionAdmissionAvailable(), + acquireAdmissionFence: (signal) => this._acquireDirectTurnAdmissionFence(signal), + normalize: (text, images, policy) => this._normalizeSubmission(text, images, policy), + settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), + canStartImmediately: () => this._canStartSessionActionImmediately(), + admit: (action, options) => this._admitSessionInput(action, options), + waitForInputIdle: () => this.waitForSessionInputIdle(), + isBusy: (point) => this._isBusyForSessionInput(point), + takeNextTurnMessages: () => this._takePendingNextTurnMessages(), + restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), + appendNextTurnMessage: (message) => this._pendingContext.appendMessages(message), + getActivity: () => this._runtimeActivity(), + suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), + observeDeferral: (action) => this._observeSessionActionDeferral(action), + rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), + cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), + emitQueueUpdate: () => this._emitQueueUpdate(), + getClearEpoch: () => this._actionQueue.clearEpoch, + queuePrompt: (schedule, text, images, options) => this._queuePreparedPrompt(schedule, text, images, options), + resetParentReply: () => { + this._childState.resetReply(); + }, + getAgent: () => this.agent, + getStore: () => this.sessionManager, + emit: (event) => this._emit(event), + }); + private readonly _inputAdmission = new SessionInputAdmission(this._actionStore, { + getScheduler: () => this._inputScheduler, + isDisposed: () => this._disposed, + isDisposing: () => this._disposing, + isStreaming: () => this.isStreaming, + rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), + emitQueueUpdate: () => this._emitQueueUpdate(), + resumeAdmission: () => this._resumeSessionInputAdmission(), + scheduleInput: () => this._scheduleSessionInputPump(), + suppressForMessage: (message) => this._markAutonomousContinuationSuppressed(message), + }); + private readonly _pendingContext = new SessionPendingContext(this._actionStore, { + getScheduler: () => this._inputScheduler, + getFence: () => this._commitFence, + isDisposed: () => this._disposed, + isDisposing: () => this._disposing, + admit: (action, options) => this._admitSessionInput(action, options), + scheduleInput: () => this._scheduleSessionInputPump(), + addCheckpointWaiter: (waiter) => this._inputCheckpoints.add(waiter), + removeCheckpointWaiter: (waiter) => this._inputCheckpoints.remove(waiter), + acquireFence: (signal) => this._acquireSessionActionCommitFence(signal), + cancelActions: (predicate, error, candidates) => this._cancelSessionActions(predicate, error, candidates), + }); + private readonly _actionQueue = new SessionActionQueue(this._actionStore, { + formatLabel: (text) => compactRlmText(text), + getScheduler: () => this._inputScheduler, + getAgent: () => this.agent, + rearmDigest: () => this._harnessContext.rearmDigest(), + restoreNextTurnMessages: (messages) => this._pendingContext.prependMessages(messages), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + emitQueueUpdate: () => this._emitQueueUpdate(), + settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), + rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), + admit: (action, options) => this._admitSessionInput(action, options), + queuePrompt: (schedule, text, images, options) => this._queuePreparedPrompt(schedule, text, images, options), + resumeQueuedWork: () => this.resumeQueuedWork(), + }); + private readonly _actionRecovery = new SessionActionRecovery(this._actionStore, { + isTerminalNoticeAction: (action) => this._isRlmTerminalNoticeAction(action), + retainTerminalNotice: (id) => this._pendingContext.retainTerminalNotice(id), + releaseTerminalNotice: (id) => this._pendingContext.releaseTerminalNotice(id), + admit: (action, options) => this._admitSessionInput(action, options), + }); + private readonly _commandExecution = new SessionCommandExecution(this._actionStore, { + getFence: () => this._commitFence, + acquireFence: () => this._acquireSessionActionCommitFence(), + getRefinement: () => this._refinement, + isDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), + getActivity: () => this._runtimeActivity(), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + emitQueueUpdate: () => this._emitQueueUpdate(), + settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), + rejectAgentMessage: (id, error) => this._rejectAgentMessage(id, error), + compact: (instructions, options) => this.compact(instructions, options), + refine: (options, internal) => this.refine(options, internal), + handleGoalCommand: (text, images) => this._handleGoalSlashCommand(text, images), + handleAutonomousCommand: (text) => this._handleAutonomousSlashCommand(text), + getGoalState: () => this._goals.state, + getStore: () => this.sessionManager, + getAgent: () => this.agent, + emit: (event) => this._emit(event), + }); + private readonly _events = new SessionEvents(this._actionStore, { + getAgent: () => this.agent, + getStore: () => this.sessionManager, + getExtensions: () => this._extensionRunner, + getRetry: () => this._retry, + getCompaction: () => this._compaction, + getRefinement: () => this._refinement, + addAutonomousUsage: (usage) => this._autonomousContinuation.recordUsage(usage), + applyLateMessages: (message) => this._applyLateIpythonSentAgentMessages(message), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), + getSnapshot: () => this.getSessionActionSnapshot(), + accountAssistantBudget: (message) => this._goalContinuation.accountAssistantBudget(message), + finishGoal: (message) => this._finishGoalForTerminalAssistantMessage(message), + checkCompaction: (message) => this._checkCompaction(message), + }); + private readonly _messageDelivery = new SessionMessageDelivery(this._actionStore, { + isDisposed: () => this._disposed, + getMessages: () => this.agent.state.messages, + getStore: () => this.sessionManager, + enqueue: (work) => this._events.enqueue(work), + emit: (event) => this._emit(event), + promptUntilAccepted: (text, options) => this.promptUntilAccepted(text, options), + cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), + }); + private readonly _submissionNormalizer = new SubmissionNormalizer({ + getExtensions: () => this._extensionRunner, + getPrompts: () => this.promptTemplates, + getSkills: () => this.resourceLoader.getSkills().skills, + }); + private readonly _refinement: SessionRefinement; + readonly agent: Agent; + readonly sessionManager: SessionManager; + readonly settingsManager: SettingsManager; + private readonly _export: SessionExport; + private readonly _contextView: SessionContextView; + private readonly _modelSelection: SessionModelSelection; + private get _scopedModels() { + return this._modelSelection.scopedModels; + } + + private readonly _inputScheduler = new SessionInputScheduler({ + canSchedule: () => !this._disposed && !this._disposing && this._hasSelectableSessionInput(), + run: (epoch) => this._inputDispatcher.run(epoch), + }); + private readonly _inputDispatcher = new SessionInputDispatcher(this._actionStore, { + isDisposed: () => this._disposed || this._disposing, + getEpoch: () => this._inputScheduler.epoch, + getActivity: () => this._runtimeActivity(), + isBusy: () => this._isBusyForSessionInput("pump"), + isHandoffDeferred: (epoch) => this._isSessionInputHandoffDeferred(epoch), + getDeliveryMode: (delivery) => (delivery === "next_turn_boundary" ? this.steeringMode : this.followUpMode), + waitForAgentIdle: () => this.agent.waitForIdle(), + hasCancelledDispatchCapture: () => this._hasCancelledDispatchCapture(), + getEventQueue: () => this._events.queue, + waitForRefinement: () => this._refinement._waitForRefineIdle(), + getTranscript: () => this.agent.state.messages, + startTurns: (actions, epoch) => this._startPreparedTurnActions(actions, epoch), + executeCommand: (action, epoch) => this._executeSelectedSessionCommand(action, epoch), + settleAgentMessage: (id, leg, error) => this._settleAgentMessage(id, leg, error), + releaseTurn: (id) => { + this._pendingContext.releaseTerminalNotice(id); + }, + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + emitQueueUpdate: () => this._emitQueueUpdate(), + surfaceError: (error) => this._surfaceSessionInputError(error), + schedule: () => this._scheduleSessionInputPump(), + }); + private readonly _commitFence = new SessionCommitFence(); + private readonly _turnPreparer = new TurnPreparer({ + hasRefinement: () => this._refinement.isApplying, + waitForRefinement: () => this._refinement._waitForRefineIdle(), + flushPendingBash: () => this._flushPendingBashMessages(), + validate: () => this._validateCanStartAgentRun(), + compact: () => this._runPreTurnCompaction(), + pendingModelSelection: () => this._pendingModelSelectEmit(), + }); + // Checkpoint, handoff, and activity waiters share lifecycle-edge notifications to avoid polling. + + private readonly _autonomousContinuation: SessionAutonomousContinuation; + private readonly _goalContinuation: SessionGoalContinuation; + private get _goals(): GoalController { + return this._goalContinuation.controller; + } + + private readonly _compaction = new SessionCompaction({ + includesCompactSkill: () => this._includeCompactSkill, + getContextUsage: () => this.getContextUsage(), + getModel: () => this.model, + isStreaming: () => this.isStreaming, + getSettings: () => this.settingsManager.getCompactionSettings(), + runAutomatic: (reason, willRetry) => this._runAutoCompaction(reason, willRetry), + queueGoalContinuation: (message) => this._queueGoalContinuationForThresholdCompaction(message), + queueAutonomousContinuation: (message) => this._queueAutonomousContinuationForThresholdCompaction(message), + beginRefinementAbort: () => this._refinement.beginAbortedTurnCleanup(), + getRequiredAuth: (model) => this._getRequiredRequestAuth(model), + getAuth: (model) => this._modelRegistry.getApiKeyAndHeaders(model), + perform: (options) => this._performCompaction(options), + disconnect: () => this._disconnectFromAgent(), + reconnect: () => this._reconnectToAgent(), + abortSession: () => this.abort(), + getContinuationState: () => ({ + scheduled: this._continuation.isScheduled, + continueAfterSessionInput: this._continuation.current?.continueAfterSessionInput ?? false, + }), + afterManualCompaction: (signal, scheduled, continueAfterInput) => + this._afterManualCompaction(signal, scheduled, continueAfterInput), + getMessages: () => this.agent.state.messages, + replaceMessages: (messages) => { + this.agent.state.messages = messages; + }, + hasAgentQueuedMessages: () => this.agent.hasQueuedMessages(), + hasPendingSessionWork: () => this.hasPendingSessionWork, + scheduleContinuation: (continueAfterInput) => this._schedulePostCompactionContinue(continueAfterInput), + scheduleRefinement: (willContinue) => this._refinement._scheduleAutoRefineAfterCompaction(willContinue), + takeThresholdAutonomousMessages: () => this._autonomousContinuation.takePendingThresholdMessages(), + getThresholdGoalContinuation: () => this._goalContinuation.thresholdContinuation, + clearAutonomousContinuations: (shouldContinue, messages) => + this._clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction(shouldContinue, messages), + clearGoalContinuation: (message) => this._clearQueuedGoalContinuationAfterCancelledThresholdCompaction(message), + getSessionStore: () => this.sessionManager, + retainUnpersistedOutcome: (message) => { + this._harnessContext.retainOutcome(message); + }, + emit: (event) => this._emit(event), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + scheduleInput: () => this._scheduleSessionInputPump(), + }); + private readonly _compactionExecution: CompactionExecutionHost = { + getSessionStore: () => this.sessionManager, + getSettings: () => this.settingsManager.getCompactionSettings(), + getSemanticEdges: () => this._semanticEdges, + getExtensions: () => this._extensionRunner, + getThinkingLevel: () => this.thinkingLevel, + getRetryPolicy: () => providerRetryPolicy(this.settingsManager), + getSessionId: () => this.sessionId, + getHarnessDigest: () => this._harnessDigest(), + rebuildContext: () => { + this.agent.state.messages = this.sessionManager.buildSessionContext().messages; + this._mergeUnpersistedOutcomes(this.agent.state.messages); + this._restoreLateIpythonSentAgentMessages(); + }, + syncKernelState: () => this._syncKernelStateAfterCompaction(), + reapDeletedChildren: () => this._reapDeletedRlmSubagentRuntimesAfterCompaction(), + }; + + private get _branchSummaryOperation(): Promise | undefined { + return this._history.operation; + } + private readonly _history: SessionHistoryNavigation; + + private readonly _retry = new SessionRetry({ + getRetrySettings: () => this.settingsManager.getRetrySettings(), + getMaxRetryDelayMs: () => this.settingsManager.getProviderRetrySettings().maxRetryDelayMs, + getContextWindow: () => this.model?.contextWindow ?? 0, + getAuthSource: (provider) => this._modelRegistry.getCurrentProviderAuthSourceToken(provider), + markAuthSourceStale: (token) => this._modelRegistry.markProviderAuthSourceStale(token), + markAuthStale: (provider) => this._modelRegistry.markProviderAuthStale(provider), + hasPayloadHooks: () => this._extensionRunner.hasHandlers("before_provider_request"), + prepareTurnRetry: () => this._semanticEdges.prepareTurnRetry(), + clearTurnRetry: () => this._semanticEdges.clearTurnRetry(), + removeLastAssistant: () => { + const messages = this.agent.state.messages; + if (messages.length > 0 && messages[messages.length - 1].role === "assistant") { + this.agent.state.messages = messages.slice(0, -1); + } + }, + continue: () => this.agent.continue(), + waitForIdle: () => this.agent.waitForIdle(), + cancelCompaction: () => { + this._compaction.abortAutomatic(); + this._cancelPostCompactionContinue(); + }, + emit: (event) => this._emit(event), + onResolved: () => { + this._notifySessionInputCheckpointChange(); + this._scheduleSessionInputPump(); + }, + }); + /** Fresh/empty contexts defer digest injection to the first committed turn so untouched sessions stay empty. */ + private readonly _harnessContext: SessionHarnessContext; + + private readonly _bash = new SessionBash({ + getCwd: () => this.sessionManager.getCwd(), + getShellCommandPrefix: () => this.settingsManager.getShellCommandPrefix(), + getShellPath: () => this.settingsManager.getShellPath(), + isStreaming: () => this.isStreaming, + intercept: (event) => this._extensionRunner.emitUserBash(event), + emit: (event) => this._emit(event), + appendMessage: (message) => { + this.agent.state.messages.push(message); + this.sessionManager.appendMessage(message); + }, + onStateChange: () => this._notifySessionInputCheckpointChange(), + onUserBashEnd: () => this._drainQueuedMessagesAfterBash(), + executeBash: (command, onChunk, options) => this.executeBash(command, onChunk, options), + recordBashResult: (command, result, options) => this.recordBashResult(command, result, options), + }); + + private readonly _resourceLoader: ResourceLoader; + private readonly _cwd: string; + private readonly _agentDir?: string; + private readonly _initialActiveToolNames?: string[]; + private readonly _includeGoals: boolean; + private readonly _includeCompactSkill: boolean; + private _rlmHeartbeatController?: AgentRlmHeartbeatController; + private readonly _agentMessageController?: AgentSessionMessageController; + private readonly _agentObserveController?: AgentObserveController; + private readonly _mcpManager?: McpManager; + private _disposed = false; + private readonly _disposeCallbacks = new Set<() => void | Promise>(); + private _disposeCallbacksPromise?: Promise; + // Set at the start of async teardown so a child finishing mid-disposeAsync doesn't + // re-populate the retained map after it's been cleared. + private _disposing = false; + private _disposeAsyncPromise?: Promise; + private readonly _semanticEdges: SemanticEdgeRecorder; + private readonly _rlmParentNodeId?: string; + private readonly _rlmParentAgent?: string; + + private readonly _modelRegistry: ModelRegistry; + + private readonly _continuation = new SessionContinuation({ + waitForAgentIdle: () => this.agent.waitForIdle(), + waitForRetry: () => this.waitForRetry(), + waitForRefinement: () => this._refinement._waitForRefineIdle(), + queuedWorkPauseCount: () => this._inputScheduler.queuedWorkPauseCount, + addCheckpointWaiter: (waiter) => { + this._inputCheckpoints.add(waiter); + }, + removeCheckpointWaiter: (waiter) => { + this._inputCheckpoints.remove(waiter); + }, + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + compactionOperation: () => this._compaction.operation, + isRefinementApplying: () => this._refinement.isApplying, + acquireCommitFence: () => this._acquireSessionActionCommitFence(), + scheduleRefinement: () => this._refinement._scheduleAutoRefineAfterAgentEnd(), + unfinishedActionCount: () => this.unfinishedActionCount, + isInputRequested: () => this._inputScheduler.requested, + scheduleInput: () => this._scheduleSessionInputPump(), + continue: () => this.agent.continue(), + waitForIdleOrSettlement: (token) => this._waitForIdleOrSettlement(token), + removeQueuedMessages: (predicate) => this.agent.removeQueuedMessages(predicate), + followUp: (message) => this.agent.followUp(message), + onMessageConsumed: (message) => { + this._autonomousContinuation.forgetSnapshot(message); + }, + }); + + constructor(config: AgentSessionConfig) { + this.agent = config.agent; + this.sessionManager = config.sessionManager; + this.settingsManager = config.settingsManager; + this._harnessContext = new SessionHarnessContext({ + sessionManager: this.sessionManager, + getMessages: () => this.messages, + getActiveToolNames: () => this.getActiveToolNames(), + getVisibleSkills: () => this._modelVisibleSkills(), + loadHarnessState: () => this._refinement._loadMergedHarnessState(), + applyLateSentMessages: (message) => this._applyLateIpythonSentAgentMessages(message), + }); + this._export = new SessionExport({ + sessionManager: this.sessionManager, + getState: () => this.state, + getTheme: () => this.settingsManager.getTheme(), + getToolDefinition: (name) => this.getToolDefinition(name), + }); + this._contextView = new SessionContextView({ + getContextUsage: () => this.getContextUsage(), + sessionManager: this.sessionManager, + getMessages: () => this.messages, + getModel: () => this.model, + findModel: (provider, modelId) => this._modelRegistry.find(provider, modelId), + subtractUnindexedChildUsage: (ownUsage, entries) => this._childUsage.subtractUnindexed(ownUsage, entries), + getRlmSessionDir: () => this._rlmSessionDirForReading(), + getLiveChildren: () => this._contextViewChildren(), + }); + + this._history = new SessionHistoryNavigation({ + getSessionId: () => this.sessionId, + getRetryPolicy: () => providerRetryPolicy(this.settingsManager), + sessionManager: this.sessionManager, + settingsManager: this.settingsManager, + getModel: () => this.model, + getExtensions: () => this._extensionRunner, + getRequiredAuth: (model) => this._getRequiredRequestAuth(model), + acquireQueuedWorkPause: () => this.acquireQueuedWorkPause(), + acquireCommitFence: () => this._acquireSessionActionCommitFence(), + runWithCommitFence: (lease, run) => this._commitFence.run(lease, run), + waitForAgentIdle: () => this.agent.waitForIdle(), + getEventQueue: () => this._events.queue, + invalidateRefinement: () => this._refinement._invalidatePendingAutoRefineForBranchChange(), + rebuildBranchContext: () => { + this.agent.state.messages = this.sessionManager.buildSessionContext().messages; + this._mergeUnpersistedOutcomes(this.agent.state.messages); + this._restoreLateIpythonSentAgentMessages(); + this._ensureHarnessDigestContext(); + this._goals.reload(); + this._reloadRlmMaxDepthFromBranch(); + this._invalidateQueuedPromptPreparation(); + }, + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + }); + + this._refinement = new SessionRefinement( + { + sessionManager: this.sessionManager, + settingsManager: this.settingsManager, + getRetryPolicy: () => providerRetryPolicy(this.settingsManager), + getSessionId: () => this.sessionId, + isDisposed: () => this._disposed, + isDisposing: () => this._disposing, + isStreaming: () => this.isStreaming, + isCompacting: () => this.isCompacting, + getDepth: () => this._childState.depth, + getRlmSessionDir: () => this._rlmSessionDir, + getModel: () => this.model, + getThinkingLevel: () => this.thinkingLevel, + getMessages: () => this.agent.state.messages, + getRequiredRequestAuth: (model) => this._getRequiredRequestAuth(model), + getExtensionRunner: () => this._extensionRunner, + getEventQueue: () => this._events.queue, + getCompactionOperation: () => this._compaction.operation, + getBranchSummaryOperation: () => this._branchSummaryOperation, + waitForAgentIdle: () => this.agent.waitForIdle(), + dispatchRefine: (options, internal) => this.refine(options, internal), + disconnect: () => this._disconnectFromAgent(), + reconnect: () => this._reconnectToAgent(), + emit: (event) => this._emit(event), + retainUnpersistedOutcome: (message) => this._harnessContext.retainOutcome(message), + notifyCheckpoints: () => this._notifySessionInputCheckpointChange(), + scheduleInputPump: () => this._scheduleSessionInputPump(), + isContinuationScheduled: () => this._continuation.isScheduled, + cancelContinuation: () => this._cancelPostCompactionContinue(), + }, + { + serializedRefine: config.serializedRefine, + autoRefineReviewer: config.autoRefineReviewer?.bind(this), + }, + ); + this._modelSelection = new SessionModelSelection( + { + getModel: () => this.model, + getState: () => this.agent.state, + setThinkingLevel: (level) => this.setThinkingLevel(level), + getAvailableThinkingLevels: () => this.getAvailableThinkingLevels(), + supportsThinking: () => this.supportsThinking(), + getRegistry: () => this._modelRegistry, + getExtensions: () => this._extensionRunner, + sessionManager: this.sessionManager, + settingsManager: this.settingsManager, + emit: (event) => this._emit(event), + }, + config.serviceTierPreference ?? config.agent.state.serviceTier, + config.scopedModels ?? [], + ); + this._resourceLoader = config.resourceLoader; + this._cwd = config.cwd; + this._agentDir = config.agentDir; + this._modelRegistry = config.modelRegistry; + this._initialActiveToolNames = config.initialActiveToolNames; + this._includeGoals = config.includeGoals ?? true; + this._includeCompactSkill = config.includeCompactSkill ?? this.settingsManager.getCompactionAgentCallable(); + this._rlmHeartbeatController = config.rlmHeartbeatController; + this._agentMessageController = config.agentMessageController; + this._agentObserveController = config.agentObserveController; + this._mcpManager = config.mcpManager; + this._childState = new SessionChildState( + { + sessionManager: this.sessionManager, + settingsManager: this.settingsManager, + getRlmMaxDepthStatus: () => this.getRlmMaxDepthStatus(), + refreshPrompt: (preserveExtensionPrompt) => { + const oldBase = this._baseSystemPrompt; + this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); + this.agent.state.systemPrompt = preserveExtensionPrompt + ? this._refreshExtensionSystemPrompt(this.agent.state.systemPrompt, oldBase) + : this._baseSystemPrompt; + }, + emitRecap: (recap) => this._emit({ type: "recap_update", recap }), + }, + config, + ); + + this._rlmParentNodeId = config.rlmParentNodeId; + this._rlmParentAgent = config.rlmParentAgent; + this._kernelEnvironment = new KernelEnvironment( + { + agentDir: this._agentDir, + authStorage: this._modelRegistry.authStorage, + resourceLoader: this._resourceLoader, + getDepth: () => this._childState.depth, + getMaxDepth: () => this._childState.maxDepth, + getArtifactDir: () => this.sessionManager.getSessionArtifactDir(), + getLocalHarnessStateDir: () => this._refinement._localHarnessStateDir(), + }, + config.rlmSessionDir, + ); + this._kernel = new SessionKernel( + { + cwd: this._cwd, + getArtifactDir: () => this.sessionManager.getSessionArtifactDir(), + getSessionId: () => this.sessionId, + getEnv: () => this._rlmKernelEnv(), + getShellCommandPrefix: () => this.settingsManager.getShellCommandPrefix(), + getShellPath: () => this.settingsManager.getShellPath(), + createHostHandlers: () => this._createKernelHostHandlers(), + recordLateSentAgentMessage: (id, message) => this._recordLateIpythonSentAgentMessage(id, message), + getMessages: () => this.agent.state.messages, + appendCustomMessageEntry: (...args) => this.sessionManager.appendCustomMessageEntry(...args), + emit: (event) => this._emit(event), + sendCustomMessage: (message, options) => this.sendCustomMessage(message, options), + }, + (config.prewarmIpythonKernel ?? false) && this._childState.depth === 0, + ); + this._tools = new SessionTools( + { + cwd: this._cwd, + resourceLoader: this._resourceLoader, + getExtensionRunner: () => this._extensionRunner, + getSessionFile: () => this.sessionManager.getSessionFile(), + getModelVisibleSkills: () => this._modelVisibleSkills(), + getDepth: () => this._childState.depth, + getMaxDepth: () => this._childState.maxDepth, + getParentAgent: () => this._rlmParentAgent, + getMcpManager: () => this._mcpManager, + getProvisioner: () => this._kernel.provisioner, + getActiveToolNames: () => this.getActiveToolNames(), + setActiveToolsByName: (names) => this.setActiveToolsByName(names), + getActiveTools: () => this.agent.state.tools, + setActiveTools: (tools) => { + this.agent.state.tools = tools; + }, + setSystemPrompt: (prompt) => { + this.agent.state.systemPrompt = prompt; + }, + isStreaming: () => this.isStreaming, + rebuildRuntime: (options) => this._buildRuntime(options), + acquireInputPause: () => this.acquireSessionInputPause(), + waitForAgentIdle: () => this.agent.waitForIdle(), + getEventQueue: () => this._events.queue, + }, + { + customTools: config.customTools, + allowedToolNames: config.allowedToolNames, + baseToolsOverride: config.baseToolsOverride, + }, + ); + this._extensions = new SessionExtensions( + { + cwd: this._cwd, + sessionManager: this.sessionManager, + resourceLoader: this._resourceLoader, + modelRegistry: this._modelRegistry, + getModelRegistry: () => this.modelRegistry, + getPromptTemplates: () => this.promptTemplates, + bindShutdownHandler: (handler) => handler?.bind(this), + getAgentMessageController: () => this._agentMessageController, + refreshCurrentModel: () => this._refreshCurrentModelFromRegistry(), + sendCustomMessage: (message, options) => this.sendCustomMessage(message, options), + sendUserMessage: (content, options) => this.sendUserMessage(content, options), + setSessionName: (name) => this.setSessionName(name), + getActiveToolNames: () => this.getActiveToolNames(), + getAllTools: () => this.getAllTools(), + setActiveToolsByName: (names) => this.setActiveToolsByName(names), + refreshTools: () => this._refreshToolRegistry(), + setModel: (model) => this.setModel(model), + getThinkingLevel: () => this.thinkingLevel, + setThinkingLevel: (level) => this.setThinkingLevel(level), + getModel: () => this.model, + isStreaming: () => this.isStreaming, + getSignal: () => this.agent.signal, + abort: () => this.abort(), + getQueuedActionCount: () => this.queuedActionCount, + getContextUsage: () => this.getContextUsage(), + compact: (instructions) => this.compact(instructions), + getSystemPrompt: () => this.systemPrompt, + rebuildSystemPrompt: () => { + this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); + this.agent.state.systemPrompt = this._baseSystemPrompt; + }, + reloadSettings: () => this.settingsManager.reload(), + getMcpManager: () => this._mcpManager, + rebuildRuntime: (options) => this._buildRuntime(options), + }, + config.sessionStartEvent ?? { type: "session_start", reason: "startup" }, + config.extensionRunnerRef, + ); + + this._semanticEdges = new SemanticEdgeRecorder({ + ledgerPath: semanticEdgeLedgerPath({ + rlmSessionDir: this._rlmSessionDir, + sessionArtifactDir: this.sessionManager.getSessionArtifactDir(), + }), + sessionId: this.sessionManager.getSessionId(), + parentSessionId: config.semanticParentSessionId, + spawnedByRequestId: config.semanticSpawnedByRequestId, + }); + this.agent.streamFn = wrapStreamFnWithSemanticEdges(this.agent.streamFn, this._semanticEdges); + this._childState.initializeParentReply(); + this._children.setRuntimeHost(config.subagentRuntimeHost); + this._autonomousContinuation = new SessionAutonomousContinuation(config.autonomous, { + getStatus: () => this.getAutonomousStatus(), + getCwd: () => this._cwd, + getAgent: () => this.agent, + getStore: () => this.sessionManager, + emit: (event) => this._emit(event), + getContinuation: () => this._continuation, + getArrivalEpoch: () => this._inputAdmission.arrivalEpoch, + admit: (action, options) => this._admitSessionInput(action, options), + cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), + emitQueueUpdate: () => this._emitQueueUpdate(), + getCompaction: () => this._compaction, + getUnfinishedActionCount: () => this.unfinishedActionCount, + cancelContinuation: () => this._cancelPostCompactionContinue(), + }); + const goalPersistence = createGoalPersistence(this.sessionManager); + this._goalContinuation = new SessionGoalContinuation( + new GoalController(goalPersistence, (goal) => this._emit({ type: "goal_update", goal })), + this._actionStore, + { + getGoalState: () => this.goalState, + queuePrompt: (schedule, text, images, options) => + this._queuePreparedPrompt(schedule, text, images, options), + getScheduler: () => this._inputScheduler, + isDisposed: () => this._disposed, + isDisposing: () => this._disposing, + hasUnsettledChildWork: () => this._hasUnsettledRlmQuiescenceWork(), + ensureRuntimeActive: (context) => this._ensureGoalRuntimeActive(context), + admit: (action, options) => this._admitSessionInput(action, options), + cancelActions: (predicate, error) => this._cancelSessionActions(predicate, error), + clearPendingGoalContexts: () => { + this._pendingContext.removeMessagesMatching( + (message) => message.customType === GOAL_CONTEXT_CUSTOM_TYPE, + ); + }, + emitQueueUpdate: () => this._emitQueueUpdate(), + emitGoalUpdate: () => this._emitGoalUpdate(), + validate: () => this._validateCanStartAgentRun(), + isStreaming: () => this.isStreaming, + includesGoals: () => this._includeGoals, + getAgent: () => this.agent, + }, + ); + // Seed initial goal from CLI --goal flag, but only for top-level sessions + // and only when the branch contains only bootstrap entry types (model_change, + // thinking_level_change, service_tier_change) and no persisted + // thread_goal_state. This prevents reseeding after clear/complete/error + // or restart/rehydration of a session that already has messages or a goal. + if (this._childState.depth === 0 && config.initialGoal && goalPersistence.canSeed()) { + this._startGoal(config.initialGoal.objective, config.initialGoal.tokenBudget); + // Goal context is the model's only source of goal visibility; action + // admission is unavailable mid-construction, so ride the next turn. + this._pendingContext.appendMessages(createGoalContextMessage(this._goals.state, "continuation")); + } + this._restoreLateIpythonSentAgentMessages(); + this._goals.restartAccounting(); + + this._events.reconnectToAgent(); + this._installAgentToolHooks(); + this._installAgentTurnHook(); + this._installAgentContinuationHook(); + + this._buildRuntime({ + activeToolNames: this._initialActiveToolNames, + includeAllExtensionTools: true, + }); + this._ensureHarnessDigestContext(); + } + + /** Refreshes MCP provider registrations without rebuilding the session runtime. */ + refreshMcpProviders(): void { + this._mcpManager?.refresh(); + } + + /** + * Set the RLM heartbeat controller after construction. Used by + * print/headless mode to attach an in-process heartbeat scheduler + * when the session is created outside the daemon. + */ + setRlmHeartbeatController(controller: AgentRlmHeartbeatController): void { + if (this._rlmHeartbeatController === controller) { + return; + } + this._rlmHeartbeatController = controller; + this._buildRuntime({ + activeToolNames: this.getActiveToolNames(), + includeAllExtensionTools: true, + }); + this._baseSystemPrompt = this._rebuildSystemPrompt(this.getActiveToolNames()); + this.agent.state.systemPrompt = this._baseSystemPrompt; + } + + replaceAcpMcpServers(servers: readonly AcpMcpServerConfig[], ownerId: string): void { + this._tools.replaceAcpMcpServers(servers, ownerId); + } + + releaseAcpMcpServers(ownerId: string, serverNames: readonly string[]): Promise { + return this._tools.releaseAcpMcpServers(ownerId, serverNames); + } + + get modelRegistry(): ModelRegistry { + return this._modelRegistry; + } + + setSubagentRuntimeHost(host?: SubagentRuntimeHost): void { + this._children.setRuntimeHost(host); + } + + private _getRequiredRequestAuth( + ...args: Parameters + ): ReturnType { + return this._modelSelection.getRequiredRequestAuth(...args); + } + + /** + * Install tool hooks once on the Agent instance. + * + * The callbacks read `this._extensionRunner` at execution time, so extension reload swaps in the + * new runner without reinstalling hooks. Extension-specific tool wrappers are still used to adapt + * registered tool execution to the extension context. Tool call and tool result interception now + * happens here instead of in wrappers. + */ + private _installAgentToolHooks(): void { + installExtensionToolHooks( + this.agent, + () => this._extensionRunner, + () => this._events.queue, + ); + } + + private _installAgentContinuationHook(): void { + this.agent.getContinuationMessages = (context, signal) => this._getContinuationMessages(context, signal); + } + + private _installAgentTurnHook(): void { + this.agent.shouldStopBeforeTurn = () => this._shouldStopBeforeTurn(); + this.agent.shouldStopAfterTurn = (context) => this._shouldStopAfterTurn(context); + } + + private _emit(...args: Parameters): ReturnType { + return this._events.emit(...args); + } + + private _emitQueueUpdate( + ...args: Parameters + ): ReturnType { + return this._events.emitQueueUpdate(...args); + } + + private _restoreLateIpythonSentAgentMessages( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.restoreLateIpythonSentAgentMessages(...args); + } + + private _applyLateIpythonSentAgentMessages( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.applyLateIpythonSentAgentMessages(...args); + } + + private _recordLateIpythonSentAgentMessage( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.recordLateIpythonSentAgentMessage(...args); + } + + private _emitGoalUpdate(): void { + this._emit({ type: "goal_update", goal: this.goalState }); + } + + private _reloadRlmMaxDepthFromBranch(): void { + this._childState.reloadFromBranch(); + } + + private _cancelSessionActions( + ...args: Parameters + ): ReturnType { + return this._actionQueue.cancelSessionActions(...args); + } + + private _startGoal( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.startGoal(...args); + } + + private _finishGoalForTerminalAssistantMessage( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.finishGoalForTerminalAssistantMessage(...args); + } + + private _stopGoalContinuationForTerminalMessage( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.stopGoalContinuationForTerminalMessage(...args); + } + + private _handleAutonomousSlashCommand( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.handleAutonomousSlashCommand(...args); + } + + private _validateCanStartAgentRun( + ...args: Parameters + ): ReturnType { + return this._modelSelection.validateCanStartAgentRun(...args); + } + + /** + * Goals are pursued through the kernel goal skill, so the only tool the + * model needs is ipython. Force-activate it (including into a live + * continuation context) so the model can always reach `goal.complete()`. + */ + private _ensureGoalRuntimeActive(context?: AgentContext): void { + if (!this._includeGoals) { + throw new Error("Goals are disabled. Enable goals before using /goal."); + } + const ipythonTool = this._toolRegistry.get("ipython"); + if (!ipythonTool) { + throw new Error("Goals require the ipython tool, which is not available in this session."); + } + const activeToolNames = new Set(this.getActiveToolNames()); + if (!activeToolNames.has("ipython")) { + activeToolNames.add("ipython"); + this.setActiveToolsByName([...activeToolNames]); + } + if (context) { + const contextTools = [...(context.tools ?? [])]; + if (!contextTools.some((tool) => tool.name === "ipython")) { + contextTools.push(ipythonTool); + context.tools = contextTools; + } + } + } + + private _maybeResumeGoalContinuationAfterRlmWork( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.maybeResumeGoalContinuationAfterRlmWork(...args); + } + + private _handleGoalSlashCommand( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.handleGoalSlashCommand(...args); + } + + private get _steeringStopPending(): boolean { + return this._actionQueue.steeringStopPending; + } + + private _shouldStopBeforeTurn( + ...args: Parameters + ): ReturnType { + return this._turnPolicy.shouldStopBeforeTurn(...args); + } + + private _shouldStopAfterTurn( + ...args: Parameters + ): ReturnType { + return this._turnPolicy.shouldStopAfterTurn(...args); + } + + private _snapshotAutonomousRuntimeState( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.snapshotAutonomousRuntimeState(...args); + } + + private _restoreAutonomousRuntimeSnapshot( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.restoreAutonomousRuntimeSnapshot(...args); + } + + private _queueAutonomousContinuationForThresholdCompaction( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.queueAutonomousContinuationForThresholdCompaction(...args); + } + + // The role heuristic reads an assistant-last threshold stop as "task finished" and + // agent.continue() cannot resume from it, so the goal continuation is queued as a session input. + private _queueGoalContinuationForThresholdCompaction( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.queueGoalContinuationForThresholdCompaction(...args); + } + + // Withdraws a goal continuation queued for a threshold compaction the user cancelled, + // rolling back the continuationsUsed increment so the next natural stop re-queues it. + private _clearQueuedGoalContinuationAfterCancelledThresholdCompaction( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.clearQueuedGoalContinuationAfterCancelledThresholdCompaction(...args); + } + + private _clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction( + ...args: Parameters< + SessionAutonomousContinuation["clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction"] + > + ): ReturnType { + return this._autonomousContinuation.clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction(...args); + } + + /** + * Handle a goal.* request from the Python kernel host bridge (the bundled + * goal skill). All goal state stays host-side; the kernel only sees the + * serialized snake_case response. + */ + handleGoalHostRequest( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.handleGoalHostRequest(...args); + } + + /** + * Handle a compact.* request from the kernel host bridge. Compaction would + * abort the run executing the requesting cell, so compact.run only schedules + * it; _checkCompaction consumes the request at the turn boundary. + */ + handleCompactHostRequest( + ...args: Parameters + ): ReturnType { + return this._compaction.handleCompactHostRequest(...args); + } + + /** + * Handle a refine.* request from the kernel host bridge. Like compact, + * refinement waits for the current turn to become idle before applying + * changes, so refine.run only schedules it; _consumePendingRequestedRefine + * fires it at the turn boundary. This prevents a deadlock that would occur + * if refine() awaited agent idle from within the active tool call. + */ + handleRefineHostRequest(type: string, payload: Record = {}): Record { + return this._refinement.handleRefineHostRequest(type, payload); + } + + /** + * Handle an rlm_heartbeat.* request from the bundled rlm-heartbeat skill. + * These heartbeats are internal to this active session and never read or + * mutate the user-level /heartbeat. + */ + handleRlmHeartbeatHostRequest(type: string, payload: Record = {}): Record { + return handleRlmHeartbeatHostRequest(this._rlmHeartbeatController, type, payload); + } + + handleAgentMessageHostRequest( + type: string, + payload: Record = {}, + ): Promise { + return handleAgentMessageHostRequest(() => this._agentMessageController, type, payload); + } + + handleAgentObserveHostRequest( + type: string, + payload: Record = {}, + ): + | AgentObserveListResult + | AgentObserveAgentSnapshot + | AgentObserveRecentMessagesResult + | Promise { + return handleAgentObserveHostRequest(this._agentObserveController, type, payload); + } + + private _getGoalContinuationMessages( + ...args: Parameters + ): ReturnType { + return this._goalContinuation.getGoalContinuationMessages(...args); + } + + private _getContinuationMessages( + ...args: Parameters + ): ReturnType { + return this._turnPolicy.getContinuationMessages(...args); + } + + /** + * Register a delivery waiter before submitting the prompt. Delivery outcomes are not retained + * for late lookup, so callers that register after admission may wait for a future use of the id. + */ + waitForAgentMessagePromptDelivery( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.waitForAgentMessagePromptDelivery(...args); + } + + private _settleAgentMessage( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.settleAgentMessage(...args); + } + + private _rejectAgentMessage( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.rejectAgentMessage(...args); + } + + private _hasCancelledDispatchCapture( + ...args: Parameters + ): ReturnType { + return this._events.hasCancelledDispatchCapture(...args); + } + + private _findLastAssistantMessage(): AssistantMessage | undefined { + return this._events.findLastAssistantInMessages(this.agent.state.messages); + } + + /** + * Subscribe to agent events. + * Session persistence is handled internally (saves messages on message_end). + * Multiple listeners can be added. Returns unsubscribe function for this listener. + */ + subscribe(...args: Parameters): ReturnType { + return this._events.subscribe(...args); + } + + /** + * Temporarily disconnect from agent events. + * User listeners are preserved and will receive events again after resubscribe(). + * Used internally during operations that need to pause event processing. + */ + private _disconnectFromAgent( + ...args: Parameters + ): ReturnType { + return this._events.disconnectFromAgent(...args); + } + + /** + * Reconnect to agent events after _disconnectFromAgent(). + * Preserves all existing listeners. + */ + private _reconnectToAgent( + ...args: Parameters + ): ReturnType { + return this._events.reconnectToAgent(...args); + } + + /** + * Remove all listeners and disconnect from agent. + * Call this when completely done with the session. + */ + /** + * Async teardown for graceful quit/switch: await the Python kernel's dispose + * (which flushes a final namespace snapshot) before the synchronous dispose, so + * the latest state reaches disk instead of racing process exit. + */ + async disposeAsync(options?: { kernelSnapshot?: boolean }): Promise { + if (this._disposed) { + return this._disposeCallbacksPromise; + } + // Concurrent callers await the same in-flight teardown so none resolves before + // the kernel snapshot flush finishes. + if (this._disposeAsyncPromise) { + return this._disposeAsyncPromise; + } + const kernelSnapshot = options?.kernelSnapshot ?? true; + this._disposeAsyncPromise = (async () => { + // Drain before marking _disposing so a refine triggered at the final + // agent_end completes instead of being aborted by dispose(). + await this._refinement._drainPendingRefinementForDisposal(); + if (this._disposed) { + return this._disposeCallbacksPromise; + } + this._disposing = true; + this._commitFence.dispose(); + await this._disposeAsyncOnce(kernelSnapshot); + })(); + return this._disposeAsyncPromise; + } + + private _disposeAsyncOnce(kernelSnapshot: boolean): Promise { + // Flush kernels/traces for both still-running and retained children; the sync + // dispose() below only tears them down synchronously. + return this._children.disposeAsync(() => + this._kernel.dispose(kernelSnapshot, () => { + this.dispose(); + return this._disposeCallbacksPromise; + }), + ); + } + + private _startDisposeCallbacks(): Promise { + if (this._disposeCallbacksPromise) { + return this._disposeCallbacksPromise; + } + const pending: Promise[] = []; + for (const callback of this._disposeCallbacks) { + try { + const result = callback(); + if (result) { + pending.push(result.catch(() => undefined)); + } + } catch { + // Disposal remains best-effort; one owner must not block the rest. + } + } + this._disposeCallbacks.clear(); + this._disposeCallbacksPromise = Promise.all(pending).then(() => undefined); + return this._disposeCallbacksPromise; + } + + dispose(): void { + if (this._disposed) { + return; + } + this._disposed = true; + this._children.beginDisposal(); + this._commitFence.dispose(); + try { + // Invalidate scheduled timers and abort any in-flight review so a late + // resolution cannot write harness state or re-subscribe handlers. + this._refinement.dispose(); + this._children.dispose(); + this._pendingContext.dispose(); + const deliveryError = new Error("Session disposed before prompt delivery."); + const completionError = new Error("Session disposed before prompt completion."); + this._messageDelivery.dispose(deliveryError, completionError); + this._cancelSessionActions(() => true, deliveryError); + this.agent.clearAllQueues(); + this._extensionRunner.invalidate( + "This extension ctx is stale after session replacement or reload. Do not use a captured pi or command ctx after ctx.newSession(), ctx.fork(), ctx.switchSession(), or ctx.reload(). For newSession, fork, and switchSession, move post-replacement work into withSession and use the ctx passed to withSession. For reload, do not use the old ctx after await ctx.reload().", + ); + this._disconnectFromAgent(); + this._events.dispose(); + cleanupSessionResources(this.sessionId); + } finally { + void this._startDisposeCallbacks(); + } + } + + registerDisposeCallback(callback: () => void | Promise): void { + if (this._disposed) { + try { + const result = callback(); + if (result) void result.catch(() => undefined); + } catch { + // Late registration follows the same best-effort disposal contract. + } + return; + } + this._disposeCallbacks.add(callback); + } + + get state(): AgentState { + return this.agent.state; + } + + get model(): Model | undefined { + return this.agent.state.model; + } + + get thinkingLevel(): ThinkingLevel { + return this.agent.state.thinkingLevel; + } + + get serviceTier(): ServiceTier { + return this.agent.state.serviceTier; + } + + get isStreaming(): boolean { + return this.agent.state.isStreaming; + } + + get systemPrompt(): string { + return this.agent.state.systemPrompt; + } + + get retryAttempt(): number { + return this._retry.attempt; + } + + getActiveToolNames(): string[] { + return this._tools.getActiveToolNames(); + } + + getAllTools(): ToolInfo[] { + return this._tools.getAllTools(); + } + + getToolDefinition(name: string): ToolDefinition | undefined { + return this._tools.getToolDefinition(name); + } + + setActiveToolsByName(toolNames: string[]): void { + this._tools.setActiveToolsByName(toolNames); + } + + get isCompacting(): boolean { + return this._compaction.isRunning || this._history.isSummarizing; + } + + get messages(): AgentMessage[] { + return this.agent.state.messages; + } + + buildSessionContext( + ...args: Parameters + ): ReturnType { + return this._harnessContext.buildSessionContext(...args); + } + + private _mergeUnpersistedOutcomes( + ...args: Parameters + ): ReturnType { + return this._harnessContext.mergeUnpersistedOutcomes(...args); + } + + get steeringMode(): "all" | "one-at-a-time" { + return this.agent.steeringMode; + } + + get followUpMode(): "all" | "one-at-a-time" { + return this.agent.followUpMode; + } + + get sessionFile(): string | undefined { + return this.sessionManager.getSessionFile(); + } + + get sessionId(): string { + return this.sessionManager.getSessionId(); + } + + get rlmDepth(): number { + return this._childState.depth; + } + + get semanticEdges(): SemanticEdgeRecorder { + return this._semanticEdges; + } + + get rlmMaxDepth(): number { + return this._childState.maxDepth; + } + + get sessionName(): string | undefined { + return this.sessionManager.getSessionName(); + } + + get goalState(): GoalState { + return this._goals.current; + } + + getAutonomousStatus( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.getAutonomousStatus(...args); + } + + recordHostAutonomousContinuation( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.recordHostAutonomousContinuation(...args); + } + + refreshAutonomousGates( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.refreshAutonomousGates(...args); + } + + private _runWithAutonomousContinuationSuppressed(fn: () => Promise): Promise { + return this._autonomousContinuation.runWithAutonomousContinuationSuppressed(fn); + } + + private _markAutonomousContinuationSuppressed( + ...args: Parameters + ): ReturnType { + return this._autonomousContinuation.markAutonomousContinuationSuppressed(...args); + } + + get scopedModels(): ReadonlyArray<{ + model: Model; + thinkingLevel?: ThinkingLevel; + }> { + return this._scopedModels; + } + + setScopedModels(scopedModels: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>): void { + this._modelSelection.setScopedModels(scopedModels); + } + + get promptTemplates(): ReadonlyArray { + return this._resourceLoader.getPrompts().prompts; + } + + private _rebuildSystemPrompt(toolNames: string[]): string { + return this._tools.rebuildSystemPrompt(toolNames); + } + + private _refreshExtensionSystemPrompt( + ...args: Parameters + ): ReturnType { + return this._tools.refreshExtensionSystemPrompt(...args); + } + + private _normalizeSubmission( + ...args: Parameters + ): ReturnType { + return this._submissionNormalizer.normalizeSubmission(...args); + } + + private async _runPreTurnCompaction(): Promise { + const lastAssistant = this._findLastAssistantMessage(); + if (lastAssistant) await this._checkCompaction(lastAssistant, false, false); + } + + private _canStartSessionActionImmediately(): boolean { + return ( + !this.isStreaming && + !this.isCompacting && + !this.isRetrying && + !this.isBashRunning && + !this._inputScheduler.suspended && + this._inputScheduler.queuedWorkPauseCount === 0 && + !this._disposed && + !this._disposing + ); + } + + /** + * Send a prompt to the agent. + * - Handles extension commands (registered via pi.registerCommand) immediately, even during streaming + * - Expands file-based prompt templates by default + * - During streaming, queues via steer() or followUp() based on streamingBehavior option + * - Validates model and API key before sending (when not streaming) + * @throws Error if streaming and no streamingBehavior specified + * @throws Error if no model selected or no API key available (when not streaming) + */ + async prompt(text: string, options?: PromptOptions): Promise { + return this._prompt(text, options); + } + + async promptUntilAccepted(text: string, options?: PromptOptions): Promise { + return this._prompt(text, { ...options, returnAfterAccepted: true }); + } + + promptAndWait( + ...args: Parameters + ): ReturnType { + return this._messageDelivery.promptAndWait(...args); + } + + acceptAgentMessagePrompt( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.acceptAgentMessagePrompt(...args); + } + + queueAgentMessagePrompt( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.queueAgentMessagePrompt(...args); + } + + async promptHeartbeat(job: AgentCronJob, options?: PromptOptions): Promise { + const message = createHeartbeatPromptMessage(job); + await this._promptInjectedMessage(message.content, message, { + ...options, + followUpQueueKey: options?.followUpQueueKey ?? `heartbeat:${job.id}`, + resumeIfIdle: true, + }); + } + + private _isRlmTerminalNoticeAction( + ...args: Parameters + ): ReturnType { + return this._pendingContext.isRlmTerminalNoticeAction(...args); + } + + private _hasDeferredRlmTerminalNotices( + ...args: Parameters + ): ReturnType { + return this._pendingContext.hasDeferredRlmTerminalNotices(...args); + } + + private _flushDeferredRlmTerminalNotices( + ...args: Parameters + ): ReturnType { + return this._pendingContext.flushDeferredRlmTerminalNotices(...args); + } + + private _deferRlmTerminalNotice( + ...args: Parameters + ): ReturnType { + return this._pendingContext.deferRlmTerminalNotice(...args); + } + + private _demoteRlmTerminalNoticeActions( + ...args: Parameters + ): ReturnType { + return this._pendingContext.demoteRlmTerminalNoticeActions(...args); + } + + private _promptInjectedMessage( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.promptInjectedMessage(...args); + } + + private _prompt( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.prompt(...args); + } + + /** + * Queue a steering message while the agent is running. + * Delivered after the current assistant turn finishes executing its tool calls, + * before the next LLM call. + * Expands skill commands and prompt templates. Errors on extension commands. + * @param images Optional image attachments to include with the message + * @throws Error if text is an extension command + */ + steer(...args: Parameters): ReturnType { + return this._promptSubmission.steer(...args); + } + + /** + * Queue a follow-up message to be processed after the agent finishes. + * Delivered only when agent has no more tool calls or steering messages. + * Expands skill commands and prompt templates. Errors on extension commands. + * @param images Optional image attachments to include with the message + * @throws Error if text is an extension command + */ + followUp(...args: Parameters): ReturnType { + return this._promptSubmission.followUp(...args); + } + + restoreSessionActions( + ...args: Parameters + ): ReturnType { + return this._actionRecovery.restoreSessionActions(...args); + } + + restoreSteeringMessage( + ...args: Parameters + ): ReturnType { + return this._actionQueue.restoreSteeringMessage(...args); + } + + restoreFollowUpMessage( + ...args: Parameters + ): ReturnType { + return this._actionQueue.restoreFollowUpMessage(...args); + } + + private _takePendingNextTurnMessages( + ...args: Parameters + ): ReturnType { + return this._pendingContext.takePendingNextTurnMessages(...args); + } + + private _assertSessionActionAdmissionAvailable( + ...args: Parameters + ): ReturnType { + return this._inputAdmission.assertSessionActionAdmissionAvailable(...args); + } + + private _admitSessionInput( + ...args: Parameters + ): ReturnType { + return this._inputAdmission.admitSessionInput(...args); + } + + private _queuePreparedPrompt( + ...args: Parameters + ): ReturnType { + return this._inputAdmission.queuePreparedPrompt(...args); + } + + private _runtimeActivity(): RuntimeActivity { + return { + lowerAgentRun: this.isStreaming, + compaction: this.isCompacting, + retry: this.isRetrying, + bash: this.isBashRunning, + refinementApply: this._refinement.isApplying, + branchMutation: this._branchSummaryOperation !== undefined, + schedulerPauseCount: this._inputScheduler.queuedWorkPauseCount + (this._inputScheduler.suspended ? 1 : 0), + disposing: this._disposed || this._disposing, + }; + } + + private _hasSelectableSessionInput(): boolean { + return this._inputDispatcher.hasSelectableInput(); + } + + get hasPendingSessionWork(): boolean { + return this._actionQueue.hasPendingSessionWork; + } + + get hasPendingAdmissionWaiters(): boolean { + return this._commitFence.hasPendingWork || this._inputCheckpoints.hasWaiters; + } + + private _scheduleSessionInputPump(): void { + this._inputScheduler.schedule(); + } + + private _executeSelectedSessionCommand( + ...args: Parameters + ): ReturnType { + return this._commandExecution.executeSelectedSessionCommand(...args); + } + + private _isBusyForSessionInput(point: "preflight" | "pump"): boolean { + const externalBusy = this.isCompacting || this.isRetrying || this.isBashRunning; + if (point === "pump") { + return ( + externalBusy || + this._disposed || + this._disposing || + this._inputScheduler.suspended || + this._inputScheduler.queuedWorkPauseCount > 0 || + this._branchSummaryOperation !== undefined + ); + } + return externalBusy || this._actionStore.unfinishedActions().length > 0; + } + + private _isSessionInputHandoffDeferred(epoch: number): boolean { + return epoch !== this._inputScheduler.epoch || this._isBusyForSessionInput("pump"); + } + + private _asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)); + } + + private _surfaceSessionInputError(error: unknown): void { + const normalized = this._asError(error); + try { + this._extensionRunner.emitError({ + extensionPath: "", + event: "session_input", + error: normalized.message, + stack: normalized.stack, + }); + } catch { + // Best-effort: a throwing error listener must not break the pump's requeue path. + } + } + + private _startPreparedTurnActions( + ...args: Parameters + ): ReturnType { + return this._turnExecution.startPreparedTurnActions(...args); + } + + /** + * Send a custom message to the session. Creates a CustomMessageEntry. + * + * Handles three cases: + * - Streaming: queues message, processed when loop pulls from queue + * - Not streaming + triggerTurn: appends to state/session, starts new turn + * - Not streaming + no trigger: appends to state/session, no turn + * + * @param message Custom message with customType, content, display, details + * @param options.triggerTurn If true and not streaming, triggers a new LLM turn + * @param options.deliverAs Delivery mode: "steer", "followUp", or "nextTurn" + */ + sendCustomMessage( + message: Pick, "customType" | "content" | "display" | "details">, + options?: { + triggerTurn?: boolean; + deliverAs?: "steer" | "followUp" | "nextTurn"; + }, + ): Promise { + return this._promptSubmission.sendCustomMessage(message, options); + } + + /** + * Send a user message to the agent. Always triggers a turn. + * When the agent is streaming, use deliverAs to specify how to queue the message. + * + * @param content User message content (string or content array) + * @param options.deliverAs Delivery mode when streaming: "steer" or "followUp" + */ + sendUserMessage( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.sendUserMessage(...args); + } + + clearQueue(...args: Parameters): ReturnType { + return this._actionQueue.clearQueue(...args); + } + + private _invalidateQueuedPromptPreparation( + ...args: Parameters + ): ReturnType { + return this._actionQueue.invalidateQueuedPromptPreparation(...args); + } + + clearQueuedAgentMessages( + ...args: Parameters + ): ReturnType { + return this._actionQueue.clearQueuedAgentMessages(...args); + } + + clearQueuedUserMessagesMatching( + ...args: Parameters + ): ReturnType { + return this._actionQueue.clearQueuedUserMessagesMatching(...args); + } + + /** + * Mutate a single visible queued message, addressed by its position in the same + * projection the session-action snapshot publishes. expectedText must match the + * item's current preview so clients never edit a shifted queue by accident. + */ + mutateQueuedMessage( + ...args: Parameters + ): ReturnType { + return this._actionQueue.mutateQueuedMessage(...args); + } + + get queuedActionCount(): number { + return visibleSessionActionProjection(this._actionStore.queuedActions()).length; + } + + get unfinishedActionCount(): number { + return this._actionStore.unfinishedActions().length; + } + + get isQueuedWorkSuspended(): boolean { + return this._inputScheduler.suspended; + } + + get isSessionActive(): boolean { + return ( + this._ipythonKernelProvisioner?.manager?.hasBackgroundWork === true || + this.isStreaming || + this.isCompacting || + this.isRetrying || + this.isBashRunning || + this._refinement.isApplying || + this._branchSummaryOperation !== undefined || + this._continuation.current !== undefined || + this.unfinishedActionCount > 0 + ); + } + + getSessionActionSnapshot( + ...args: Parameters + ): ReturnType { + return this._actionQueue.getSessionActionSnapshot(...args); + } + + getSteeringMessages( + ...args: Parameters + ): ReturnType { + return this._actionQueue.getSteeringMessages(...args); + } + + getSteeringMessagePreviews( + ...args: Parameters + ): ReturnType { + return this._actionQueue.getSteeringMessagePreviews(...args); + } + + getFollowUpMessages( + ...args: Parameters + ): ReturnType { + return this._actionQueue.getFollowUpMessages(...args); + } + + getFollowUpMessagePreviews( + ...args: Parameters + ): ReturnType { + return this._actionQueue.getFollowUpMessagePreviews(...args); + } + + getSessionActionRecoverySnapshot( + ...args: Parameters + ): ReturnType { + return this._actionRecovery.getSessionActionRecoverySnapshot(...args); + } + + private _notifySessionInputCheckpointChange( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.notifySessionInputCheckpointChange(...args); + } + + private _waitForSessionActivityChange( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.waitForSessionActivityChange(...args); + } + + private _observeSessionActionDeferral( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.observeSessionActionDeferral(...args); + } + + waitForSessionInputCheckpoint( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.waitForSessionInputCheckpoint(...args); + } + + acquireSessionInputPause(): { release(): void } { + return this._inputScheduler.acquireAdmissionPause(() => { + this._notifySessionInputCheckpointChange(); + this._flushDeferredRlmTerminalNotices(); + this._maybeResumeGoalContinuationAfterRlmWork(); + this._scheduleSessionInputPump(); + }); + } + + acquireQueuedWorkPause(): { release(): void } { + return this._inputScheduler.acquireQueuedWorkPause(() => { + this._notifySessionInputCheckpointChange(); + this._flushDeferredRlmTerminalNotices(); + this._scheduleSessionInputPump(); + }); + } + + private _acquireDirectTurnAdmissionFence( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.acquireDirectTurnAdmissionFence(...args); + } + + private _acquireSessionActionCommitFence(signal?: AbortSignal): Promise { + return this._commitFence.acquire(signal); + } + + private _resumeSessionInputAdmission(): void { + if (!this._inputScheduler.resume()) return; + this._notifySessionInputCheckpointChange(); + this._flushDeferredRlmTerminalNotices(); + } + + /** Resume the scheduler after requestAbort/abortForUpdateRestart suspended it; owned pause leases are unaffected. */ + resumeQueuedWork(): boolean { + this._resumeSessionInputAdmission(); + this._maybeResumeGoalContinuationAfterRlmWork(); + this._scheduleSessionInputPump(); + return this._hasSelectableSessionInput(); + } + + waitForSessionInputIdle(): Promise { + return this._inputScheduler.waitForIdle(); + } + + async waitForIdle(): Promise { + await this._waitForIdleOrSettlement(); + } + + /** + * {@link waitForIdle} loop; with a settlement, returns once that settlement is + * superseded so a cancelled post-compaction runner cannot keep a checkpoint + * waiter registered (a leaked waiter holds hasPendingAdmissionWaiters true and + * blocks daemon passivation). + */ + private _waitForIdleOrSettlement( + ...args: Parameters + ): ReturnType { + return this._inputCheckpoints.waitForIdleOrSettlement(...args); + } + + /** Waits out any owned post-compaction continuation and rejects when one cannot start; {@link waitForIdle} never rejects. */ + waitForHeadlessIdle(): Promise { + return this._inputCheckpoints.waitForHeadlessIdle(); + } + + getPendingNextTurnMessageSnapshots( + ...args: Parameters + ): ReturnType { + return this._pendingContext.getPendingNextTurnMessageSnapshots(...args); + } + + restorePendingNextTurnMessages( + ...args: Parameters + ): ReturnType { + return this._pendingContext.restorePendingNextTurnMessages(...args); + } + + removeQueuedFollowUp( + ...args: Parameters + ): ReturnType { + return this._actionQueue.removeQueuedFollowUp(...args); + } + + get resourceLoader(): ResourceLoader { + return this._resourceLoader; + } + + requestAbort(): void { + this._children.requestAbort(); + this._inputScheduler.suspend("abort"); + this._demoteRlmTerminalNoticeActions(); + this._cancelSessionActions( + (action) => + action.payload.kind === "turn" && + !action.payload.queueVisible && + !this._pendingContext.isRetainedTerminalNotice(action.id), + new Error("Prompt aborted before delivery."), + ); + this._cancelPostCompactionContinue(); + this.abortRetry(); + this.abortCompaction(); + this.abortBranchSummary(); + this.abortBash(); + this._refinement.requestAbort(); + this.agent.abort(); + } + + async abort(): Promise { + const compactionOperation = this._compaction.operation; + const branchSummaryOperation = this._branchSummaryOperation; + this.requestAbort(); + this._cancelActiveRlmChildRuns("Parent session aborted"); + this._goalContinuation.beginAbort(); + try { + await Promise.allSettled([ + this.agent.waitForIdle(), + this._events.queue, + ...(compactionOperation ? [compactionOperation] : []), + ...(branchSummaryOperation ? [branchSummaryOperation] : []), + ]); + } finally { + this._goalContinuation.finishAbort(); + } + } + + abortForUpdateRestart(): void { + // Cancel scheduled pumps and suspend new ones: queued inputs must survive + // into the restart manifest instead of starting a turn during teardown. + this._inputScheduler.suspend("update-restart"); + this._cancelPostCompactionContinue(); + this.abortRetry(); + this._children.cancelQuiescenceWaits(); + this._cancelActiveRlmChildRuns("Parent session aborted for update restart"); + this._goalContinuation.beginAbort(); + this.agent.abort(); + if (this._goalContinuation.abortInProgress) { + void this.agent + .waitForIdle() + .then(() => this._events.queue) + .catch(() => undefined) + .finally(() => { + this._goalContinuation.finishAbort(); + }); + } + } + + setModel(...args: Parameters): ReturnType { + return this._modelSelection.setModel(...args); + } + + private _pendingModelSelectEmit( + ...args: Parameters + ): ReturnType { + return this._modelSelection.pendingModelSelectEmit(...args); + } + + cycleModel( + ...args: Parameters + ): ReturnType { + return this._modelSelection.cycleModel(...args); + } + + setThinkingLevel( + ...args: Parameters + ): ReturnType { + return this._modelSelection.setThinkingLevel(...args); + } + + setServiceTier( + ...args: Parameters + ): ReturnType { + return this._modelSelection.setServiceTier(...args); + } + + cycleThinkingLevel( + ...args: Parameters + ): ReturnType { + return this._modelSelection.cycleThinkingLevel(...args); + } + + getAvailableThinkingLevels( + ...args: Parameters + ): ReturnType { + return this._modelSelection.getAvailableThinkingLevels(...args); + } + + supportsThinking( + ...args: Parameters + ): ReturnType { + return this._modelSelection.supportsThinking(...args); + } + + private _syncKernelStateAfterCompaction(): Promise { + return this._kernel.syncAfterCompaction(); + } + + setSteeringMode(mode: "all" | "one-at-a-time"): void { + this.agent.steeringMode = mode; + this.settingsManager.setSteeringMode(mode); + } + + setFollowUpMode(mode: "all" | "one-at-a-time"): void { + this.agent.followUpMode = mode; + this.settingsManager.setFollowUpMode(mode); + } + + compact(customInstructions?: string, options: { skipAbort?: boolean } = {}): Promise { + return this._compaction.compact(customInstructions, options); + } + + private _afterManualCompaction( + signal: AbortSignal, + hadPostCompactionContinue: boolean, + continueAfterSessionInput: boolean, + ): void { + this._refinement._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); + if (this._goals.state.status === "active" && !signal.aborted) { + if (!this._goalContinuation.awaitsChildWork && !this.agent.hasQueuedMessages()) { + this._goalContinuation.deferUntilChildSettlement(); + } + this.resumeQueuedWork(); + if (this.agent.hasQueuedMessages()) this._schedulePostCompactionContinue(); + } + if (hadPostCompactionContinue) { + this._schedulePostCompactionContinue(continueAfterSessionInput); + } + // Queued agent or session-owned inputs resume the loop; defer refine + // behind them instead of interleaving it before their turns. + this._refinement._scheduleAutoRefineAfterCompaction( + this._goalContinuation.awaitsChildWork || + hadPostCompactionContinue || + this.agent.hasQueuedMessages() || + this.unfinishedActionCount > 0, + ); + } + + /** + * Shared compaction core behind /compact, auto-compaction, and the compact + * skill. Throws CompactionSkippedError when there is nothing to compact and + * Error("Compaction cancelled") on abort or extension cancel. + */ + private _performCompaction(options: CompactionExecutionOptions): Promise { + return performSessionCompaction(this._compactionExecution, options); + } + + private _reapDeletedRlmSubagentRuntimesAfterCompaction(): Promise { + return this._children.reapAfterCompaction(); + } + + abortCompaction(): void { + this._compaction.abort(); + } + + private _cancelPostCompactionContinue(): void { + this._continuation.cancel(); + } + + private _schedulePostCompactionContinue(continueAfterSessionInput = false): void { + this._continuation.schedule(continueAfterSessionInput); + } + + private _forgetConsumedPostCompactionContinuations(messages: AgentMessage[]): void { + this._continuation.forgetConsumed(messages); + } + + /** The compact harness digest delivered at cold context boundaries (session start, resume, compaction head). */ + private _harnessDigest( + ...args: Parameters + ): ReturnType { + return this._harnessContext.harnessDigest(...args); + } + + /** Cold-boundary digest delivery: empty contexts defer to the first committed turn (untouched sessions must stay empty); non-empty contexts append only when the newest in-context digest mismatches disk. */ + private _ensureHarnessDigestContext( + ...args: Parameters + ): ReturnType { + return this._harnessContext.ensureHarnessDigestContext(...args); + } + + private _latestContextHarnessDigest( + ...args: Parameters + ): ReturnType { + return this._harnessContext.latestContextHarnessDigest(...args); + } + + /** + * Refine editable continual harness state: prompt notes, memory, skills, and subagent specs. + * The base system prompt is intentionally not editable through this path. + * + * Planning runs in the background and does NOT block turn entry points + * (`_waitForRefineIdle` only waits for `_refineInFlight`). Only the fast + * application phase (disk I/O + in-memory mutation) blocks turn entry points. + */ + refine( + options: { instructions?: string; rollbackId?: string; global?: boolean } = {}, + internal: { skipAbort?: boolean; trigger?: "manual" | "auto"; source?: RefinementSource } = {}, + ): Promise { + return this._refinement.refine(options, internal); + } + + abortBranchSummary( + ...args: Parameters + ): ReturnType { + return this._history.abortBranchSummary(...args); + } + + /** + * Check if compaction is needed and run it. + * Called after agent_end and before prompt submission. + * + * Two cases: + * 1. Overflow: LLM returned context overflow error, remove error message from agent state, compact, auto-retry + * 2. Threshold: Context over threshold, compact, and continue only for stopped in-progress loops or queued messages + * + * @param assistantMessage The assistant message to check + * @param skipAbortedCheck If false, include aborted messages (for pre-prompt check). Default: true + */ + private _checkCompaction( + assistantMessage: AssistantMessage, + skipAbortedCheck = true, + queueAutonomousContinuation = true, + ): Promise { + return this._compaction.check(assistantMessage, skipAbortedCheck, queueAutonomousContinuation); + } + + /** + * Internal: Run automatic (threshold/overflow) or model-requested compaction + * with events. + */ + + private _runAutoCompaction(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise { + return this._compaction.runAutomatic(reason, willRetry); + } + + setAutoCompactionEnabled(enabled: boolean): void { + this.settingsManager.setCompactionEnabled(enabled); + } + + get autoCompactionEnabled(): boolean { + return this.settingsManager.getCompactionEnabled(); + } + + /** + * Set the provider for extra env vars merged over process.env in extension + * pi.exec() subprocesses. The function is read at exec time, so a host (e.g. + * the daemon) can update the underlying value per attach without rebinding. + */ + setExecEnvProvider(provider: (() => Record | undefined) | undefined): void { + this._extensions.setExecEnvProvider(provider); + } + + bindExtensions(bindings: ExtensionBindings): Promise { + return this._extensions.bindExtensions(bindings); + } + + refreshModelMetadata(): void { + this._modelSelection.refreshModelMetadata(); + } + + private _refreshCurrentModelFromRegistry( + ...args: Parameters + ): ReturnType { + return this._modelSelection.refreshCurrentModelFromRegistry(...args); + } + + private _refreshToolRegistry(options?: { activeToolNames?: string[]; includeAllExtensionTools?: boolean }): void { + this._tools.refreshToolRegistry(options); + } + + private _buildRuntime(options: { + activeToolNames?: string[]; + flagValues?: Map; + includeAllExtensionTools?: boolean; + }): void { + const pythonSkills = getPythonSkillRuntimeInfo(this._modelVisibleSkills()); + this._tools.setBaseDefinitions(this._tools.buildBaseOverrides() ?? this._kernel.build(pythonSkills)); + this._extensions.build(options.flagValues); + this._tools.updateAcpDefinitions(); + const baseActiveToolNames = [...(options.activeToolNames ?? this._tools.defaultActiveToolNames)]; + if (this._goals.state.status === "active" && this._includeGoals) baseActiveToolNames.push("ipython"); + this._refreshToolRegistry({ + activeToolNames: [...new Set(baseActiveToolNames)], + includeAllExtensionTools: options.includeAllExtensionTools, + }); + this._kernel.finishBuild(this.getActiveToolNames()); + } + + /** + * Skills exposed to the model (system prompt + kernel). The bundled goal + * and compact skills are withheld when disabled for this session. + */ + private _modelVisibleSkills(): Skill[] { + let skills = this._resourceLoader.getSkills().skills; + if (!this._includeGoals) { + skills = skills.filter((skill) => skill.name !== GOAL_SKILL_NAME); + } + if (!this._includeCompactSkill) { + skills = skills.filter((skill) => skill.name !== COMPACT_SKILL_NAME); + } + if (!this._refinement._autoRefineAllowedForSession()) { + skills = skills.filter((skill) => skill.name !== REFINE_SKILL_NAME); + } + if (!this._agentMessageController) { + skills = skills.filter((skill) => skill.name !== AGENT_MESSAGE_SKILL_NAME); + } + if (!this._agentObserveController) { + skills = skills.filter((skill) => skill.name !== AGENT_OBSERVE_SKILL_NAME); + } + if (!this._agentObserveController || !this._rlmHeartbeatController) { + skills = skills.filter((skill) => skill.name !== ORCHESTRATION_HEARTBEAT_SKILL_NAME); + } + return skills; + } + + private _createKernelHostHandlers(): HostRequestHandlers { + return createSessionKernelHostHandlers({ + runChild: (prompt, kwargs, code) => this.runRlmChild(prompt, kwargs, code), + createSession: (prompt, kwargs) => this.createRlmSession(prompt, kwargs), + findModels: (query, limit) => this.findRlmModels(query, limit), + listSubagents: () => this.listRlmSubagents(), + deleteSubagent: (target) => this.deleteRlmSubagent(target), + handleBashCompletion: (details) => this._handleKernelBashCompletion(details), + withdrawBashCompletion: (details) => this._actionQueue.withdrawAsyncBashCompletionNotice(details), + getModel: () => this.model, + includeGoals: this._includeGoals, + includeCompactSkill: this._includeCompactSkill, + isRefineAllowed: () => this._refinement._autoRefineAllowedForSession(), + hasHeartbeatController: () => !!this._rlmHeartbeatController, + getModelVisibleSkills: () => this._modelVisibleSkills(), + getAgentMessageController: () => this._agentMessageController, + hasObserveController: () => !!this._agentObserveController, + getMcpManager: () => this._mcpManager, + getDepth: () => this._childState.depth, + awaitChildPublication: (selector) => this._awaitPendingRlmChildPublication(selector), + recordParentReply: () => this._childState.recordReply(), + handleGoal: (type, payload) => this.handleGoalHostRequest(type, payload), + handleCompact: (type, payload) => this.handleCompactHostRequest(type, payload), + handleRefine: (type, payload) => this.handleRefineHostRequest(type, payload), + handleHeartbeat: (type, payload) => this.handleRlmHeartbeatHostRequest(type, payload), + handleMessage: (type, payload) => this.handleAgentMessageHostRequest(type, payload), + handleObserve: (type, payload) => this.handleAgentObserveHostRequest(type, payload), + }); + } + + private _handleKernelBashCompletion( + ...args: Parameters + ): ReturnType { + return this._promptSubmission.handleKernelBashCompletion(...args); + } + + reload(): Promise { + return this._extensions.reload(); + } + + // Undefined when there's no persistent artifact dir (e.g. the viewer client): + // don't mkdtemp here, since this runs on every kernel build but a viewer never + // does RLM work. The temp dir is created lazily in _createChildRlmSessionDir. + + private _createChildRlmSessionDir(): string { + return createChildSessionDir(() => this._ensureRlmSessionDir() ?? this._createEphemeralRlmSessionDir()); + } + + private _rlmKernelEnv(): Record { + return this._kernelEnvironment.buildEnv(); + } + private _ensureRlmSessionDir(): string | undefined { + return this._kernelEnvironment.ensureSessionDir(); + } + private _createEphemeralRlmSessionDir(): string { + return this._kernelEnvironment.createEphemeralSessionDir(); + } + + _contextTokensForCurrentMessages(): number | undefined { + const last = this._findLastAssistantMessage(); + return last ? calculateContextTokens(last.usage) : undefined; + } + + setCurrentRecap(recap: string | undefined): void { + this._childState.setCurrentRecap(recap); + } + + get repliedToParentSinceTask(): boolean | undefined { + return this._childState.repliedSinceTask; + } + + getCurrentRecap(): string | undefined { + return this._childState.getCurrentRecap(); + } + + private _createRlmSubagentRuntimeOptions(options: { + id: string; + prompt: string; + sessionName: string; + spawnCode?: string; + sessionDir: string; + model: Model; + thinkingLevel?: ThinkingLevel; + spawnedByRequestId?: string; + }): CreateRlmSubagentRuntimeOptions { + return { + parentSession: this, + id: options.id, + prompt: options.prompt, + sessionName: options.sessionName, + spawnCode: options.spawnCode, + sessionDir: options.sessionDir, + model: options.model, + thinkingLevel: + options.thinkingLevel ?? (clampThinkingLevel(options.model, this.thinkingLevel) as ThinkingLevel), + serviceTier: + this.serviceTier === "priority" && !supportsFastMode(options.model) ? "default" : this.serviceTier, + scopedModels: [...this._scopedModels], + activeToolNames: this.getActiveToolNames(), + allowedToolNames: this._allowedToolNames ? [...this._allowedToolNames] : undefined, + customTools: [...this._customTools], + includeGoals: this._includeGoals, + includeCompactSkill: this._includeCompactSkill, + rlmDepth: this._childState.depth + 1, + rlmMaxDepth: this._childState.maxDepth, + rlmParentNodeId: options.id, + spawnedByRequestId: options.spawnedByRequestId, + }; + } + + private async _createRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): Promise { + const host = this._children.getRuntimeHost(); + if (host) { + return await host.createRlmSubagentRuntime(options); + } + + return this._createInlineRlmSubagentRuntime(options); + } + + private _createInlineRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): RlmSubagentRuntime { + return createInlineChildRuntime( + { + cwd: this._cwd, + agentDir: this._agentDir, + agent: this.agent, + settingsManager: this.settingsManager, + resourceLoader: this._resourceLoader, + modelRegistry: this._modelRegistry, + }, + options, + ); + } + + private _cancelActiveRlmChildRuns(reason: string): void { + this._children.cancelActiveRuns(reason); + } + + getRlmChildRunStatus(childId: string): RlmChildAgentStatus | undefined { + return this._children.getRlmChildRunStatus(childId); + } + + private _awaitPendingRlmChildPublication(selector: string): Promise { + return this._children.awaitPublication(selector); + } + + listRlmSubagents(): Promise { + return this._children.listRlmSubagents(); + } + + deleteInactiveRlmSubagent( + childId: string, + isExternallyRunning: () => boolean = () => false, + ): Promise<"deleted" | "not_found" | "running"> { + return this._children.deleteInactiveRlmSubagent(childId, isExternallyRunning); + } + + deleteRlmSubagent(target: string): Promise { + return this._children.deleteRlmSubagent(target); + } + + /** + * Retain a finished child session for the parent lifetime so inspectors and + * daemon-hosted agent messaging can keep addressing it. Returns false (and disposes + * the child) when the parent is already tearing down, so the caller can drop the + * matching event forwarder too. + */ + registerRlmChildSession(childId: string, session: AgentSession, unsubscribe?: () => void): boolean { + return this._children.registerRlmChildSession(childId, session, unsubscribe); + } + + releaseRlmChildSession(childId: string, session: AgentSession): (() => void) | false { + return this._children.releaseRlmChildSession(childId, session); + } + + /** Live recursive child roster from lifecycle state, including nested work under retained parents. */ + getRlmChildSnapshots(): RlmChildAgentSnapshot[] { + return this._children.getRlmChildSnapshots(); + } + + /** True when any direct or nested subagent is still running or queued. */ + hasRunningRlmChildren(): boolean { + return this._children.hasRunningRlmChildren(); + } + + private _hasUnsettledRlmQuiescenceWork(): boolean { + return this._children.hasUnsettledWork(); + } + + /** + * Wait for every admitted descendant run to publish its terminal parent + * message and for the resulting parent turns to drain. Re-snapshotting after + * each drain includes descendants spawned while earlier results were consumed. + */ + waitForRlmQuiescence(externalSignal?: AbortSignal): Promise { + return this._children.waitForRlmQuiescence(externalSignal); + } + + // Inline (non-daemon) mode only; daemon clients attach to the child session directly. + getRlmChildSession(childId: string): AgentSession | undefined { + return this._children.getRlmChildSession(childId); + } + + /** + * Cancel a single RLM child run by id, searching nested child sessions. + * + * @returns true when a live run was cancelled or its unsettled terminal notice + * was suppressed; false when the id is unknown or the run already settled. + */ + cancelRlmChildRun(childId: string, reason = "Cancelled by user"): boolean { + return this._children.cancelRlmChildRun(childId, reason); + } + + /** Cancel every running or queued run in this session's subtree. */ + cancelRunningRlmDescendants(reason = "Cancelled by user"): boolean { + return this._children.cancelRunningRlmDescendants(reason); + } + + findRlmModels( + ...args: Parameters + ): ReturnType { + return this._modelSelection.findRlmModels(...args); + } + + private _resolveRlmSubagentModel( + ...args: Parameters + ): ReturnType { + return this._modelSelection.resolveRlmSubagentModel(...args); + } + + createRlmSession(prompt: string, kwargs: Record = {}): Promise { + return this._children.createRlmSession(prompt, kwargs); + } + + async runRlmChild( + prompt: string, + kwargs: Record = {}, + spawnCode?: string, + ): Promise { + return this._children.run(prompt, kwargs, spawnCode); + } + + abortRetry(): void { + this._retry.abortRetry(); + } + + private waitForRetry(): Promise { + return this._retry.waitForRetry(); + } + + get isRetrying(): boolean { + return this._retry.isRetrying; + } + + get hasAcceptedPromptInFlight(): boolean { + return this._actionQueue.hasAcceptedPromptInFlight; + } + + get autoRetryEnabled(): boolean { + return this.settingsManager.getRetryEnabled(); + } + + setAutoRetryEnabled(enabled: boolean): void { + this.settingsManager.setRetryEnabled(enabled); + } + + /** Execute a shell command and record its result unless transient. */ + executeBash(command: string, onChunk?: (chunk: string) => void, options?: ExecuteBashOptions): Promise { + return this._bash.executeBash(command, onChunk, options); + } + + /** Run ! / !! input with extension interception and bash lifecycle events. */ + runUserBash(command: string, options?: RunUserBashOptions): Promise { + return this._bash.runUserBash(command, options); + } + + private async _drainQueuedMessagesAfterBash(): Promise { + await this.agent.waitForIdle(); + this._scheduleSessionInputPump(); + } + + recordBashResult(command: string, result: BashResult, options?: { excludeFromContext?: boolean }): void { + this._bash.recordBashResult(command, result, options); + } + + /** Cancel every in-flight shell command, including pending extension dispatch. */ + abortBash(): void { + this._bash.abortBash(); + } + + get isBashRunning(): boolean { + return this._bash.isBashRunning; + } + + get hasPendingBashMessages(): boolean { + return this._bash.hasPendingBashMessages; + } + + private _flushPendingBashMessages(): void { + this._bash.flushPendingMessages(); + } + + getRlmMaxDepthStatus(): RlmMaxDepthStatus { + return this._childState.getRlmMaxDepthStatus(); + } + + setRlmMaxDepth(maxDepth: number, options: { global?: boolean } = {}): Promise { + return this._childState.setRlmMaxDepth(maxDepth, options); + } + + setSessionName(name: string): void { + this.sessionManager.appendSessionInfo(name); + this._emit({ + type: "session_info_changed", + name: this.sessionManager.getSessionName(), + }); + } + + /** + * Navigate to a different node in the session tree. + * Unlike fork() which creates a new session file, this stays in the same file. + * + * @param targetId The entry ID to navigate to + * @param options.summarize Whether user wants to summarize abandoned branch + * @param options.customInstructions Custom instructions for summarizer + * @param options.replaceInstructions If true, customInstructions replaces the default prompt + * @param options.label Label to attach to the branch summary entry + * @returns Result with editorText (if user message) and cancelled status + */ + + navigateTree( + ...args: Parameters + ): ReturnType { + return this._history.navigateTree(...args); + } + + getUserMessagesForForking( + ...args: Parameters + ): ReturnType { + return this._history.getUserMessagesForForking(...args); + } + + getSessionStats( + ...args: Parameters + ): ReturnType { + return this._contextView.getSessionStats(...args); + } + + getContextUsage( + ...args: Parameters + ): ReturnType { + return this._contextView.getContextUsage(...args); + } + + private _rlmSessionDirForReading(): string | undefined { + return this._rlmSessionDir ?? this.sessionManager.getSessionArtifactDir(); + } + + private *_contextViewChildren(): Generator { + for (const run of this._children.getActiveRuns()) { + yield { + id: run.id, + get label() { + return rlmChildLabel(run.prompt); + }, + get status() { + return run.status; + }, + sessionDir: run.sessionDir, + getContextTree: run.session ? () => run.session!.getContextTree() : undefined, + }; + } + } + + private _invalidateOwnUsage(): void { + this._contextView.invalidateOwnUsage(); + } + + // Whole-file own spend, identical to the catalog scan so rows never shift at passivation. + getOwnUsageSummary( + ...args: Parameters + ): ReturnType { + return this._contextView.getOwnUsageSummary(...args); + } + + /** + * Build the agent context overview for /context: this session as the root + * plus one node per RLM sub-agent, recursively. Running children are read + * from their live sessions; completed children from their persisted session + * dirs, so the tree survives child disposal and session resume. + */ + getContextTree( + ...args: Parameters + ): ReturnType { + return this._contextView.getContextTree(...args); + } + + /** + * Export session to HTML. + * @param outputPath Optional output path (defaults to session directory) + * @returns Path to exported file + */ + exportToHtml(...args: Parameters): ReturnType { + return this._export.exportToHtml(...args); + } + + /** + * Export the current session branch to a JSONL file. + * Writes the session header followed by all entries on the current branch path. + * @param outputPath Target file path. If omitted, generates a timestamped file in cwd. + * @returns The resolved output file path. + */ + exportToJsonl(...args: Parameters): ReturnType { + return this._export.exportToJsonl(...args); + } + + /** + * Get text content of last assistant message. + * Useful for /copy command. + * @returns Text content, or undefined if no assistant message exists + */ + getLastAssistantText( + ...args: Parameters + ): ReturnType { + return this._contextView.getLastAssistantText(...args); + } + + // ================================================================== // Extension System + // ================================================================== + createReplacedSessionContext(): ReplacedSessionContext { + const context = Object.defineProperties( + {}, + Object.getOwnPropertyDescriptors(this._extensionRunner.createCommandContext()), + ) as ReplacedSessionContext; + context.sendMessage = (message, options) => this.sendCustomMessage(message, options); + context.sendUserMessage = (content, options) => this.sendUserMessage(content, options); + return context; + } + + hasExtensionHandlers(eventType: string): boolean { + return this._extensionRunner.hasHandlers(eventType); + } + + get extensionRunner(): ExtensionRunner { + return this._extensionRunner; + } +} diff --git a/packages/coding-agent/src/session/autonomy/autonomous.ts b/packages/coding-agent/src/session/autonomy/autonomous.ts new file mode 100644 index 0000000000..c62d8b1219 --- /dev/null +++ b/packages/coding-agent/src/session/autonomy/autonomous.ts @@ -0,0 +1,627 @@ +import { createHash } from "node:crypto"; +import { createReadStream } from "node:fs"; +import { lstat, readlink } from "node:fs/promises"; +import { resolve } from "node:path"; +import type { AssistantMessage, Usage, UserMessage } from "@earendil-works/pi-ai"; +import { spawnHidden, waitForChildProcess } from "../../utils/child-process.js"; +import { killProcessTree, trackDetachedChildPid, untrackDetachedChildPid } from "../../utils/shell.js"; + +export interface AgentAutonomousConfig { + enabled?: boolean; + maxContinuations?: number; + maxTurns?: number; + maxTokens?: number; + timeoutMs?: number; + continuationPrompt?: string; + gates?: AgentAutonomousGateConfig; +} + +export interface AgentAutonomousGateConfig { + commands?: string[]; + maxRetries?: number; + timeoutMs?: number; +} + +export interface AgentAutonomousGateFailure { + command: string; + attempt: number; + exitText: string; + output: string; +} + +export interface AgentAutonomousStatus { + enabled: boolean; + continuationsUsed: number; + turnsUsed: number; + tokensUsed: number; + startedAt?: number; + limits: Required>; + gates: Required; + gateAttempts: Record; + lastGateFailure?: AgentAutonomousGateFailure; +} + +export const DEFAULT_AUTONOMOUS_CONTINUATION_PROMPT = + "No human input is available in autonomous mode. Continue working until the host evaluator, verifier, or configured autonomous limits stop the run. If you were asking the user a question, make a reasonable assumption and verify it. If you believe you are blocked, prove it with host-observable evidence, preserve that evidence, and keep looking for safe progress while budget remains. Do not end the session yourself; the verifier/evaluator decides completion when configured gates pass."; + +export const DEFAULT_AUTONOMOUS_LIMITS: Required< + Omit +> = { + maxContinuations: 3, + maxTurns: 12, + maxTokens: 80_000, + timeoutMs: 30 * 60 * 1000, +}; + +export const DEFAULT_AUTONOMOUS_GATES: Required = { + commands: [], + maxRetries: 3, + timeoutMs: 5 * 60 * 1000, +}; + +/** + * JSON-safe sentinel meaning "no cap". Limit checks compare usage against the + * configured value, so this stays finite and serializes to JSON while no + * realistic run can ever reach it. + */ +export const UNLIMITED_AUTONOMOUS_LIMIT = Number.MAX_SAFE_INTEGER; + +export function isUnlimitedAutonomousLimit(value: number): boolean { + return value >= UNLIMITED_AUTONOMOUS_LIMIT; +} + +const MAX_GATE_OUTPUT_CHARS = 6000; +const MAX_CHILD_PROCESS_OUTPUT_CHARS = 1024 * 1024; + +export interface AutonomousRuntimeState { + enabled: boolean; + continuationsUsed: number; + turnsUsed: number; + tokensUsed: number; + startedAt?: number; + limits: Required>; + continuationPrompt: string; + gates: Required; + gateAttempts: Record; + lastGateFailure?: GateFailure; + lastGateFailureSnapshot?: GitWorktreeSnapshot; +} + +export type AutonomousLimitReason = "maxContinuations" | "maxTurns" | "maxTokens" | "timeoutMs"; +export type AutonomousGateResult = "passed" | "failed" | "retry_exhausted"; + +type AutonomousLimitState = Pick< + AgentAutonomousStatus, + "continuationsUsed" | "turnsUsed" | "tokensUsed" | "startedAt" | "limits" +>; + +export interface AutonomousDecision { + shouldContinue: boolean; + reason: "missing_terminal_evidence" | "gate_failed" | "not_needed" | "limit_reached"; +} + +interface GitWorktreeSnapshot { + status: string; + diff: string; + untrackedHash: string; +} + +interface AutonomousOperationOptions { + cwd?: string; + signal?: AbortSignal; +} + +type GateFailure = AgentAutonomousGateFailure; + +export function createAutonomousRuntimeState( + config?: AgentAutonomousConfig, + _options: { cwd?: string } = {}, +): AutonomousRuntimeState { + const enabled = config?.enabled === true; + return { + enabled, + continuationsUsed: 0, + turnsUsed: 0, + tokensUsed: 0, + startedAt: enabled ? Date.now() : undefined, + limits: { + maxContinuations: normalizeLimit(config?.maxContinuations, DEFAULT_AUTONOMOUS_LIMITS.maxContinuations), + maxTurns: normalizeLimit(config?.maxTurns, DEFAULT_AUTONOMOUS_LIMITS.maxTurns), + maxTokens: normalizeLimit(config?.maxTokens, DEFAULT_AUTONOMOUS_LIMITS.maxTokens), + timeoutMs: normalizeLimit(config?.timeoutMs, DEFAULT_AUTONOMOUS_LIMITS.timeoutMs), + }, + continuationPrompt: config?.continuationPrompt?.trim() || DEFAULT_AUTONOMOUS_CONTINUATION_PROMPT, + gates: { + commands: [...(config?.gates?.commands ?? DEFAULT_AUTONOMOUS_GATES.commands)], + maxRetries: normalizeLimit(config?.gates?.maxRetries, DEFAULT_AUTONOMOUS_GATES.maxRetries), + timeoutMs: normalizeLimit(config?.gates?.timeoutMs, DEFAULT_AUTONOMOUS_GATES.timeoutMs), + }, + gateAttempts: {}, + lastGateFailure: undefined, + lastGateFailureSnapshot: undefined, + }; +} + +export function setAutonomousEnabled( + state: AutonomousRuntimeState, + enabled: boolean, + _options: { cwd?: string } = {}, +): void { + state.enabled = enabled; + if (enabled) { + state.continuationsUsed = 0; + state.turnsUsed = 0; + state.tokensUsed = 0; + state.startedAt = Date.now(); + state.gateAttempts = {}; + state.lastGateFailure = undefined; + state.lastGateFailureSnapshot = undefined; + } else { + state.startedAt = undefined; + state.gateAttempts = {}; + state.lastGateFailure = undefined; + state.lastGateFailureSnapshot = undefined; + } +} + +/** + * Apply user-provided budget and gate options to a live runtime state. + * Only fields present in `config` change; unspecified fields keep the state's + * current values, which come from the session/CLI configuration or defaults. + */ +export function setAutonomousLimits(state: AutonomousRuntimeState, config?: AgentAutonomousConfig): void { + if (!config) { + return; + } + state.limits.maxContinuations = normalizeLimit(config.maxContinuations, state.limits.maxContinuations); + state.limits.maxTurns = normalizeLimit(config.maxTurns, state.limits.maxTurns); + state.limits.maxTokens = normalizeLimit(config.maxTokens, state.limits.maxTokens); + state.limits.timeoutMs = normalizeLimit(config.timeoutMs, state.limits.timeoutMs); + if (config.continuationPrompt?.trim()) { + state.continuationPrompt = config.continuationPrompt.trim(); + } + if (config.gates) { + if (config.gates.commands !== undefined) { + state.gates.commands = [...config.gates.commands]; + } + state.gates.maxRetries = normalizeLimit(config.gates.maxRetries, state.gates.maxRetries); + state.gates.timeoutMs = normalizeLimit(config.gates.timeoutMs, state.gates.timeoutMs); + } +} + +export function autonomousStatus(state: AutonomousRuntimeState): AgentAutonomousStatus { + return { + enabled: state.enabled, + continuationsUsed: state.continuationsUsed, + turnsUsed: state.turnsUsed, + tokensUsed: state.tokensUsed, + startedAt: state.startedAt, + limits: { ...state.limits }, + gates: { ...state.gates, commands: [...state.gates.commands] }, + gateAttempts: { ...state.gateAttempts }, + lastGateFailure: state.lastGateFailure ? { ...state.lastGateFailure } : undefined, + }; +} + +export function addAutonomousUsage(state: AutonomousRuntimeState, usage: Usage | undefined): void { + if (!state.enabled) { + return; + } + state.turnsUsed++; + state.tokensUsed += autonomousTokenDelta(usage); +} + +export function addAutonomousContinuation(state: AutonomousRuntimeState): void { + if (!state.enabled) { + return; + } + state.continuationsUsed++; +} + +function autonomousTokenDelta(usage: Usage | undefined): number { + if (!usage) { + return 0; + } + // Cache-read tokens are repeated context served from provider cache. Counting them + // cumulatively makes long autonomous verifier loops exhaust their host-side token + // budget far before the non-cached work reaches the configured cap. + return usage.input + usage.output + usage.cacheWrite; +} + +export async function nextAutonomousContinuation( + state: AutonomousRuntimeState, + message: AssistantMessage, + options: AutonomousOperationOptions = {}, + now = Date.now(), +): Promise { + options.signal?.throwIfAborted(); + if (!state.enabled) { + return undefined; + } + const decision = await shouldAutonomouslyContinue(state, message, options, now); + options.signal?.throwIfAborted(); + if (!decision.shouldContinue) { + return undefined; + } + state.continuationsUsed++; + const gateFailureText = decision.reason === "gate_failed" ? buildGateFailureContinuation(state, now) : undefined; + return { + role: "user", + content: [ + { + type: "text", + text: gateFailureText ?? `[autonomous-continuation]\n\n${state.continuationPrompt}`, + }, + ], + timestamp: now, + }; +} + +export async function shouldAutonomouslyContinue( + state: AutonomousRuntimeState, + message: AssistantMessage, + options: AutonomousOperationOptions = {}, + now = Date.now(), +): Promise { + options.signal?.throwIfAborted(); + if (!state.enabled || message.stopReason === "error" || message.stopReason === "aborted") { + return { shouldContinue: false, reason: "not_needed" }; + } + const gateResult = await refreshAutonomousQualityGates(state, options); + options.signal?.throwIfAborted(); + if (gateResult) { + if (gateResult === "passed") { + return { shouldContinue: false, reason: "not_needed" }; + } + if (gateResult === "retry_exhausted" || autonomousLimitReason(state, now)) { + return { shouldContinue: false, reason: "limit_reached" }; + } + return { shouldContinue: true, reason: "gate_failed" }; + } + if (autonomousLimitReason(state, now)) { + return { shouldContinue: false, reason: "limit_reached" }; + } + return { shouldContinue: true, reason: "missing_terminal_evidence" }; +} + +export function autonomousLimitReason( + state: AutonomousLimitState, + now = Date.now(), +): AutonomousLimitReason | undefined { + if (state.continuationsUsed >= state.limits.maxContinuations) { + return "maxContinuations"; + } + if (state.turnsUsed >= state.limits.maxTurns) { + return "maxTurns"; + } + if (state.tokensUsed >= state.limits.maxTokens) { + return "maxTokens"; + } + if (state.startedAt !== undefined && now - state.startedAt >= state.limits.timeoutMs) { + return "timeoutMs"; + } + return undefined; +} + +export async function refreshAutonomousQualityGates( + state: AutonomousRuntimeState, + options: AutonomousOperationOptions = {}, +): Promise { + options.signal?.throwIfAborted(); + if (!state.enabled || state.gates.commands.length === 0) { + return undefined; + } + return await runAutonomousQualityGates(state, options.cwd, options.signal); +} + +async function runAutonomousQualityGates( + state: AutonomousRuntimeState, + cwd: string | undefined, + signal: AbortSignal | undefined, +): Promise { + signal?.throwIfAborted(); + if (!cwd) { + return "failed"; + } + for (const command of state.gates.commands) { + const currentSnapshot = await captureGitWorktreeSnapshot(cwd, signal); + signal?.throwIfAborted(); + if ( + state.lastGateFailure?.command === command && + state.lastGateFailureSnapshot && + gitWorktreeSnapshotsEqual(currentSnapshot, state.lastGateFailureSnapshot) + ) { + const attempt = (state.gateAttempts[command] ?? state.lastGateFailure.attempt) + 1; + state.gateAttempts[command] = attempt; + state.lastGateFailure = { + ...state.lastGateFailure, + attempt, + exitText: "not rerun: workspace unchanged since previous failed gate", + output: + "The autonomous gate was not rerun because the workspace has not changed since this failure. Edit source files, tests, or a blocker artifact before attempting to finish again.", + }; + return attempt > state.gates.maxRetries ? "retry_exhausted" : "failed"; + } + const result = await runChildProcess(command, [], { + cwd, + shell: true, + timeoutMs: state.gates.timeoutMs, + maxOutputChars: MAX_GATE_OUTPUT_CHARS, + signal, + }); + signal?.throwIfAborted(); + const postRunSnapshot = await captureGitWorktreeSnapshot(cwd, signal); + signal?.throwIfAborted(); + if (result.status === 0 && !result.error && !result.timedOut) { + state.gateAttempts[command] = 0; + if (state.lastGateFailure?.command === command) { + state.lastGateFailure = undefined; + state.lastGateFailureSnapshot = undefined; + } + continue; + } + const attempt = (state.gateAttempts[command] ?? 0) + 1; + state.gateAttempts[command] = attempt; + const exitText = formatProcessExit(result); + state.lastGateFailure = { + command, + attempt, + exitText, + output: truncateGateOutput( + [result.stdout, result.stderr].filter(Boolean).join("\n").trim(), + result.outputTruncated, + ), + }; + state.lastGateFailureSnapshot = postRunSnapshot; + return attempt > state.gates.maxRetries ? "retry_exhausted" : "failed"; + } + state.lastGateFailure = undefined; + state.lastGateFailureSnapshot = undefined; + return "passed"; +} + +export function buildAutonomousGateFailureContinuation( + failure: AgentAutonomousGateFailure, + maxRetries: number, + timestamp = Date.now(), +): string { + return ( + `[autonomous-continuation: gate-failed]\n\n` + + `Autonomous quality gate failed (attempt ${failure.attempt}/${maxRetries}): \`${failure.command}\` ${failure.exitText}.\n` + + (failure.output ? `\nOutput:\n${failure.output}\n` : "\n") + + `\nContinue working. Fix the failure, then produce terminal evidence. Timestamp: ${new Date(timestamp).toISOString()}.` + ); +} + +function buildGateFailureContinuation(state: AutonomousRuntimeState, timestamp: number): string | undefined { + const failure = state.lastGateFailure; + if (!failure) { + return undefined; + } + return buildAutonomousGateFailureContinuation(failure, state.gates.maxRetries, timestamp); +} + +function gitWorktreeSnapshotsEqual(a: GitWorktreeSnapshot | undefined, b: GitWorktreeSnapshot | undefined): boolean { + return !!a && !!b && a.status === b.status && a.diff === b.diff && a.untrackedHash === b.untrackedHash; +} + +async function captureGitWorktreeSnapshot( + cwd: string | undefined, + signal?: AbortSignal, +): Promise { + signal?.throwIfAborted(); + if (!cwd) { + return undefined; + } + const pathspec = [ + "--", + ".", + ":(exclude)verification", + ":(exclude)target", + ":(exclude).vf-prime-agent", + ":(exclude)Cargo.lock", + ":(exclude)submission.tar.gz", + ":(exclude)runner_args.log", + ]; + const status = await runChildProcess( + "git", + ["--no-optional-locks", "status", "--porcelain=v1", "-z", "-uall", "--no-renames", ...pathspec], + { + cwd, + timeoutMs: 10_000, + signal, + }, + ); + signal?.throwIfAborted(); + if (status.status !== 0 || status.error || status.timedOut || status.outputTruncated) { + return undefined; + } + const diff = await runChildProcess( + "git", + ["--no-optional-locks", "diff", "--no-ext-diff", "--binary", "HEAD", ...pathspec], + { + cwd, + timeoutMs: 10_000, + signal, + }, + ); + signal?.throwIfAborted(); + if (diff.status !== 0 || diff.error || diff.timedOut || diff.outputTruncated) { + return undefined; + } + return { + status: status.stdout, + diff: diff.stdout, + untrackedHash: await hashUntrackedFiles(cwd, status.stdout, signal), + }; +} + +function untrackedPathsFromStatus(status: string): string[] { + return status + .split("\0") + .filter((entry) => entry.startsWith("?? ")) + .map((entry) => entry.slice(3)) + .sort(); +} + +async function hashUntrackedFiles(cwd: string, status: string, signal?: AbortSignal): Promise { + const aggregate = createHash("sha256"); + for (const path of untrackedPathsFromStatus(status)) { + signal?.throwIfAborted(); + aggregate.update(path); + aggregate.update("\0"); + aggregate.update(await hashUntrackedPath(resolve(cwd, path), signal)); + aggregate.update("\0"); + } + signal?.throwIfAborted(); + return aggregate.digest("hex"); +} + +async function hashUntrackedPath(path: string, signal?: AbortSignal): Promise { + try { + signal?.throwIfAborted(); + const stat = await lstat(path); + signal?.throwIfAborted(); + if (stat.isSymbolicLink()) { + const target = await readlink(path); + signal?.throwIfAborted(); + return `symlink:${target}`; + } + if (!stat.isFile()) { + return `other:${stat.mode}:${stat.size}:${stat.mtimeMs}`; + } + const hash = createHash("sha256"); + for await (const chunk of createReadStream(path, { signal })) { + hash.update(chunk); + } + signal?.throwIfAborted(); + return `file:${hash.digest("hex")}`; + } catch (error) { + signal?.throwIfAborted(); + return `error:${error instanceof Error ? error.message : String(error)}`; + } +} + +interface ChildProcessResult { + status: number | null; + signal: NodeJS.Signals | null; + stdout: string; + stderr: string; + error?: Error; + timedOut?: boolean; + outputTruncated: boolean; +} + +function runChildProcess( + command: string, + args: string[], + options: { + cwd?: string; + shell?: boolean; + timeoutMs?: number; + maxOutputChars?: number; + signal?: AbortSignal; + } = {}, +): Promise { + options.signal?.throwIfAborted(); + return new Promise((resolve) => { + const child = spawnHidden(command, args, { + cwd: options.cwd, + detached: process.platform !== "win32", + shell: options.shell === true, + stdio: ["ignore", "pipe", "pipe"], + }); + if (child.pid) { + trackDetachedChildPid(child.pid); + } + let stdout = ""; + let stderr = ""; + let error: Error | undefined; + let timedOut = false; + let outputTruncated = false; + let settled = false; + const maxOutputChars = options.maxOutputChars ?? MAX_CHILD_PROCESS_OUTPUT_CHARS; + const finish = (result: Pick) => { + if (settled) { + return; + } + settled = true; + if (timer) { + clearTimeout(timer); + } + options.signal?.removeEventListener("abort", abort); + if (child.pid) { + untrackDetachedChildPid(child.pid); + } + resolve({ ...result, stdout, stderr, error, timedOut, outputTruncated }); + }; + const timer = options.timeoutMs + ? setTimeout(() => { + timedOut = true; + if (child.pid) { + killProcessTree(child.pid); + } else { + child.kill("SIGKILL"); + } + }, options.timeoutMs) + : undefined; + const abort = () => { + if (child.pid) { + killProcessTree(child.pid); + } else { + child.kill("SIGKILL"); + } + }; + options.signal?.addEventListener("abort", abort, { once: true }); + if (options.signal?.aborted) { + abort(); + } + child.stdout?.setEncoding("utf8"); + child.stderr?.setEncoding("utf8"); + child.stdout?.on("data", (chunk: string) => { + const remaining = maxOutputChars - stdout.length; + if (remaining > 0) { + stdout += chunk.slice(0, remaining); + } + outputTruncated ||= chunk.length > remaining; + }); + child.stderr?.on("data", (chunk: string) => { + const remaining = maxOutputChars - stderr.length; + if (remaining > 0) { + stderr += chunk.slice(0, remaining); + } + outputTruncated ||= chunk.length > remaining; + }); + void waitForChildProcess(child).then( + (status) => finish({ status, signal: child.signalCode }), + (err: Error) => { + error = err; + finish({ status: child.exitCode, signal: child.signalCode }); + }, + ); + }); +} + +function formatProcessExit(result: ChildProcessResult): string { + if (result.timedOut) { + return "timed out"; + } + if (result.error) { + return result.error.message; + } + return result.signal ? `terminated by ${result.signal}` : `exited ${result.status ?? "unknown"}`; +} + +function truncateGateOutput(output: string, outputAlreadyTruncated = false, maxChars = MAX_GATE_OUTPUT_CHARS): string { + if (output.length <= maxChars && !outputAlreadyTruncated) { + return output; + } + return `${output.slice(0, maxChars)}\n... [truncated]`; +} + +function normalizeLimit(value: number | undefined, fallback: number): number { + if (!Number.isFinite(value) || value === undefined || value <= 0) { + return fallback; + } + return Math.trunc(value); +} diff --git a/packages/coding-agent/src/session/autonomy/continuation.ts b/packages/coding-agent/src/session/autonomy/continuation.ts new file mode 100644 index 0000000000..73d5d161d3 --- /dev/null +++ b/packages/coding-agent/src/session/autonomy/continuation.ts @@ -0,0 +1,391 @@ +import type { Agent, AgentMessage } from "@earendil-works/pi-agent-core"; +import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; +import { parseCommandArgs } from "../../core/prompt-templates.js"; +import type { SessionManager } from "../../core/session-manager.js"; +import { parseSessionSlashCommand } from "../../core/slash-commands.js"; +import type { AgentSessionEvent } from "../agent-session.js"; +import type { SessionCompaction } from "../compaction/controller.js"; +import type { CustomMessage } from "../context/messages.js"; +import type { SessionInputAdmission } from "../input/input-admission.js"; +import { + createPreparedTurnAction, + primaryDeliveryRecord, + type QueuedSessionAction, +} from "../input/prepared-actions.js"; +import type { SessionContinuation } from "../turns/continuation.js"; +import { + type AgentAutonomousConfig, + type AgentAutonomousStatus, + type AutonomousRuntimeState, + addAutonomousContinuation, + addAutonomousUsage, + autonomousStatus, + createAutonomousRuntimeState, + isUnlimitedAutonomousLimit, + nextAutonomousContinuation, + refreshAutonomousQualityGates, + setAutonomousEnabled, + setAutonomousLimits, + UNLIMITED_AUTONOMOUS_LIMIT, +} from "./autonomous.js"; + +type AutonomousSlashCommand = { kind: "status" } | { kind: "on"; config?: AgentAutonomousConfig } | { kind: "off" }; +type AutonomousRuntimeSnapshot = Pick< + AutonomousRuntimeState, + "continuationsUsed" | "gateAttempts" | "lastGateFailure" | "lastGateFailureSnapshot" +>; + +const AUTONOMOUS_STATUS_NUMBER_FORMAT = new Intl.NumberFormat("en-US"); + +const AUTONOMOUS_BUDGET_USAGE = + "Usage: /autonomous [status|off] or /autonomous on [--max-continuations ] [--max-turns ] [--max-tokens ] [--timeout-ms ] [--gate ] [--gate-retries ] [--gate-timeout-ms ]"; + +// `/autonomous` budget flags mirror the `--autonomous-*` CLI options. The CLI +// spelling (`--autonomous-max-continuations`) is accepted as an alias so the +// exact CLI budget flags also work from the slash command. +const AUTONOMOUS_BUDGET_FLAGS: ReadonlySet = new Set([ + "max-continuations", + "max-turns", + "max-tokens", + "timeout-ms", + "gate", + "gate-retries", + "gate-timeout-ms", +]); + +function parseAutonomousBudgetInt(flag: string, value: string, allowUnlimited = false): number { + if (allowUnlimited && value.toLowerCase() === "unlimited") { + return UNLIMITED_AUTONOMOUS_LIMIT; + } + // Commas and underscores are accepted as digit separators (100,000,000). + const digits = value.replace(/[,_]/g, ""); + if (!/^[1-9]\d*$/.test(digits)) { + throw new Error( + `--${flag} must be a positive integer${allowUnlimited ? ' or "unlimited"' : ""}. ${AUTONOMOUS_BUDGET_USAGE}`, + ); + } + return Number(digits); +} + +function parseAutonomousBudgetOptions(tokens: string[]): AgentAutonomousConfig { + const config: AgentAutonomousConfig = {}; + const gateCommands: string[] = []; + for (let i = 0; i < tokens.length; i++) { + const token = tokens[i]!; + if (!token.startsWith("--")) { + throw new Error(`Unexpected autonomous argument: ${token}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + const equalsIndex = token.indexOf("="); + const rawFlag = equalsIndex === -1 ? token : token.slice(0, equalsIndex); + const inlineValue = equalsIndex === -1 ? undefined : token.slice(equalsIndex + 1); + const flag = rawFlag.startsWith("--autonomous-") ? rawFlag.slice("--autonomous-".length) : rawFlag.slice(2); + if (!AUTONOMOUS_BUDGET_FLAGS.has(flag)) { + throw new Error(`Unknown autonomous budget flag: ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + let value = inlineValue; + if (value === undefined) { + const next = tokens[i + 1]; + if (next === undefined || next.startsWith("--")) { + throw new Error(`Missing value for ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + value = next; + i++; + } + if (value === "") { + throw new Error(`Missing value for ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + switch (flag) { + case "gate": + gateCommands.push(value); + break; + case "gate-retries": + config.gates = config.gates ?? {}; + config.gates.maxRetries = parseAutonomousBudgetInt(flag, value); + break; + case "gate-timeout-ms": + config.gates = config.gates ?? {}; + config.gates.timeoutMs = parseAutonomousBudgetInt(flag, value); + break; + case "max-continuations": + config.maxContinuations = parseAutonomousBudgetInt(flag, value, true); + break; + case "max-turns": + config.maxTurns = parseAutonomousBudgetInt(flag, value, true); + break; + case "max-tokens": + config.maxTokens = parseAutonomousBudgetInt(flag, value, true); + break; + case "timeout-ms": + config.timeoutMs = parseAutonomousBudgetInt(flag, value, true); + break; + } + } + if (gateCommands.length > 0) { + config.gates = { ...config.gates, commands: gateCommands }; + } + // Named budget flags define the whole budget: any limit the user did not + // name stops cutting the run short. With no budget flags at all, the + // configured or default limits still apply. + if ( + config.maxContinuations !== undefined || + config.maxTurns !== undefined || + config.maxTokens !== undefined || + config.timeoutMs !== undefined + ) { + config.maxContinuations ??= UNLIMITED_AUTONOMOUS_LIMIT; + config.maxTurns ??= UNLIMITED_AUTONOMOUS_LIMIT; + config.maxTokens ??= UNLIMITED_AUTONOMOUS_LIMIT; + config.timeoutMs ??= UNLIMITED_AUTONOMOUS_LIMIT; + } + return config; +} + +export interface SessionAutonomousContinuationHost { + getStatus(): AgentAutonomousStatus; + getCwd(): string; + getAgent(): Pick; + getStore(): Pick; + emit(event: AgentSessionEvent): void; + getContinuation(): Pick; + getArrivalEpoch(): number; + admit: SessionInputAdmission["admitSessionInput"]; + cancelActions(predicate: (action: QueuedSessionAction) => boolean, error: Error): QueuedSessionAction[]; + emitQueueUpdate(): void; + getCompaction(): Pick; + getUnfinishedActionCount(): number; + cancelContinuation(): void; +} +export class SessionAutonomousContinuation { + private readonly state: AutonomousRuntimeState; + private suppressionDepth = 0; + private readonly suppressedMessages = new WeakSet(); + private readonly thresholdContinuations = new WeakMap(); + private readonly snapshots = new WeakMap(); + private pendingThresholdMessages: AgentMessage[] = []; + constructor( + config: AgentAutonomousConfig | undefined, + private readonly host: SessionAutonomousContinuationHost, + ) { + this.state = createAutonomousRuntimeState(config, { cwd: host.getCwd() }); + } + forgetSnapshot(message: AgentMessage): void { + this.snapshots.delete(message); + } + + takePendingThresholdMessages(): AgentMessage[] { + return this.pendingThresholdMessages.splice(0); + } + + recordUsage(usage: Usage): void { + addAutonomousUsage(this.state, usage); + } + isSuppressed(messages: AgentMessage[]): boolean { + return this.suppressionDepth > 0 || messages.some((message) => this.suppressedMessages.has(message)); + } + next(message: AssistantMessage, signal?: AbortSignal): Promise { + return nextAutonomousContinuation(this.state, message, { cwd: this.host.getCwd(), signal }); + } + + parseAutonomousSlashCommand(text: string): AutonomousSlashCommand | undefined { + const command = parseSessionSlashCommand(text); + if (command?.name !== "autonomous") return undefined; + const tokens = parseCommandArgs(command.args); + if (tokens.length === 0 || tokens[0]!.toLowerCase() === "status") { + if (tokens.length > 1) { + throw new Error(`Unexpected autonomous argument: ${tokens[1]}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + return { kind: "status" }; + } + const subcommand = tokens[0]!.toLowerCase(); + if (subcommand === "on" || subcommand === "enable" || subcommand === "enabled") { + return { kind: "on", config: parseAutonomousBudgetOptions(tokens.slice(1)) }; + } + if (subcommand === "off" || subcommand === "disable" || subcommand === "disabled") { + if (tokens.length > 1) { + throw new Error(`Unexpected autonomous argument: ${tokens[1]}. ${AUTONOMOUS_BUDGET_USAGE}`); + } + return { kind: "off" }; + } + throw new Error(AUTONOMOUS_BUDGET_USAGE); + } + + formatAutonomousStatus(): string { + const status = this.host.getStatus(); + const state = status.enabled ? "on" : "off"; + const elapsedSeconds = status.startedAt ? Math.round((Date.now() - status.startedAt) / 1000) : 0; + const gateSummary = + status.gates.commands.length > 0 ? status.gates.commands.map((command) => `"${command}"`).join(", ") : "none"; + const formatCount = (value: number): string => + isUnlimitedAutonomousLimit(value) ? "unlimited" : AUTONOMOUS_STATUS_NUMBER_FORMAT.format(value); + const timeBudget = isUnlimitedAutonomousLimit(status.limits.timeoutMs) + ? "unlimited" + : `${AUTONOMOUS_STATUS_NUMBER_FORMAT.format(Math.round(status.limits.timeoutMs / 1000))}s`; + return `[autonomous-status: ${state}]\n\nContinuations: ${formatCount(status.continuationsUsed)}/${formatCount(status.limits.maxContinuations)}. Turns: ${formatCount(status.turnsUsed)}/${formatCount(status.limits.maxTurns)}. Tokens: ${formatCount(status.tokensUsed)}/${formatCount(status.limits.maxTokens)}. Time: ${elapsedSeconds}s/${timeBudget}. Gates: ${gateSummary}.`; + } + + emitAutonomousStatus(): void { + const message = { + role: "custom" as const, + customType: "autonomous_status", + content: this.formatAutonomousStatus(), + display: true, + details: this.host.getStatus(), + timestamp: Date.now(), + } satisfies CustomMessage; + this.host.getAgent().state.messages.push(message); + this.host + .getStore() + .appendCustomMessageEntry(message.customType, message.content, message.display, message.details); + this.host.emit({ type: "message_start", message }); + this.host.emit({ type: "message_end", message }); + } + + async handleAutonomousSlashCommand(text: string): Promise { + const command = this.parseAutonomousSlashCommand(text); + if (!command) { + return false; + } + if (command.kind === "on") { + setAutonomousEnabled(this.state, true, { cwd: this.host.getCwd() }); + setAutonomousLimits(this.state, command.config); + } else if (command.kind === "off") { + setAutonomousEnabled(this.state, false); + this.clearQueuedAutonomousContinuations(); + } + this.emitAutonomousStatus(); + return true; + } + + snapshotAutonomousRuntimeState(): AutonomousRuntimeSnapshot { + return { + continuationsUsed: this.state.continuationsUsed, + gateAttempts: { ...this.state.gateAttempts }, + lastGateFailure: this.state.lastGateFailure ? { ...this.state.lastGateFailure } : undefined, + lastGateFailureSnapshot: this.state.lastGateFailureSnapshot + ? { ...this.state.lastGateFailureSnapshot } + : undefined, + }; + } + + restoreAutonomousRuntimeSnapshot(snapshot: AutonomousRuntimeSnapshot): void { + this.state.continuationsUsed = snapshot.continuationsUsed; + this.state.gateAttempts = { ...snapshot.gateAttempts }; + this.state.lastGateFailure = snapshot.lastGateFailure ? { ...snapshot.lastGateFailure } : undefined; + this.state.lastGateFailureSnapshot = snapshot.lastGateFailureSnapshot + ? { ...snapshot.lastGateFailureSnapshot } + : undefined; + } + + async queueAutonomousContinuationForThresholdCompaction( + message: AssistantMessage, + ): Promise { + const queuedMessage = this.thresholdContinuations.get(message); + if (queuedMessage && this.host.getContinuation().messages.includes(queuedMessage)) { + return queuedMessage; + } + const snapshot = this.snapshotAutonomousRuntimeState(); + const arrivalEpoch = this.host.getArrivalEpoch(); + const autonomousMessage = await nextAutonomousContinuation(this.state, message, { + cwd: this.host.getCwd(), + signal: this.host.getAgent().signal, + }); + if (!autonomousMessage) { + return undefined; + } + if (this.host.getArrivalEpoch() !== arrivalEpoch) { + this.restoreAutonomousRuntimeSnapshot(snapshot); + return undefined; + } + this.thresholdContinuations.set(message, autonomousMessage); + this.snapshots.set(autonomousMessage, snapshot); + this.host.getContinuation().track(autonomousMessage); + this.pendingThresholdMessages.push(autonomousMessage); + const text = + typeof autonomousMessage.content === "string" + ? autonomousMessage.content + : autonomousMessage.content.map((block) => (block.type === "text" ? block.text : "")).join("\n"); + this.host.admit( + createPreparedTurnAction("followUp", text, undefined, { + message: autonomousMessage, + }), + ); + return autonomousMessage; + } + + clearQueuedAutonomousContinuations( + options: { restoreAutonomousState?: boolean; messages?: AgentMessage[] } = {}, + ): void { + const requestedMessages = options.messages ?? [...this.host.getContinuation().messages]; + const requestedMessageSet = new Set(requestedMessages); + const queuedMessages = this.host.getContinuation().messages.filter((message) => requestedMessageSet.has(message)); + if (queuedMessages.length === 0) { + return; + } + const queuedMessageSet = new Set(queuedMessages); + this.host.getContinuation().remove(queuedMessageSet); + this.host.getAgent().removeQueuedMessages((message) => queuedMessageSet.has(message)); + this.host.cancelActions( + (action) => action.payload.kind === "turn" && queuedMessageSet.has(primaryDeliveryRecord(action).message), + new Error("Queued autonomous continuation was cleared before delivery."), + ); + this.host.emitQueueUpdate(); + if (options.restoreAutonomousState) { + for (const queuedMessage of queuedMessages) { + const snapshot = this.snapshots.get(queuedMessage); + if (snapshot) { + this.restoreAutonomousRuntimeSnapshot(snapshot); + break; + } + } + } + for (const queuedMessage of queuedMessages) { + this.snapshots.delete(queuedMessage); + } + this.pendingThresholdMessages = this.pendingThresholdMessages.filter((message) => !queuedMessageSet.has(message)); + if (options.messages === undefined) { + this.host.getCompaction().resetContinuation(); + } + if (!this.host.getAgent().hasQueuedMessages() && this.host.getUnfinishedActionCount() === 0) { + this.host.cancelContinuation(); + } + } + + clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction( + shouldContinueAfterThreshold: boolean, + queuedMessages: AgentMessage[], + ): void { + if (shouldContinueAfterThreshold) { + this.clearQueuedAutonomousContinuations({ + restoreAutonomousState: true, + messages: queuedMessages, + }); + } + } + + getAutonomousStatus(): AgentAutonomousStatus { + return autonomousStatus(this.state); + } + + recordHostAutonomousContinuation(): void { + addAutonomousContinuation(this.state); + } + + async refreshAutonomousGates(): Promise { + await refreshAutonomousQualityGates(this.state, { + cwd: this.host.getCwd(), + }); + } + + async runWithAutonomousContinuationSuppressed(fn: () => Promise): Promise { + this.suppressionDepth++; + try { + return await fn(); + } finally { + this.suppressionDepth--; + } + } + + markAutonomousContinuationSuppressed(message: AgentMessage): void { + this.suppressedMessages.add(message); + } +} diff --git a/packages/coding-agent/src/session/children/child-projection.ts b/packages/coding-agent/src/session/children/child-projection.ts index 067ebaf99a..1a6dc113b4 100644 --- a/packages/coding-agent/src/session/children/child-projection.ts +++ b/packages/coding-agent/src/session/children/child-projection.ts @@ -1,6 +1,5 @@ import type { AgentSessionMessageAgentSummary, AgentSessionMessageListResult } from "../../core/agent-messages.js"; -import type { AgentSession } from "../../core/agent-session.js"; -import { createDefaultRlmSubagentSessionName, type RlmListSubagentsResult } from "../../core/rlm-runtime.js"; +import type { AgentSession } from "../agent-session.js"; import { compactRlmText, type RetainedRlmChild, @@ -9,6 +8,8 @@ import { readAssistantText, rlmChildLabel, } from "./child-types.js"; +import type { RlmListSubagentsResult } from "./runtime-contracts.js"; +import { createDefaultRlmSubagentSessionName } from "./spawn-options.js"; interface ChildVisibility { isDeleting(id: string): boolean; diff --git a/packages/coding-agent/src/session/children/child-run.ts b/packages/coding-agent/src/session/children/child-run.ts index 4bfdd30dfb..60d7d632ac 100644 --- a/packages/coding-agent/src/session/children/child-run.ts +++ b/packages/coding-agent/src/session/children/child-run.ts @@ -1,18 +1,11 @@ import type { AssistantMessage } from "@earendil-works/pi-ai"; import { AGENT_MESSAGE_CUSTOM_TYPE, type AgentSessionMessage } from "../../core/agent-messages.js"; -import type { AgentSession } from "../../core/agent-session.js"; +import type { AgentSession } from "../agent-session.js"; import { type CustomMessage, createRlmChildFailureMessage, createRlmChildTerminalNoticeMessage, -} from "../../core/messages.js"; -import type { - CreateRlmSubagentRuntimeOptions, - RlmSpawnHandle, - RlmSubagentRegistryEntry, - RlmSubagentRuntime, - SubagentRuntimeHost, -} from "../../core/rlm-runtime.js"; +} from "../context/messages.js"; import { compactRlmText, createChildDeferred, @@ -22,6 +15,13 @@ import { readAssistantText, } from "./child-types.js"; import type { ChildRuntimeRequest, SessionChildrenHost } from "./children.js"; +import type { + CreateRlmSubagentRuntimeOptions, + RlmSpawnHandle, + RlmSubagentRegistryEntry, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "./runtime-contracts.js"; interface ChildTaskLifecycle { admitRun(run: RlmChildRun): void; diff --git a/packages/coding-agent/src/session/children/child-runtime.ts b/packages/coding-agent/src/session/children/child-runtime.ts index e9048b5cd3..c46734f93a 100644 --- a/packages/coding-agent/src/session/children/child-runtime.ts +++ b/packages/coding-agent/src/session/children/child-runtime.ts @@ -2,12 +2,12 @@ import { randomUUID } from "node:crypto"; import { mkdirSync } from "node:fs"; import { join } from "node:path"; import { Agent } from "@earendil-works/pi-agent-core"; -import { AgentSession } from "../../core/agent-session.js"; import type { ModelRegistry } from "../../core/model-registry.js"; import type { ResourceLoader } from "../../core/resource-loader.js"; -import type { CreateRlmSubagentRuntimeOptions, RlmSubagentRuntime } from "../../core/rlm-runtime.js"; import { SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; +import { AgentSession } from "../agent-session.js"; +import type { CreateRlmSubagentRuntimeOptions, RlmSubagentRuntime } from "./runtime-contracts.js"; export interface InlineChildRuntimeHost { cwd: string; diff --git a/packages/coding-agent/src/session/children/child-state.ts b/packages/coding-agent/src/session/children/child-state.ts index 034eb55bc1..0029183de2 100644 --- a/packages/coding-agent/src/session/children/child-state.ts +++ b/packages/coding-agent/src/session/children/child-state.ts @@ -1,6 +1,6 @@ -import type { RlmMaxDepthSource, RlmMaxDepthStatus, SetRlmMaxDepthResult } from "../../core/rlm-max-depth.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; +import type { RlmMaxDepthSource, RlmMaxDepthStatus, SetRlmMaxDepthResult } from "./max-depth.js"; interface PersistedRlmMaxDepthState { maxDepth: number; diff --git a/packages/coding-agent/src/session/children/child-types.ts b/packages/coding-agent/src/session/children/child-types.ts index 0a645bbef5..59bd6744c8 100644 --- a/packages/coding-agent/src/session/children/child-types.ts +++ b/packages/coding-agent/src/session/children/child-types.ts @@ -1,6 +1,6 @@ import type { Api, AssistantMessage, Model } from "@earendil-works/pi-ai"; -import type { AgentSession } from "../../core/agent-session.js"; -import type { RlmSubagentRegistryEntry } from "../../core/rlm-runtime.js"; +import type { AgentSession } from "../agent-session.js"; +import type { RlmSubagentRegistryEntry } from "./runtime-contracts.js"; export type RlmChildAgentStatus = "queued" | "running" | "done" | "error" | "cancelled"; diff --git a/packages/coding-agent/src/session/children/child-usage.ts b/packages/coding-agent/src/session/children/child-usage.ts index f33f120bf2..8108089c63 100644 --- a/packages/coding-agent/src/session/children/child-usage.ts +++ b/packages/coding-agent/src/session/children/child-usage.ts @@ -7,7 +7,7 @@ import type { SessionManager, SessionMessageEntry, } from "../../core/session-manager.js"; -import { addAssistantUsage, cloneUsage, emptyUsage, subtractAssistantUsage } from "../../core/usage.js"; +import { addAssistantUsage, cloneUsage, emptyUsage, subtractAssistantUsage } from "../context/usage.js"; export interface ChildUsageHost { sessionManager: Pick; diff --git a/packages/coding-agent/src/session/children/children.ts b/packages/coding-agent/src/session/children/children.ts index a73ae6be96..75e66e85b1 100644 --- a/packages/coding-agent/src/session/children/children.ts +++ b/packages/coding-agent/src/session/children/children.ts @@ -9,23 +9,9 @@ import { assertDirectAgentMessageTarget, formatAgentSessionNameUnavailable, } from "../../core/agent-messages.js"; -import type { AgentSession, AgentSessionEvent } from "../../core/agent-session.js"; -import type { CustomMessage } from "../../core/messages.js"; -import { - type CreateRlmSubagentRuntimeOptions, - createDefaultRlmSubagentSessionName, - normalizeRequestedRlmSubagentModel, - normalizeRequestedRlmSubagentSessionName, - normalizeRequestedRlmSubagentThinkingLevel, - type RlmCreateSessionResult, - type RlmDeleteSubagentResult, - type RlmListSubagentsResult, - type RlmSpawnHandle, - type RlmSubagentRegistryEntry, - type RlmSubagentRuntime, - type SubagentRuntimeHost, -} from "../../core/rlm-runtime.js"; import type { SemanticEdgeRecorder } from "../../core/semantic-edges.js"; +import type { AgentSession, AgentSessionEvent } from "../agent-session.js"; +import type { CustomMessage } from "../context/messages.js"; import { buildChildList, snapshotChildRun, snapshotRetainedChild } from "./child-projection.js"; import { launchChildTask } from "./child-run.js"; import { @@ -38,6 +24,22 @@ import { type RlmChildRun, } from "./child-types.js"; import type { ChildUsageTracker } from "./child-usage.js"; +import type { + CreateRlmSubagentRuntimeOptions, + RlmCreateSessionResult, + RlmDeleteSubagentResult, + RlmListSubagentsResult, + RlmSpawnHandle, + RlmSubagentRegistryEntry, + RlmSubagentRuntime, + SubagentRuntimeHost, +} from "./runtime-contracts.js"; +import { + createDefaultRlmSubagentSessionName, + normalizeRequestedRlmSubagentModel, + normalizeRequestedRlmSubagentSessionName, + normalizeRequestedRlmSubagentThinkingLevel, +} from "./spawn-options.js"; export interface ChildRuntimeRequest { id: string; diff --git a/packages/coding-agent/src/session/children/host-requests.ts b/packages/coding-agent/src/session/children/host-requests.ts new file mode 100644 index 0000000000..56e4824848 --- /dev/null +++ b/packages/coding-agent/src/session/children/host-requests.ts @@ -0,0 +1,58 @@ +import type { HostRequestHandler } from "../../core/kernel/index.js"; +import type { + RlmCreateSessionHandler, + RlmDeleteSubagentHandler, + RlmListSubagentsHandler, + RlmRunHandler, +} from "./runtime-contracts.js"; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +export function createRlmCreateSessionHostHandler(handler: RlmCreateSessionHandler): HostRequestHandler { + return async (payload) => { + if (typeof payload.prompt !== "string") { + throw new Error("rlm.create_session prompt must be a string"); + } + const kwargs = isRecord(payload.kwargs) ? payload.kwargs : {}; + const result = await handler({ prompt: payload.prompt, kwargs }); + return result as unknown as Record; + }; +} + +/** Adapt an RlmRunHandler into the typed `rlm.run` kernel host handler. */ +export function createRlmRunHostHandler(handler: RlmRunHandler): HostRequestHandler { + return async (payload) => { + if (typeof payload.prompt !== "string") { + throw new Error("rlm.spawn prompt must be a string"); + } + const kwargs = isRecord(payload.kwargs) ? payload.kwargs : {}; + const cellSourceCode = typeof payload.cellSourceCode === "string" ? payload.cellSourceCode : undefined; + const result = await handler({ + prompt: payload.prompt, + kwargs, + cellSourceCode, + }); + return result as unknown as Record; + }; +} + +/** Expose the current parent session's direct RLM child registry to its kernel. */ +export function createRlmListSubagentsHostHandler(handler: RlmListSubagentsHandler): HostRequestHandler { + return async () => { + const { subagents } = await handler(); + return { subagents }; + }; +} + +/** Delete one direct child selected from the current parent session's registry. */ +export function createRlmDeleteSubagentHostHandler(handler: RlmDeleteSubagentHandler): HostRequestHandler { + return async (payload) => { + if (typeof payload.target !== "string" || !payload.target.trim()) { + throw new Error("rlm.delete_subagent target must be a non-empty string"); + } + const { subagent, outcome } = await handler(payload.target.trim()); + return outcome === undefined ? { subagent } : { subagent, outcome }; + }; +} diff --git a/packages/coding-agent/src/session/children/max-depth.ts b/packages/coding-agent/src/session/children/max-depth.ts new file mode 100644 index 0000000000..e51e6f65b9 --- /dev/null +++ b/packages/coding-agent/src/session/children/max-depth.ts @@ -0,0 +1,13 @@ +/** Wire-safe types for the immediate /rlm-max-depth state APIs. */ + +export type RlmMaxDepthSource = "default" | "env" | "global" | "inherited" | "chat"; + +export interface RlmMaxDepthStatus { + maxDepth: number; + source: RlmMaxDepthSource; +} + +export interface SetRlmMaxDepthResult extends RlmMaxDepthStatus { + globalSaved: boolean; + globalError?: string; +} diff --git a/packages/coding-agent/src/session/children/runtime-contracts.ts b/packages/coding-agent/src/session/children/runtime-contracts.ts new file mode 100644 index 0000000000..06fe042cc2 --- /dev/null +++ b/packages/coding-agent/src/session/children/runtime-contracts.ts @@ -0,0 +1,110 @@ +import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Api, Model, ServiceTier } from "@earendil-works/pi-ai"; +import type { ToolDefinition } from "../../core/extensions/index.js"; +import type { AgentSession } from "../agent-session.js"; + +/** Request emitted by `rlm.spawn`; cellSourceCode preserves the spawning cell for display. */ +export interface RlmRunRequest { + prompt: string; + kwargs: Record; + cellSourceCode?: string; +} + +interface RlmCreateSessionRequest { + prompt: string; + kwargs: Record; +} + +export interface RlmCreateSessionResult { + active_session_id: string; + session_id: string; + name: string; + session_file: string; + model: string; +} + +export interface RlmSpawnHandle { + rlm_child_id: string; + name: string; + session_dir: string; + model: string; +} + +export type RlmSubagentRegistryStatus = "running" | "completed" | "error"; + +export interface RlmSubagentRegistryEntry { + rlm_child_id: string; + active_session_id: string | null; + session_id: string | null; + session_name: string; + session_dir: string; + status: RlmSubagentRegistryStatus; +} + +export interface RlmListSubagentsResult { + subagents: RlmSubagentRegistryEntry[]; +} + +export interface RlmDeleteSubagentResult { + subagent: RlmSubagentRegistryEntry; + outcome?: "deleted" | "skipped_running"; +} + +export type RlmRunHandler = (request: RlmRunRequest) => Promise>; +export type RlmCreateSessionHandler = (request: RlmCreateSessionRequest) => Promise; + +export type RlmListSubagentsHandler = () => RlmListSubagentsResult | Promise; +export type RlmDeleteSubagentHandler = (target: string) => Promise; +export interface RlmSubagentRuntime { + session: AgentSession; +} + +export interface CreateRlmSubagentRuntimeOptions { + parentSession: AgentSession; + id: string; + prompt: string; + sessionName: string; + sessionDir: string; + model: Model; + thinkingLevel: ThinkingLevel; + serviceTier: ServiceTier; + scopedModels: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>; + activeToolNames: string[]; + allowedToolNames?: string[]; + customTools: ToolDefinition[]; + includeGoals: boolean; + includeCompactSkill: boolean; + rlmDepth: number; + rlmMaxDepth: number; + rlmParentNodeId: string; + /** Request ID of the parent model call whose tool call caused this spawn. */ + spawnedByRequestId?: string; + /** Source of the Python cell that spawned this subagent, for display. */ + spawnCode?: string; + /** Publish the session to the parent before a host makes the runtime addressable. */ + onSessionPublished?: (session: AgentSession) => void; +} + +export interface CreateRlmRootSessionOptions { + prompt: string; + sessionName?: string; + cwd: string; + model: Model; + thinkingLevel: ThinkingLevel; +} + +export interface SubagentRuntimeHost { + createRlmSubagentRuntime(options: CreateRlmSubagentRuntimeOptions): Promise; + createRlmRootSession?(options: CreateRlmRootSessionOptions): Promise; + /** Persist host-owned completion before the child becomes passivation-eligible. */ + completeRlmSubagentRuntime?(childId: string, session: AgentSession): boolean; + /** Release a host-owned child after its detached initial task settles. */ + releaseRlmSubagentRuntime?: ( + runtime: RlmSubagentRuntime, + options: CreateRlmSubagentRuntimeOptions, + status: "done" | "error" | "cancelled", + ) => Promise; + /** Close or remove the host-owned child; session is absent when a persisted child is still passive. */ + deleteRlmSubagentRuntime(childId: string, session?: AgentSession): Promise; + disposeRlmSubagentRuntimes?(): Promise; +} diff --git a/packages/coding-agent/src/session/children/spawn-options.ts b/packages/coding-agent/src/session/children/spawn-options.ts new file mode 100644 index 0000000000..643caeb1b3 --- /dev/null +++ b/packages/coding-agent/src/session/children/spawn-options.ts @@ -0,0 +1,71 @@ +import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; +import { THINKING_LEVELS } from "../../core/thinking-levels.js"; + +const RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH = 64; +export function normalizeRequestedRlmSubagentSessionName(value: unknown, operation = "rlm.spawn"): string | undefined { + if (value === undefined) { + return undefined; + } + if (typeof value !== "string") { + throw new Error(`${operation} name must be a string`); + } + const name = value.trim(); + if (!name) { + throw new Error(`${operation} name must not be empty`); + } + if (name.length > RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH) { + throw new Error(`${operation} name must be at most ${RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH} characters`); + } + return name; +} + +export function normalizeRequestedRlmSubagentThinkingLevel( + value: unknown, + operation = "rlm.spawn", +): ThinkingLevel | undefined { + if (value === undefined) { + return undefined; + } + if (typeof value !== "string") { + throw new Error(`${operation} thinking must be a string`); + } + const level = value.trim().toLowerCase(); + if (!THINKING_LEVELS.includes(level as ThinkingLevel)) { + throw new Error(`${operation} thinking must be one of: ${THINKING_LEVELS.join(", ")}`); + } + return level as ThinkingLevel; +} + +export function normalizeRequestedRlmSubagentModel(value: unknown, operation = "rlm.spawn"): string | undefined { + if (value === undefined) { + return undefined; + } + if (typeof value !== "string") { + throw new Error(`${operation} model must be a string`); + } + const model = value.trim(); + if (!model) { + throw new Error(`${operation} model must not be empty`); + } + return model; +} + +/** Create a readable, collision-resistant default name usable as an agent-message selector. */ +export function createDefaultRlmSubagentSessionName(prompt: string, childId: string): string { + const promptSlug = prompt + .normalize("NFKD") + .replace(/[\u0300-\u036f]/g, "") + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, ""); + const idSuffix = + childId + .replace(/^sub-/, "") + .replace(/[^A-Za-z0-9]+/g, "") + .slice(-8) || "child"; + const fixedLength = "subagent--".length + idSuffix.length; + const promptPart = (promptSlug || "worker") + .slice(0, Math.max(1, RLM_SUBAGENT_SESSION_NAME_MAX_LENGTH - fixedLength)) + .replace(/-+$/g, ""); + return `subagent-${promptPart || "worker"}-${idSuffix}`; +} diff --git a/packages/coding-agent/src/session/compaction/compaction-execution.ts b/packages/coding-agent/src/session/compaction/compaction-execution.ts index 818825d303..00fdad033c 100644 --- a/packages/coding-agent/src/session/compaction/compaction-execution.ts +++ b/packages/coding-agent/src/session/compaction/compaction-execution.ts @@ -1,189 +1,7 @@ -import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { Api, Model } from "@earendil-works/pi-ai"; -import { - type CompactionResult, - type CompactionSettings, - compact, - prepareCompaction, -} from "../../core/compaction/index.js"; -import type { ExtensionRunner, SessionBeforeCompactResult } from "../../core/extensions/index.js"; -import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; -import { modelRequestHeaders, type SemanticEdgeRecorder } from "../../core/semantic-edges.js"; -import type { CompactionEntry, SessionManager } from "../../core/session-manager.js"; - -export class CompactionSkippedError extends Error {} - -export interface CompactionExecutionOptions { - model: Model; - apiKey: string; - headers?: Record; - customInstructions?: string; - signal: AbortSignal; -} - -export interface CompactionExecutionHost { - getSessionStore(): Pick; - getSettings(): CompactionSettings; - getSemanticEdges(): Pick< - SemanticEdgeRecorder, - "beginCompaction" | "startCompactionRequest" | "failRequest" | "finishRequest" | "finishCompaction" - >; - getExtensions(): Pick; - getThinkingLevel(): ThinkingLevel; - getRetryPolicy(): ProviderRetryPolicy; - getSessionId?(): string; - getHarnessDigest(): string; - rebuildContext(): void; - syncKernelState(): Promise; - reapDeletedChildren(): Promise; -} - -export async function performSessionCompaction( - host: CompactionExecutionHost, - options: CompactionExecutionOptions, -): Promise { - const { model, apiKey, headers, customInstructions, signal } = options; - const pathEntries = host.getSessionStore().getBranch(); - const settings = host.getSettings(); - - const preparation = prepareCompaction(pathEntries, settings); - if (!preparation) { - const lastEntry = pathEntries[pathEntries.length - 1]; - if (lastEntry?.type === "compaction") { - throw new CompactionSkippedError("Already compacted"); - } - throw new CompactionSkippedError("Session is too short to compact — try again once it grows"); - } - - let extensionCompaction: CompactionResult | undefined; - let fromExtension = false; - - const semanticCompaction = host.getSemanticEdges().beginCompaction(); - let compactionRecorded = false; - const uncommittedSlices: string[] = []; - let compactionSettled = false; - let summary: string; - let firstKeptEntryId: string; - let tokensBefore: number; - let details: CompactionResult["details"]; - let usage: CompactionResult["usage"]; - try { - if (host.getExtensions().hasHandlers("session_before_compact")) { - const result = (await host.getExtensions().emit({ - type: "session_before_compact", - preparation, - branchEntries: pathEntries, - customInstructions, - signal, - })) as SessionBeforeCompactResult | undefined; - - if (result?.cancel) { - throw new Error("Compaction cancelled"); - } - - if (result?.compaction) { - extensionCompaction = result.compaction; - fromExtension = true; - } - } - - if (extensionCompaction) { - ({ summary, firstKeptEntryId, tokensBefore, details, usage } = extensionCompaction); - } else { - // Each summary wire call gets its own request ID: split turns send two - // different bodies, and one Idempotency-Key must never cover both. A slice - // that succeeds on the wire stays uncommitted until the compaction itself - // commits: a racing sibling's failure (or an abort) must leave no committed - // summary request for the next turn's continuation edge to attach to. - const summaryCall = async ( - call: (callHeaders: Record | undefined) => Promise, - ): Promise => { - const requestId = host.getSemanticEdges().startCompactionRequest(semanticCompaction.compactionId); - if (requestId === undefined) { - return call(headers); - } - try { - const result = await call({ ...headers, ...modelRequestHeaders(requestId) }); - // A slice resolving after a sibling's rejection already settled the - // compaction would push into a drained list and stay in-flight forever. - if (compactionSettled) { - host.getSemanticEdges().failRequest(requestId); - } else { - uncommittedSlices.push(requestId); - } - return result; - } catch (error) { - host.getSemanticEdges().failRequest(requestId); - throw error; - } - }; - ({ summary, firstKeptEntryId, tokensBefore, details, usage } = await compact( - preparation, - model, - apiKey, - headers, - customInstructions, - signal, - host.getThinkingLevel(), - summaryCall, - host.getRetryPolicy(), - host.getSessionId?.(), - )); - } - - if (signal.aborted) { - throw new Error("Compaction cancelled"); - } - - // Ledger-before-effect: the compaction outcome is durable before the transcript - // commits it. Marked first: the ID is consumed even when the write throws, and a - // second finish attempt would mask the original I/O error. - compactionRecorded = true; - compactionSettled = true; - for (const requestId of uncommittedSlices.splice(0)) { - host.getSemanticEdges().finishRequest(requestId); - } - host.getSemanticEdges().finishCompaction(semanticCompaction.compactionId, "completed"); - // Attached mechanically; the digest never flows through the summarizer LLM. - host - .getSessionStore() - .appendCompaction( - summary, - firstKeptEntryId, - tokensBefore, - details, - fromExtension, - customInstructions, - usage, - host.getHarnessDigest(), - ); - } catch (error) { - compactionSettled = true; - for (const requestId of uncommittedSlices.splice(0)) { - host.getSemanticEdges().failRequest(requestId); - } - if (!compactionRecorded) { - const cancelled = - error instanceof Error && (error.name === "AbortError" || error.message === "Compaction cancelled"); - host.getSemanticEdges().finishCompaction(semanticCompaction.compactionId, cancelled ? "cancelled" : "failed"); - } - throw error; - } - const newEntries = host.getSessionStore().getEntries(); - host.rebuildContext(); - - const savedCompactionEntry = newEntries.find((e) => e.type === "compaction" && e.summary === summary) as - | CompactionEntry - | undefined; - if (savedCompactionEntry) { - await host.getExtensions().emit({ - type: "session_compact", - compactionEntry: savedCompactionEntry, - fromExtension, - }); - } - await host.syncKernelState(); - await host.reapDeletedChildren(); - - return { summary, firstKeptEntryId, tokensBefore, details }; -} +// Compatibility exports; implementation lives with its session owner. +export { + type CompactionExecutionHost, + type CompactionExecutionOptions, + CompactionSkippedError, + performSessionCompaction, +} from "./execution.js"; diff --git a/packages/coding-agent/src/session/compaction/compaction.ts b/packages/coding-agent/src/session/compaction/compaction.ts index cb4dfc916b..70b987e773 100644 --- a/packages/coding-agent/src/session/compaction/compaction.ts +++ b/packages/coding-agent/src/session/compaction/compaction.ts @@ -1,560 +1,7 @@ -import type { AgentEvent, AgentMessage } from "@earendil-works/pi-agent-core"; -import { type Api, type AssistantMessage, isContextOverflow, type Model } from "@earendil-works/pi-ai"; -import { formatNoModelSelectedMessage } from "../../core/auth-guidance.js"; -import { - type CompactionResult, - type CompactionSettings, - calculateContextTokens, - estimateContextTokens, - prepareCompaction, - shouldCompact, -} from "../../core/compaction/index.js"; -import type { ContextUsage } from "../../core/extensions/index.js"; -import { - type CompactionOutcome, - type CompactionOutcomeReason, - type CustomMessage, - createCompactionOutcomeMessage, -} from "../../core/messages.js"; -import type { ModelRegistry } from "../../core/model-registry.js"; -import { getLatestCompactionEntry, type SessionManager } from "../../core/session-manager.js"; -import { type CompactionExecutionOptions, CompactionSkippedError } from "./compaction-execution.js"; - -export type CompactionReason = "manual" | "threshold" | "overflow" | "requested"; -export type SessionCompactionEvent = - | { type: "compaction_start"; reason: CompactionReason; customInstructions?: string } - | { - type: "compaction_end"; - reason: CompactionReason; - result: CompactionResult | undefined; - aborted: boolean; - willRetry: boolean; - errorMessage?: string; - errorSeverity?: "warning" | "error"; - customInstructions?: string; - }; - -export interface SessionCompactionHost { - includesCompactSkill(): boolean; - getContextUsage(): ContextUsage | undefined; - getSettings(): CompactionSettings; - runAutomatic(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise; - queueGoalContinuation(message: AssistantMessage): boolean; - queueAutonomousContinuation(message: AssistantMessage): Promise; - beginRefinementAbort(): { promise: Promise; finish(): void } | undefined; - getModel(): Model | undefined; - isStreaming(): boolean; - getRequiredAuth( - model: Model, - ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; - getAuth(model: Model): ReturnType; - perform(options: CompactionExecutionOptions): Promise; - disconnect(): void; - reconnect(): void; - abortSession(): Promise; - getContinuationState(): { scheduled: boolean; continueAfterSessionInput: boolean }; - afterManualCompaction(signal: AbortSignal, wasScheduled: boolean, continueAfterSessionInput: boolean): void; - getMessages(): AgentMessage[]; - replaceMessages(messages: AgentMessage[]): void; - hasAgentQueuedMessages(): boolean; - hasPendingSessionWork(): boolean; - scheduleContinuation(continueAfterSessionInput?: boolean): void; - scheduleRefinement(willContinue: boolean): void; - takeThresholdAutonomousMessages(): AgentMessage[]; - getThresholdGoalContinuation(): AgentMessage | undefined; - clearAutonomousContinuations(shouldContinue: boolean, messages: AgentMessage[]): void; - clearGoalContinuation(message: AgentMessage | undefined): void; - getSessionStore(): Pick; - retainUnpersistedOutcome(message: CustomMessage): void; - emit(event: SessionCompactionEvent | Extract): void; - notifyCheckpoints(): void; - scheduleInput(): void; -} - -export class SessionCompaction { - private manualAbort: AbortController | undefined; - private automaticAbort: AbortController | undefined; - private activeOperation: Promise | undefined; - private overflowStage: "idle" | "attempted" | "reported" = "idle"; - private pendingRequest: { customInstructions?: string } | undefined; - private continueAfterThreshold = false; - - constructor(private readonly host: SessionCompactionHost) {} - - handleCompactHostRequest(type: string, payload: Record = {}): Record { - if (!this.host.includesCompactSkill()) { - throw new Error("the compact skill is disabled in this session"); - } - switch (type) { - case "compact.status": { - const usage = this.host.getContextUsage(); - return { - tokens: usage?.tokens ?? null, - context_window: usage?.contextWindow ?? null, - percent: usage?.percent ?? null, - scheduled: this.hasPendingRequest, - }; - } - case "compact.run": { - const instructions = payload.instructions; - if (instructions !== undefined && typeof instructions !== "string") { - throw new Error("compact.run instructions must be a string when provided"); - } - if (!this.host.isStreaming()) { - return { - scheduled: false, - reason: "no active turn; compaction can only be requested while a turn is running", - }; - } - const preparation = prepareCompaction(this.host.getSessionStore().getBranch(), this.host.getSettings()); - if (!preparation) { - const lastEntry = this.host.getSessionStore().getBranch().at(-1); - return { - scheduled: false, - reason: lastEntry?.type === "compaction" ? "already compacted" : "session is too short to compact", - }; - } - this.request(instructions); - return { - scheduled: true, - note: "Compaction runs when the current turn ends; you resume automatically afterwards. Continue working normally.", - }; - } - default: - throw new Error(`unknown compact request type "${type}"`); - } - } - - private get model(): Model | undefined { - return this.host.getModel(); - } - get operation(): Promise | undefined { - return this.activeOperation; - } - get isRunning(): boolean { - return this.automaticAbort !== undefined || this.manualAbort !== undefined; - } - get hasPendingRequest(): boolean { - return this.pendingRequest !== undefined; - } - get overflowRecovery(): "idle" | "attempted" | "reported" { - return this.overflowStage; - } - - request(customInstructions?: string): void { - this.pendingRequest = { customInstructions }; - } - clearRequest(): void { - this.pendingRequest = undefined; - } - requestContinuation(): void { - this.continueAfterThreshold = true; - } - resetContinuation(): void { - this.continueAfterThreshold = false; - } - resetOverflowRecovery(): void { - this.overflowStage = "idle"; - } - markOverflowAttempted(): void { - this.overflowStage = "attempted"; - } - markOverflowReported(): void { - this.overflowStage = "reported"; - } - abort(): void { - this.manualAbort?.abort(); - this.automaticAbort?.abort(); - } - abortAutomatic(): void { - this.automaticAbort?.abort(); - } - - getThresholdContextTokens( - assistantMessage: AssistantMessage, - compactionTimestamp: number | undefined, - ): number | undefined { - const messages = this.host.getMessages(); - const estimate = estimateContextTokens(messages); - if (estimate.lastUsageIndex !== null) { - // Verify the usage source is post-compaction. Kept pre-compaction messages - // have stale usage reflecting the old (larger) context and would falsely - // trigger compaction right after one just finished. - const usageMsg = messages[estimate.lastUsageIndex]; - if ( - compactionTimestamp !== undefined && - usageMsg.role === "assistant" && - (usageMsg as AssistantMessage).timestamp <= compactionTimestamp - ) { - return undefined; - } - return estimate.tokens; - } - if (assistantMessage.stopReason === "error") return undefined; - return calculateContextTokens(assistantMessage.usage); - } - - async check( - assistantMessage: AssistantMessage, - skipAbortedCheck = true, - queueAutonomousContinuation = true, - ): Promise { - // An abort drops any compaction the model requested this turn, even on the - // pre-prompt path (skipAbortedCheck=false) which continues to threshold checks. - if (assistantMessage.stopReason === "aborted") { - this.clearRequest(); - const refinementAbort = this.host.beginRefinementAbort(); - if (refinementAbort) { - await refinementAbort.promise.catch(() => undefined); - refinementAbort.finish(); - } - if (skipAbortedCheck) return false; - } - - const settings = this.host.getSettings(); - const contextWindow = this.model?.contextWindow ?? 0; - - // Skip overflow check if the message came from a different model. - // This handles the case where user switched from a smaller-context model (e.g. opus) - // to a larger-context model (e.g. codex) - the overflow error from the old model - // shouldn't trigger compaction for the new model. - const sameModel = - this.model && assistantMessage.provider === this.model.provider && assistantMessage.model === this.model.id; - - // Skip overflow/threshold checks if this assistant message is older than the - // latest compaction boundary. This prevents a stale pre-compaction usage/error - // from retriggering compaction on the first prompt after compaction. - const compactionEntry = getLatestCompactionEntry(this.host.getSessionStore().getBranch()); - const compactionTimestamp = compactionEntry ? new Date(compactionEntry.timestamp).getTime() : undefined; - const assistantIsFromBeforeCompaction = - compactionTimestamp !== undefined && assistantMessage.timestamp <= compactionTimestamp; - - // Case 1: Overflow - takes priority over a pending model request so the error - // strip + retry still happen; the compaction it runs consumes the request. - if ( - !assistantIsFromBeforeCompaction && - (settings.enabled || this.hasPendingRequest) && - sameModel && - isContextOverflow(assistantMessage, contextWindow) - ) { - if (this.overflowRecovery !== "idle") { - if (this.overflowRecovery === "attempted") { - this.markOverflowReported(); - this.endUnsuccessfully( - "overflow", - "failed", - "Context overflow recovery failed after one compact-and-retry attempt. Try reducing context or switching to a larger-context model.", - ); - } - return false; - } - - this.markOverflowAttempted(); - // Remove the error message from agent state (it IS saved to session for history, - // but we don't want it in context for the retry) - const messages = this.host.getMessages(); - if (messages.length > 0 && messages[messages.length - 1].role === "assistant") { - this.host.replaceMessages(messages.slice(0, -1)); - } - return await this.host.runAutomatic("overflow", true); - } - - if (this.hasPendingRequest) { - return await this.host.runAutomatic("requested", false); - } - - if (!settings.enabled || assistantIsFromBeforeCompaction) return false; - - // Case 3: Threshold - context is getting large. - // Use the full-session estimate so messages appended after the last successful - // assistant usage are included, matching the /usage context display. - const contextTokens = this.getThresholdContextTokens(assistantMessage, compactionTimestamp); - if (contextTokens === undefined) return false; - if (shouldCompact(contextTokens, contextWindow, settings)) { - if (queueAutonomousContinuation && this.host.queueGoalContinuation(assistantMessage)) { - this.requestContinuation(); - } else if (queueAutonomousContinuation && (await this.host.queueAutonomousContinuation(assistantMessage))) { - this.requestContinuation(); - } - return await this.host.runAutomatic("threshold", false); - } - return false; - } - - async compact(customInstructions?: string, options: { skipAbort?: boolean } = {}): Promise { - if (options.skipAbort && this.host.isStreaming()) { - throw new Error("Cannot compact without aborting while the agent is running."); - } - const { scheduled: hadPostCompactionContinue, continueAfterSessionInput } = this.host.getContinuationState(); - this.host.disconnect(); - if (!options.skipAbort) await this.host.abortSession(); - let didCompact = false; - const compactionAbort = new AbortController(); - this.manualAbort = compactionAbort; - let resolveCompactionOperation: () => void = () => {}; - const compactionOperation = new Promise((resolve) => { - resolveCompactionOperation = resolve; - }); - this.activeOperation = compactionOperation; - this.host.emit({ - type: "compaction_start", - reason: "manual", - customInstructions, - }); - - try { - if (!this.model) { - throw new Error(formatNoModelSelectedMessage()); - } - - const { apiKey, headers, requestModel } = await this.host.getRequiredAuth(this.model); - const result = await this.host.perform({ - model: requestModel ?? this.model, - apiKey, - headers, - customInstructions, - signal: compactionAbort.signal, - }); - - this.host.emit({ - type: "compaction_end", - reason: "manual", - result, - aborted: false, - willRetry: false, - customInstructions, - }); - didCompact = true; - // A manual compaction satisfies any pending model request; on failure the - // request stays scheduled for the next turn boundary. - this.pendingRequest = undefined; - return result; - } catch (error) { - const message = error instanceof Error ? error.message : String(error); - const aborted = message === "Compaction cancelled" || (error instanceof Error && error.name === "AbortError"); - const skipped = error instanceof CompactionSkippedError; - this.host.emit({ - type: "compaction_end", - reason: "manual", - result: undefined, - aborted, - willRetry: false, - errorMessage: aborted ? undefined : skipped ? message : `Compaction failed: ${message}`, - errorSeverity: skipped ? "warning" : "error", - customInstructions, - }); - throw error; - } finally { - this.manualAbort = undefined; - this.host.reconnect(); - if (this.activeOperation === compactionOperation) { - this.activeOperation = undefined; - } - resolveCompactionOperation(); - this.host.notifyCheckpoints(); - this.host.scheduleInput(); - if (didCompact) { - this.host.afterManualCompaction( - compactionAbort.signal, - hadPostCompactionContinue, - continueAfterSessionInput, - ); - } - } - } - - endUnsuccessfully( - reason: CompactionOutcomeReason, - outcome: CompactionOutcome, - message: string, - options: { - aborted?: boolean; - errorSeverity?: "warning" | "error"; - customInstructions?: string; - } = {}, - ): void { - this.persistOutcome(reason, outcome, message); - this.host.emit({ - type: "compaction_end", - reason, - result: undefined, - aborted: options.aborted ?? false, - willRetry: false, - // Aborts are user-initiated; they carry no error message on the event. - errorMessage: options.aborted ? undefined : message, - errorSeverity: options.errorSeverity, - customInstructions: options.customInstructions, - }); - } - - private persistOutcome(reason: CompactionOutcomeReason, outcome: CompactionOutcome, message: string): void { - let outcomeMessage = createCompactionOutcomeMessage(message, { - reason, - outcome, - }); - try { - this.host - .getSessionStore() - .appendCustomMessageEntryWithRollback( - outcomeMessage.customType, - outcomeMessage.content, - outcomeMessage.display, - outcomeMessage.details, - ); - } catch (error) { - const persistenceError = error instanceof Error ? error.message : String(error); - outcomeMessage = createCompactionOutcomeMessage( - `${message}\n\nThis compaction outcome could not be saved to session history: ${persistenceError}`, - { reason, outcome }, - ); - // Not in the session file, so context rebuilds would drop the disclosure. - this.host.retainUnpersistedOutcome(outcomeMessage); - } - this.host.getMessages().push(outcomeMessage); - this.host.emit({ type: "message_start", message: outcomeMessage }); - this.host.emit({ type: "message_end", message: outcomeMessage }); - } - - async runAutomatic(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise { - // Any compaction consumes a pending model request and honors its instructions - // (overflow recovery can fire first and take the request with it). - const pending = this.pendingRequest; - this.pendingRequest = undefined; - const customInstructions = pending?.customInstructions; - const shouldContinueAfterCompaction = - (reason === "threshold" || reason === "requested") && this.continueAfterThreshold; - const queuedAutonomousContinuationsForThisCompaction = - reason === "threshold" && shouldContinueAfterCompaction ? this.host.takeThresholdAutonomousMessages() : []; - const queuedGoalContinuationForThisCompaction = - reason === "threshold" && shouldContinueAfterCompaction ? this.host.getThresholdGoalContinuation() : undefined; - this.continueAfterThreshold = false; - - // Requested/threshold stop the loop on purpose, so a failed or skipped compaction must not stall it. - // Overflow stays excluded: a failed overflow recovery must not re-issue the overflowing request. - const resumeAfterFailure = () => { - if ( - (reason === "requested" || reason === "threshold") && - (shouldContinueAfterCompaction || this.host.hasAgentQueuedMessages() || this.host.hasPendingSessionWork()) - ) { - this.host.scheduleContinuation(shouldContinueAfterCompaction); - } - }; - - this.host.emit({ type: "compaction_start", reason, customInstructions }); - this.automaticAbort = new AbortController(); - let resolveCompactionOperation: () => void = () => {}; - const compactionOperation = new Promise((resolve) => { - resolveCompactionOperation = resolve; - }); - this.activeOperation = compactionOperation; - - try { - const authResult = this.model ? await this.host.getAuth(this.model) : undefined; - if (!this.model || !authResult || !authResult.ok || !authResult.apiKey) { - const detail = - !this.model || !authResult - ? "no model is selected" - : authResult.ok - ? "no API key is available" - : authResult.error; - this.endUnsuccessfully(reason, "failed", `Compaction failed: ${detail}`); - this.host.clearAutonomousContinuations( - reason === "threshold" && shouldContinueAfterCompaction, - queuedAutonomousContinuationsForThisCompaction, - ); - resumeAfterFailure(); - return false; - } - - const result = await this.host.perform({ - model: authResult.requestModel ?? this.model, - apiKey: authResult.apiKey, - headers: authResult.headers, - customInstructions, - signal: this.automaticAbort.signal, - }); - - this.host.emit({ - type: "compaction_end", - reason, - result, - aborted: false, - willRetry, - customInstructions, - }); - // Queued work lives in both the agent queues and the session-owned queues. - const hasQueuedMessages = this.host.hasAgentQueuedMessages() || this.host.hasPendingSessionWork(); - const willContinueAfterCompaction = willRetry || shouldContinueAfterCompaction || hasQueuedMessages; - - if (willRetry) { - const messages = this.host.getMessages(); - const lastMsg = messages[messages.length - 1]; - if (lastMsg?.role === "assistant" && (lastMsg as AssistantMessage).stopReason === "error") { - this.host.replaceMessages(messages.slice(0, -1)); - } - - this.host.scheduleContinuation(true); - this.host.scheduleRefinement(willContinueAfterCompaction); - return true; - } else if (shouldContinueAfterCompaction || hasQueuedMessages) { - // Compaction can intentionally stop a tool loop between turns. - // Queued follow-up/steering/custom messages can also be waiting. - this.host.scheduleContinuation(shouldContinueAfterCompaction); - this.host.scheduleRefinement(willContinueAfterCompaction); - } else { - this.host.scheduleRefinement(willContinueAfterCompaction); - } - return false; - } catch (error) { - this.host.clearAutonomousContinuations( - reason === "threshold" && shouldContinueAfterCompaction, - queuedAutonomousContinuationsForThisCompaction, - ); - const errorMessage = error instanceof Error ? error.message : "compaction failed"; - const aborted = - errorMessage === "Compaction cancelled" || (error instanceof Error && error.name === "AbortError"); - if (aborted) { - this.host.clearGoalContinuation(queuedGoalContinuationForThisCompaction); - this.endUnsuccessfully( - reason, - "cancelled", - `${reason === "requested" ? "Requested c" : "C"}ompaction cancelled`, - { aborted: true, customInstructions }, - ); - return false; - } - if (error instanceof CompactionSkippedError) { - this.endUnsuccessfully( - reason, - "skipped", - reason === "requested" - ? `Requested compaction skipped: ${errorMessage}` - : `Auto-compaction skipped: ${errorMessage}`, - { errorSeverity: "warning", customInstructions }, - ); - resumeAfterFailure(); - return false; - } - this.endUnsuccessfully( - reason, - "failed", - reason === "overflow" - ? `Context overflow recovery failed: ${errorMessage}` - : reason === "requested" - ? `Requested compaction failed: ${errorMessage}` - : `Auto-compaction failed: ${errorMessage}`, - { customInstructions }, - ); - resumeAfterFailure(); - return false; - } finally { - this.automaticAbort = undefined; - if (this.activeOperation === compactionOperation) { - this.activeOperation = undefined; - } - resolveCompactionOperation(); - this.host.notifyCheckpoints(); - this.host.scheduleInput(); - } - } -} +// Compatibility exports; implementation lives with its session owner. +export { + type CompactionReason, + SessionCompaction, + type SessionCompactionEvent, + type SessionCompactionHost, +} from "./controller.js"; diff --git a/packages/coding-agent/src/session/compaction/controller.ts b/packages/coding-agent/src/session/compaction/controller.ts new file mode 100644 index 0000000000..4ef4bf316b --- /dev/null +++ b/packages/coding-agent/src/session/compaction/controller.ts @@ -0,0 +1,555 @@ +import type { AgentEvent, AgentMessage } from "@earendil-works/pi-agent-core"; +import { type Api, type AssistantMessage, isContextOverflow, type Model } from "@earendil-works/pi-ai"; +import { formatNoModelSelectedMessage } from "../../core/auth-guidance.js"; +import type { ContextUsage } from "../../core/extensions/index.js"; +import type { ModelRegistry } from "../../core/model-registry.js"; +import { getLatestCompactionEntry, type SessionManager } from "../../core/session-manager.js"; +import { + type CompactionOutcome, + type CompactionOutcomeReason, + type CustomMessage, + createCompactionOutcomeMessage, +} from "../context/messages.js"; +import { calculateContextTokens, estimateContextTokens } from "../context/token-estimate.js"; +import { type CompactionExecutionOptions, CompactionSkippedError } from "./execution.js"; +import { prepareCompaction, shouldCompact } from "./summary.js"; +import type { CompactionResult, CompactionSettings } from "./types.js"; + +export type CompactionReason = "manual" | "threshold" | "overflow" | "requested"; +export type SessionCompactionEvent = + | { type: "compaction_start"; reason: CompactionReason; customInstructions?: string } + | { + type: "compaction_end"; + reason: CompactionReason; + result: CompactionResult | undefined; + aborted: boolean; + willRetry: boolean; + errorMessage?: string; + errorSeverity?: "warning" | "error"; + customInstructions?: string; + }; + +export interface SessionCompactionHost { + includesCompactSkill(): boolean; + getContextUsage(): ContextUsage | undefined; + getSettings(): CompactionSettings; + runAutomatic(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise; + queueGoalContinuation(message: AssistantMessage): boolean; + queueAutonomousContinuation(message: AssistantMessage): Promise; + beginRefinementAbort(): { promise: Promise; finish(): void } | undefined; + getModel(): Model | undefined; + isStreaming(): boolean; + getRequiredAuth( + model: Model, + ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; + getAuth(model: Model): ReturnType; + perform(options: CompactionExecutionOptions): Promise; + disconnect(): void; + reconnect(): void; + abortSession(): Promise; + getContinuationState(): { scheduled: boolean; continueAfterSessionInput: boolean }; + afterManualCompaction(signal: AbortSignal, wasScheduled: boolean, continueAfterSessionInput: boolean): void; + getMessages(): AgentMessage[]; + replaceMessages(messages: AgentMessage[]): void; + hasAgentQueuedMessages(): boolean; + hasPendingSessionWork(): boolean; + scheduleContinuation(continueAfterSessionInput?: boolean): void; + scheduleRefinement(willContinue: boolean): void; + takeThresholdAutonomousMessages(): AgentMessage[]; + getThresholdGoalContinuation(): AgentMessage | undefined; + clearAutonomousContinuations(shouldContinue: boolean, messages: AgentMessage[]): void; + clearGoalContinuation(message: AgentMessage | undefined): void; + getSessionStore(): Pick; + retainUnpersistedOutcome(message: CustomMessage): void; + emit(event: SessionCompactionEvent | Extract): void; + notifyCheckpoints(): void; + scheduleInput(): void; +} + +export class SessionCompaction { + private manualAbort: AbortController | undefined; + private automaticAbort: AbortController | undefined; + private activeOperation: Promise | undefined; + private overflowStage: "idle" | "attempted" | "reported" = "idle"; + private pendingRequest: { customInstructions?: string } | undefined; + private continueAfterThreshold = false; + + constructor(private readonly host: SessionCompactionHost) {} + + handleCompactHostRequest(type: string, payload: Record = {}): Record { + if (!this.host.includesCompactSkill()) { + throw new Error("the compact skill is disabled in this session"); + } + switch (type) { + case "compact.status": { + const usage = this.host.getContextUsage(); + return { + tokens: usage?.tokens ?? null, + context_window: usage?.contextWindow ?? null, + percent: usage?.percent ?? null, + scheduled: this.hasPendingRequest, + }; + } + case "compact.run": { + const instructions = payload.instructions; + if (instructions !== undefined && typeof instructions !== "string") { + throw new Error("compact.run instructions must be a string when provided"); + } + if (!this.host.isStreaming()) { + return { + scheduled: false, + reason: "no active turn; compaction can only be requested while a turn is running", + }; + } + const preparation = prepareCompaction(this.host.getSessionStore().getBranch(), this.host.getSettings()); + if (!preparation) { + const lastEntry = this.host.getSessionStore().getBranch().at(-1); + return { + scheduled: false, + reason: lastEntry?.type === "compaction" ? "already compacted" : "session is too short to compact", + }; + } + this.request(instructions); + return { + scheduled: true, + note: "Compaction runs when the current turn ends; you resume automatically afterwards. Continue working normally.", + }; + } + default: + throw new Error(`unknown compact request type "${type}"`); + } + } + + private get model(): Model | undefined { + return this.host.getModel(); + } + get operation(): Promise | undefined { + return this.activeOperation; + } + get isRunning(): boolean { + return this.automaticAbort !== undefined || this.manualAbort !== undefined; + } + get hasPendingRequest(): boolean { + return this.pendingRequest !== undefined; + } + get overflowRecovery(): "idle" | "attempted" | "reported" { + return this.overflowStage; + } + + request(customInstructions?: string): void { + this.pendingRequest = { customInstructions }; + } + clearRequest(): void { + this.pendingRequest = undefined; + } + requestContinuation(): void { + this.continueAfterThreshold = true; + } + resetContinuation(): void { + this.continueAfterThreshold = false; + } + resetOverflowRecovery(): void { + this.overflowStage = "idle"; + } + markOverflowAttempted(): void { + this.overflowStage = "attempted"; + } + markOverflowReported(): void { + this.overflowStage = "reported"; + } + abort(): void { + this.manualAbort?.abort(); + this.automaticAbort?.abort(); + } + abortAutomatic(): void { + this.automaticAbort?.abort(); + } + + getThresholdContextTokens( + assistantMessage: AssistantMessage, + compactionTimestamp: number | undefined, + ): number | undefined { + const messages = this.host.getMessages(); + const estimate = estimateContextTokens(messages); + if (estimate.lastUsageIndex !== null) { + // Verify the usage source is post-compaction. Kept pre-compaction messages + // have stale usage reflecting the old (larger) context and would falsely + // trigger compaction right after one just finished. + const usageMsg = messages[estimate.lastUsageIndex]; + if ( + compactionTimestamp !== undefined && + usageMsg.role === "assistant" && + (usageMsg as AssistantMessage).timestamp <= compactionTimestamp + ) { + return undefined; + } + return estimate.tokens; + } + if (assistantMessage.stopReason === "error") return undefined; + return calculateContextTokens(assistantMessage.usage); + } + + async check( + assistantMessage: AssistantMessage, + skipAbortedCheck = true, + queueAutonomousContinuation = true, + ): Promise { + // An abort drops any compaction the model requested this turn, even on the + // pre-prompt path (skipAbortedCheck=false) which continues to threshold checks. + if (assistantMessage.stopReason === "aborted") { + this.clearRequest(); + const refinementAbort = this.host.beginRefinementAbort(); + if (refinementAbort) { + await refinementAbort.promise.catch(() => undefined); + refinementAbort.finish(); + } + if (skipAbortedCheck) return false; + } + + const settings = this.host.getSettings(); + const contextWindow = this.model?.contextWindow ?? 0; + + // Skip overflow check if the message came from a different model. + // This handles the case where user switched from a smaller-context model (e.g. opus) + // to a larger-context model (e.g. codex) - the overflow error from the old model + // shouldn't trigger compaction for the new model. + const sameModel = + this.model && assistantMessage.provider === this.model.provider && assistantMessage.model === this.model.id; + + // Skip overflow/threshold checks if this assistant message is older than the + // latest compaction boundary. This prevents a stale pre-compaction usage/error + // from retriggering compaction on the first prompt after compaction. + const compactionEntry = getLatestCompactionEntry(this.host.getSessionStore().getBranch()); + const compactionTimestamp = compactionEntry ? new Date(compactionEntry.timestamp).getTime() : undefined; + const assistantIsFromBeforeCompaction = + compactionTimestamp !== undefined && assistantMessage.timestamp <= compactionTimestamp; + + // Case 1: Overflow - takes priority over a pending model request so the error + // strip + retry still happen; the compaction it runs consumes the request. + if ( + !assistantIsFromBeforeCompaction && + (settings.enabled || this.hasPendingRequest) && + sameModel && + isContextOverflow(assistantMessage, contextWindow) + ) { + if (this.overflowRecovery !== "idle") { + if (this.overflowRecovery === "attempted") { + this.markOverflowReported(); + this.endUnsuccessfully( + "overflow", + "failed", + "Context overflow recovery failed after one compact-and-retry attempt. Try reducing context or switching to a larger-context model.", + ); + } + return false; + } + + this.markOverflowAttempted(); + // Remove the error message from agent state (it IS saved to session for history, + // but we don't want it in context for the retry) + const messages = this.host.getMessages(); + if (messages.length > 0 && messages[messages.length - 1].role === "assistant") { + this.host.replaceMessages(messages.slice(0, -1)); + } + return await this.host.runAutomatic("overflow", true); + } + + if (this.hasPendingRequest) { + return await this.host.runAutomatic("requested", false); + } + + if (!settings.enabled || assistantIsFromBeforeCompaction) return false; + + // Case 3: Threshold - context is getting large. + // Use the full-session estimate so messages appended after the last successful + // assistant usage are included, matching the /usage context display. + const contextTokens = this.getThresholdContextTokens(assistantMessage, compactionTimestamp); + if (contextTokens === undefined) return false; + if (shouldCompact(contextTokens, contextWindow, settings)) { + if (queueAutonomousContinuation && this.host.queueGoalContinuation(assistantMessage)) { + this.requestContinuation(); + } else if (queueAutonomousContinuation && (await this.host.queueAutonomousContinuation(assistantMessage))) { + this.requestContinuation(); + } + return await this.host.runAutomatic("threshold", false); + } + return false; + } + + async compact(customInstructions?: string, options: { skipAbort?: boolean } = {}): Promise { + if (options.skipAbort && this.host.isStreaming()) { + throw new Error("Cannot compact without aborting while the agent is running."); + } + const { scheduled: hadPostCompactionContinue, continueAfterSessionInput } = this.host.getContinuationState(); + this.host.disconnect(); + if (!options.skipAbort) await this.host.abortSession(); + let didCompact = false; + const compactionAbort = new AbortController(); + this.manualAbort = compactionAbort; + let resolveCompactionOperation: () => void = () => {}; + const compactionOperation = new Promise((resolve) => { + resolveCompactionOperation = resolve; + }); + this.activeOperation = compactionOperation; + this.host.emit({ + type: "compaction_start", + reason: "manual", + customInstructions, + }); + + try { + if (!this.model) { + throw new Error(formatNoModelSelectedMessage()); + } + + const { apiKey, headers, requestModel } = await this.host.getRequiredAuth(this.model); + const result = await this.host.perform({ + model: requestModel ?? this.model, + apiKey, + headers, + customInstructions, + signal: compactionAbort.signal, + }); + + this.host.emit({ + type: "compaction_end", + reason: "manual", + result, + aborted: false, + willRetry: false, + customInstructions, + }); + didCompact = true; + // A manual compaction satisfies any pending model request; on failure the + // request stays scheduled for the next turn boundary. + this.pendingRequest = undefined; + return result; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + const aborted = message === "Compaction cancelled" || (error instanceof Error && error.name === "AbortError"); + const skipped = error instanceof CompactionSkippedError; + this.host.emit({ + type: "compaction_end", + reason: "manual", + result: undefined, + aborted, + willRetry: false, + errorMessage: aborted ? undefined : skipped ? message : `Compaction failed: ${message}`, + errorSeverity: skipped ? "warning" : "error", + customInstructions, + }); + throw error; + } finally { + this.manualAbort = undefined; + this.host.reconnect(); + if (this.activeOperation === compactionOperation) { + this.activeOperation = undefined; + } + resolveCompactionOperation(); + this.host.notifyCheckpoints(); + this.host.scheduleInput(); + if (didCompact) { + this.host.afterManualCompaction( + compactionAbort.signal, + hadPostCompactionContinue, + continueAfterSessionInput, + ); + } + } + } + + endUnsuccessfully( + reason: CompactionOutcomeReason, + outcome: CompactionOutcome, + message: string, + options: { + aborted?: boolean; + errorSeverity?: "warning" | "error"; + customInstructions?: string; + } = {}, + ): void { + this.persistOutcome(reason, outcome, message); + this.host.emit({ + type: "compaction_end", + reason, + result: undefined, + aborted: options.aborted ?? false, + willRetry: false, + // Aborts are user-initiated; they carry no error message on the event. + errorMessage: options.aborted ? undefined : message, + errorSeverity: options.errorSeverity, + customInstructions: options.customInstructions, + }); + } + + private persistOutcome(reason: CompactionOutcomeReason, outcome: CompactionOutcome, message: string): void { + let outcomeMessage = createCompactionOutcomeMessage(message, { + reason, + outcome, + }); + try { + this.host + .getSessionStore() + .appendCustomMessageEntryWithRollback( + outcomeMessage.customType, + outcomeMessage.content, + outcomeMessage.display, + outcomeMessage.details, + ); + } catch (error) { + const persistenceError = error instanceof Error ? error.message : String(error); + outcomeMessage = createCompactionOutcomeMessage( + `${message}\n\nThis compaction outcome could not be saved to session history: ${persistenceError}`, + { reason, outcome }, + ); + // Not in the session file, so context rebuilds would drop the disclosure. + this.host.retainUnpersistedOutcome(outcomeMessage); + } + this.host.getMessages().push(outcomeMessage); + this.host.emit({ type: "message_start", message: outcomeMessage }); + this.host.emit({ type: "message_end", message: outcomeMessage }); + } + + async runAutomatic(reason: "overflow" | "threshold" | "requested", willRetry: boolean): Promise { + // Any compaction consumes a pending model request and honors its instructions + // (overflow recovery can fire first and take the request with it). + const pending = this.pendingRequest; + this.pendingRequest = undefined; + const customInstructions = pending?.customInstructions; + const shouldContinueAfterCompaction = + (reason === "threshold" || reason === "requested") && this.continueAfterThreshold; + const queuedAutonomousContinuationsForThisCompaction = + reason === "threshold" && shouldContinueAfterCompaction ? this.host.takeThresholdAutonomousMessages() : []; + const queuedGoalContinuationForThisCompaction = + reason === "threshold" && shouldContinueAfterCompaction ? this.host.getThresholdGoalContinuation() : undefined; + this.continueAfterThreshold = false; + + // Requested/threshold stop the loop on purpose, so a failed or skipped compaction must not stall it. + // Overflow stays excluded: a failed overflow recovery must not re-issue the overflowing request. + const resumeAfterFailure = () => { + if ( + (reason === "requested" || reason === "threshold") && + (shouldContinueAfterCompaction || this.host.hasAgentQueuedMessages() || this.host.hasPendingSessionWork()) + ) { + this.host.scheduleContinuation(shouldContinueAfterCompaction); + } + }; + + this.host.emit({ type: "compaction_start", reason, customInstructions }); + this.automaticAbort = new AbortController(); + let resolveCompactionOperation: () => void = () => {}; + const compactionOperation = new Promise((resolve) => { + resolveCompactionOperation = resolve; + }); + this.activeOperation = compactionOperation; + + try { + const authResult = this.model ? await this.host.getAuth(this.model) : undefined; + if (!this.model || !authResult || !authResult.ok || !authResult.apiKey) { + const detail = + !this.model || !authResult + ? "no model is selected" + : authResult.ok + ? "no API key is available" + : authResult.error; + this.endUnsuccessfully(reason, "failed", `Compaction failed: ${detail}`); + this.host.clearAutonomousContinuations( + reason === "threshold" && shouldContinueAfterCompaction, + queuedAutonomousContinuationsForThisCompaction, + ); + resumeAfterFailure(); + return false; + } + + const result = await this.host.perform({ + model: authResult.requestModel ?? this.model, + apiKey: authResult.apiKey, + headers: authResult.headers, + customInstructions, + signal: this.automaticAbort.signal, + }); + + this.host.emit({ + type: "compaction_end", + reason, + result, + aborted: false, + willRetry, + customInstructions, + }); + // Queued work lives in both the agent queues and the session-owned queues. + const hasQueuedMessages = this.host.hasAgentQueuedMessages() || this.host.hasPendingSessionWork(); + const willContinueAfterCompaction = willRetry || shouldContinueAfterCompaction || hasQueuedMessages; + + if (willRetry) { + const messages = this.host.getMessages(); + const lastMsg = messages[messages.length - 1]; + if (lastMsg?.role === "assistant" && (lastMsg as AssistantMessage).stopReason === "error") { + this.host.replaceMessages(messages.slice(0, -1)); + } + + this.host.scheduleContinuation(true); + this.host.scheduleRefinement(willContinueAfterCompaction); + return true; + } else if (shouldContinueAfterCompaction || hasQueuedMessages) { + // Compaction can intentionally stop a tool loop between turns. + // Queued follow-up/steering/custom messages can also be waiting. + this.host.scheduleContinuation(shouldContinueAfterCompaction); + this.host.scheduleRefinement(willContinueAfterCompaction); + } else { + this.host.scheduleRefinement(willContinueAfterCompaction); + } + return false; + } catch (error) { + this.host.clearAutonomousContinuations( + reason === "threshold" && shouldContinueAfterCompaction, + queuedAutonomousContinuationsForThisCompaction, + ); + const errorMessage = error instanceof Error ? error.message : "compaction failed"; + const aborted = + errorMessage === "Compaction cancelled" || (error instanceof Error && error.name === "AbortError"); + if (aborted) { + this.host.clearGoalContinuation(queuedGoalContinuationForThisCompaction); + this.endUnsuccessfully( + reason, + "cancelled", + `${reason === "requested" ? "Requested c" : "C"}ompaction cancelled`, + { aborted: true, customInstructions }, + ); + return false; + } + if (error instanceof CompactionSkippedError) { + this.endUnsuccessfully( + reason, + "skipped", + reason === "requested" + ? `Requested compaction skipped: ${errorMessage}` + : `Auto-compaction skipped: ${errorMessage}`, + { errorSeverity: "warning", customInstructions }, + ); + resumeAfterFailure(); + return false; + } + this.endUnsuccessfully( + reason, + "failed", + reason === "overflow" + ? `Context overflow recovery failed: ${errorMessage}` + : reason === "requested" + ? `Requested compaction failed: ${errorMessage}` + : `Auto-compaction failed: ${errorMessage}`, + { customInstructions }, + ); + resumeAfterFailure(); + return false; + } finally { + this.automaticAbort = undefined; + if (this.activeOperation === compactionOperation) { + this.activeOperation = undefined; + } + resolveCompactionOperation(); + this.host.notifyCheckpoints(); + this.host.scheduleInput(); + } + } +} diff --git a/packages/coding-agent/src/session/compaction/execution.ts b/packages/coding-agent/src/session/compaction/execution.ts new file mode 100644 index 0000000000..af6590ec49 --- /dev/null +++ b/packages/coding-agent/src/session/compaction/execution.ts @@ -0,0 +1,185 @@ +import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Api, Model } from "@earendil-works/pi-ai"; +import type { ExtensionRunner, SessionBeforeCompactResult } from "../../core/extensions/index.js"; +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; +import { modelRequestHeaders, type SemanticEdgeRecorder } from "../../core/semantic-edges.js"; +import type { CompactionEntry, SessionManager } from "../../core/session-manager.js"; +import { compact, prepareCompaction } from "./summary.js"; +import type { CompactionResult, CompactionSettings } from "./types.js"; + +export class CompactionSkippedError extends Error {} + +export interface CompactionExecutionOptions { + model: Model; + apiKey: string; + headers?: Record; + customInstructions?: string; + signal: AbortSignal; +} + +export interface CompactionExecutionHost { + getSessionStore(): Pick; + getSettings(): CompactionSettings; + getSemanticEdges(): Pick< + SemanticEdgeRecorder, + "beginCompaction" | "startCompactionRequest" | "failRequest" | "finishRequest" | "finishCompaction" + >; + getExtensions(): Pick; + getThinkingLevel(): ThinkingLevel; + getRetryPolicy(): ProviderRetryPolicy; + getSessionId?(): string; + getHarnessDigest(): string; + rebuildContext(): void; + syncKernelState(): Promise; + reapDeletedChildren(): Promise; +} + +export async function performSessionCompaction( + host: CompactionExecutionHost, + options: CompactionExecutionOptions, +): Promise { + const { model, apiKey, headers, customInstructions, signal } = options; + const pathEntries = host.getSessionStore().getBranch(); + const settings = host.getSettings(); + + const preparation = prepareCompaction(pathEntries, settings); + if (!preparation) { + const lastEntry = pathEntries[pathEntries.length - 1]; + if (lastEntry?.type === "compaction") { + throw new CompactionSkippedError("Already compacted"); + } + throw new CompactionSkippedError("Session is too short to compact — try again once it grows"); + } + + let extensionCompaction: CompactionResult | undefined; + let fromExtension = false; + + const semanticCompaction = host.getSemanticEdges().beginCompaction(); + let compactionRecorded = false; + const uncommittedSlices: string[] = []; + let compactionSettled = false; + let summary: string; + let firstKeptEntryId: string; + let tokensBefore: number; + let details: CompactionResult["details"]; + let usage: CompactionResult["usage"]; + try { + if (host.getExtensions().hasHandlers("session_before_compact")) { + const result = (await host.getExtensions().emit({ + type: "session_before_compact", + preparation, + branchEntries: pathEntries, + customInstructions, + signal, + })) as SessionBeforeCompactResult | undefined; + + if (result?.cancel) { + throw new Error("Compaction cancelled"); + } + + if (result?.compaction) { + extensionCompaction = result.compaction; + fromExtension = true; + } + } + + if (extensionCompaction) { + ({ summary, firstKeptEntryId, tokensBefore, details, usage } = extensionCompaction); + } else { + // Each summary wire call gets its own request ID: split turns send two + // different bodies, and one Idempotency-Key must never cover both. A slice + // that succeeds on the wire stays uncommitted until the compaction itself + // commits: a racing sibling's failure (or an abort) must leave no committed + // summary request for the next turn's continuation edge to attach to. + const summaryCall = async ( + call: (callHeaders: Record | undefined) => Promise, + ): Promise => { + const requestId = host.getSemanticEdges().startCompactionRequest(semanticCompaction.compactionId); + if (requestId === undefined) { + return call(headers); + } + try { + const result = await call({ ...headers, ...modelRequestHeaders(requestId) }); + // A slice resolving after a sibling's rejection already settled the + // compaction would push into a drained list and stay in-flight forever. + if (compactionSettled) { + host.getSemanticEdges().failRequest(requestId); + } else { + uncommittedSlices.push(requestId); + } + return result; + } catch (error) { + host.getSemanticEdges().failRequest(requestId); + throw error; + } + }; + ({ summary, firstKeptEntryId, tokensBefore, details, usage } = await compact( + preparation, + model, + apiKey, + headers, + customInstructions, + signal, + host.getThinkingLevel(), + summaryCall, + host.getRetryPolicy(), + host.getSessionId?.(), + )); + } + + if (signal.aborted) { + throw new Error("Compaction cancelled"); + } + + // Ledger-before-effect: the compaction outcome is durable before the transcript + // commits it. Marked first: the ID is consumed even when the write throws, and a + // second finish attempt would mask the original I/O error. + compactionRecorded = true; + compactionSettled = true; + for (const requestId of uncommittedSlices.splice(0)) { + host.getSemanticEdges().finishRequest(requestId); + } + host.getSemanticEdges().finishCompaction(semanticCompaction.compactionId, "completed"); + // Attached mechanically; the digest never flows through the summarizer LLM. + host + .getSessionStore() + .appendCompaction( + summary, + firstKeptEntryId, + tokensBefore, + details, + fromExtension, + customInstructions, + usage, + host.getHarnessDigest(), + ); + } catch (error) { + compactionSettled = true; + for (const requestId of uncommittedSlices.splice(0)) { + host.getSemanticEdges().failRequest(requestId); + } + if (!compactionRecorded) { + const cancelled = + error instanceof Error && (error.name === "AbortError" || error.message === "Compaction cancelled"); + host.getSemanticEdges().finishCompaction(semanticCompaction.compactionId, cancelled ? "cancelled" : "failed"); + } + throw error; + } + const newEntries = host.getSessionStore().getEntries(); + host.rebuildContext(); + + const savedCompactionEntry = newEntries.find((e) => e.type === "compaction" && e.summary === summary) as + | CompactionEntry + | undefined; + if (savedCompactionEntry) { + await host.getExtensions().emit({ + type: "session_compact", + compactionEntry: savedCompactionEntry, + fromExtension, + }); + } + await host.syncKernelState(); + await host.reapDeletedChildren(); + + return { summary, firstKeptEntryId, tokensBefore, details }; +} diff --git a/packages/coding-agent/src/session/compaction/summary.ts b/packages/coding-agent/src/session/compaction/summary.ts new file mode 100644 index 0000000000..d64068d1aa --- /dev/null +++ b/packages/coding-agent/src/session/compaction/summary.ts @@ -0,0 +1,636 @@ +import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Model, Usage } from "@earendil-works/pi-ai"; +import { completeSimple } from "@earendil-works/pi-ai"; +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; +import { completeWithProviderRetry } from "../../core/provider-retry.js"; +import type { CompactionEntry, SessionEntry } from "../../core/session-manager.js"; +import { buildSessionContext } from "../../core/session-manager.js"; +import { SUMMARIZATION_SYSTEM_PROMPT, serializeConversation } from "../context/conversation-text.js"; +import type { FileOperations } from "../context/file-tracking.js"; +import { + computeFileLists, + createFileOps, + extractFileOpsFromMessage, + formatFileOperations, +} from "../context/file-tracking.js"; +import { + convertToLlm, + createBranchSummaryMessage, + createCompactionSummaryMessage, + createCustomMessage, + HARNESS_DIGEST_CUSTOM_TYPE, +} from "../context/messages.js"; +import { estimateContextTokens, estimateTokens } from "../context/token-estimate.js"; +import { addAssistantUsage, emptyUsage } from "../context/usage.js"; +import type { + CompactionDetails, + CompactionPreparation, + CompactionResult, + CompactionSettings, + CutPointResult, + SummaryCallRunner, + SummarySlice, +} from "./types.js"; + +/** + * Extract file operations from messages and previous compaction entries. + */ +/** Preserve file operations recorded by prior compactions and current tool calls. */ +function extractFileOperations( + messages: AgentMessage[], + entries: SessionEntry[], + prevCompactionIndex: number, +): FileOperations { + const fileOps = createFileOps(); + if (prevCompactionIndex >= 0) { + const prevCompaction = entries[prevCompactionIndex] as CompactionEntry; + if (!prevCompaction.fromHook && prevCompaction.details) { + // fromHook field kept for session file compatibility + const details = prevCompaction.details as CompactionDetails; + if (Array.isArray(details.readFiles)) { + for (const f of details.readFiles) fileOps.read.add(f); + } + if (Array.isArray(details.modifiedFiles)) { + for (const f of details.modifiedFiles) fileOps.edited.add(f); + } + } + } + for (const msg of messages) { + extractFileOpsFromMessage(msg, fileOps); + } + + return fileOps; +} + +/** + * Extract AgentMessage from an entry if it produces one. + * Returns undefined for entries that don't contribute to LLM context. + */ +function getMessageFromEntry(entry: SessionEntry): AgentMessage | undefined { + if (entry.type === "message") { + return entry.message; + } + if (entry.type === "custom_message") { + return createCustomMessage(entry.customType, entry.content, entry.display, entry.details, entry.timestamp); + } + if (entry.type === "branch_summary") { + return createBranchSummaryMessage(entry.summary, entry.fromId, entry.timestamp); + } + if (entry.type === "compaction") { + return createCompactionSummaryMessage( + entry.summary, + entry.tokensBefore, + entry.timestamp, + entry.customInstructions, + ); + } + return undefined; +} + +function getMessageFromEntryForCompaction(entry: SessionEntry): AgentMessage | undefined { + if (entry.type === "compaction") { + return undefined; + } + // Harness digests are regenerated on the new compaction head; never summarizer input. + if (entry.type === "custom_message" && entry.customType === HARNESS_DIGEST_CUSTOM_TYPE) { + return undefined; + } + return getMessageFromEntry(entry); +} + +/** + * Check if compaction should trigger based on context usage. + */ +export function shouldCompact(contextTokens: number, contextWindow: number, settings: CompactionSettings): boolean { + if (!settings.enabled) return false; + if (contextWindow <= 0) return false; + return contextTokens > contextWindow - settings.reserveTokens; +} + +/** + * Find valid cut points: indices of user, assistant, custom, or bashExecution messages. + * Never cut at tool results (they must follow their tool call). + * When we cut at an assistant message with tool calls, its tool results follow it + * and will be kept. + * BashExecutionMessage is treated like a user message (user-initiated context). + */ +function findValidCutPoints(entries: SessionEntry[], startIndex: number, endIndex: number): number[] { + const cutPoints: number[] = []; + for (let i = startIndex; i < endIndex; i++) { + const entry = entries[i]; + switch (entry.type) { + case "message": { + const role = entry.message.role; + switch (role) { + case "bashExecution": + case "custom": + case "branchSummary": + case "compactionSummary": + case "user": + case "assistant": + cutPoints.push(i); + break; + case "toolResult": + break; + } + break; + } + case "thinking_level_change": + case "model_change": + case "compaction": + case "branch_summary": + case "custom": + case "custom_message": + case "label": + case "session_info": + break; + } + // Branch summaries and custom messages are user-role turn boundaries. + if (entry.type === "branch_summary" || entry.type === "custom_message") { + cutPoints.push(i); + } + } + return cutPoints; +} + +/** + * Find the user message (or bashExecution) that starts the turn containing the given entry index. + * Returns -1 if no turn start found before the index. + * BashExecutionMessage is treated like a user message for turn boundaries. + */ +export function findTurnStartIndex(entries: SessionEntry[], entryIndex: number, startIndex: number): number { + for (let i = entryIndex; i >= startIndex; i--) { + const entry = entries[i]; + if (entry.type === "branch_summary" || entry.type === "custom_message") { + return i; + } + if (entry.type === "message") { + const role = entry.message.role; + if (role === "user" || role === "bashExecution") { + return i; + } + } + } + return -1; +} + +/** + * Find the cut point in session entries that keeps approximately `keepRecentTokens`. + * + * Algorithm: Walk backwards from newest, accumulating estimated message sizes. + * Stop when we've accumulated >= keepRecentTokens. Cut at that point. + * + * Can cut at user OR assistant messages (never tool results). When cutting at an + * assistant message with tool calls, its tool results come after and will be kept. + * + * Returns CutPointResult with: + * - firstKeptEntryIndex: the entry index to start keeping from + * - turnStartIndex: if cutting mid-turn, the user message that started that turn + * - isSplitTurn: whether we're cutting in the middle of a turn + * + * Only considers entries between `startIndex` and `endIndex` (exclusive). + */ +export function findCutPoint( + entries: SessionEntry[], + startIndex: number, + endIndex: number, + keepRecentTokens: number, +): CutPointResult { + const cutPoints = findValidCutPoints(entries, startIndex, endIndex); + + if (cutPoints.length === 0) { + return { firstKeptEntryIndex: startIndex, turnStartIndex: -1, isSplitTurn: false }; + } + let accumulatedTokens = 0; + let cutIndex = cutPoints[0]; // Default: keep from first message (not header) + + for (let i = endIndex - 1; i >= startIndex; i--) { + const entry = entries[i]; + if (entry.type !== "message") continue; + const messageTokens = estimateTokens(entry.message); + accumulatedTokens += messageTokens; + if (accumulatedTokens >= keepRecentTokens) { + // No cut point at/after i (trailing tool results): keep only the final turn, not everything. + cutIndex = cutPoints[cutPoints.length - 1]; + for (let c = 0; c < cutPoints.length; c++) { + if (cutPoints[c] >= i) { + cutIndex = cutPoints[c]; + break; + } + } + break; + } + } + while (cutIndex > startIndex) { + const prevEntry = entries[cutIndex - 1]; + if (prevEntry.type === "compaction") { + break; + } + if (prevEntry.type === "message") { + break; + } + cutIndex--; + } + const cutEntry = entries[cutIndex]; + const isUserMessage = cutEntry.type === "message" && cutEntry.message.role === "user"; + // A cut in a non-user turn requires a prefix summary. + const turnStartIndex = isUserMessage ? -1 : findTurnStartIndex(entries, cutIndex, startIndex); + + return { + firstKeptEntryIndex: cutIndex, + turnStartIndex, + isSplitTurn: !isUserMessage && turnStartIndex !== -1, + }; +} + +const SUMMARIZATION_PROMPT = `The messages above are a conversation to summarize. Create a structured context checkpoint summary that another LLM will use to continue the work. + +Use this EXACT format: + +## Goal +[What is the user trying to accomplish? Can be multiple items if the session covers different tasks.] + +## Constraints & Preferences +- [Any constraints, preferences, or requirements mentioned by user] +- [Or "(none)" if none were mentioned] + +## Progress +### Done +- [x] [Completed tasks/changes] + +### In Progress +- [ ] [Current work] + +### Blocked +- [Issues preventing progress, if any] + +## Key Decisions +- **[Decision]**: [Brief rationale] + +## Next Steps +1. [Ordered list of what should happen next] + +## Critical Context +- [Any data, examples, or references needed to continue] +- [Or "(none)" if not applicable] + +Keep each section concise. Preserve exact file paths, function names, and error messages.`; + +const KERNEL_PERSIST_SUMMARY_NOTE = + "Note: the Python kernel keeps running after this summary — every Python variable, import, and helper you defined stays available. The cells that defined them won't appear above, so record in the summary any names worth remembering so you reuse them instead of redefining them."; + +const UPDATE_SUMMARIZATION_PROMPT = `The messages above are NEW conversation messages to incorporate into the existing summary provided in tags. + +Update the existing structured summary with new information. RULES: +- PRESERVE all existing information from the previous summary +- ADD new progress, decisions, and context from the new messages +- UPDATE the Progress section: move items from "In Progress" to "Done" when completed +- UPDATE "Next Steps" based on what was accomplished +- PRESERVE exact file paths, function names, and error messages +- If something is no longer relevant, you may remove it + +Use this EXACT format: + +## Goal +[Preserve existing goals, add new ones if the task expanded] + +## Constraints & Preferences +- [Preserve existing, add new ones discovered] + +## Progress +### Done +- [x] [Include previously done items AND newly completed items] + +### In Progress +- [ ] [Current work - update based on progress] + +### Blocked +- [Current blockers - remove if resolved] + +## Key Decisions +- **[Decision]**: [Brief rationale] (preserve all previous, add new) + +## Next Steps +1. [Update based on current state] + +## Critical Context +- [Preserve important context, add new if needed] + +Keep each section concise. Preserve exact file paths, function names, and error messages.`; + +/** + * Build the instruction portion of the summarization prompt: the initial or + * update template, optional user instructions, and the kernel persistence note. + */ +export function buildSummarizationPrompt(customInstructions?: string, previousSummary?: string): string { + let basePrompt = previousSummary ? UPDATE_SUMMARIZATION_PROMPT : SUMMARIZATION_PROMPT; + if (customInstructions) { + basePrompt += `\n\n\nThe user provided these instructions for this summary. Follow them with high priority while keeping the section format above: emphasize what they ask to focus on, and preserve verbatim anything they ask to remember.\n${customInstructions}\n`; + } + return `${basePrompt}\n\n${KERNEL_PERSIST_SUMMARY_NOTE}`; +} + +/** + * Generate a summary of the conversation using the LLM. + * If previousSummary is provided, uses the update prompt to merge. + */ +export async function generateSummary( + currentMessages: AgentMessage[], + model: Model, + reserveTokens: number, + apiKey: string, + headers?: Record, + signal?: AbortSignal, + customInstructions?: string, + previousSummary?: string, + thinkingLevel?: ThinkingLevel, + retry?: ProviderRetryPolicy, + sessionId?: string, +): Promise { + const maxTokens = Math.floor(0.8 * reserveTokens); + + const basePrompt = buildSummarizationPrompt(customInstructions, previousSummary); + // Serialize before the LLM call so it summarizes rather than continues this conversation. + const llmMessages = convertToLlm(currentMessages); + const conversationText = serializeConversation(llmMessages); + let promptText = `\n${conversationText}\n\n\n`; + if (previousSummary) { + promptText += `\n${previousSummary}\n\n\n`; + } + promptText += basePrompt; + + const summarizationMessages = [ + { + role: "user" as const, + content: [{ type: "text" as const, text: promptText }], + timestamp: Date.now(), + }, + ]; + + const completionOptions = + model.reasoning && thinkingLevel && thinkingLevel !== "off" + ? { maxTokens, signal, apiKey, headers, sessionId, reasoning: thinkingLevel } + : { maxTokens, signal, apiKey, headers, sessionId }; + + const response = await completeWithProviderRetry( + () => + completeSimple( + model, + { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, + completionOptions, + ), + { policy: retry, signal }, + ); + + if (response.stopReason === "error") { + throw new Error(`Summarization failed: ${response.errorMessage || "Unknown error"}`); + } + + const textContent = response.content + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join("\n"); + + return { summary: textContent, usage: response.usage }; +} + +export function prepareCompaction( + pathEntries: SessionEntry[], + settings: CompactionSettings, +): CompactionPreparation | undefined { + if (pathEntries.length > 0 && pathEntries[pathEntries.length - 1].type === "compaction") { + return undefined; + } + + let prevCompactionIndex = -1; + for (let i = pathEntries.length - 1; i >= 0; i--) { + if (pathEntries[i].type === "compaction") { + prevCompactionIndex = i; + break; + } + } + + let previousSummary: string | undefined; + let boundaryStart = 0; + if (prevCompactionIndex >= 0) { + const prevCompaction = pathEntries[prevCompactionIndex] as CompactionEntry; + previousSummary = prevCompaction.summary; + const firstKeptEntryIndex = pathEntries.findIndex((entry) => entry.id === prevCompaction.firstKeptEntryId); + boundaryStart = firstKeptEntryIndex >= 0 ? firstKeptEntryIndex : prevCompactionIndex + 1; + } + const boundaryEnd = pathEntries.length; + + const tokensBefore = estimateContextTokens(buildSessionContext(pathEntries).messages).tokens; + + const cutPoint = findCutPoint(pathEntries, boundaryStart, boundaryEnd, settings.keepRecentTokens); + const firstKeptEntry = pathEntries[cutPoint.firstKeptEntryIndex]; + if (!firstKeptEntry?.id) { + return undefined; // Session needs migration + } + const firstKeptEntryId = firstKeptEntry.id; + + const historyEnd = cutPoint.isSplitTurn ? cutPoint.turnStartIndex : cutPoint.firstKeptEntryIndex; + const messagesToSummarize: AgentMessage[] = []; + for (let i = boundaryStart; i < historyEnd; i++) { + const msg = getMessageFromEntryForCompaction(pathEntries[i]); + if (msg) messagesToSummarize.push(msg); + } + const turnPrefixMessages: AgentMessage[] = []; + if (cutPoint.isSplitTurn) { + for (let i = cutPoint.turnStartIndex; i < cutPoint.firstKeptEntryIndex; i++) { + const msg = getMessageFromEntryForCompaction(pathEntries[i]); + if (msg) turnPrefixMessages.push(msg); + } + } + + // Avoid a compaction that would summarize no history. + if (messagesToSummarize.length === 0 && turnPrefixMessages.length === 0 && !previousSummary) { + return undefined; + } + const fileOps = extractFileOperations(messagesToSummarize, pathEntries, prevCompactionIndex); + // Split turns retain their suffix, but their prefix file operations still belong in the summary. + if (cutPoint.isSplitTurn) { + for (const msg of turnPrefixMessages) { + extractFileOpsFromMessage(msg, fileOps); + } + } + + return { + firstKeptEntryId, + messagesToSummarize, + turnPrefixMessages, + isSplitTurn: cutPoint.isSplitTurn, + tokensBefore, + previousSummary, + fileOps, + settings, + }; +} + +const TURN_PREFIX_SUMMARIZATION_PROMPT = `This is the PREFIX of a turn that was too large to keep. The SUFFIX (recent work) is retained. + +Summarize the prefix to provide context for the retained suffix: + +## Original Request +[What did the user ask for in this turn?] + +## Early Progress +- [Key decisions and work done in the prefix] + +## Context for Suffix +- [Information needed to understand the retained recent work] + +Be concise. Focus on what's needed to understand the kept suffix.`; + +export async function compact( + preparation: CompactionPreparation, + model: Model, + apiKey: string, + headers?: Record, + customInstructions?: string, + signal?: AbortSignal, + thinkingLevel?: ThinkingLevel, + summaryCall: SummaryCallRunner = (call) => call(headers), + retry?: ProviderRetryPolicy, + sessionId?: string, +): Promise { + const { + firstKeptEntryId, + messagesToSummarize, + turnPrefixMessages, + isSplitTurn, + tokensBefore, + previousSummary, + fileOps, + settings, + } = preparation; + let summary: string; + const slices: SummarySlice[] = []; + + if (isSplitTurn && turnPrefixMessages.length > 0) { + // Split turns make two wire calls with different bodies; each needs its own identity. + const [historyResult, turnPrefixResult] = await Promise.all([ + messagesToSummarize.length > 0 + ? summaryCall((callHeaders) => + generateSummary( + messagesToSummarize, + model, + settings.reserveTokens, + apiKey, + callHeaders, + signal, + customInstructions, + previousSummary, + thinkingLevel, + retry, + sessionId, + ), + ) + : Promise.resolve({ summary: "No prior history." }), + summaryCall((callHeaders) => + generateTurnPrefixSummary( + turnPrefixMessages, + model, + settings.reserveTokens, + apiKey, + callHeaders, + signal, + thinkingLevel, + retry, + sessionId, + ), + ), + ]); + slices.push(historyResult, turnPrefixResult); + summary = `${historyResult.summary}\n\n---\n\n**Turn Context (split turn):**\n\n${turnPrefixResult.summary}`; + } else { + const result = await summaryCall((callHeaders) => + generateSummary( + messagesToSummarize, + model, + settings.reserveTokens, + apiKey, + callHeaders, + signal, + customInstructions, + previousSummary, + thinkingLevel, + retry, + sessionId, + ), + ); + slices.push(result); + summary = result.summary; + } + const { readFiles, modifiedFiles } = computeFileLists(fileOps); + summary += formatFileOperations(readFiles, modifiedFiles); + + if (!firstKeptEntryId) { + throw new Error("First kept entry has no UUID - session may need migration"); + } + + let usage: Usage | undefined; + for (const slice of slices) { + if (!slice.usage) continue; + usage ??= emptyUsage(); + addAssistantUsage(usage, slice.usage); + } + return { + summary, + firstKeptEntryId, + tokensBefore, + details: { readFiles, modifiedFiles } as CompactionDetails, + usage, + }; +} + +/** + * Generate a summary for a turn prefix (when splitting a turn). + */ +async function generateTurnPrefixSummary( + messages: AgentMessage[], + model: Model, + reserveTokens: number, + apiKey: string, + headers?: Record, + signal?: AbortSignal, + thinkingLevel?: ThinkingLevel, + retry?: ProviderRetryPolicy, + sessionId?: string, +): Promise { + const maxTokens = Math.floor(0.5 * reserveTokens); // Smaller budget for turn prefix + const llmMessages = convertToLlm(messages); + const conversationText = serializeConversation(llmMessages); + const promptText = `\n${conversationText}\n\n\n${TURN_PREFIX_SUMMARIZATION_PROMPT}`; + const summarizationMessages = [ + { + role: "user" as const, + content: [{ type: "text" as const, text: promptText }], + timestamp: Date.now(), + }, + ]; + + const response = await completeWithProviderRetry( + () => + completeSimple( + model, + { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, + model.reasoning && thinkingLevel && thinkingLevel !== "off" + ? { maxTokens, signal, apiKey, headers, sessionId, reasoning: thinkingLevel } + : { maxTokens, signal, apiKey, headers, sessionId }, + ), + { policy: retry, signal }, + ); + + if (response.stopReason === "error") { + throw new Error(`Turn prefix summarization failed: ${response.errorMessage || "Unknown error"}`); + } + + return { + summary: response.content + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join("\n"), + usage: response.usage, + }; +} diff --git a/packages/coding-agent/src/session/compaction/types.ts b/packages/coding-agent/src/session/compaction/types.ts new file mode 100644 index 0000000000..fd4610fe1a --- /dev/null +++ b/packages/coding-agent/src/session/compaction/types.ts @@ -0,0 +1,78 @@ +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { Usage } from "@earendil-works/pi-ai"; +import type { FileOperations } from "../context/file-tracking.js"; + +/** Details stored in CompactionEntry.details for file tracking */ +export interface CompactionDetails { + readFiles: string[]; + modifiedFiles: string[]; +} + +export interface SummarySlice { + summary: string; + usage?: Usage; +} + +/** Result from compact() - SessionManager adds uuid/parentUuid when saving */ +export interface CompactionResult { + summary: string; + firstKeptEntryId: string; + tokensBefore: number; + /** Extension-specific data (e.g., ArtifactIndex, version markers for structured compaction) */ + details?: T; + /** What the summarization call(s) billed; persisted on the compaction entry. */ + usage?: Usage; +} + +export const COMPACT_SKILL_NAME = "compact"; + +export interface CompactionSettings { + enabled: boolean; + reserveTokens: number; + keepRecentTokens: number; +} + +export const DEFAULT_COMPACTION_SETTINGS: CompactionSettings = { + enabled: true, + reserveTokens: 16384, + keepRecentTokens: 20000, +}; + +export interface CutPointResult { + /** Index of first entry to keep */ + firstKeptEntryIndex: number; + /** Index of user message that starts the turn being split, or -1 if not splitting */ + turnStartIndex: number; + /** Whether this cut splits a turn (cut point is not a user message) */ + isSplitTurn: boolean; +} + +export interface CompactionPreparation { + /** UUID of first entry to keep */ + firstKeptEntryId: string; + /** Messages that will be summarized and discarded */ + messagesToSummarize: AgentMessage[]; + /** Messages that will be turned into turn prefix summary (if splitting) */ + turnPrefixMessages: AgentMessage[]; + /** Whether this is a split turn (cut point in middle of turn) */ + isSplitTurn: boolean; + tokensBefore: number; + /** Summary from previous compaction, for iterative update */ + previousSummary?: string; + /** File operations extracted from messagesToSummarize */ + fileOps: FileOperations; + /** Compaction settions from settings.jsonl */ + settings: CompactionSettings; +} + +/** + * Generate summaries for compaction using prepared data. + * Returns CompactionResult - SessionManager adds uuid/parentUuid when saving. + * + * @param preparation - Pre-calculated preparation from prepareCompaction() + * @param customInstructions - Optional custom focus for the summary + */ +/** Runs one summary wire call; hosts decorate each call with its own request identity. */ +export type SummaryCallRunner = ( + call: (callHeaders: Record | undefined) => Promise, +) => Promise; diff --git a/packages/coding-agent/src/session/context/branch-summary.ts b/packages/coding-agent/src/session/context/branch-summary.ts new file mode 100644 index 0000000000..57b452e08a --- /dev/null +++ b/packages/coding-agent/src/session/context/branch-summary.ts @@ -0,0 +1,329 @@ +/** + * Branch summarization for tree navigation. + * + * When navigating to a different point in the session tree, this generates + * a summary of the branch being left so context isn't lost. + */ + +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { Model, Usage } from "@earendil-works/pi-ai"; +import { completeSimple } from "@earendil-works/pi-ai"; +import { completeWithProviderRetry, type ProviderRetryPolicy } from "../../core/provider-retry.js"; +import type { ReadonlySessionManager, SessionEntry } from "../../core/session-manager.js"; +import { SUMMARIZATION_SYSTEM_PROMPT, serializeConversation } from "./conversation-text.js"; +import { + computeFileLists, + createFileOps, + extractFileOpsFromMessage, + type FileOperations, + formatFileOperations, +} from "./file-tracking.js"; +import { + convertToLlm, + createBranchSummaryMessage, + createCompactionSummaryMessage, + createCustomMessage, + HARNESS_DIGEST_CUSTOM_TYPE, +} from "./messages.js"; +import { estimateTokens } from "./token-estimate.js"; +export interface BranchSummaryResult { + summary?: string; + readFiles?: string[]; + modifiedFiles?: string[]; + aborted?: boolean; + error?: string; + usage?: Usage; +} + +/** Details stored in BranchSummaryEntry.details for file tracking */ +export interface BranchSummaryDetails { + readFiles: string[]; + modifiedFiles: string[]; +} + +export type { FileOperations } from "./file-tracking.js"; + +export interface BranchPreparation { + /** Messages extracted for summarization, in chronological order */ + messages: AgentMessage[]; + /** File operations extracted from tool calls */ + fileOps: FileOperations; + /** Total estimated tokens in messages */ + totalTokens: number; +} + +export interface CollectEntriesResult { + /** Entries to summarize, in chronological order */ + entries: SessionEntry[]; + /** Common ancestor between old and new position, if any */ + commonAncestorId: string | null; +} + +export interface GenerateBranchSummaryOptions { + /** Model to use for summarization */ + model: Model; + /** API key for the model */ + apiKey: string; + /** Request headers for the model */ + headers?: Record; + /** Owning conversation identity for provider routing and caching. */ + sessionId?: string; + /** Abort signal for cancellation */ + signal: AbortSignal; + /** Optional custom instructions for summarization */ + customInstructions?: string; + /** If true, customInstructions replaces the default prompt instead of being appended */ + replaceInstructions?: boolean; + retry?: ProviderRetryPolicy; + /** Tokens reserved for prompt + LLM response (default 16384) */ + reserveTokens?: number; +} +/** + * Collect entries that should be summarized when navigating from one position to another. + * + * Walks from oldLeafId back to the common ancestor with targetId, collecting entries + * along the way. Does NOT stop at compaction boundaries - those are included and their + * summaries become context. + * + * @param session - Session manager (read-only access) + * @param oldLeafId - Current position (where we're navigating from) + * @param targetId - Target position (where we're navigating to) + * @returns Entries to summarize and the common ancestor + */ +export function collectEntriesForBranchSummary( + session: ReadonlySessionManager, + oldLeafId: string | null, + targetId: string, +): CollectEntriesResult { + if (!oldLeafId) { + return { entries: [], commonAncestorId: null }; + } + const oldPath = new Set(session.getBranch(oldLeafId).map((e) => e.id)); + const targetPath = session.getBranch(targetId); + let commonAncestorId: string | null = null; + for (let i = targetPath.length - 1; i >= 0; i--) { + if (oldPath.has(targetPath[i].id)) { + commonAncestorId = targetPath[i].id; + break; + } + } + const entries: SessionEntry[] = []; + let current: string | null = oldLeafId; + + while (current && current !== commonAncestorId) { + const entry = session.getEntry(current); + if (!entry) break; + entries.push(entry); + current = entry.parentId; + } + entries.reverse(); + + return { entries, commonAncestorId }; +} +/** + * Extract AgentMessage from a session entry. + * Similar to getMessageFromEntry in compaction.ts but also handles compaction entries. + */ +function getMessageFromEntry(entry: SessionEntry): AgentMessage | undefined { + switch (entry.type) { + case "message": + // Tool-result context remains attached to its assistant tool call. + if (entry.message.role === "toolResult") return undefined; + return entry.message; + + case "custom_message": + // Harness digests are regenerated at cold boundaries; never summarizer input. + if (entry.customType === HARNESS_DIGEST_CUSTOM_TYPE) return undefined; + return createCustomMessage(entry.customType, entry.content, entry.display, entry.details, entry.timestamp); + + case "branch_summary": + return createBranchSummaryMessage(entry.summary, entry.fromId, entry.timestamp); + + case "compaction": + return createCompactionSummaryMessage( + entry.summary, + entry.tokensBefore, + entry.timestamp, + entry.customInstructions, + ); + case "thinking_level_change": + case "model_change": + case "custom": + case "label": + case "session_info": + return undefined; + } +} + +/** + * Prepare entries for summarization with token budget. + * + * Walks entries from NEWEST to OLDEST, adding messages until we hit the token budget. + * This ensures we keep the most recent context when the branch is too long. + * + * Also collects file operations from: + * - Tool calls in assistant messages + * - Existing branch_summary entries' details (for cumulative tracking) + * + * @param entries - Entries in chronological order + * @param tokenBudget - Maximum tokens to include (0 = no limit) + */ +export function prepareBranchEntries(entries: SessionEntry[], tokenBudget: number = 0): BranchPreparation { + const messages: AgentMessage[] = []; + const fileOps = createFileOps(); + let totalTokens = 0; + + // First pass: collect file ops from ALL entries (even if they don't fit in token budget) + // This ensures we capture cumulative file tracking from nested branch summaries + // Only extract from pi-generated summaries (fromHook !== true), not extension-generated ones + for (const entry of entries) { + if (entry.type === "branch_summary" && !entry.fromHook && entry.details) { + const details = entry.details as BranchSummaryDetails; + if (Array.isArray(details.readFiles)) { + for (const f of details.readFiles) fileOps.read.add(f); + } + if (Array.isArray(details.modifiedFiles)) { + for (const f of details.modifiedFiles) { + fileOps.edited.add(f); + } + } + } + } + for (let i = entries.length - 1; i >= 0; i--) { + const entry = entries[i]; + const message = getMessageFromEntry(entry); + if (!message) continue; + extractFileOpsFromMessage(message, fileOps); + + const tokens = estimateTokens(message); + if (tokenBudget > 0 && totalTokens + tokens > tokenBudget) { + if (entry.type === "compaction" || entry.type === "branch_summary") { + if (totalTokens < tokenBudget * 0.9) { + messages.unshift(message); + totalTokens += tokens; + } + } + break; + } + + messages.unshift(message); + totalTokens += tokens; + } + + return { messages, fileOps, totalTokens }; +} +const BRANCH_SUMMARY_PREAMBLE = `The user explored a different conversation branch before returning here. +Summary of that exploration: + +`; + +const BRANCH_SUMMARY_PROMPT = `Create a structured summary of this conversation branch for context when returning later. + +Use this EXACT format: + +## Goal +[What was the user trying to accomplish in this branch?] + +## Constraints & Preferences +- [Any constraints, preferences, or requirements mentioned] +- [Or "(none)" if none were mentioned] + +## Progress +### Done +- [x] [Completed tasks/changes] + +### In Progress +- [ ] [Work that was started but not finished] + +### Blocked +- [Issues preventing progress, if any] + +## Key Decisions +- **[Decision]**: [Brief rationale] + +## Next Steps +1. [What should happen next to continue this work] + +Keep each section concise. Preserve exact file paths, function names, and error messages.`; + +/** + * Generate a summary of abandoned branch entries. + * + * @param entries - Session entries to summarize (chronological order) + * @param options - Generation options + */ +export async function generateBranchSummary( + entries: SessionEntry[], + options: GenerateBranchSummaryOptions, +): Promise { + const { + model, + apiKey, + headers, + sessionId, + signal, + customInstructions, + replaceInstructions, + retry, + reserveTokens = 16384, + } = options; + const contextWindow = model.contextWindow || 128000; + const tokenBudget = contextWindow - reserveTokens; + + const { messages, fileOps } = prepareBranchEntries(entries, tokenBudget); + + // Nothing model-visible remains after filtering. + if (messages.length === 0) { + return { summary: "No content to summarize" }; + } + // Serialize before the LLM call so it summarizes rather than continues this branch. + const llmMessages = convertToLlm(messages); + const conversationText = serializeConversation(llmMessages); + let instructions: string; + if (replaceInstructions && customInstructions) { + instructions = customInstructions; + } else if (customInstructions) { + instructions = `${BRANCH_SUMMARY_PROMPT}\n\nAdditional focus: ${customInstructions}`; + } else { + instructions = BRANCH_SUMMARY_PROMPT; + } + const promptText = `\n${conversationText}\n\n\n${instructions}`; + + const summarizationMessages = [ + { + role: "user" as const, + content: [{ type: "text" as const, text: promptText }], + timestamp: Date.now(), + }, + ]; + const response = await completeWithProviderRetry( + () => + completeSimple( + model, + { systemPrompt: SUMMARIZATION_SYSTEM_PROMPT, messages: summarizationMessages }, + { apiKey, headers, sessionId, signal, maxTokens: 2048 }, + ), + { policy: retry, signal }, + ); + if (response.stopReason === "aborted") { + return { aborted: true }; + } + if (response.stopReason === "error") { + return { error: response.errorMessage || "Summarization failed" }; + } + + let summary = response.content + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join("\n"); + summary = BRANCH_SUMMARY_PREAMBLE + summary; + const { readFiles, modifiedFiles } = computeFileLists(fileOps); + summary += formatFileOperations(readFiles, modifiedFiles); + + return { + summary: summary || "No summary generated", + readFiles, + modifiedFiles, + usage: response.usage, + }; +} diff --git a/packages/coding-agent/src/session/context/context-tree.ts b/packages/coding-agent/src/session/context/context-tree.ts new file mode 100644 index 0000000000..4f10682ba4 --- /dev/null +++ b/packages/coding-agent/src/session/context/context-tree.ts @@ -0,0 +1,326 @@ +import { existsSync, readdirSync, statSync } from "node:fs"; +import { basename, join } from "node:path"; +import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; +import type { ContextUsage } from "../../core/extensions/index.js"; +import { + buildSessionContext, + type FileEntry, + loadEntriesFromFile, + type SessionEntry, +} from "../../core/session-manager.js"; +import type { RlmChildAgentStatus } from "../agent-session.js"; +import { calculateContextTokens, estimateContextTokens } from "./token-estimate.js"; +import { addAssistantUsage, cloneUsage, emptyUsage, subtractAssistantUsage } from "./usage.js"; + +/** Resolves a model's context window so disk-only nodes can report utilization. */ +export type ContextWindowResolver = (provider: string, modelId: string) => number | undefined; + +/** + * One agent in the context overview: the main session or an RLM (sub-)agent. + * `ownUsage` excludes descendants; `totalUsage` includes completed descendants, matching /usage. + */ +export interface ContextTreeNode { + /** "root" for the session itself; sub-xxxx for an RLM child. */ + id: string; + label: string; + status: "active" | RlmChildAgentStatus; + model?: { provider: string; id: string }; + ownUsage: Usage; + totalUsage: Usage; + contextUsage?: ContextUsage; + children: ContextTreeNode[]; +} + +function isAssistantEntry(entry: SessionEntry): entry is SessionEntry & { + type: "message"; + message: AssistantMessage; +} { + return entry.type === "message" && entry.message.role === "assistant"; +} + +function readUserMessageText(content: unknown): string { + if (typeof content === "string") { + return content; + } + if (!Array.isArray(content)) { + return ""; + } + return content + .filter( + (block): block is { type: "text"; text: string } => + typeof block === "object" && + block !== null && + (block as { type?: unknown }).type === "text" && + typeof (block as { text?: unknown }).text === "string", + ) + .map((block) => block.text) + .join("\n"); +} + +function compactLabel(text: string, maxLength = 80): string { + const compact = text.replace(/\s+/g, " ").trim(); + if (compact.length <= maxLength) { + return compact; + } + return `${compact.slice(0, Math.max(0, maxLength - 3)).trimEnd()}...`; +} + +/** + * Usage totals for one agent: `totalUsage` sums the branch's assistant usage + * (attributed aggregates, so descendants are included), `ownUsage` removes the + * attributions targeting those assistants. Attribution entries are matched by + * target across ALL entries, not just the branch: attributions rewrite the + * target assistant's usage no matter which branch they were appended on, so a + * fork that keeps the assistant but drops the attribution entry must still + * subtract it. + * + * Totals are deliberately cumulative across compactions: compaction shrinks + * the model-facing context, not what the session has spent, so assistants + * dropped from the resolved context still count here. + */ +export function computeOwnAndTotalUsage( + branch: SessionEntry[], + allEntries: SessionEntry[], +): { ownUsage: Usage; totalUsage: Usage } { + const totalUsage = emptyUsage(); + const branchAssistantIds = new Set(); + for (const entry of branch) { + if (isAssistantEntry(entry)) { + branchAssistantIds.add(entry.id); + addAssistantUsage(totalUsage, entry.message.usage); + } else if ((entry.type === "compaction" || entry.type === "branch_summary") && entry.usage) { + addAssistantUsage(totalUsage, entry.usage); + } + } + const ownUsage = cloneUsage(totalUsage); + for (const entry of allEntries) { + if (entry.type === "child_usage_attributed" && branchAssistantIds.has(entry.targetId)) { + subtractAssistantUsage(ownUsage, entry.childUsage); + } + } + return { ownUsage, totalUsage }; +} + +/** + * Current context utilization from persisted entries, mirroring + * AgentSession.getContextUsage(): unknown right after a compaction until the + * next assistant response, otherwise the last assistant usage plus an + * estimate for trailing messages (tool results, queued user input) that have + * not hit the model yet. + */ +function computeContextUsageFromEntries( + allEntries: SessionEntry[], + branch: SessionEntry[], + contextWindow: number | undefined, +): ContextUsage | undefined { + if (!contextWindow || contextWindow <= 0) { + return undefined; + } + + let latestCompactionIndex = -1; + for (let i = branch.length - 1; i >= 0; i--) { + if (branch[i].type === "compaction") { + latestCompactionIndex = i; + break; + } + } + + if (latestCompactionIndex >= 0) { + let hasPostCompactionUsage = false; + for (let i = branch.length - 1; i > latestCompactionIndex; i--) { + const entry = branch[i]; + if (!isAssistantEntry(entry)) { + continue; + } + const assistant = entry.message; + if (assistant.stopReason === "aborted" || assistant.stopReason === "error") { + continue; + } + if (calculateContextTokens(assistant.usage) > 0) { + hasPostCompactionUsage = true; + } + break; + } + if (!hasPostCompactionUsage) { + return { tokens: null, contextWindow, percent: null }; + } + } + + const estimate = estimateContextTokens(buildSessionContext(allEntries).messages); + if (estimate.tokens <= 0) { + return undefined; + } + return { tokens: estimate.tokens, contextWindow, percent: (estimate.tokens / contextWindow) * 100 }; +} + +function sessionEntriesFromFile(file: string): SessionEntry[] { + return loadEntriesFromFile(file).filter((entry: FileEntry): entry is SessionEntry => entry.type !== "session"); +} + +/** + * Entries on the current branch, root to leaf, mirroring + * SessionManager.getBranch(): the leaf is the last appended entry and the + * branch is its parentId chain. Keeps forked/abandoned paths out of usage + * sums so disk nodes match what a live session would report. + */ +function branchEntries(entries: SessionEntry[]): SessionEntry[] { + if (entries.length === 0) { + return []; + } + const byId = new Map(entries.map((entry) => [entry.id, entry])); + const branch: SessionEntry[] = []; + const seen = new Set(); + let current: SessionEntry | undefined = entries[entries.length - 1]; + while (current && !seen.has(current.id)) { + seen.add(current.id); + branch.push(current); + current = current.parentId ? byId.get(current.parentId) : undefined; + } + return branch.reverse(); +} + +/** + * Terminal status for a persisted child, inferred from how its last assistant + * turn ended: errored and aborted runs should not render as successful. + */ +function statusFromBranch(entries: SessionEntry[]): "done" | "error" | "cancelled" { + for (let i = entries.length - 1; i >= 0; i--) { + const entry = entries[i]; + if (!isAssistantEntry(entry)) { + continue; + } + if (entry.message.stopReason === "error") { + return "error"; + } + if (entry.message.stopReason === "aborted") { + return "cancelled"; + } + return "done"; + } + return "done"; +} + +function findSessionFile(dir: string): string | undefined { + let newest: { path: string; mtime: number } | undefined; + for (const name of readdirSync(dir)) { + if (!name.endsWith(".jsonl")) { + continue; + } + const path = join(dir, name); + try { + const mtime = statSync(path).mtime.getTime(); + if (!newest || mtime > newest.mtime) { + newest = { path, mtime }; + } + } catch { + // Skip unreadable files. + } + } + return newest?.path; +} + +function listChildSessionDirs(rlmSessionDir: string): string[] { + let names: string[]; + try { + names = readdirSync(rlmSessionDir); + } catch { + return []; + } + return names + .filter((name) => name.startsWith("sub-")) + .map((name) => join(rlmSessionDir, name)) + .filter((path) => { + try { + return statSync(path).isDirectory(); + } catch { + return false; + } + }) + .sort((a, b) => { + try { + return statSync(a).mtime.getTime() - statSync(b).mtime.getTime(); + } catch { + return 0; + } + }); +} + +/** + * Build a context node for a completed RLM child from its persisted session + * dir (sub-xxxx/). Children that already attributed grandchild usage carry the + * aggregate on their assistant messages (applyChildUsageAttributions), so own + * usage is recovered by subtracting the attribution entries. Returns undefined + * when the dir holds no readable session. + */ +export function loadContextTreeChildFromDisk( + childSessionDir: string, + resolveContextWindow: ContextWindowResolver, +): ContextTreeNode | undefined { + const sessionFile = findSessionFile(childSessionDir); + if (!sessionFile) { + return undefined; + } + const allEntries = sessionEntriesFromFile(sessionFile); + const branch = branchEntries(allEntries); + if (branch.length === 0) { + return undefined; + } + + const { ownUsage, totalUsage } = computeOwnAndTotalUsage(branch, allEntries); + + let model: { provider: string; id: string } | undefined; + for (const entry of branch) { + if (entry.type === "model_change") { + model = { provider: entry.provider, id: entry.modelId }; + } + } + + let label = ""; + for (const entry of branch) { + if (entry.type === "message" && entry.message.role === "user") { + label = compactLabel(readUserMessageText(entry.message.content)); + if (label) { + break; + } + } + } + + const contextWindow = model ? resolveContextWindow(model.provider, model.id) : undefined; + + return { + id: basename(childSessionDir), + label: label || "child agent", + status: statusFromBranch(branch), + model, + ownUsage, + totalUsage, + contextUsage: computeContextUsageFromEntries(allEntries, branch, contextWindow), + children: loadContextTreeChildrenFromDisk(childSessionDir, resolveContextWindow), + }; +} + +/** + * Build context nodes for all persisted RLM children under an RLM session + * dir, recursing into nested sub-* dirs for grandchildren. `skipIds` + * excludes children that are already represented live. + */ +export function loadContextTreeChildrenFromDisk( + rlmSessionDir: string | undefined, + resolveContextWindow: ContextWindowResolver, + skipIds?: ReadonlySet, +): ContextTreeNode[] { + if (!rlmSessionDir || !existsSync(rlmSessionDir)) { + return []; + } + const nodes: ContextTreeNode[] = []; + for (const childDir of listChildSessionDirs(rlmSessionDir)) { + if (skipIds?.has(basename(childDir))) { + continue; + } + const node = loadContextTreeChildFromDisk(childDir, resolveContextWindow); + if (node) { + nodes.push(node); + } + } + return nodes; +} diff --git a/packages/coding-agent/src/session/context/context-view.ts b/packages/coding-agent/src/session/context/context-view.ts index e30311417d..f1f7e63d3f 100644 --- a/packages/coding-agent/src/session/context/context-view.ts +++ b/packages/coding-agent/src/session/context/context-view.ts @@ -1,17 +1,17 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; import type { Api, AssistantMessage, Model, Usage } from "@earendil-works/pi-ai"; -import { calculateContextTokens, estimateContextTokens } from "../../core/compaction/index.js"; +import type { ContextUsage } from "../../core/extensions/index.js"; +import { getLatestCompactionEntry, type SessionEntry, type SessionManager } from "../../core/session-manager.js"; import { type ContextTreeNode, type ContextWindowResolver, computeOwnAndTotalUsage, loadContextTreeChildFromDisk, loadContextTreeChildrenFromDisk, -} from "../../core/context-tree.js"; -import type { ContextUsage } from "../../core/extensions/index.js"; -import { getLatestCompactionEntry, type SessionEntry, type SessionManager } from "../../core/session-manager.js"; -import type { SessionStats } from "../../core/session-stats.js"; -import { emptyUsage, type SessionUsageSummary, sessionUsageSummaryFrom } from "../../core/usage.js"; +} from "./context-tree.js"; +import type { SessionStats } from "./session-stats.js"; +import { calculateContextTokens, estimateContextTokens } from "./token-estimate.js"; +import { emptyUsage, type SessionUsageSummary, sessionUsageSummaryFrom } from "./usage.js"; export interface ContextViewChild { id: string; diff --git a/packages/coding-agent/src/session/context/conversation-text.ts b/packages/coding-agent/src/session/context/conversation-text.ts new file mode 100644 index 0000000000..077631fbb0 --- /dev/null +++ b/packages/coding-agent/src/session/context/conversation-text.ts @@ -0,0 +1,81 @@ +import type { Message } from "@earendil-works/pi-ai"; + +/** Maximum characters for a tool result in serialized summaries. */ +const TOOL_RESULT_MAX_CHARS = 2000; + +/** + * Truncate text to a maximum character length for summarization. + * Keeps the beginning and appends a truncation marker. + */ +function truncateForSummary(text: string, maxChars: number): string { + if (text.length <= maxChars) return text; + const truncatedChars = text.length - maxChars; + return `${text.slice(0, maxChars)}\n\n[... ${truncatedChars} more characters truncated]`; +} + +/** + * Serialize LLM messages to text for summarization. + * This prevents the model from treating it as a conversation to continue. + * Call convertToLlm() first to handle custom message types. + * + * Tool results are truncated to keep the summarization request within + * reasonable token budgets. Full content is not needed for summarization. + */ +export function serializeConversation(messages: Message[]): string { + const parts: string[] = []; + + for (const msg of messages) { + if (msg.role === "user") { + const content = + typeof msg.content === "string" + ? msg.content + : msg.content + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join(""); + if (content) parts.push(`[User]: ${content}`); + } else if (msg.role === "assistant") { + const textParts: string[] = []; + const thinkingParts: string[] = []; + const toolCalls: string[] = []; + + for (const block of msg.content) { + if (block.type === "text") { + textParts.push(block.text); + } else if (block.type === "thinking") { + thinkingParts.push(block.thinking); + } else if (block.type === "toolCall") { + const args = block.arguments as Record; + const argsStr = Object.entries(args) + .map(([k, v]) => `${k}=${JSON.stringify(v)}`) + .join(", "); + toolCalls.push(`${block.name}(${argsStr})`); + } + } + + if (thinkingParts.length > 0) { + parts.push(`[Assistant thinking]: ${thinkingParts.join("\n")}`); + } + if (textParts.length > 0) { + parts.push(`[Assistant]: ${textParts.join("\n")}`); + } + if (toolCalls.length > 0) { + parts.push(`[Assistant tool calls]: ${toolCalls.join("; ")}`); + } + } else if (msg.role === "toolResult") { + const content = msg.content + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join(""); + if (content) { + parts.push(`[Tool result]: ${truncateForSummary(content, TOOL_RESULT_MAX_CHARS)}`); + } + } + } + + return parts.join("\n\n"); +} + +export const SUMMARIZATION_SYSTEM_PROMPT = `You are a context summarization assistant. Your task is to read a conversation between a user and an AI coding assistant, then produce a structured summary following the exact format specified. + +Do NOT continue the conversation. Do NOT respond to any questions in the conversation. ONLY output the structured summary.`; diff --git a/packages/coding-agent/src/session/context/file-tracking.ts b/packages/coding-agent/src/session/context/file-tracking.ts new file mode 100644 index 0000000000..42bea9fe2c --- /dev/null +++ b/packages/coding-agent/src/session/context/file-tracking.ts @@ -0,0 +1,67 @@ +import type { AgentMessage } from "@earendil-works/pi-agent-core"; + +export interface FileOperations { + read: Set; + written: Set; + edited: Set; +} + +export function createFileOps(): FileOperations { + return { + read: new Set(), + written: new Set(), + edited: new Set(), + }; +} + +/** + * Extract file operations from tool calls in an assistant message. + */ +export function extractFileOpsFromMessage(message: AgentMessage, fileOps: FileOperations): void { + if (message.role !== "assistant") return; + if (!("content" in message) || !Array.isArray(message.content)) return; + + for (const block of message.content) { + if (typeof block !== "object" || block === null) continue; + if (!("type" in block) || block.type !== "toolCall") continue; + if (!("arguments" in block) || !("name" in block)) continue; + + const args = block.arguments as Record | undefined; + if (!args) continue; + + const path = typeof args.path === "string" ? args.path : undefined; + if (!path) continue; + + switch (block.name) { + case "edit": + fileOps.edited.add(path); + break; + } + } +} + +/** + * Compute final file lists from file operations. + * Returns readFiles (files only read, not modified) and modifiedFiles. + */ +export function computeFileLists(fileOps: FileOperations): { readFiles: string[]; modifiedFiles: string[] } { + const modified = new Set([...fileOps.edited, ...fileOps.written]); + const readOnly = [...fileOps.read].filter((f) => !modified.has(f)).sort(); + const modifiedFiles = [...modified].sort(); + return { readFiles: readOnly, modifiedFiles }; +} + +/** + * Format file operations as XML tags for summary. + */ +export function formatFileOperations(readFiles: string[], modifiedFiles: string[]): string { + const sections: string[] = []; + if (readFiles.length > 0) { + sections.push(`\n${readFiles.join("\n")}\n`); + } + if (modifiedFiles.length > 0) { + sections.push(`\n${modifiedFiles.join("\n")}\n`); + } + if (sections.length === 0) return ""; + return `\n\n${sections.join("\n\n")}`; +} diff --git a/packages/coding-agent/src/session/context/harness-context.ts b/packages/coding-agent/src/session/context/harness-context.ts index 5a9995436b..df5fb01048 100644 --- a/packages/coding-agent/src/session/context/harness-context.ts +++ b/packages/coding-agent/src/session/context/harness-context.ts @@ -1,13 +1,14 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { SessionContext, SessionManager } from "../../core/session-manager.js"; +import type { Skill } from "../../core/skills.js"; +import { formatHarnessStateForPrompt } from "../refinement/format.js"; +import { type HarnessState, REFINE_SKILL_NAME } from "../refinement/types.js"; import { type CustomMessage, createHarnessDigestMessage, HARNESS_DIGEST_CUSTOM_TYPE, type HarnessDigestDetails, -} from "../../core/messages.js"; -import { formatHarnessStateForPrompt, type HarnessState, REFINE_SKILL_NAME } from "../../core/refinement/index.js"; -import type { SessionContext, SessionManager } from "../../core/session-manager.js"; -import type { Skill } from "../../core/skills.js"; +} from "./messages.js"; export interface HarnessContextHost { sessionManager: Pick; getMessages(): AgentMessage[]; diff --git a/packages/coding-agent/src/session/context/history-navigation.ts b/packages/coding-agent/src/session/context/history-navigation.ts index ee75bcf701..51adfb6a97 100644 --- a/packages/coding-agent/src/session/context/history-navigation.ts +++ b/packages/coding-agent/src/session/context/history-navigation.ts @@ -1,10 +1,10 @@ import type { Api, Model, Usage } from "@earendil-works/pi-ai"; -import { collectEntriesForBranchSummary, generateBranchSummary } from "../../core/compaction/index.js"; import type { ExtensionRunner, SessionBeforeTreeResult, TreePreparation } from "../../core/extensions/index.js"; import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; import type { BranchSummaryEntry, SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; import type { SessionCommitLease } from "../input/commit-fence.js"; +import { collectEntriesForBranchSummary, generateBranchSummary } from "./branch-summary.js"; export interface HistoryNavigationHost { getSessionId?(): string; diff --git a/packages/coding-agent/src/session/context/messages.ts b/packages/coding-agent/src/session/context/messages.ts new file mode 100644 index 0000000000..c559df8602 --- /dev/null +++ b/packages/coding-agent/src/session/context/messages.ts @@ -0,0 +1,670 @@ +/** + * Custom message types and transformers for the coding agent. + * + * Extends the base AgentMessage type with coding-agent specific message types, + * and provides a transformer to convert them to LLM-compatible messages. + */ + +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { ImageContent, Message, TextContent } from "@earendil-works/pi-ai"; +import type { AgentCronJob } from "../../core/cron-jobs.js"; +import { + isSessionSlashCommandName, + parseSessionSlashCommand, + type SessionSlashCommand, +} from "../../core/slash-commands.js"; +import { formatRefinementNoticeBody } from "../refinement/format.js"; +import type { AppliedRefinementEdit, HarnessScope, RefinementResult } from "../refinement/types.js"; + +export const COMPACTION_SUMMARY_PREFIX = `[compaction-summary] + +The conversation history before this point was compacted into the following summary: + + +`; + +export const COMPACTION_SUMMARY_SUFFIX = ` +`; + +export const BRANCH_SUMMARY_PREFIX = `[branch-summary] + +The following is a summary of a branch that this conversation came back from: + + +`; + +export const BRANCH_SUMMARY_SUFFIX = ``; + +export const HEARTBEAT_PROMPT_CUSTOM_TYPE = "heartbeat_prompt"; +export const HEARTBEAT_PROMPT_PREVIEW_LABEL = "Heartbeat prompt"; +export const IPYTHON_STATE_RESTORED_CUSTOM_TYPE = "ipython_state_restored"; +export const SESSION_SLASH_COMMAND_CUSTOM_TYPE = "session_slash_command"; +export const SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE = "session_slash_command_result"; +export const COMPACTION_OUTCOME_CUSTOM_TYPE = "compaction_outcome"; +export const REFINEMENT_OUTCOME_CUSTOM_TYPE = "refinement_outcome"; +export const REFINEMENT_NOTICE_CUSTOM_TYPE = "refinement_notice"; +export const HARNESS_DIGEST_CUSTOM_TYPE = "harness_digest"; +export const RLM_CHILD_FAILURE_CUSTOM_TYPE = "rlm_child_failure"; +export const RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE = "rlm_child_terminal_notice"; +export const ASYNC_BASH_COMPLETION_CUSTOM_TYPE = "async_bash_completion"; +export const ASYNC_BASH_COMPLETION_PREVIEW_LABEL = "Background command finished"; + +/** + * Names and other metadata interpolated into a `[ ...]` header line must not + * carry the characters that delimit the header itself (brackets, newlines, commas, + * or the relationship separator ":"). + */ +export function sanitizeMessageHeaderValue(value: string): string { + return value.replace(/[\s,:[\]]+/g, " ").trim(); +} + +export interface SessionSlashCommandDetails { + command: SessionSlashCommand; + commandEntryId?: string; +} + +export interface SessionSlashCommandResultDetails { + command: SessionSlashCommand; + success: boolean; + severity: "info" | "warning" | "error"; + error?: string; + commandEntryId?: string; +} + +export interface SessionSlashCommandMessage extends CustomMessage { + customType: typeof SESSION_SLASH_COMMAND_CUSTOM_TYPE; + content: string; + details: SessionSlashCommandDetails; +} + +export interface SessionSlashCommandResultMessage extends CustomMessage { + customType: typeof SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE; + content: string; + details: SessionSlashCommandResultDetails; +} + +export type CompactionOutcomeReason = "threshold" | "overflow" | "requested"; +export type CompactionOutcome = "skipped" | "cancelled" | "failed"; + +export interface CompactionOutcomeDetails { + reason: CompactionOutcomeReason; + outcome: CompactionOutcome; +} + +export interface CompactionOutcomeMessage extends CustomMessage { + customType: typeof COMPACTION_OUTCOME_CUSTOM_TYPE; + content: string; + details: CompactionOutcomeDetails; +} + +export interface RefinementOutcomeDetails { + refinementId: string; + summary: string; + scope: HarnessScope; + rollbackOf?: string; + edits: AppliedRefinementEdit[]; +} + +export interface RefinementOutcomeMessage extends CustomMessage { + customType: typeof REFINEMENT_OUTCOME_CUSTOM_TYPE; + content: string; + details: RefinementOutcomeDetails; +} + +/** How a refinement was initiated: reviewer-triggered auto-refine, the /refine slash command, or the model's own refine.run(). */ +export type RefinementSource = "auto" | "user" | "self"; + +export interface RefinementNoticeDetails extends RefinementOutcomeDetails { + source: RefinementSource; +} + +export interface RefinementNoticeMessage extends CustomMessage { + customType: typeof REFINEMENT_NOTICE_CUSTOM_TYPE; + content: string; + details: RefinementNoticeDetails; +} + +export interface HarnessDigestDetails { + digest: string; +} + +export const HARNESS_DIGEST_PREFIX = `[harness-digest] + +The persistent memories produced across this session so far: + + +`; + +export const HARNESS_DIGEST_SUFFIX = ` +`; + +export function createHarnessDigestMessage( + digest: string, + timestamp = Date.now(), +): CustomMessage { + return { + role: "custom", + customType: HARNESS_DIGEST_CUSTOM_TYPE, + content: HARNESS_DIGEST_PREFIX + digest + HARNESS_DIGEST_SUFFIX, + display: false, + details: { digest }, + timestamp, + }; +} + +export interface RlmChildFailureDetails { + childId: string; + sessionName: string; + error: string; +} + +export type RlmChildTerminalNoticeDetails = + | { + kind: "cancelled"; + childId: string; + sessionName: string; + reason?: string; + } + | { + kind: "completed_without_reply"; + childId: string; + sessionName: string; + lastAssistantTextPreview?: string; + }; + +export interface AsyncBashCompletionDetails { + pid: number; + command: string; + exitCode: number; +} + +interface AsyncBashCompletionMessage extends CustomMessage { + customType: typeof ASYNC_BASH_COMPLETION_CUSTOM_TYPE; + content: string; +} + +export function createAsyncBashCompletionMessage( + details: AsyncBashCompletionDetails, + timestamp = Date.now(), +): AsyncBashCompletionMessage { + return { + role: "custom", + customType: ASYNC_BASH_COMPLETION_CUSTOM_TYPE, + content: `[bash-done pid:${details.pid} exit:${details.exitCode}] + +Command: ${JSON.stringify(details.command)}`, + display: true, + details, + timestamp, + }; +} + +export function createRlmChildFailureMessage( + details: RlmChildFailureDetails, + timestamp = Date.now(), +): CustomMessage { + return { + role: "custom", + customType: RLM_CHILD_FAILURE_CUSTOM_TYPE, + content: `[child-failed child:${sanitizeMessageHeaderValue(details.sessionName)}] + +${details.error}`, + display: true, + details, + timestamp, + }; +} + +export function createRlmChildTerminalNoticeMessage( + details: RlmChildTerminalNoticeDetails, + timestamp = Date.now(), +): CustomMessage { + const childName = sanitizeMessageHeaderValue(details.sessionName); + const content = + details.kind === "cancelled" + ? `[child-exited: cancelled child:${childName}]${details.reason ? `\n\n${details.reason}` : ""}` + : `[child-exited: no-reply child:${childName}]${details.lastAssistantTextPreview ? `\n\nLast assistant text: ${details.lastAssistantTextPreview}` : ""}`; + return { + role: "custom", + customType: RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, + content, + display: true, + details, + timestamp, + }; +} + +/** + * Message type for bash executions via the ! command. + */ +export interface BashExecutionMessage { + role: "bashExecution"; + command: string; + output: string; + exitCode: number | undefined; + cancelled: boolean; + truncated: boolean; + fullOutputPath?: string; + timestamp: number; + /** If true, this message is excluded from LLM context (!! prefix) */ + excludeFromContext?: boolean; +} + +/** + * Message type for extension-injected messages via sendMessage(). + * These are custom messages that extensions can inject into the conversation. + */ +export interface CustomMessage { + role: "custom"; + customType: string; + content: string | (TextContent | ImageContent)[]; + display: boolean; + details?: T; + timestamp: number; +} + +export interface HeartbeatPromptDetails { + jobId: string; + schedule: string; + status: AgentCronJob["status"]; + runCount: number; + nextRunAt?: string; + lastRunAt?: string; +} + +export interface IpythonStateRestoredDetails { + restored: boolean; +} + +export interface BranchSummaryMessage { + role: "branchSummary"; + summary: string; + fromId: string; + timestamp: number; +} + +export interface CompactionSummaryMessage { + role: "compactionSummary"; + summary: string; + tokensBefore: number; + /** Number of retained messages that precede this summary in transcript presentation. */ + retainedMessageCount?: number; + /** User instructions that guided the summary (from `/compact `) */ + customInstructions?: string; + /** Harness digest snapshot rendered before the summary in LLM context. Attached mechanically at compaction, never summarized. */ + harnessDigest?: string; + timestamp: number; +} + +declare module "@earendil-works/pi-agent-core" { + interface CustomAgentMessages { + bashExecution: BashExecutionMessage; + custom: CustomMessage; + branchSummary: BranchSummaryMessage; + compactionSummary: CompactionSummaryMessage; + } +} + +/** + * Format bash output for LLM context. The fence must be longer than any + * backtick run in the output so command output cannot terminate it early. + */ +export function bashOutputToText( + msg: Pick, +): string { + let text = ""; + if (msg.output) { + let longestBacktickRun = 0; + for (const match of msg.output.matchAll(/`+/g)) { + longestBacktickRun = Math.max(longestBacktickRun, match[0].length); + } + const fence = "`".repeat(Math.max(3, longestBacktickRun + 1)); + text += `${fence}\n${msg.output}\n${fence}`; + } else { + text += "(no output)"; + } + if (msg.cancelled) { + text += "\n\n(command cancelled)"; + } else if (msg.exitCode !== null && msg.exitCode !== undefined && msg.exitCode !== 0) { + text += `\n\nCommand exited with code ${msg.exitCode}`; + } + if (msg.truncated) { + text += msg.fullOutputPath + ? `\n\n[Output truncated. Full output: ${msg.fullOutputPath}]` + : "\n\n[Output truncated.]"; + } + return text; +} + +/** + * Convert a BashExecutionMessage to user message text for LLM context. + */ +export function bashExecutionToText(msg: BashExecutionMessage): string { + return `Ran \`${msg.command}\`\n${bashOutputToText(msg)}`; +} + +export function createBranchSummaryMessage(summary: string, fromId: string, timestamp: string): BranchSummaryMessage { + return { + role: "branchSummary", + summary, + fromId, + timestamp: new Date(timestamp).getTime(), + }; +} + +export function createCompactionSummaryMessage( + summary: string, + tokensBefore: number, + timestamp: string, + customInstructions?: string, + retainedMessageCount?: number, + harnessDigest?: string, +): CompactionSummaryMessage { + return { + role: "compactionSummary", + summary, + tokensBefore, + retainedMessageCount, + customInstructions, + harnessDigest, + timestamp: new Date(timestamp).getTime(), + }; +} + +/** Convert CustomMessageEntry to AgentMessage format */ +export function createCustomMessage( + customType: string, + content: string | (TextContent | ImageContent)[], + display: boolean, + details: unknown | undefined, + timestamp: string, +): CustomMessage { + return { + role: "custom", + customType, + content, + display, + details, + timestamp: new Date(timestamp).getTime(), + }; +} + +export function createSessionSlashCommandMessage( + command: SessionSlashCommand, + details: Omit = {}, + display = true, + timestamp = Date.now(), +): SessionSlashCommandMessage { + return { + role: "custom", + customType: SESSION_SLASH_COMMAND_CUSTOM_TYPE, + content: command.text, + display, + details: { ...details, command: { ...command } }, + timestamp, + }; +} + +export function createSessionSlashCommandResultMessage( + content: string, + details: SessionSlashCommandResultDetails, + display = true, + timestamp = Date.now(), +): SessionSlashCommandResultMessage { + return { + role: "custom", + customType: SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, + content, + display, + details: { ...details, command: { ...details.command } }, + timestamp, + }; +} + +export function createCompactionOutcomeMessage( + content: string, + details: CompactionOutcomeDetails, + display = true, + timestamp = Date.now(), +): CompactionOutcomeMessage { + return { + role: "custom", + customType: COMPACTION_OUTCOME_CUSTOM_TYPE, + content, + display, + details: { ...details }, + timestamp, + }; +} + +export function createRefinementOutcomeMessage( + result: RefinementResult, + display = true, + timestamp = Date.now(), +): RefinementOutcomeMessage { + return { + role: "custom", + customType: REFINEMENT_OUTCOME_CUSTOM_TYPE, + content: `Refinement complete: ${result.summary}`, + display, + details: { + refinementId: result.id, + summary: result.summary, + scope: result.scope ?? "local", + ...(result.rollbackOf ? { rollbackOf: result.rollbackOf } : {}), + edits: result.appliedEdits, + }, + timestamp, + }; +} + +/** Model-facing refinement notice: passes convertToLlm (unlike the refinement_outcome audit entry); display false because the TUI renders the outcome message. */ +export function createRefinementNoticeMessage( + result: RefinementResult, + source: RefinementSource, + timestamp = Date.now(), +): RefinementNoticeMessage { + return { + role: "custom", + customType: REFINEMENT_NOTICE_CUSTOM_TYPE, + content: `[${source}-refinement]\n\n${formatRefinementNoticeBody(result)}`, + display: false, + details: { + refinementId: result.id, + summary: result.summary, + scope: result.scope ?? "local", + ...(result.rollbackOf ? { rollbackOf: result.rollbackOf } : {}), + edits: result.appliedEdits, + source, + }, + timestamp, + }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null; +} + +function hasValidCustomMessageEnvelope(message: Record, customType: string): boolean { + return ( + message.role === "custom" && + message.customType === customType && + typeof message.content === "string" && + typeof message.display === "boolean" && + typeof message.timestamp === "number" && + Number.isFinite(message.timestamp) + ); +} + +export function isSessionSlashCommand(value: unknown): value is SessionSlashCommand { + if ( + !isRecord(value) || + !isSessionSlashCommandName(value.name) || + typeof value.args !== "string" || + typeof value.text !== "string" + ) { + return false; + } + const parsed = parseSessionSlashCommand(value.text); + return ( + parsed !== undefined && parsed.name === value.name && parsed.args === value.args && parsed.text === value.text + ); +} + +function isValidCommandEntryId(value: unknown): value is string | undefined { + return value === undefined || (typeof value === "string" && value.length > 0); +} + +export function isSessionSlashCommandMessage(message: unknown): message is SessionSlashCommandMessage { + if ( + !isRecord(message) || + !hasValidCustomMessageEnvelope(message, SESSION_SLASH_COMMAND_CUSTOM_TYPE) || + typeof message.content !== "string" + ) { + return false; + } + if (!isRecord(message.details) || !isSessionSlashCommand(message.details.command)) return false; + return message.content === message.details.command.text && isValidCommandEntryId(message.details.commandEntryId); +} + +export function isSessionSlashCommandResultMessage(message: unknown): message is SessionSlashCommandResultMessage { + if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE)) + return false; + if (!isRecord(message.details) || !isSessionSlashCommand(message.details.command)) return false; + return ( + typeof message.details.success === "boolean" && + (message.details.severity === "info" || + message.details.severity === "warning" || + message.details.severity === "error") && + (message.details.error === undefined || typeof message.details.error === "string") && + isValidCommandEntryId(message.details.commandEntryId) + ); +} + +export function isCompactionOutcomeMessage(message: unknown): message is CompactionOutcomeMessage { + if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, COMPACTION_OUTCOME_CUSTOM_TYPE)) return false; + if (!isRecord(message.details)) return false; + return ( + (message.details.reason === "threshold" || + message.details.reason === "overflow" || + message.details.reason === "requested") && + (message.details.outcome === "skipped" || + message.details.outcome === "cancelled" || + message.details.outcome === "failed") + ); +} + +function isAppliedRefinementEdit(value: unknown): value is AppliedRefinementEdit { + return ( + isRecord(value) && + (value.action === "create" || value.action === "update" || value.action === "delete") && + typeof value.kind === "string" && + typeof value.id === "string" && + typeof value.applied === "boolean" + ); +} + +export function isRefinementOutcomeMessage(message: unknown): message is RefinementOutcomeMessage { + if (!isRecord(message) || !hasValidCustomMessageEnvelope(message, REFINEMENT_OUTCOME_CUSTOM_TYPE)) return false; + if (!isRecord(message.details)) return false; + return ( + typeof message.details.summary === "string" && + (message.details.scope === "local" || message.details.scope === "global") && + Array.isArray(message.details.edits) && + message.details.edits.every(isAppliedRefinementEdit) + ); +} + +export interface HeartbeatPromptMessage extends CustomMessage { + customType: typeof HEARTBEAT_PROMPT_CUSTOM_TYPE; + content: string; +} + +export function createHeartbeatPromptMessage(job: AgentCronJob, timestamp = Date.now()): HeartbeatPromptMessage { + return { + role: "custom", + customType: HEARTBEAT_PROMPT_CUSTOM_TYPE, + content: `[heartbeat: ${sanitizeMessageHeaderValue(job.schedule.expression)} run#${job.runCount}]\n\n${job.prompt}`, + display: true, + details: { + jobId: job.id, + schedule: job.schedule.expression, + status: job.status, + runCount: job.runCount, + nextRunAt: job.nextRunAt, + lastRunAt: job.lastRunAt, + }, + timestamp, + }; +} + +/** + * Transform AgentMessages (including custom types) to LLM-compatible Messages. + * + * This is used by: + * - Agent's transormToLlm option (for prompt calls and queued messages) + * - Compaction's generateSummary (for summarization) + * - Custom extensions and tools + */ +export function convertToLlm(messages: AgentMessage[]): Message[] { + return messages + .map((m): Message | undefined => { + switch (m.role) { + case "bashExecution": + if (m.excludeFromContext) { + return undefined; + } + return { + role: "user", + content: [{ type: "text", text: bashExecutionToText(m) }], + timestamp: m.timestamp, + }; + case "custom": { + if ( + m.customType === SESSION_SLASH_COMMAND_CUSTOM_TYPE || + m.customType === SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE || + m.customType === COMPACTION_OUTCOME_CUSTOM_TYPE || + m.customType === REFINEMENT_OUTCOME_CUSTOM_TYPE + ) { + return undefined; + } + const content = typeof m.content === "string" ? [{ type: "text" as const, text: m.content }] : m.content; + return { + role: "user", + content, + timestamp: m.timestamp, + }; + } + case "branchSummary": + return { + role: "user", + content: [{ type: "text" as const, text: BRANCH_SUMMARY_PREFIX + m.summary + BRANCH_SUMMARY_SUFFIX }], + timestamp: m.timestamp, + }; + case "compactionSummary": { + const digestBlock = m.harnessDigest + ? `${HARNESS_DIGEST_PREFIX}${m.harnessDigest}${HARNESS_DIGEST_SUFFIX}\n\n` + : ""; + return { + role: "user", + content: [ + { + type: "text" as const, + text: digestBlock + COMPACTION_SUMMARY_PREFIX + m.summary + COMPACTION_SUMMARY_SUFFIX, + }, + ], + timestamp: m.timestamp, + }; + } + case "user": + case "assistant": + case "toolResult": + return m; + default: + // biome-ignore lint/correctness/noSwitchDeclarations: fine + const _exhaustiveCheck: never = m; + return undefined; + } + }) + .filter((m) => m !== undefined); +} diff --git a/packages/coding-agent/src/session/context/pending-context.ts b/packages/coding-agent/src/session/context/pending-context.ts index 2676642a01..cc4a3e84bb 100644 --- a/packages/coding-agent/src/session/context/pending-context.ts +++ b/packages/coding-agent/src/session/context/pending-context.ts @@ -1,10 +1,5 @@ -import { - type CustomMessage, - RLM_CHILD_FAILURE_CUSTOM_TYPE, - RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, -} from "../../core/messages.js"; -import type { ActionStore, DeliveryRecord } from "../../core/session-action-store.js"; import { waitForPromiseOrAbort } from "../../utils/wait-for-abort.js"; +import type { ActionStore, DeliveryRecord } from "../input/action-store.js"; import type { SessionCommitFence, SessionCommitLease } from "../input/commit-fence.js"; import type { SessionInputScheduler } from "../input/input-scheduler.js"; import { @@ -12,8 +7,13 @@ import { createPreparedTurnAction, primaryDeliveryRecord, type QueuedSessionAction, -} from "../prepared-actions.js"; +} from "../input/prepared-actions.js"; import { createTurnExecutionPolicy } from "../turns/turn-preparation.js"; +import { + type CustomMessage, + RLM_CHILD_FAILURE_CUSTOM_TYPE, + RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, +} from "./messages.js"; export interface SessionPendingContextHost { getScheduler(): Pick; diff --git a/packages/coding-agent/src/session/context/prompts/index.ts b/packages/coding-agent/src/session/context/prompts/index.ts new file mode 100644 index 0000000000..ecf670098f --- /dev/null +++ b/packages/coding-agent/src/session/context/prompts/index.ts @@ -0,0 +1,7 @@ +export { + buildChildAgentDoctrine, + buildRlmPrompt, + buildSubagentGuidance, + type ChildAgentDoctrineOptions, + type RlmPromptOptions, +} from "./rlm.js"; diff --git a/packages/coding-agent/src/session/context/prompts/rlm.ts b/packages/coding-agent/src/session/context/prompts/rlm.ts new file mode 100644 index 0000000000..c40731026b --- /dev/null +++ b/packages/coding-agent/src/session/context/prompts/rlm.ts @@ -0,0 +1,232 @@ +import { DEFAULT_RLM_EXTRA_IMPORT_LABELS } from "../../../core/kernel/bootstrap.js"; + +export interface RlmPromptOptions { + cwd: string; + skillsDir?: string; + installedSkills?: string[]; + messagesPath: string; + allowRecursion?: boolean; + depth?: number; + parentAgent?: string; + activeTools?: string[]; +} + +const LONG_RUNNING_WORK_PROMPT = [ + "For slow or independently completing work, use a nonblocking control loop: start the work, record its handle or output location, then end your turn. A `bash()` handle left running beyond its creating cell sends a completion follow-up; when it arrives, inspect the saved handle and continue. Reading a finished handle's result first cancels that follow-up.", + "When delegation is available and useful, assign independent substantive tasks to separate workers. Start independent workers without waiting for each one sequentially, and let them run in parallel.", + "Do not keep the turn open by polling with `time.sleep()` or shell `sleep`, and do not replace polling with a long blocking `await`. Await only the short operation needed to start work or inspect a result that is already available; otherwise end the turn.", +].join("\n"); + +const USER_PROGRESS_PROMPT = + "As the user-facing root agent, when work follows a plan, uses many subagents, or spans multiple turns, proactively give regular concise progress updates so the user does not have to ask. State the current plan, what has completed, any blockers, the proposed fixes, and the next actions. Lead with user-visible outcomes rather than internal process or gate names. Mention internal details only when they explain a blocker or decision. Send an update at meaningful milestones and before ending a turn while work is still running. Do not repeat unchanged status or interrupt short work with unnecessary updates."; + +const SIMPLIFIED_TECHNICAL_ENGLISH_PROMPT = [ + "Use simplified technical English by default for user-facing prose.", + "Prefer short sentences, common words, and concrete verbs. State one main action or fact per sentence when practical. Use lists for steps or conditions.", + "Keep necessary technical terms, names, commands, code, paths, and exact quoted text unchanged. State uncertainty directly.", + "Treat this as clarity guidance, not a claim of formal ASD-STE100 compliance. Preserve a user-requested format, tone, terminology, and necessary precision.", +].join("\n"); + +const REPL_CONTROL_PROMPT = [ + "The `ipython` tool is a persistent Python REPL — the agent's long-lived control environment for reasoning, context management, state, tool orchestration, and recursive subcalls. Top-level `await` works directly. Use it to keep intermediate variables, inspect and transform outputs, and write small helper functions. Compaction removes individual variables whose serialized form exceeds 16 MiB; keep large source data on disk and reload it when needed.", + "", + "Python is the orchestration language: use Python for loops, conditionals, parsing, and state. Use `bash()` to invoke programs, not to write shell programs — no shell loops or heredocs; do those in Python.", + "", + "Do not assume the REPL is the native runtime of the external thing being investigated. A repository, package, service, dataset, paper, website, benchmark, or API may have its own environment and normal interface. Evaluate external systems through their own interface, then use the REPL to coordinate the process and analyze what comes back.", + "", + "`bash(command)` starts a shell command in the background and returns a handle immediately: `h = bash('npm test')`. Use `h.pid` / `h.running` for liveness, `h.tail(n)` / `h.output()` for combined stdout+stderr so far, `h.poll()` for a non-blocking result, `h.kill()` to terminate (SIGTERM, escalating to SIGKILL; on Windows kill() uses taskkill /T and detached or reparented descendants may survive), and `await h` (or `await bash('cmd')`) for the completed result with exit_code, output, and duration. Prefer bash() for long-running commands so the turn keeps working. Run shell commands with `bash()`, not `subprocess`/`os.system`: subprocess calls block the kernel, show the user nothing while they run, and spawn processes the harness cannot see or stop.", + "", + "Important: do not install dependencies into the kernel just to make an external project import or run there. If a project import, test, script, CLI, or dependency check is needed, run it through that project's own environment and normal command interface. For example, in a Python repo use its documented commands, `uv run ...`, `.venv/bin/python ...`, or the active project interpreter from the repo root. Treat failures from that native environment as the relevant result.", + "", + "Use Python for reading, searching, and editing files — it gives you reusable variables you can slice, filter, and act on without re-reading. Always assign read/search results to named variables so you can revisit them later.", + "", + "Each `bash()` call is its own process, so shell state does not persist between calls; use `os.chdir(...)` for the working directory and `os.environ[...]` for environment variables — both persist in the REPL and apply to later `bash()` calls.", + "", + "Python state in the kernel persists across cells: named variables, helper functions, classes, imports, notes, parsed outputs, and helper data structures all remain available in every later turn. Tool calls are themselves Python `await` expressions, so their return values can be bound to variables and composed into program logic just like any other call.", + "", + "Continual harness state is available as `rlm.harness` and `rlm.get_harness_state()`. CRUD calls are local to this Prime Agent session by default: `rlm.harness.create_memory(...)`, `rlm.harness.update_memory(...)`, `rlm.harness.delete_memory(...)`, `rlm.harness.create_skill(...)`, `rlm.harness.update_skill(...)`, `rlm.harness.delete_skill(...)`, `rlm.harness.create_subagent(...)`, `rlm.harness.update_subagent(...)`, `rlm.harness.delete_subagent(...)`, `rlm.harness.create_prompt_note(...)`, `rlm.harness.update_prompt_note(...)`, `rlm.harness.delete_prompt_note(...)`, plus `rlm.harness.record_refinement(...)` and `rlm.harness.overview()`. Use `global_=True` only for stable cross-session lessons; Python reserves `global`, so literal `global=True` is invalid syntax.", + "", + "Terminology: continual harness names the persisted prompt, memory, skill, and subagent layer; RLM names the runtime, Python REPL kernel, and native call interface exposed to the model.", + "", + "RLM-native call contract: installed Python skills are pre-imported modules. Read the matching SKILL.md and call its documented function, such as `await .(...)`; when a CLI exists, use ` ...` from shell. Continual harness skill entries are Python REPL skills with an explicit Python `reference` and `arguments` contract. Spawn a reusable delegation spec with `await rlm.spawn('sub-task', name='worker')`; admission returns a child handle immediately. Results arrive only through an available messaging capability or files, never as an `rlm.spawn()` return value. Do not invent non-native wrappers such as `call_skill(...)` or `run_subagent(...)`.", +].join("\n"); + +export interface ChildAgentDoctrineOptions { + depth?: number; + parentAgent?: string; + installedSkills?: string[]; + activeTools?: string[]; +} + +export function buildChildAgentDoctrine(options: ChildAgentDoctrineOptions): string | undefined { + const depth = options.depth ?? 0; + const hasIpython = options.activeTools === undefined || options.activeTools.includes("ipython"); + const hasAgentMessage = options.installedSkills?.includes("agent_message") ?? false; + if (depth <= 0) return undefined; + + const lines = [ + `You are a child agent spawned by ${options.parentAgent ?? "your parent agent"}. Task prompts are labeled \`[task from parent]\`.`, + ]; + if (hasAgentMessage && hasIpython) { + lines.push( + 'When a task calls for an answer, reply explicitly with `await agent_message.send(message, receiver_role="parent")`. Not every message or task needs a reply; continue cleanup after sending and go idle normally.', + ); + } + return lines.join("\n"); +} + +export function buildRlmPrompt(options: RlmPromptOptions): string { + const { cwd, skillsDir, messagesPath } = options; + const installedSkills = options.installedSkills ?? []; + const hasAgentMessage = installedSkills.includes("agent_message"); + const hasAgentObserve = installedSkills.includes("agent_observe"); + const allowRecursion = options.allowRecursion ?? true; + const depth = options.depth ?? 0; + const activeTools = options.activeTools ?? []; + const hasIpython = options.activeTools === undefined ? true : activeTools.includes("ipython"); + const canRunShellSkills = hasIpython || activeTools.includes("bash"); + const parts = [ + "You are a general purpose agent that uses code to solve tasks.", + "You solve tasks by breaking down problems into sub-tasks, writing and executing code, observing results, and iterating one step at a time.", + "When you are done, stop calling tools and state your final answer.", + "", + LONG_RUNNING_WORK_PROMPT, + "", + ...(depth === 0 ? [USER_PROGRESS_PROMPT, ""] : []), + SIMPLIFIED_TECHNICAL_ENGLISH_PROMPT, + "", + `Working directory: ${cwd}`, + `Conversation log: ${messagesPath}`, + `Recursive agent depth: ${depth}`, + `Pre-installed Python packages: ${DEFAULT_RLM_EXTRA_IMPORT_LABELS.join(", ")}.`, + "Install additional packages with `uv pip install ` (this is a uv-managed venv with no pip module).", + ]; + + const childDoctrine = buildChildAgentDoctrine(options); + if (childDoctrine) { + parts.push("", childDoctrine); + } + + const skillLines: string[] = []; + if (skillsDir) { + skillLines.push(`Local skills live under ${skillsDir}. Read their SKILL.md files when helpful.`); + } + if (installedSkills.length > 0) { + const installed = installedSkills.map((skill) => `\`${skill}\``).join(", "); + if (hasIpython) { + skillLines.push(`Installed Python skill modules (pre-imported): ${installed}.`); + skillLines.push( + "Read each skill's SKILL.md for its API. Inspect a module with `help()` or `dir()`, then inspect a documented callable with `inspect.signature(.)`.", + ); + } else if (canRunShellSkills) { + skillLines.push(`Installed skills available as shell commands: ${installed}.`); + } + if (canRunShellSkills) { + skillLines.push( + "Each skill is also available as a shell command by the same name: ` ...`. Discover its CLI usage with ` --help`.", + ); + } + if (hasIpython && installedSkills.includes("edit")) { + skillLines.push( + "For targeted existing-file edits, prefer the pre-imported async `edit` skill from the REPL: `old = '''...'''; new = '''...'''; await edit(path=\"pkg/file.py\", old_str=old, new_str=new)`. Use exact old/new strings; if the text contains triple double quotes, use triple single-quoted variables or build `old`/`new` from inspected file slices.", + ); + } + } + if (skillLines.length > 0) { + parts.push("", ...skillLines); + } + if (hasAgentMessage) { + parts.push( + "Agent messaging is restricted to your parent, siblings, and direct children; roots are siblings, and deeper communication relays through the intermediate child.", + ); + } + if (hasAgentObserve) { + parts.push( + "Agent observation is restricted to your parent, siblings, and direct children; roots are siblings, and deeper inspection relays through the intermediate child.", + ); + } + + if (depth === 0 && hasIpython) { + parts.push( + "", + "From a daemon-backed depth-0 session, use `await rlm.create_session('task', name='researcher')` to start a separate top-level session. The call returns after the daemon creates the session and accepts its first prompt. Inline and nested sessions cannot use it. `rlm.spawn(...)` still creates a child.", + ); + } + + if (allowRecursion && hasIpython) { + parts.push( + "", + "An `rlm` object is already in your global namespace. `await rlm.spawn('sub-task', name='api-reviewer')` spawns a child and returns immediately after task admission with `rlm_child_id`, `name`, `session_dir`, and `model`; it never waits for or returns the child's answer.", + "`name` is required: choose a stable child name that is unique among siblings.", + "A child inherits your model. If a different model is explicitly requested, use `await rlm.find_models(...)` and an exact returned selector. An unavailable requested model fails spawn; decide whether to retry or omit `model`. Children also inherit your thinking level; the `thinking` option overrides it with any level the resolved child model supports, and an unsupported level fails spawn.", + ); + parts.push( + hasAgentObserve + ? "Use `await agent_observe.list_agents()` to discover family, including inactive members, and `await rlm.list_subagents()` to recover direct child handles." + : "Use `await rlm.list_subagents()` to recover direct child handles after admission.", + ); + if (hasAgentMessage) { + parts.push( + "Children reply explicitly with `await agent_message.send(message, receiver_role='parent')` when an answer is needed. Replies and follow-ups arrive as ordinary agent messages; not every task requires a reply.", + "Use `agent_message.send(..., receiver_role='child', receiver_name=child.name)` for follow-ups.", + ); + } + if (hasAgentObserve) { + parts.push( + "Use `agent_observe` to inspect a child's rollout. Observation is restricted to your parent, siblings, and direct children; relay through the intermediate child for deeper descendants.", + ); + } else { + parts.push("Inspect files a child wrote when you need to collect its work without an observation capability."); + } + parts.push( + "Spawn independent children in separate calls and end your turn instead of awaiting completion. Multiple replies may arrive over multiple turns. Delete a direct child explicitly with `await rlm.delete_subagent(child)` when it is no longer needed.", + ); + } + + if (hasIpython) { + parts.push("", REPL_CONTROL_PROMPT); + if (installedSkills.includes("refine")) { + parts.push( + "", + "Treat continual harness refinement as a small, evidence-backed update after observing a repeated failure or reusable tactic: diagnose the issue, update the smallest relevant continual harness component, validate on the next action, then record the outcome. Use `await refine.run()` to turn repeated delegation patterns into reusable subagent specs, repeated procedures into skills, durable facts/preferences into memories, and narrow behavioral policies into prompt addendums. It returns immediately and runs when the current turn ends, so continue working normally after calling it. Do not rewrite the whole continual harness when a focused memory, skill, prompt note, or subagent spec is enough.", + ); + } + } + + return parts.join("\n"); +} + +/** + * Supplemental sub-agent delegation guidance, appended after the base RLM + * prompt (see system-prompt.ts). The recursion block covers the mechanics + * (`rlm.spawn(...)` admission and handle management); this block adds the + * when and why in the same When -> Why -> menu order Claude Code's Agent tool + * uses. The subagent-spec menu itself renders just after this, inside the + * harness-state block. + */ +export function buildSubagentGuidance( + options: { includeRefineExamples?: boolean; hasAgentMessage?: boolean; hasAgentObserve?: boolean } = {}, +): string { + const lines = [ + "# Delegating to sub-agents", + "", + "Spawn independent, self-contained work with `handle = await rlm.spawn('task', name='worker')`. This returns at admission, not completion; keep the handle to stop or inspect the child later.", + ]; + if (options.hasAgentMessage) { + lines.push( + "Ask for an explicit reply when needed. A child replies with `await agent_message.send(message, receiver_role='parent')`; parent follow-ups use `receiver_role='child'` plus the child's name or id. Not every message needs a reply.", + ); + } + lines.push("Use `await rlm.list_subagents()` after kernel restart or compaction."); + if (options.hasAgentObserve) { + lines.push("Use `agent_observe` for bounded transcript inspection."); + } + lines.push( + "Have children write files and read those files for fan-in.", + "Delegate parallel context-heavy research or independent implementation; do a single known lookup, edit, or command inline.", + ); + if (options.includeRefineExamples ?? true) { + lines.push("Persist genuinely reusable delegation patterns with `await refine.run()`."); + } + return lines.join("\n"); +} diff --git a/packages/coding-agent/src/session/context/session-stats.ts b/packages/coding-agent/src/session/context/session-stats.ts new file mode 100644 index 0000000000..39c45d1a7f --- /dev/null +++ b/packages/coding-agent/src/session/context/session-stats.ts @@ -0,0 +1,20 @@ +import type { ContextUsage } from "../../core/extensions/index.js"; + +export interface SessionStats { + sessionFile: string | undefined; + sessionId: string; + userMessages: number; + assistantMessages: number; + toolCalls: number; + toolResults: number; + totalMessages: number; + tokens: { + input: number; + output: number; + cacheRead: number; + cacheWrite: number; + total: number; + }; + cost: number; + contextUsage?: ContextUsage; +} diff --git a/packages/coding-agent/src/session/context/system-prompt.ts b/packages/coding-agent/src/session/context/system-prompt.ts new file mode 100644 index 0000000000..e54cf8d9a7 --- /dev/null +++ b/packages/coding-agent/src/session/context/system-prompt.ts @@ -0,0 +1,197 @@ +/** + * System prompt construction and project context loading + */ + +import { formatSkillsForPrompt, getPythonSkillRuntimeInfo, type Skill } from "../../core/skills.js"; +import { REFINE_SKILL_NAME } from "../refinement/types.js"; +import { buildChildAgentDoctrine, buildRlmPrompt, buildSubagentGuidance } from "./prompts/index.js"; + +export interface BuildSystemPromptOptions { + /** Custom system prompt (replaces default). */ + customPrompt?: string; + /** Active tools. Tool schemas carry tool descriptions outside the prompt body. */ + selectedTools?: string[]; + /** Optional one-line tool snippets keyed by tool name. Used only for custom prompts. */ + toolSnippets?: Record; + /** Additional guideline bullets appended to the system prompt. */ + promptGuidelines?: string[]; + /** Text to append to system prompt. */ + appendSystemPrompt?: string; + /** Working directory. */ + cwd: string; + /** Conversation log path. */ + messagesPath?: string; + /** Pre-loaded context files. */ + contextFiles?: Array<{ path: string; content: string }>; + /** Pre-loaded skills. */ + skills?: Skill[]; + /** Whether to include the model-facing rlm recursion guidance. */ + allowRecursion?: boolean; + /** Fixed recursive-agent depth for this session. */ + rlmDepth?: number; + /** Human-readable parent name or id for child communication doctrine. */ + rlmParentAgent?: string; + /** Enabled user-configured servers available through the generic kernel MCP API. */ + genericMcpServers?: string[]; +} + +/** Build the system prompt with tools, guidelines, and context */ +export function buildSystemPrompt(options: BuildSystemPromptOptions): string { + const { + customPrompt, + selectedTools, + promptGuidelines, + appendSystemPrompt, + cwd, + messagesPath, + contextFiles: providedContextFiles, + skills: providedSkills, + allowRecursion, + } = options; + const promptCwd = cwd.replace(/\\/g, "/"); + const promptMessagesPath = (messagesPath ?? "not persisted").replace(/\\/g, "/"); + + const now = new Date(); + const year = now.getFullYear(); + const month = String(now.getMonth() + 1).padStart(2, "0"); + const day = String(now.getDate()).padStart(2, "0"); + const date = `${year}-${month}-${day}`; + + const appendSection = appendSystemPrompt ? `\n\n${appendSystemPrompt}` : ""; + + const contextFiles = providedContextFiles ?? []; + const skills = providedSkills ?? []; + const tools = selectedTools ?? ["ipython"]; + const hasIpython = tools.includes("ipython"); + const visibleSkills = skills.filter((skill) => !skill.disableModelInvocation); + const visiblePythonSkillImportNames = getPythonSkillRuntimeInfo(visibleSkills).map((skill) => skill.importName); + const hasRefineSkill = visibleSkills.some((skill) => skill.name === REFINE_SKILL_NAME); + const genericMcpSection = hasIpython ? formatGenericMcpGuidance(options.genericMcpServers) : ""; + + if (customPrompt) { + let prompt = customPrompt; + + // Append project context files + if (contextFiles.length > 0) { + prompt += "\n\n# Project Context\n\n"; + prompt += "Project-specific instructions and guidelines:\n\n"; + for (const { path: filePath, content } of contextFiles) { + prompt += `## ${filePath}\n\n${content}\n\n`; + } + } + + // Append skills section only when the model has a way to inspect skill files. + const customPromptHasFileAccess = + !selectedTools || selectedTools.includes("ipython") || selectedTools.includes("bash"); + if (customPromptHasFileAccess && skills.length > 0) { + prompt += formatSkillsForPrompt(skills); + } + + // Add date and working directory last + prompt += `\nCurrent date: ${date}`; + prompt += `\nCurrent working directory: ${promptCwd}`; + + const childDoctrine = buildChildAgentDoctrine({ + depth: options.rlmDepth, + parentAgent: options.rlmParentAgent, + installedSkills: visiblePythonSkillImportNames, + activeTools: tools, + }); + if (childDoctrine) { + prompt += `\n\n${childDoctrine}`; + } + + if (genericMcpSection) { + prompt += `\n\n${genericMcpSection}`; + } + + if (appendSection) { + prompt += appendSection; + } + + return prompt; + } + + let prompt = buildRlmPrompt({ + cwd: promptCwd, + messagesPath: promptMessagesPath, + installedSkills: visiblePythonSkillImportNames, + activeTools: tools.filter((name) => name === "ipython" || name === "bash" || name === "edit"), + allowRecursion, + depth: options.rlmDepth, + parentAgent: options.rlmParentAgent, + }); + + // Appended AFTER the trained buildRlmPrompt prefix: delegation doctrine precedes the subagent specs delivered via the harness digest. + if ((allowRecursion ?? true) && hasIpython) { + const visiblePythonSkillNames = new Set( + getPythonSkillRuntimeInfo(visibleSkills).map((skill) => skill.importName), + ); + prompt += `\n\n${buildSubagentGuidance({ + includeRefineExamples: hasRefineSkill, + hasAgentMessage: visiblePythonSkillNames.has("agent_message"), + hasAgentObserve: visiblePythonSkillNames.has("agent_observe"), + })}`; + } + + if (genericMcpSection) { + prompt += `\n\n${genericMcpSection}`; + } + + const guidelines = formatPromptGuidelines(promptGuidelines); + if (guidelines) { + prompt += `\n\n# Additional Guidance\n\n${guidelines}`; + } + + // Append project context files + if (contextFiles.length > 0) { + prompt += "\n\n# Project Context\n\n"; + prompt += "Project-specific instructions and guidelines:\n\n"; + for (const { path: filePath, content } of contextFiles) { + prompt += `## ${filePath}\n\n${content}\n\n`; + } + } + + // Append skills section only when the model has a way to inspect skill files. + const hasFileAccess = tools.includes("ipython") || tools.includes("bash"); + if (hasFileAccess && skills.length > 0) { + prompt += formatSkillsForPrompt(skills); + } + + if (appendSection) { + prompt += appendSection; + } + + return prompt; +} + +function formatGenericMcpGuidance(servers: string[] | undefined): string { + const enabledServers = [...new Set(servers ?? [])].sort((left, right) => left.localeCompare(right)); + if (enabledServers.length === 0) return ""; + + return [ + "# Generic MCP Connections", + "", + "Generic MCP connections are accessed through the pre-imported Python `mcp` object in the Python REPL, not as top-level native tool namespaces or installed Python skills.", + `Enabled generic MCP servers: ${enabledServers.map((server) => `\`${server}\``).join(", ")}.`, + ...enabledServers.map( + (server) => + `For \`${server}\`, first discover its tools with \`await mcp.list_tools("${server}")\`, then call one with \`await mcp.call_tool("${server}", "", arguments)\`.`, + ), + ].join("\n"); +} + +function formatPromptGuidelines(promptGuidelines: string[] | undefined): string { + const guidelinesList: string[] = []; + const guidelinesSet = new Set(); + + for (const guideline of promptGuidelines ?? []) { + const normalized = guideline.trim(); + if (normalized.length > 0 && !guidelinesSet.has(normalized)) { + guidelinesSet.add(normalized); + guidelinesList.push(normalized); + } + } + + return guidelinesList.map((guideline) => `- ${guideline}`).join("\n"); +} diff --git a/packages/coding-agent/src/session/context/token-estimate.ts b/packages/coding-agent/src/session/context/token-estimate.ts new file mode 100644 index 0000000000..2f1e9777b2 --- /dev/null +++ b/packages/coding-agent/src/session/context/token-estimate.ts @@ -0,0 +1,155 @@ +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; +import type { SessionEntry } from "../../core/session-manager.js"; + +/** + * Calculate total context tokens from usage. + * Uses the native totalTokens field when available, falls back to computing from components. + * + * Includes output: the assistant's response becomes part of the prompt on the next + * request, so it counts toward the context the next turn will send. + */ +export function calculateContextTokens(usage: Usage): number { + return usage.totalTokens || usage.input + usage.output + usage.cacheRead + usage.cacheWrite; +} + +/** + * Get usage from an assistant message if available. + * Skips aborted and error messages as they don't have valid usage data. + */ +function getAssistantUsage(msg: AgentMessage): Usage | undefined { + if (msg.role === "assistant" && "usage" in msg) { + const assistantMsg = msg as AssistantMessage; + if (assistantMsg.stopReason !== "aborted" && assistantMsg.stopReason !== "error" && assistantMsg.usage) { + return assistantMsg.usage; + } + } + return undefined; +} + +/** + * Find the last non-aborted assistant message usage from session entries. + */ +export function getLastAssistantUsage(entries: SessionEntry[]): Usage | undefined { + for (let i = entries.length - 1; i >= 0; i--) { + const entry = entries[i]; + if (entry.type === "message") { + const usage = getAssistantUsage(entry.message); + if (usage) return usage; + } + } + return undefined; +} + +export interface ContextUsageEstimate { + tokens: number; + usageTokens: number; + trailingTokens: number; + lastUsageIndex: number | null; +} + +function getLastAssistantUsageInfo(messages: AgentMessage[]): { usage: Usage; index: number } | undefined { + for (let i = messages.length - 1; i >= 0; i--) { + const usage = getAssistantUsage(messages[i]); + if (usage) return { usage, index: i }; + } + return undefined; +} + +/** + * Estimate context tokens from messages, using the last assistant usage when available. + * If there are messages after the last usage, estimate their tokens with estimateTokens. + */ +export function estimateContextTokens(messages: AgentMessage[]): ContextUsageEstimate { + const usageInfo = getLastAssistantUsageInfo(messages); + + if (!usageInfo) { + let estimated = 0; + for (const message of messages) { + estimated += estimateTokens(message); + } + return { + tokens: estimated, + usageTokens: 0, + trailingTokens: estimated, + lastUsageIndex: null, + }; + } + + const usageTokens = calculateContextTokens(usageInfo.usage); + let trailingTokens = 0; + for (let i = usageInfo.index + 1; i < messages.length; i++) { + trailingTokens += estimateTokens(messages[i]); + } + + return { + tokens: usageTokens + trailingTokens, + usageTokens, + trailingTokens, + lastUsageIndex: usageInfo.index, + }; +} + +/** + * Estimate token count for a message using chars/4 heuristic. + * This is conservative (overestimates tokens). + */ +export function estimateTokens(message: AgentMessage): number { + let chars = 0; + + switch (message.role) { + case "user": { + const content = (message as { content: string | Array<{ type: string; text?: string }> }).content; + if (typeof content === "string") { + chars = content.length; + } else if (Array.isArray(content)) { + for (const block of content) { + if (block.type === "text" && block.text) { + chars += block.text.length; + } + } + } + return Math.ceil(chars / 4); + } + case "assistant": { + const assistant = message as AssistantMessage; + for (const block of assistant.content) { + if (block.type === "text") { + chars += block.text.length; + } else if (block.type === "thinking") { + chars += block.thinking.length; + } else if (block.type === "toolCall") { + chars += block.name.length + JSON.stringify(block.arguments).length; + } + } + return Math.ceil(chars / 4); + } + case "custom": + case "toolResult": { + if (typeof message.content === "string") { + chars = message.content.length; + } else { + for (const block of message.content) { + if (block.type === "text" && block.text) { + chars += block.text.length; + } + if (block.type === "image") { + chars += 4800; // Estimate images as 4000 chars, or 1200 tokens + } + } + } + return Math.ceil(chars / 4); + } + case "bashExecution": { + chars = message.command.length + message.output.length; + return Math.ceil(chars / 4); + } + case "branchSummary": + case "compactionSummary": { + chars = message.summary.length; + return Math.ceil(chars / 4); + } + } + + return 0; +} diff --git a/packages/coding-agent/src/session/context/usage.ts b/packages/coding-agent/src/session/context/usage.ts new file mode 100644 index 0000000000..ae45dbc923 --- /dev/null +++ b/packages/coding-agent/src/session/context/usage.ts @@ -0,0 +1,76 @@ +import type { Usage } from "@earendil-works/pi-ai"; + +export interface SessionUsageSummary { + inputTokens: number; + outputTokens: number; + cost: number; +} + +export function sessionUsageSummaryFrom(usage: Usage): SessionUsageSummary | undefined { + const inputTokens = usage.input + usage.cacheRead + usage.cacheWrite; + if (inputTokens === 0 && usage.output === 0 && usage.cost.total === 0) { + return undefined; + } + return { inputTokens, outputTokens: usage.output, cost: usage.cost.total }; +} + +export function emptyUsage(): Usage { + return { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 0, + cost: { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + total: 0, + }, + }; +} + +export function addAssistantUsage(total: Usage, usage: Usage): void { + total.input += usage.input; + total.output += usage.output; + total.cacheRead += usage.cacheRead; + total.cacheWrite += usage.cacheWrite; + total.totalTokens += usage.totalTokens; + total.cost.input += usage.cost.input; + total.cost.output += usage.cost.output; + total.cost.cacheRead += usage.cost.cacheRead; + total.cost.cacheWrite += usage.cost.cacheWrite; + total.cost.total += usage.cost.total; +} + +/** Remove a previously added usage, clamping at zero to absorb attribution drift. */ +export function subtractAssistantUsage(total: Usage, usage: Usage): void { + total.input = Math.max(0, total.input - usage.input); + total.output = Math.max(0, total.output - usage.output); + total.cacheRead = Math.max(0, total.cacheRead - usage.cacheRead); + total.cacheWrite = Math.max(0, total.cacheWrite - usage.cacheWrite); + total.totalTokens = Math.max(0, total.totalTokens - usage.totalTokens); + total.cost.input = Math.max(0, total.cost.input - usage.cost.input); + total.cost.output = Math.max(0, total.cost.output - usage.cost.output); + total.cost.cacheRead = Math.max(0, total.cost.cacheRead - usage.cost.cacheRead); + total.cost.cacheWrite = Math.max(0, total.cost.cacheWrite - usage.cost.cacheWrite); + total.cost.total = Math.max(0, total.cost.total - usage.cost.total); +} + +export function cloneUsage(usage: Usage): Usage { + return { + input: usage.input, + output: usage.output, + cacheRead: usage.cacheRead, + cacheWrite: usage.cacheWrite, + totalTokens: usage.totalTokens, + cost: { + input: usage.cost.input, + output: usage.cost.output, + cacheRead: usage.cost.cacheRead, + cacheWrite: usage.cost.cacheWrite, + total: usage.cost.total, + }, + }; +} diff --git a/packages/coding-agent/src/session/extensions/extensions.ts b/packages/coding-agent/src/session/extensions/extensions.ts index eee3aa3593..421d219f5d 100644 --- a/packages/coding-agent/src/session/extensions/extensions.ts +++ b/packages/coding-agent/src/session/extensions/extensions.ts @@ -2,7 +2,6 @@ import { basename, dirname } from "node:path"; import type { Agent, ThinkingLevel } from "@earendil-works/pi-agent-core"; import { type Api, type Model, resetApiProviders } from "@earendil-works/pi-ai"; import type { AgentSessionMessageController } from "../../core/agent-messages.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; import { type ContextUsage, type ExtensionActions, @@ -21,6 +20,7 @@ import type { PromptTemplate } from "../../core/prompt-templates.js"; import type { ResourceExtensionPaths, ResourceLoader } from "../../core/resource-loader.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { SlashCommandInfo } from "../../core/slash-commands.js"; +import type { CompactionResult } from "../compaction/types.js"; export interface ExtensionBindings { uiContext?: ExtensionUIContext; diff --git a/packages/coding-agent/src/session/goals/continuation.ts b/packages/coding-agent/src/session/goals/continuation.ts index 1e9440d609..2160686bb2 100644 --- a/packages/coding-agent/src/session/goals/continuation.ts +++ b/packages/coding-agent/src/session/goals/continuation.ts @@ -1,6 +1,6 @@ import type { Agent, AgentContext, AgentMessage, GetContinuationMessagesContext } from "@earendil-works/pi-agent-core"; import type { AssistantMessage, ImageContent } from "@earendil-works/pi-ai"; -import type { ActionStore } from "../../core/session-action-store.js"; +import type { ActionStore } from "../input/action-store.js"; import type { SessionInputAdmission } from "../input/input-admission.js"; import type { SessionInputScheduler } from "../input/input-scheduler.js"; import { @@ -8,7 +8,7 @@ import { normalizeMessageContent, primaryDeliveryRecord, type QueuedSessionAction, -} from "../prepared-actions.js"; +} from "../input/prepared-actions.js"; import { parseGoalSlashCommand } from "./commands.js"; import { createGoalContextMessage, diff --git a/packages/coding-agent/src/session/goals/contracts.ts b/packages/coding-agent/src/session/goals/contracts.ts index 1be3f30eda..38b240779e 100644 --- a/packages/coding-agent/src/session/goals/contracts.ts +++ b/packages/coding-agent/src/session/goals/contracts.ts @@ -1,5 +1,5 @@ import type { ImageContent, TextContent } from "@earendil-works/pi-ai"; -import type { CustomMessage } from "../../core/messages.js"; +import type { CustomMessage } from "../context/messages.js"; export const GOAL_STATE_CUSTOM_TYPE = "thread_goal_state"; export const GOAL_CONTEXT_CUSTOM_TYPE = "goal_context"; diff --git a/packages/coding-agent/src/session/input/action-queue.ts b/packages/coding-agent/src/session/input/action-queue.ts index b0d7c41ab7..2b3c748d50 100644 --- a/packages/coding-agent/src/session/input/action-queue.ts +++ b/packages/coding-agent/src/session/input/action-queue.ts @@ -1,13 +1,14 @@ import type { Agent } from "@earendil-works/pi-agent-core"; import type { ImageContent, TextContent } from "@earendil-works/pi-ai"; import { isAgentSessionMessage, isAgentSessionMessagePrompt } from "../../core/agent-messages.js"; +import { parseSessionSlashCommand } from "../../core/slash-commands.js"; import { ASYNC_BASH_COMPLETION_CUSTOM_TYPE, type AsyncBashCompletionDetails, type CustomMessage, HARNESS_DIGEST_CUSTOM_TYPE, isSessionSlashCommandMessage, -} from "../../core/messages.js"; +} from "../context/messages.js"; import { type ActionStore, type DeliveryPolicy, @@ -18,8 +19,8 @@ import { queuedMessageLaneDeliveryPolicy, type SessionAction, type SessionActionSnapshot, -} from "../../core/session-action-store.js"; -import { parseSessionSlashCommand } from "../../core/slash-commands.js"; +} from "./action-store.js"; +import type { SessionInputScheduler } from "./input-scheduler.js"; import { cloneCustomMessage, type createPreparedTurnAction, @@ -31,8 +32,7 @@ import { type RestoredPromptInput, type SessionInputSchedule, visibleSessionActionProjection, -} from "../prepared-actions.js"; -import type { SessionInputScheduler } from "./input-scheduler.js"; +} from "./prepared-actions.js"; export interface SessionActionQueueHost { formatLabel(text: string): string; diff --git a/packages/coding-agent/src/session/input/action-recovery.ts b/packages/coding-agent/src/session/input/action-recovery.ts index 0793420376..49eecf4428 100644 --- a/packages/coding-agent/src/session/input/action-recovery.ts +++ b/packages/coding-agent/src/session/input/action-recovery.ts @@ -1,4 +1,4 @@ -import type { ActionStore } from "../../core/session-action-store.js"; +import type { ActionStore } from "./action-store.js"; import { cloneCustomMessage, cloneQueuedAgentMessage, @@ -7,7 +7,7 @@ import { type QueuedSessionAction, SESSION_ACTION_RECOVERY_FORMAT_VERSION, type SessionActionRecoverySnapshot, -} from "../prepared-actions.js"; +} from "./prepared-actions.js"; export interface SessionActionRecoveryHost { isTerminalNoticeAction(action: QueuedSessionAction): boolean; diff --git a/packages/coding-agent/src/session/input/action-store.ts b/packages/coding-agent/src/session/input/action-store.ts new file mode 100644 index 0000000000..27c1b72b13 --- /dev/null +++ b/packages/coding-agent/src/session/input/action-store.ts @@ -0,0 +1,359 @@ +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { ImageContent, UserMessage } from "@earendil-works/pi-ai"; +import type { InputSource } from "../../core/extensions/index.js"; +import type { SessionSlashCommand } from "../../core/slash-commands.js"; +import type { CustomMessage } from "../context/messages.js"; + +export type DeliveryPolicy = "next_turn_boundary" | "when_run_idle"; +export type WakePolicy = "immediate" | "on_lower_boundary" | "external_resume"; + +export type QueuedMessageLane = "steering" | "followUp"; + +export function queuedMessageLaneDeliveryPolicy(lane: QueuedMessageLane): DeliveryPolicy { + return lane === "steering" ? "next_turn_boundary" : "when_run_idle"; +} + +export type QueuedMessageMutation = + | { type: "delete" } + | { type: "move"; direction: -1 | 1 } + | { type: "replace"; text: string; images?: ImageContent[]; lane: QueuedMessageLane }; +export type QueuedMessageMutationStatus = "applied" | "rejected" | "invalid"; + +export interface SessionActionSnapshot { + queuedCount: number; + steering: readonly string[]; + followUps: readonly string[]; + active?: { + kind: "turn" | "session_command"; + phase: "preparing" | "committing" | "running"; + label?: string; + }; +} + +export interface DeliveryRecord { + id: string; + role: "primary" | "prefix" | "next_turn"; + message: UserMessage | CustomMessage; + started: boolean; + durable: boolean; + ownerActionId: string; +} + +export interface SessionTurnPayload { + kind: "turn"; + records: DeliveryRecord[]; + text: string; + preview?: string; +} + +export interface SessionCommandPayload { + kind: "session_command"; + command: SessionSlashCommand; + text: string; +} + +export type SessionActionPayload = SessionTurnPayload | SessionCommandPayload; + +export type ActionLifecycle = + | { state: "queued" } + | { state: "selected" } + | { state: "preparing"; preparation?: object } + | { state: "committing" } + | { state: "running"; execution: "agent_turn" | "session_command" } + | { state: "completed" } + | { state: "failed"; error: Error } + | { state: "cancelled" }; + +export interface SessionAction { + id: string; + source: InputSource | "internal"; + delivery: DeliveryPolicy; + wake: WakePolicy; + payload: TPayload; + lifecycle: ActionLifecycle; + queueKey?: string; + agentMessageId?: string; + suppressAutonomousContinuation?: boolean; +} + +export interface RollbackProof { + dispatchSettled: true; + transcript: readonly AgentMessage[]; +} + +const TERMINAL_STATES = new Set(["completed", "failed", "cancelled"]); +const ACTIVE_STATES = new Set(["selected", "preparing", "committing", "running"]); +const CLEARABLE_STATES = new Set(["queued", "selected", "preparing"]); + +function isClearable(action: SessionAction): boolean { + return CLEARABLE_STATES.has(action.lifecycle.state); +} + +const LEGAL_TRANSITIONS: Readonly>> = { + queued: new Set(["selected", "failed", "cancelled"]), + selected: new Set(["queued", "preparing", "running", "failed", "cancelled"]), + preparing: new Set(["queued", "committing", "failed", "cancelled"]), + committing: new Set(["queued", "running", "failed", "cancelled"]), + running: new Set(["completed", "failed", "cancelled"]), + completed: new Set(), + failed: new Set(), + cancelled: new Set(), +}; + +function primaryRecords(action: SessionAction): readonly DeliveryRecord[] { + return action.payload.kind === "turn" ? action.payload.records.filter((record) => record.role === "primary") : []; +} + +export function transitionSessionAction( + action: SessionAction, + next: ActionLifecycle, + options: { rollbackProof?: RollbackProof } = {}, +): void { + const previous = action.lifecycle.state; + if (!LEGAL_TRANSITIONS[previous].has(next.state)) { + throw new Error(`Illegal session action lifecycle transition: ${previous} -> ${next.state}`); + } + if (previous === "committing" && next.state === "queued") { + const proof = options.rollbackProof; + if (!proof?.dispatchSettled) { + throw new Error("Committing session action rollback requires a settled dispatch and transcript proof"); + } + const transcript = new Set(proof.transcript); + if (primaryRecords(action).some((record) => transcript.has(record.message))) { + throw new Error("Cannot roll back a session action whose primary message is durable in the transcript"); + } + } + action.lifecycle = next; +} + +export type AdmissionDisposition = "starts_when_admitted" | "queued"; +export type SubmissionOutcome = + | { status: "accepted"; actionId: string; disposition: AdmissionDisposition } + | { status: "coalesced"; existingActionId: string } + | { status: "handled_without_turn" } + | { status: "extension_command"; completion: Promise }; +export type DeliveryOutcome = { status: "delivered" } | { status: "not_applicable" }; + +export interface ActionTicket { + id: string; + accepted: Promise; + delivered: Promise; + completed: Promise; +} + +interface Deferred { + promise: Promise; + settle(value: T): boolean; + reject(error: Error): boolean; +} + +function createDeferred(): Deferred { + let settled = false; + let resolvePromise!: (value: T) => void; + let rejectPromise!: (error: Error) => void; + const promise = new Promise((resolve, reject) => { + resolvePromise = resolve; + rejectPromise = reject; + }); + void promise.catch(() => undefined); + return { + promise, + settle: (value) => { + if (settled) return false; + settled = true; + resolvePromise(value); + return true; + }, + reject: (error) => { + if (settled) return false; + settled = true; + rejectPromise(error); + return true; + }, + }; +} + +export class ActionTicketController { + readonly ticket: ActionTicket; + private readonly accepted = createDeferred(); + private readonly delivered = createDeferred(); + private readonly completed = createDeferred(); + + constructor(id: string) { + this.ticket = { + id, + accepted: this.accepted.promise, + delivered: this.delivered.promise, + completed: this.completed.promise, + }; + } + + settleAccepted(outcome: SubmissionOutcome): boolean { + return this.accepted.settle(outcome); + } + + settleDelivered(outcome: DeliveryOutcome): boolean { + return this.delivered.settle(outcome); + } + + rejectDelivered(error: Error): boolean { + return this.delivered.reject(error); + } + + settleCompleted(error?: Error): boolean { + return error ? this.completed.reject(error) : this.completed.settle(); + } +} + +export class ActionStore { + private readonly nextTurnBoundary: TAction[] = []; + private readonly whenRunIdle: TAction[] = []; + private readonly tickets = new Map(); + + enqueue(action: TAction): void { + this.assertNewAction(action); + this.list(action.delivery).push(action); + this.tickets.set(action.id, new ActionTicketController(action.id)); + } + + enqueueFront(action: TAction): void { + this.assertNewAction(action); + const list = this.list(action.delivery); + const firstQueued = list.findIndex((item) => item.lifecycle.state === "queued"); + list.splice(firstQueued < 0 ? list.length : firstQueued, 0, action); + this.tickets.set(action.id, new ActionTicketController(action.id)); + } + + selectFirst(): TAction | undefined { + const action = + this.nextTurnBoundary.find((item) => item.lifecycle.state === "queued") ?? + this.whenRunIdle.find((item) => item.lifecycle.state === "queued"); + if (action) transitionSessionAction(action, { state: "selected" }); + return action; + } + + remove(predicate: (action: TAction) => boolean, candidates = this.clearableActions()): TAction[] { + const removed = candidates.filter(predicate); + for (const action of removed) transitionSessionAction(action, { state: "cancelled" }); + return removed; + } + + rollback(action: TAction, proof?: RollbackProof): void { + transitionSessionAction(action, { state: "queued" }, { rollbackProof: proof }); + } + + swapQueued(left: TAction, right: TAction): void { + if (left.lifecycle.state !== "queued" || right.lifecycle.state !== "queued" || left.delivery !== right.delivery) { + throw new Error("Only queued actions in the same lane can be swapped"); + } + const list = this.list(left.delivery); + const leftIndex = list.indexOf(left); + const rightIndex = list.indexOf(right); + if (leftIndex < 0 || rightIndex < 0) throw new Error("Queued action is not owned by this store"); + [list[leftIndex], list[rightIndex]] = [right, left]; + } + + moveQueued(action: TAction, delivery: DeliveryPolicy, index: number): void { + if (action.lifecycle.state !== "queued") throw new Error("Only queued actions can be moved"); + const source = this.list(action.delivery); + const sourceIndex = source.indexOf(action); + if (sourceIndex < 0) throw new Error(`Session action ${action.id} is not owned by this store`); + source.splice(sourceIndex, 1); + action.delivery = delivery; + const target = this.list(delivery); + const queued = target.filter((item) => item.lifecycle.state === "queued"); + const before = queued[Math.max(0, Math.min(index, queued.length))]; + target.splice(before ? target.indexOf(before) : target.length, 0, action); + } + + queuedActions(policy?: DeliveryPolicy): readonly TAction[] { + return this.actions(policy).filter((action) => action.lifecycle.state === "queued"); + } + + clearableActions(policy?: DeliveryPolicy): readonly TAction[] { + return this.actions(policy).filter(isClearable); + } + + snapshotActions(): readonly TAction[] { + return this.queuedActions(); + } + + unfinishedActions(policy?: DeliveryPolicy): readonly TAction[] { + return this.actions(policy).filter((action) => !TERMINAL_STATES.has(action.lifecycle.state)); + } + + activeActions(policy?: DeliveryPolicy): readonly TAction[] { + return this.actions(policy).filter((action) => ACTIVE_STATES.has(action.lifecycle.state)); + } + + queuePreview(policy: DeliveryPolicy): readonly string[] { + return this.queuedActions(policy).map((action) => + action.payload.kind === "turn" ? (action.payload.preview ?? action.payload.text) : action.payload.text, + ); + } + + ticketFor(action: TAction): ActionTicketController { + const ticket = this.tickets.get(action.id); + if (!ticket) throw new Error(`Session action ${action.id} is not owned by this store`); + return ticket; + } + + ownedActions(): readonly TAction[] { + return this.actions(); + } + + actionsForMessage(message: UserMessage | CustomMessage): readonly TAction[] { + return this.actions().filter( + (action) => + action.payload.kind === "turn" && action.payload.records.some((record) => record.message === message), + ); + } + + releaseTerminal(action: TAction): void { + if (!TERMINAL_STATES.has(action.lifecycle.state)) { + throw new Error(`Cannot release nonterminal session action ${action.id}`); + } + const list = this.list(action.delivery); + const index = list.indexOf(action); + if (index >= 0) list.splice(index, 1); + this.tickets.delete(action.id); + } + + private actions(policy?: DeliveryPolicy): readonly TAction[] { + if (policy) return this.list(policy); + return [...this.nextTurnBoundary, ...this.whenRunIdle]; + } + + private list(policy: DeliveryPolicy): TAction[] { + return policy === "next_turn_boundary" ? this.nextTurnBoundary : this.whenRunIdle; + } + + private assertNewAction(action: TAction): void { + if (action.lifecycle.state !== "queued") throw new Error("Only queued session actions can be enqueued"); + if (this.tickets.has(action.id)) throw new Error(`Duplicate session action id: ${action.id}`); + } +} + +export interface RuntimeActivity { + lowerAgentRun: boolean; + compaction: boolean; + retry: boolean; + bash: boolean; + refinementApply: boolean; + branchMutation: boolean; + schedulerPauseCount: number; + disposing: boolean; +} + +export function canSelectSessionAction(activity: RuntimeActivity): boolean { + return ( + !activity.lowerAgentRun && + !activity.compaction && + !activity.retry && + !activity.bash && + !activity.refinementApply && + !activity.branchMutation && + activity.schedulerPauseCount === 0 && + !activity.disposing + ); +} diff --git a/packages/coding-agent/src/session/input/bash-host-requests.ts b/packages/coding-agent/src/session/input/bash-host-requests.ts new file mode 100644 index 0000000000..04487dbe34 --- /dev/null +++ b/packages/coding-agent/src/session/input/bash-host-requests.ts @@ -0,0 +1,48 @@ +import type { HostRequestHandler } from "../../core/kernel/index.js"; + +interface AsyncBashCompletionRequest { + pid: number; + command: string; + exitCode: number; +} + +type AsyncBashCompletionHandler = (request: AsyncBashCompletionRequest) => void | Promise; + +interface AsyncBashConsumedRequest { + pid: number; + command: string; +} + +type AsyncBashConsumedHandler = (request: AsyncBashConsumedRequest) => void | Promise; +/** Adapt detached kernel bash completions into a validated host notification. */ +export function createAsyncBashCompletionHostHandler(handler: AsyncBashCompletionHandler): HostRequestHandler { + return async (payload) => { + const { pid, command, exitCode } = payload; + if (typeof pid !== "number" || !Number.isInteger(pid) || pid <= 0) { + throw new Error("bash.completed pid must be a positive integer"); + } + if (typeof command !== "string" || !command) { + throw new Error("bash.completed command must be a non-empty string"); + } + if (typeof exitCode !== "number" || !Number.isInteger(exitCode)) { + throw new Error("bash.completed exitCode must be an integer"); + } + await handler({ pid, command, exitCode }); + return {}; + }; +} + +/** The kernel read a finished command's result, so its completion notice is stale. */ +export function createAsyncBashConsumedHostHandler(handler: AsyncBashConsumedHandler): HostRequestHandler { + return async (payload) => { + const { pid, command } = payload; + if (typeof pid !== "number" || !Number.isInteger(pid) || pid <= 0) { + throw new Error("bash.consumed pid must be a positive integer"); + } + if (typeof command !== "string" || !command) { + throw new Error("bash.consumed command must be a non-empty string"); + } + await handler({ pid, command }); + return {}; + }; +} diff --git a/packages/coding-agent/src/session/input/input-admission.ts b/packages/coding-agent/src/session/input/input-admission.ts index c2f89f936a..d826632c3d 100644 --- a/packages/coding-agent/src/session/input/input-admission.ts +++ b/packages/coding-agent/src/session/input/input-admission.ts @@ -6,8 +6,9 @@ import { isAgentSessionMessage, } from "../../core/agent-messages.js"; import type { InputSource } from "../../core/extensions/index.js"; -import type { CustomMessage } from "../../core/messages.js"; -import type { ActionStore, ActionTicket } from "../../core/session-action-store.js"; +import type { CustomMessage } from "../context/messages.js"; +import type { ActionStore, ActionTicket } from "./action-store.js"; +import type { SessionInputScheduler } from "./input-scheduler.js"; import { createPreparedTurnAction, primaryDeliveryRecord, @@ -15,8 +16,7 @@ import { type QueuedSessionAction, SessionInputAdmissionPausedError, type SessionInputSchedule, -} from "../prepared-actions.js"; -import type { SessionInputScheduler } from "./input-scheduler.js"; +} from "./prepared-actions.js"; export interface SessionInputAdmissionHost { getScheduler(): Pick; diff --git a/packages/coding-agent/src/session/input/input-checkpoints.ts b/packages/coding-agent/src/session/input/input-checkpoints.ts index 66987c89ae..aad4f16221 100644 --- a/packages/coding-agent/src/session/input/input-checkpoints.ts +++ b/packages/coding-agent/src/session/input/input-checkpoints.ts @@ -1,15 +1,16 @@ import type { Agent } from "@earendil-works/pi-agent-core"; -import type { ActionStore } from "../../core/session-action-store.js"; import type { SessionManager } from "../../core/session-manager.js"; import { waitForPromiseOrAbort } from "../../utils/wait-for-abort.js"; -import { primaryDeliveryRecord, type QueuedSessionAction } from "../prepared-actions.js"; import type { ContinuationToken, SessionContinuation } from "../turns/continuation.js"; +import type { ActionStore } from "./action-store.js"; import type { SessionCommitFence, SessionCommitLease } from "./commit-fence.js"; import type { SessionInputScheduler } from "./input-scheduler.js"; +import { primaryDeliveryRecord, type QueuedSessionAction } from "./prepared-actions.js"; export interface SessionInputCheckpointsHost { getFence(): Pick; - getScheduler(): Pick; + getScheduler(): Pick; + isBusyForInputPump(): boolean; getEventQueue(): Promise; acquireFence(signal?: AbortSignal): Promise; getStore(): Pick; @@ -174,7 +175,8 @@ export class SessionInputCheckpoints { async waitForIdleOrSettlement(settlement?: ContinuationToken): Promise { while (settlement === undefined || this.host.getContinuation().current === settlement) { if (this.actions.queuedActions().length > 0) { - if (this.host.getScheduler().suspended || this.host.getScheduler().queuedWorkPauseCount > 0) { + // A blocked pump must yield to the work that clears its busy state. + if (this.host.isBusyForInputPump()) { let wake = () => {}; const changed = new Promise((resolve) => { wake = resolve; diff --git a/packages/coding-agent/src/session/input/input-dispatcher.ts b/packages/coding-agent/src/session/input/input-dispatcher.ts index 965e1e72ab..be15e75e94 100644 --- a/packages/coding-agent/src/session/input/input-dispatcher.ts +++ b/packages/coding-agent/src/session/input/input-dispatcher.ts @@ -1,13 +1,13 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import { turnExecutionPoliciesEqual } from "../turns/turn-preparation.js"; import { type ActionStore, canSelectSessionAction, type DeliveryPolicy, type RuntimeActivity, transitionSessionAction, -} from "../../core/session-action-store.js"; -import { DeferredSessionInputError, primaryDeliveryRecord, type QueuedSessionAction } from "../prepared-actions.js"; -import { turnExecutionPoliciesEqual } from "../turns/turn-preparation.js"; +} from "./action-store.js"; +import { DeferredSessionInputError, primaryDeliveryRecord, type QueuedSessionAction } from "./prepared-actions.js"; export interface SessionInputDispatcherHost { isDisposed(): boolean; diff --git a/packages/coding-agent/src/session/input/message-delivery.ts b/packages/coding-agent/src/session/input/message-delivery.ts index 401f7cf267..ea11698ca2 100644 --- a/packages/coding-agent/src/session/input/message-delivery.ts +++ b/packages/coding-agent/src/session/input/message-delivery.ts @@ -1,10 +1,10 @@ import { randomUUID } from "node:crypto"; import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { AgentSessionEvent, PromptOptions } from "../../core/agent-session.js"; import type { KernelSentAgentMessage } from "../../core/kernel/index.js"; -import type { ActionStore } from "../../core/session-action-store.js"; import type { SessionManager } from "../../core/session-manager.js"; -import type { QueuedSessionAction } from "../prepared-actions.js"; +import type { AgentSessionEvent, PromptOptions } from "../agent-session.js"; +import type { ActionStore } from "./action-store.js"; +import type { QueuedSessionAction } from "./prepared-actions.js"; interface AgentMessageDeferred { promise: Promise; diff --git a/packages/coding-agent/src/session/input/prepared-actions.ts b/packages/coding-agent/src/session/input/prepared-actions.ts new file mode 100644 index 0000000000..008357f65c --- /dev/null +++ b/packages/coding-agent/src/session/input/prepared-actions.ts @@ -0,0 +1,277 @@ +import { randomUUID } from "node:crypto"; +import type { AgentMessage } from "@earendil-works/pi-agent-core"; +import type { ImageContent, TextContent, UserMessage } from "@earendil-works/pi-ai"; +import { AGENT_MESSAGE_RECEIVED_PREVIEW_LABEL, isAgentSessionMessage } from "../../core/agent-messages.js"; +import type { ExtensionRunner, InputSource } from "../../core/extensions/index.js"; +import type { SessionSlashCommand } from "../../core/slash-commands.js"; +import { + ASYNC_BASH_COMPLETION_CUSTOM_TYPE, + ASYNC_BASH_COMPLETION_PREVIEW_LABEL, + type AsyncBashCompletionDetails, + type CustomMessage, +} from "../context/messages.js"; +import { createTurnExecutionPolicy, type TurnExecutionPolicy } from "../turns/turn-preparation.js"; +import type { + DeliveryPolicy, + DeliveryRecord, + SessionAction, + SessionCommandPayload, + SessionTurnPayload, + WakePolicy, +} from "./action-store.js"; + +export type QueuedAgentMessage = UserMessage | CustomMessage; +export type SessionInputSchedule = "steer" | "followUp"; + +export interface PreparedTurnPayload extends SessionTurnPayload { + images?: ImageContent[]; + content?: (TextContent | ImageContent)[]; + customMessage?: CustomMessage; + prepared?: PreparedPromptPreparation; + executionPolicy: TurnExecutionPolicy; + queueVisible: boolean; + acceptedAgentMessage: boolean; + acceptedBeforeCompletion: boolean; + captureRunMessages?: Set; + cancelledDispatchEnded?: boolean; +} + +export interface PreparedCommandPayload extends SessionCommandPayload { + images?: ImageContent[]; +} + +export type QueuedSessionAction = SessionAction; + +export interface PreparedPromptPreparation { + result: Awaited>; + basePromptSnapshot: string; +} + +export class DeferredSessionInputError extends Error {} + +export class SessionInputAdmissionPausedError extends Error {} + +export interface RestoredPromptInput { + text: string; + content?: (TextContent | ImageContent)[]; + images?: ImageContent[]; + queueKey?: string; + agentMessageId?: string; + customMessage?: CustomMessage; + prefixMessages?: CustomMessage[]; +} + +export const SESSION_ACTION_RECOVERY_FORMAT_VERSION = 1; + +export interface SessionActionRecoveryRecord { + id: string; + role: DeliveryRecord["role"]; + message: QueuedAgentMessage; + ownerActionId: string; +} + +export type SessionActionRecoveryPayload = + | { + kind: "turn"; + text: string; + preview?: string; + records: SessionActionRecoveryRecord[]; + images?: ImageContent[]; + content?: (TextContent | ImageContent)[]; + customMessage?: CustomMessage; + executionPolicy: TurnExecutionPolicy; + queueVisible: boolean; + acceptedAgentMessage: boolean; + acceptedBeforeCompletion: boolean; + } + | { + kind: "session_command"; + text: string; + command: SessionSlashCommand; + images?: ImageContent[]; + }; + +export interface SessionActionRecoveryAction { + id: string; + source: InputSource | "internal"; + delivery: DeliveryPolicy; + wake: WakePolicy; + payload: SessionActionRecoveryPayload; + queueKey?: string; + agentMessageId?: string; + suppressAutonomousContinuation?: boolean; +} + +export interface SessionActionRecoverySnapshot { + formatVersion: typeof SESSION_ACTION_RECOVERY_FORMAT_VERSION; + actions: SessionActionRecoveryAction[]; +} + +export function cloneCustomMessage(message: CustomMessage): CustomMessage { + return { + ...message, + content: Array.isArray(message.content) ? message.content.map((block) => ({ ...block })) : message.content, + }; +} + +export function cloneQueuedAgentMessage(message: QueuedAgentMessage): QueuedAgentMessage { + if (message.role === "custom") return cloneCustomMessage(message); + return { + ...message, + content: Array.isArray(message.content) ? message.content.map((block) => ({ ...block })) : message.content, + }; +} + +export function primaryDeliveryRecord(action: QueuedSessionAction): DeliveryRecord { + if (action.payload.kind !== "turn") throw new Error(`Session action ${action.id} is not a turn`); + const record = action.payload.records.find((candidate) => candidate.role === "primary"); + if (!record) throw new Error(`Turn action ${action.id} has no primary delivery record`); + return record; +} + +export function normalizeMessageContent(content: string | (TextContent | ImageContent)[]): { + text: string; + images?: ImageContent[]; +} { + if (typeof content === "string") return { text: content }; + const text = content + .filter((part): part is TextContent => part.type === "text") + .map((part) => part.text) + .join("\n"); + const images = content.filter((part): part is ImageContent => part.type === "image"); + return { text, ...(images.length > 0 ? { images } : {}) }; +} + +export function queuedAgentMessagePreview(action: QueuedSessionAction): string { + const payload = action.payload; + if (payload.kind === "session_command") return payload.text; + if (payload.customMessage && isAgentSessionMessage(payload.customMessage)) { + return `${AGENT_MESSAGE_RECEIVED_PREVIEW_LABEL}: ${payload.customMessage.details.message}`; + } + if (payload.customMessage?.customType === ASYNC_BASH_COMPLETION_CUSTOM_TYPE) { + const details = payload.customMessage.details as AsyncBashCompletionDetails | undefined; + return details + ? `${ASYNC_BASH_COMPLETION_PREVIEW_LABEL}: pid ${details.pid}, exit ${details.exitCode}` + : ASYNC_BASH_COMPLETION_PREVIEW_LABEL; + } + return payload.preview ?? payload.text; +} + +export function visibleSessionActionProjection( + actions: readonly QueuedSessionAction[], +): readonly QueuedSessionAction[] { + return actions.filter( + (action) => + action.payload.kind === "session_command" || + action.payload.queueVisible || + action.payload.acceptedAgentMessage, + ); +} + +export function buildPromptContent(text: string, images?: ImageContent[]): (TextContent | ImageContent)[] { + const content: (TextContent | ImageContent)[] = []; + content.push({ type: "text", text }); + if (images) content.push(...images); + return content; +} + +function deliveryPolicy(schedule: SessionInputSchedule): DeliveryPolicy { + return schedule === "steer" ? "next_turn_boundary" : "when_run_idle"; +} + +export function createDeliveryRecord( + actionId: string, + role: DeliveryRecord["role"], + message: QueuedAgentMessage, +): DeliveryRecord { + return { + id: randomUUID(), + role, + message, + started: false, + durable: false, + ownerActionId: actionId, + }; +} + +export function createPreparedTurnAction( + schedule: SessionInputSchedule, + text: string, + images: ImageContent[] | undefined, + options: { + agentMessageId?: string; + queueKey?: string; + content?: (TextContent | ImageContent)[]; + message?: QueuedAgentMessage; + prefixMessages?: CustomMessage[]; + previewLabel?: string; + suppressAutonomousContinuation?: boolean; + resumeIfIdle?: boolean; + source?: InputSource | "internal"; + executionPolicy?: TurnExecutionPolicy; + queueVisible?: boolean; + acceptedAgentMessage?: boolean; + acceptedBeforeCompletion?: boolean; + }, +): QueuedSessionAction { + const id = randomUUID(); + const content = options.content ?? buildPromptContent(text, images); + const message = + options.message ?? + ({ + role: "user", + content: content.map((block) => ({ ...block })), + timestamp: Date.now(), + } satisfies UserMessage); + const prefixMessages = options.prefixMessages?.map((prefix) => cloneCustomMessage(prefix)) ?? []; + const preview = options.previewLabel ? `${options.previewLabel}: ${text}` : undefined; + const payload: PreparedTurnPayload = { + kind: "turn", + text, + records: [ + ...prefixMessages.map((prefix) => createDeliveryRecord(id, "prefix", prefix)), + createDeliveryRecord(id, "primary", message), + ], + preview, + images: images?.map((image) => ({ ...image })), + content: content.map((block) => ({ ...block })), + customMessage: options.message?.role === "custom" ? cloneCustomMessage(options.message) : undefined, + executionPolicy: options.executionPolicy ?? createTurnExecutionPolicy("queued"), + queueVisible: options.queueVisible ?? true, + acceptedAgentMessage: options.acceptedAgentMessage ?? false, + acceptedBeforeCompletion: options.acceptedBeforeCompletion ?? false, + }; + return { + id, + source: options.source ?? "internal", + delivery: deliveryPolicy(schedule), + wake: + options.resumeIfIdle === true ? "immediate" : schedule === "steer" ? "on_lower_boundary" : "external_resume", + payload, + lifecycle: { state: "queued" }, + queueKey: options.queueKey, + agentMessageId: options.agentMessageId, + suppressAutonomousContinuation: options.suppressAutonomousContinuation, + }; +} + +export function createSessionCommandAction( + text: string, + command: SessionSlashCommand, + images: ImageContent[] | undefined, + schedule: SessionInputSchedule, + options: { + agentMessageId?: string; + source?: InputSource | "internal"; + } = {}, +): QueuedSessionAction { + return { + id: randomUUID(), + source: options.source ?? "internal", + delivery: deliveryPolicy(schedule), + wake: "immediate", + payload: { kind: "session_command", text, command, images }, + lifecycle: { state: "queued" }, + agentMessageId: options.agentMessageId, + }; +} diff --git a/packages/coding-agent/src/session/input/prompt-admission.ts b/packages/coding-agent/src/session/input/prompt-admission.ts new file mode 100644 index 0000000000..7b7908ba78 --- /dev/null +++ b/packages/coding-agent/src/session/input/prompt-admission.ts @@ -0,0 +1,42 @@ +export class PromptAdmissionCancelledError extends Error { + constructor() { + super("Prompt admission was cancelled."); + this.name = "PromptAdmissionCancelledError"; + } +} + +export function throwIfPromptAdmissionCancelled(signal: AbortSignal | undefined): void { + if (signal?.aborted) throw new PromptAdmissionCancelledError(); +} + +/** + * Await `promise` unless `signal` aborts first. Always observes the supplied + * work's rejection so a cancelled admission never leaks an unhandled rejection. + */ +export function waitForPromptAdmission(promise: Promise, signal: AbortSignal | undefined): Promise { + if (!signal) return promise; + if (signal.aborted) { + void promise.catch(() => {}); + return Promise.reject(new PromptAdmissionCancelledError()); + } + return new Promise((resolve, reject) => { + const cleanup = () => signal.removeEventListener("abort", onAbort); + const onAbort = () => { + cleanup(); + reject(new PromptAdmissionCancelledError()); + }; + signal.addEventListener("abort", onAbort, { once: true }); + // Close the listener-registration race before observing the awaited work. + if (signal.aborted) return onAbort(); + promise.then( + (value) => { + cleanup(); + resolve(value); + }, + (error: unknown) => { + cleanup(); + reject(error); + }, + ); + }); +} diff --git a/packages/coding-agent/src/session/input/prompt-submission.ts b/packages/coding-agent/src/session/input/prompt-submission.ts index d78ebe1559..05a3ef744a 100644 --- a/packages/coding-agent/src/session/input/prompt-submission.ts +++ b/packages/coding-agent/src/session/input/prompt-submission.ts @@ -6,6 +6,7 @@ import { parseAgentSessionMessagePromptId, } from "../../core/agent-messages.js"; import type { InputSource } from "../../core/extensions/index.js"; +import type { SessionManager } from "../../core/session-manager.js"; import { ASYNC_BASH_COMPLETION_CUSTOM_TYPE, ASYNC_BASH_COMPLETION_PREVIEW_LABEL, @@ -14,11 +15,14 @@ import { createAsyncBashCompletionMessage, HEARTBEAT_PROMPT_CUSTOM_TYPE, HEARTBEAT_PROMPT_PREVIEW_LABEL, -} from "../../core/messages.js"; -import { throwIfPromptAdmissionCancelled } from "../../core/prompt-admission.js"; -import { type ActionStore, canSelectSessionAction, type RuntimeActivity } from "../../core/session-action-store.js"; -import type { SessionManager } from "../../core/session-manager.js"; +} from "../context/messages.js"; import { GOAL_CONTEXT_CUSTOM_TYPE, GOAL_CONTEXT_PREVIEW_LABEL } from "../goals/contracts.js"; +import type { AgentSessionEvent } from "../turns/events.js"; +import { createTurnExecutionPolicy, type TurnExecutionPolicy } from "../turns/turn-preparation.js"; +import { type ActionStore, canSelectSessionAction, type RuntimeActivity } from "./action-store.js"; +import type { SessionCommitFence, SessionCommitLease } from "./commit-fence.js"; +import type { SessionInputAdmission } from "./input-admission.js"; +import type { SessionInputScheduler } from "./input-scheduler.js"; import { buildPromptContent, cloneCustomMessage, @@ -28,12 +32,8 @@ import { primaryDeliveryRecord, type QueuedSessionAction, SessionInputAdmissionPausedError, -} from "../prepared-actions.js"; -import type { AgentSessionEvent } from "../turns/events.js"; -import { createTurnExecutionPolicy, type TurnExecutionPolicy } from "../turns/turn-preparation.js"; -import type { SessionCommitFence, SessionCommitLease } from "./commit-fence.js"; -import type { SessionInputAdmission } from "./input-admission.js"; -import type { SessionInputScheduler } from "./input-scheduler.js"; +} from "./prepared-actions.js"; +import { throwIfPromptAdmissionCancelled } from "./prompt-admission.js"; import type { SubmissionNormalizer } from "./submission-normalization.js"; export interface PromptOptions { expandPromptTemplates?: boolean; diff --git a/packages/coding-agent/src/session/kernel/kernel-environment.ts b/packages/coding-agent/src/session/kernel/kernel-environment.ts index a9363392a7..bf29f8c2da 100644 --- a/packages/coding-agent/src/session/kernel/kernel-environment.ts +++ b/packages/coding-agent/src/session/kernel/kernel-environment.ts @@ -2,10 +2,10 @@ import { mkdirSync, mkdtempSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import type { AuthStorage } from "../../core/auth-storage.js"; -import { getGlobalHarnessStateDir, getLocalHarnessStateDir } from "../../core/refinement/index.js"; import { resolveConfigValue } from "../../core/resolve-config-value.js"; import type { ResourceLoader } from "../../core/resource-loader.js"; import { SERPER_CREDENTIAL_ID, SERPER_ENV_VAR, WEBSEARCH_SKILL_NAME } from "../../core/websearch-credential.js"; +import { getGlobalHarnessStateDir, getLocalHarnessStateDir } from "../refinement/harness-state.js"; export interface KernelEnvironmentHost { agentDir?: string; diff --git a/packages/coding-agent/src/session/kernel/kernel-host-handlers.ts b/packages/coding-agent/src/session/kernel/kernel-host-handlers.ts index 7803096efe..25e5474575 100644 --- a/packages/coding-agent/src/session/kernel/kernel-host-handlers.ts +++ b/packages/coding-agent/src/session/kernel/kernel-host-handlers.ts @@ -14,22 +14,25 @@ import { } from "../../core/agent-observe.js"; import type { HostRequestHandlers } from "../../core/kernel/index.js"; import type { McpManager } from "../../core/mcp/mcp-manager.js"; -import type { AsyncBashCompletionDetails } from "../../core/messages.js"; +import type { Skill } from "../../core/skills.js"; import { - createAsyncBashCompletionHostHandler, - createAsyncBashConsumedHostHandler, createRlmCreateSessionHostHandler, createRlmDeleteSubagentHostHandler, - createRlmFindModelsHostHandler, createRlmListSubagentsHostHandler, createRlmRunHostHandler, - type RlmCreateSessionResult, - type RlmDeleteSubagentResult, - type RlmFindModelsResult, - type RlmListSubagentsResult, - type RlmSpawnHandle, -} from "../../core/rlm-runtime.js"; -import type { Skill } from "../../core/skills.js"; +} from "../children/host-requests.js"; +import type { + RlmCreateSessionResult, + RlmDeleteSubagentResult, + RlmListSubagentsResult, + RlmSpawnHandle, +} from "../children/runtime-contracts.js"; +import type { AsyncBashCompletionDetails } from "../context/messages.js"; +import { + createAsyncBashCompletionHostHandler, + createAsyncBashConsumedHostHandler, +} from "../input/bash-host-requests.js"; +import { createRlmFindModelsHostHandler, type RlmFindModelsResult } from "../models/model-search.js"; type ObserveResult = AgentObserveListResult | AgentObserveAgentSnapshot | AgentObserveRecentMessagesResult; export interface SessionKernelOperations { diff --git a/packages/coding-agent/src/session/kernel/kernel.ts b/packages/coding-agent/src/session/kernel/kernel.ts index 77ec6db5da..3a10cae35d 100644 --- a/packages/coding-agent/src/session/kernel/kernel.ts +++ b/packages/coding-agent/src/session/kernel/kernel.ts @@ -4,11 +4,11 @@ import type { AssistantMessage } from "@earendil-works/pi-ai"; import type { ToolDefinition } from "../../core/extensions/index.js"; import type { HostRequestHandlers, KernelSentAgentMessage } from "../../core/kernel/index.js"; import { type RestoreResult, snapshotPathIn } from "../../core/kernel/state-snapshot.js"; -import { type CustomMessage, IPYTHON_STATE_RESTORED_CUSTOM_TYPE } from "../../core/messages.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { PythonSkillRuntimeInfo } from "../../core/skills.js"; import { createAllToolDefinitions } from "../../core/tools/index.js"; import { IpythonKernelProvisioner } from "../../core/tools/ipython.js"; +import { type CustomMessage, IPYTHON_STATE_RESTORED_CUSTOM_TYPE } from "../context/messages.js"; const KERNEL_STATE_LISTING_TIMEOUT_MS = 5000; export interface SessionKernelHost { diff --git a/packages/coding-agent/src/session/models/model-search.ts b/packages/coding-agent/src/session/models/model-search.ts new file mode 100644 index 0000000000..14cde057ef --- /dev/null +++ b/packages/coding-agent/src/session/models/model-search.ts @@ -0,0 +1,65 @@ +import type { Api, Model } from "@earendil-works/pi-ai"; +import type { HostRequestHandler } from "../../core/kernel/index.js"; + +export interface RlmModelMatch { + provider: string; + id: string; + name: string; + selector: string; +} + +export interface RlmFindModelsResult { + models: RlmModelMatch[]; +} + +export type RlmFindModelsHandler = (query: string, limit: number) => RlmFindModelsResult | Promise; + +export const DEFAULT_RLM_MODEL_SEARCH_LIMIT = 8; +export const MAX_RLM_MODEL_SEARCH_LIMIT = 20; + +function normalizeModelSearchText(value: string): string { + return value.toLowerCase().replace(/[^a-z0-9]+/g, ""); +} + +export function findRlmModelMatches(query: string, models: Model[], limit: number): RlmModelMatch[] { + const normalizedQuery = normalizeModelSearchText(query.trim()); + return models + .map((model) => { + const selector = `${model.provider}/${model.id}`; + const fields = [selector, model.id, model.name || model.id]; + const normalizedFields = fields.map(normalizeModelSearchText); + let score = normalizedQuery ? Number.POSITIVE_INFINITY : 0; + if (normalizedQuery) { + const exactIndex = normalizedFields.indexOf(normalizedQuery); + const prefixIndex = normalizedFields.findIndex((field) => field.startsWith(normalizedQuery)); + const partialIndex = normalizedFields.findIndex((field) => field.includes(normalizedQuery)); + if (exactIndex >= 0) score = exactIndex; + else if (prefixIndex >= 0) score = 3 + prefixIndex; + else if (partialIndex >= 0) score = 6 + partialIndex; + } + return { model, selector, score }; + }) + .filter((candidate) => Number.isFinite(candidate.score)) + .sort((a, b) => a.score - b.score || a.selector.localeCompare(b.selector)) + .slice(0, limit) + .map(({ model, selector }) => ({ + provider: model.provider, + id: model.id, + name: model.name || model.id, + selector, + })); +} + +/** Search a bounded authenticated model catalog without adding it to the system prompt. */ +export function createRlmFindModelsHostHandler(handler: RlmFindModelsHandler): HostRequestHandler { + return async (payload) => { + if (typeof payload.query !== "string") { + throw new Error("rlm.find_models query must be a string"); + } + const limit = payload.limit === undefined ? DEFAULT_RLM_MODEL_SEARCH_LIMIT : payload.limit; + if (!Number.isInteger(limit) || (limit as number) < 1 || (limit as number) > MAX_RLM_MODEL_SEARCH_LIMIT) { + throw new Error(`rlm.find_models limit must be an integer from 1 to ${MAX_RLM_MODEL_SEARCH_LIMIT}`); + } + return { models: (await handler(payload.query, limit as number)).models }; + }; +} diff --git a/packages/coding-agent/src/session/models/model-selection.ts b/packages/coding-agent/src/session/models/model-selection.ts index 3fed0bdbdc..329a664091 100644 --- a/packages/coding-agent/src/session/models/model-selection.ts +++ b/packages/coding-agent/src/session/models/model-selection.ts @@ -17,10 +17,10 @@ import { import { DEFAULT_THINKING_LEVEL } from "../../core/defaults.js"; import type { ExtensionRunner } from "../../core/extensions/index.js"; import type { ModelRegistry } from "../../core/model-registry.js"; -import { findRlmModelMatches, type RlmFindModelsResult } from "../../core/rlm-runtime.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; import { THINKING_LEVELS } from "../../core/thinking-levels.js"; +import { findRlmModelMatches, type RlmFindModelsResult } from "./model-search.js"; export interface ModelCycleResult { model: Model; diff --git a/packages/coding-agent/src/session/prepared-actions.ts b/packages/coding-agent/src/session/prepared-actions.ts index fb19ed1e13..24e08dc809 100644 --- a/packages/coding-agent/src/session/prepared-actions.ts +++ b/packages/coding-agent/src/session/prepared-actions.ts @@ -1,277 +1,26 @@ -import { randomUUID } from "node:crypto"; -import type { AgentMessage } from "@earendil-works/pi-agent-core"; -import type { ImageContent, TextContent, UserMessage } from "@earendil-works/pi-ai"; -import { AGENT_MESSAGE_RECEIVED_PREVIEW_LABEL, isAgentSessionMessage } from "../core/agent-messages.js"; -import type { ExtensionRunner, InputSource } from "../core/extensions/index.js"; -import { - ASYNC_BASH_COMPLETION_CUSTOM_TYPE, - ASYNC_BASH_COMPLETION_PREVIEW_LABEL, - type AsyncBashCompletionDetails, - type CustomMessage, -} from "../core/messages.js"; -import type { - DeliveryPolicy, - DeliveryRecord, - SessionAction, - SessionCommandPayload, - SessionTurnPayload, - WakePolicy, -} from "../core/session-action-store.js"; -import type { SessionSlashCommand } from "../core/slash-commands.js"; -import { createTurnExecutionPolicy, type TurnExecutionPolicy } from "./turns/turn-preparation.js"; - -export type QueuedAgentMessage = UserMessage | CustomMessage; -export type SessionInputSchedule = "steer" | "followUp"; - -export interface PreparedTurnPayload extends SessionTurnPayload { - images?: ImageContent[]; - content?: (TextContent | ImageContent)[]; - customMessage?: CustomMessage; - prepared?: PreparedPromptPreparation; - executionPolicy: TurnExecutionPolicy; - queueVisible: boolean; - acceptedAgentMessage: boolean; - acceptedBeforeCompletion: boolean; - captureRunMessages?: Set; - cancelledDispatchEnded?: boolean; -} - -export interface PreparedCommandPayload extends SessionCommandPayload { - images?: ImageContent[]; -} - -export type QueuedSessionAction = SessionAction; - -export interface PreparedPromptPreparation { - result: Awaited>; - basePromptSnapshot: string; -} - -export class DeferredSessionInputError extends Error {} - -export class SessionInputAdmissionPausedError extends Error {} - -export interface RestoredPromptInput { - text: string; - content?: (TextContent | ImageContent)[]; - images?: ImageContent[]; - queueKey?: string; - agentMessageId?: string; - customMessage?: CustomMessage; - prefixMessages?: CustomMessage[]; -} - -export const SESSION_ACTION_RECOVERY_FORMAT_VERSION = 1; - -export interface SessionActionRecoveryRecord { - id: string; - role: DeliveryRecord["role"]; - message: QueuedAgentMessage; - ownerActionId: string; -} - -export type SessionActionRecoveryPayload = - | { - kind: "turn"; - text: string; - preview?: string; - records: SessionActionRecoveryRecord[]; - images?: ImageContent[]; - content?: (TextContent | ImageContent)[]; - customMessage?: CustomMessage; - executionPolicy: TurnExecutionPolicy; - queueVisible: boolean; - acceptedAgentMessage: boolean; - acceptedBeforeCompletion: boolean; - } - | { - kind: "session_command"; - text: string; - command: SessionSlashCommand; - images?: ImageContent[]; - }; - -export interface SessionActionRecoveryAction { - id: string; - source: InputSource | "internal"; - delivery: DeliveryPolicy; - wake: WakePolicy; - payload: SessionActionRecoveryPayload; - queueKey?: string; - agentMessageId?: string; - suppressAutonomousContinuation?: boolean; -} - -export interface SessionActionRecoverySnapshot { - formatVersion: typeof SESSION_ACTION_RECOVERY_FORMAT_VERSION; - actions: SessionActionRecoveryAction[]; -} - -export function cloneCustomMessage(message: CustomMessage): CustomMessage { - return { - ...message, - content: Array.isArray(message.content) ? message.content.map((block) => ({ ...block })) : message.content, - }; -} - -export function cloneQueuedAgentMessage(message: QueuedAgentMessage): QueuedAgentMessage { - if (message.role === "custom") return cloneCustomMessage(message); - return { - ...message, - content: Array.isArray(message.content) ? message.content.map((block) => ({ ...block })) : message.content, - }; -} - -export function primaryDeliveryRecord(action: QueuedSessionAction): DeliveryRecord { - if (action.payload.kind !== "turn") throw new Error(`Session action ${action.id} is not a turn`); - const record = action.payload.records.find((candidate) => candidate.role === "primary"); - if (!record) throw new Error(`Turn action ${action.id} has no primary delivery record`); - return record; -} - -export function normalizeMessageContent(content: string | (TextContent | ImageContent)[]): { - text: string; - images?: ImageContent[]; -} { - if (typeof content === "string") return { text: content }; - const text = content - .filter((part): part is TextContent => part.type === "text") - .map((part) => part.text) - .join("\n"); - const images = content.filter((part): part is ImageContent => part.type === "image"); - return { text, ...(images.length > 0 ? { images } : {}) }; -} - -export function queuedAgentMessagePreview(action: QueuedSessionAction): string { - const payload = action.payload; - if (payload.kind === "session_command") return payload.text; - if (payload.customMessage && isAgentSessionMessage(payload.customMessage)) { - return `${AGENT_MESSAGE_RECEIVED_PREVIEW_LABEL}: ${payload.customMessage.details.message}`; - } - if (payload.customMessage?.customType === ASYNC_BASH_COMPLETION_CUSTOM_TYPE) { - const details = payload.customMessage.details as AsyncBashCompletionDetails | undefined; - return details - ? `${ASYNC_BASH_COMPLETION_PREVIEW_LABEL}: pid ${details.pid}, exit ${details.exitCode}` - : ASYNC_BASH_COMPLETION_PREVIEW_LABEL; - } - return payload.preview ?? payload.text; -} - -export function visibleSessionActionProjection( - actions: readonly QueuedSessionAction[], -): readonly QueuedSessionAction[] { - return actions.filter( - (action) => - action.payload.kind === "session_command" || - action.payload.queueVisible || - action.payload.acceptedAgentMessage, - ); -} - -export function buildPromptContent(text: string, images?: ImageContent[]): (TextContent | ImageContent)[] { - const content: (TextContent | ImageContent)[] = []; - content.push({ type: "text", text }); - if (images) content.push(...images); - return content; -} - -function deliveryPolicy(schedule: SessionInputSchedule): DeliveryPolicy { - return schedule === "steer" ? "next_turn_boundary" : "when_run_idle"; -} - -export function createDeliveryRecord( - actionId: string, - role: DeliveryRecord["role"], - message: QueuedAgentMessage, -): DeliveryRecord { - return { - id: randomUUID(), - role, - message, - started: false, - durable: false, - ownerActionId: actionId, - }; -} - -export function createPreparedTurnAction( - schedule: SessionInputSchedule, - text: string, - images: ImageContent[] | undefined, - options: { - agentMessageId?: string; - queueKey?: string; - content?: (TextContent | ImageContent)[]; - message?: QueuedAgentMessage; - prefixMessages?: CustomMessage[]; - previewLabel?: string; - suppressAutonomousContinuation?: boolean; - resumeIfIdle?: boolean; - source?: InputSource | "internal"; - executionPolicy?: TurnExecutionPolicy; - queueVisible?: boolean; - acceptedAgentMessage?: boolean; - acceptedBeforeCompletion?: boolean; - }, -): QueuedSessionAction { - const id = randomUUID(); - const content = options.content ?? buildPromptContent(text, images); - const message = - options.message ?? - ({ - role: "user", - content: content.map((block) => ({ ...block })), - timestamp: Date.now(), - } satisfies UserMessage); - const prefixMessages = options.prefixMessages?.map((prefix) => cloneCustomMessage(prefix)) ?? []; - const preview = options.previewLabel ? `${options.previewLabel}: ${text}` : undefined; - const payload: PreparedTurnPayload = { - kind: "turn", - text, - records: [ - ...prefixMessages.map((prefix) => createDeliveryRecord(id, "prefix", prefix)), - createDeliveryRecord(id, "primary", message), - ], - preview, - images: images?.map((image) => ({ ...image })), - content: content.map((block) => ({ ...block })), - customMessage: options.message?.role === "custom" ? cloneCustomMessage(options.message) : undefined, - executionPolicy: options.executionPolicy ?? createTurnExecutionPolicy("queued"), - queueVisible: options.queueVisible ?? true, - acceptedAgentMessage: options.acceptedAgentMessage ?? false, - acceptedBeforeCompletion: options.acceptedBeforeCompletion ?? false, - }; - return { - id, - source: options.source ?? "internal", - delivery: deliveryPolicy(schedule), - wake: - options.resumeIfIdle === true ? "immediate" : schedule === "steer" ? "on_lower_boundary" : "external_resume", - payload, - lifecycle: { state: "queued" }, - queueKey: options.queueKey, - agentMessageId: options.agentMessageId, - suppressAutonomousContinuation: options.suppressAutonomousContinuation, - }; -} - -export function createSessionCommandAction( - text: string, - command: SessionSlashCommand, - images: ImageContent[] | undefined, - schedule: SessionInputSchedule, - options: { - agentMessageId?: string; - source?: InputSource | "internal"; - } = {}, -): QueuedSessionAction { - return { - id: randomUUID(), - source: options.source ?? "internal", - delivery: deliveryPolicy(schedule), - wake: "immediate", - payload: { kind: "session_command", text, command, images }, - lifecycle: { state: "queued" }, - agentMessageId: options.agentMessageId, - }; -} +export { + buildPromptContent, + cloneCustomMessage, + cloneQueuedAgentMessage, + createDeliveryRecord, + createPreparedTurnAction, + createSessionCommandAction, + DeferredSessionInputError, + normalizeMessageContent, + type PreparedCommandPayload, + type PreparedPromptPreparation, + type PreparedTurnPayload, + primaryDeliveryRecord, + type QueuedAgentMessage, + type QueuedSessionAction, + queuedAgentMessagePreview, + type RestoredPromptInput, + SESSION_ACTION_RECOVERY_FORMAT_VERSION, + type SessionActionRecoveryAction, + type SessionActionRecoveryPayload, + type SessionActionRecoveryRecord, + type SessionActionRecoverySnapshot, + SessionInputAdmissionPausedError, + type SessionInputSchedule, + visibleSessionActionProjection, +} from "./input/prepared-actions.js"; diff --git a/packages/coding-agent/src/session/refinement/auto-refinement.ts b/packages/coding-agent/src/session/refinement/auto-refinement.ts index 4f3c7bdbb7..cd729c66a3 100644 --- a/packages/coding-agent/src/session/refinement/auto-refinement.ts +++ b/packages/coding-agent/src/session/refinement/auto-refinement.ts @@ -1,365 +1,8 @@ -import type { Api, Model } from "@earendil-works/pi-ai"; -import type { AutoRefineReason, AutoRefineReview, RefinementResult } from "../../core/refinement/index.js"; -import type { SettingsManager } from "../../core/settings-manager.js"; -import { RefineSkippedError } from "./refinement-execution.js"; -export interface AutoRefineReviewRequest { - reason: AutoRefineReason; - turnsSinceLastReview: number; -} -export type AutoRefineReviewer = (request: AutoRefineReviewRequest, signal?: AbortSignal) => Promise; - -export function autoRefineInstructions(reason: AutoRefineReason, review: AutoRefineReview): string { - const detail = review.instructions - ? ` -Reviewer instructions: ${review.instructions}` - : ""; - return `Automatic refine review triggered by ${reason}. Only create/update/delete local harness entries if there is clear evidence that should help this session continue. Prefer an empty edits array over speculative or one-off memories. Do not promote anything global unless explicitly requested. Reviewer rationale: ${review.rationale}${detail}`; -} - -export interface AutoRefinementHost { - settingsManager: Pick; - isDisposed(): boolean; - isDisposing(): boolean; - isStreaming(): boolean; - isCompacting(): boolean; - isAllowed(): boolean; - getModel(): Model | undefined; - isContinuationScheduled(): boolean; - cancelContinuation(): void; - refine(options: { instructions?: string }, internal: { trigger: "auto" }): Promise; - runSerialized(options: { instructions?: string }, source: "auto"): Promise; - emitFailure(error: unknown): void; - review(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise; -} - -/** Owns automatic review triggers, cooldowns, cancellation, and scheduled work. */ -export class AutoRefinement { - private _assistantTurnsSinceAutoRefine = 0; - private _lastAutoRefineReviewAt = 0; - private _autoRefineInProgress = false; - private readonly _autoRefineOperations = new Set>(); - private readonly _scheduledAutoRefineTimers = new Set>(); - private _compactAutoRefinePending = false; - private _turnIntervalAutoRefinePending = false; - private _pendingAutoRefineReview: { reason: AutoRefineReason; review: AutoRefineReview } | undefined; - private _autoRefineBranchVersion = 0; - private _autoRefineReviewAbort?: AbortController; - private readonly _autoRefineReviewer?: AutoRefineReviewer; - - constructor( - private readonly _host: AutoRefinementHost, - private readonly _serialized: boolean, - reviewer?: AutoRefineReviewer, - ) { - this._autoRefineReviewer = reviewer; - } - - get branchVersion(): number { - return this._autoRefineBranchVersion; - } - get turnsSinceReview(): number { - return this._assistantTurnsSinceAutoRefine; - } - get lastReviewAt(): number { - return this._lastAutoRefineReviewAt; - } - get hasPendingCompact(): boolean { - return this._compactAutoRefinePending; - } - observeAssistantEnd(): void { - this._assistantTurnsSinceAutoRefine++; - } - resetTurns(): void { - this._assistantTurnsSinceAutoRefine = 0; - } - stampCooldown(): void { - this._lastAutoRefineReviewAt = Date.now(); - } - invalidatePlans(): void { - this._autoRefineBranchVersion++; - } - abortReview(): void { - this._autoRefineReviewAbort?.abort(); - } - discardCompact(): void { - this._compactAutoRefinePending = false; - } - cancelScheduled(): void { - for (const timer of this._scheduledAutoRefineTimers) clearTimeout(timer); - this._scheduledAutoRefineTimers.clear(); - } - pendingOperations(): Promise[] { - return [...this._autoRefineOperations]; - } - - async _runSerializedAutoRefineReview(reason: "compact" | "turn_interval", branchVersion: number): Promise { - const reviewAbort = new AbortController(); - this._autoRefineReviewAbort = reviewAbort; - this._autoRefineInProgress = true; - try { - const review = await this._reviewAutoRefine( - { reason, turnsSinceLastReview: this._assistantTurnsSinceAutoRefine }, - reviewAbort.signal, - ); - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { - return; - } - if (!review.shouldRefine) { - this._lastAutoRefineReviewAt = Date.now(); - this._assistantTurnsSinceAutoRefine = 0; - return; - } - await this._host.runSerialized({ instructions: autoRefineInstructions(reason, review) }, "auto"); - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { - return; - } - this._lastAutoRefineReviewAt = Date.now(); - this._assistantTurnsSinceAutoRefine = 0; - } catch (error) { - if (branchVersion === this._autoRefineBranchVersion) { - this._lastAutoRefineReviewAt = Date.now(); - // An extension skip is an intentional non-round, not a failure. - if (error instanceof RefineSkippedError) { - this._assistantTurnsSinceAutoRefine = 0; - } else { - this._host.emitFailure(error); - } - } - } finally { - if (this._autoRefineReviewAbort === reviewAbort) { - this._autoRefineReviewAbort = undefined; - } - this._autoRefineInProgress = false; - } - } - - _discardPendingAutoRefine(options: { cancelPostCompactionContinue?: boolean } = {}): void { - this._compactAutoRefinePending = false; - this._turnIntervalAutoRefinePending = false; - this._pendingAutoRefineReview = undefined; - if (options.cancelPostCompactionContinue) { - this._host.cancelContinuation(); - } - } - - _scheduleAutoRefineAfterAgentEnd(): void { - if (!this._host.isAllowed()) { - return; - } - if (this._pendingAutoRefineReview) { - this._scheduleAutoRefine(this._pendingAutoRefineReview.reason); - return; - } - if (this._compactAutoRefinePending) { - if (this._host.isContinuationScheduled()) { - return; - } - this._scheduleAutoRefine("compact"); - return; - } - - this._scheduleAutoRefine("turn_interval"); - } - - _scheduleAutoRefineAfterCompaction(willContinueAfterCompaction: boolean): void { - if (!this._host.isAllowed()) { - return; - } - if (this._serialized) { - // Serialized sessions must service compaction-triggered refinement at - // shouldStopAfterTurn (or disposal), never through the interactive path. - this._compactAutoRefinePending = true; - return; - } - if (willContinueAfterCompaction) { - this._compactAutoRefinePending = true; - return; - } - - this._scheduleAutoRefine("compact"); - } - - private _shouldSkipAutoRefineForActiveAgent(): boolean { - return this._host.isStreaming() || this._host.isCompacting(); - } - - private _scheduleDeferredAutoRefineIfIdle(): void { - if (this._autoRefineInProgress || this._shouldSkipAutoRefineForActiveAgent() || this._pendingAutoRefineReview) { - return; - } - if (this._turnIntervalAutoRefinePending) { - this._turnIntervalAutoRefinePending = false; - this._scheduleAutoRefine("turn_interval"); - } - } - - _scheduleAutoRefine(reason: AutoRefineReason, branchVersion = this._autoRefineBranchVersion): void { - const timer = setTimeout(() => { - this._scheduledAutoRefineTimers.delete(timer); - if (branchVersion !== this._autoRefineBranchVersion) { - return; - } - const operation = this._maybeAutoRefine(reason); - this._autoRefineOperations.add(operation); - void operation.finally(() => this._autoRefineOperations.delete(operation)).catch(() => undefined); - }, 0); - this._scheduledAutoRefineTimers.add(timer); - } - - async _maybeAutoRefine(reason: AutoRefineReason): Promise { - if (this._host.isDisposed() || this._host.isDisposing()) { - this._discardPendingAutoRefine(); - return; - } - if (!this._host.isAllowed()) { - this._discardPendingAutoRefine(); - return; - } - - const settings = this._host.settingsManager.getAutoRefineSettings(); - if (!settings.enabled) { - this._discardPendingAutoRefine(); - return; - } - if (this._autoRefineInProgress || this._shouldSkipAutoRefineForActiveAgent()) { - if (reason === "compact") { - this._compactAutoRefinePending = true; - } else { - this._turnIntervalAutoRefinePending = true; - } - return; - } - - const nowMs = Date.now(); - const underCooldown = - this._lastAutoRefineReviewAt > 0 && nowMs - this._lastAutoRefineReviewAt < settings.cooldownMs; - - const pendingReview = this._pendingAutoRefineReview; - if (pendingReview) { - // A failed refine stamps the cooldown; keep the pending review for later. - if (underCooldown) { - return; - } - await this._runApprovedRefine(pendingReview.reason, pendingReview.review); - return; - } - - if (reason === "compact" && !settings.compact) { - this._compactAutoRefinePending = false; - reason = "turn_interval"; - } - if (reason === "turn_interval" && this._assistantTurnsSinceAutoRefine < settings.turnInterval) { - return; - } - if (underCooldown) { - if (reason === "compact") { - this._compactAutoRefinePending = true; - } else { - this._turnIntervalAutoRefinePending = true; - } - return; - } - if (reason === "turn_interval") { - this._turnIntervalAutoRefinePending = false; - } - if (!this._host.getModel()) { - if (reason === "compact") { - this._compactAutoRefinePending = true; - } - return; - } - this._autoRefineInProgress = true; - const turnsSinceLastReview = this._assistantTurnsSinceAutoRefine; - const branchVersion = this._autoRefineBranchVersion; - const reviewAbort = new AbortController(); - this._autoRefineReviewAbort = reviewAbort; - let approvedReview: AutoRefineReview | undefined; - try { - const review = await this._reviewAutoRefine({ reason, turnsSinceLastReview }, reviewAbort.signal); - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { - return; - } - if (!review.shouldRefine) { - const preserveTurnIntervalReview = - reason === "compact" && this._assistantTurnsSinceAutoRefine >= settings.turnInterval; - if (preserveTurnIntervalReview) { - this._turnIntervalAutoRefinePending = true; - } else { - this._lastAutoRefineReviewAt = nowMs; - this._assistantTurnsSinceAutoRefine = 0; - } - if (reason === "compact") { - this._compactAutoRefinePending = false; - } - return; - } - if (this._shouldSkipAutoRefineForActiveAgent()) { - this._pendingAutoRefineReview = { reason, review }; - return; - } - approvedReview = review; - } catch { - // Failed review: stamp the cooldown so a persistent failure (bad auth, - // unparseable output) doesn't retry a full review on every agent end. - if (branchVersion === this._autoRefineBranchVersion) { - this._lastAutoRefineReviewAt = Date.now(); - } - } finally { - if (this._autoRefineReviewAbort === reviewAbort) { - this._autoRefineReviewAbort = undefined; - } - this._autoRefineInProgress = false; - // When a refine follows, _runApprovedRefine schedules the deferred pass. - if (!approvedReview) { - this._scheduleDeferredAutoRefineIfIdle(); - } - } - if (approvedReview) { - await this._runApprovedRefine(reason, approvedReview); - } - } - - private async _runApprovedRefine(reason: AutoRefineReason, review: AutoRefineReview): Promise { - this._autoRefineInProgress = true; - try { - await this._host.refine({ instructions: autoRefineInstructions(reason, review) }, { trigger: "auto" }); - this._pendingAutoRefineReview = undefined; - this._turnIntervalAutoRefinePending = false; - this._lastAutoRefineReviewAt = Date.now(); - this._assistantTurnsSinceAutoRefine = 0; - if (reason === "compact") { - this._compactAutoRefinePending = false; - } - } catch (error) { - // Auto-refine is opportunistic; manual /refine remains available. - // Stamp the cooldown so a persistently failing refine doesn't retry - // (via a retained pending review) on every agent end. - this._lastAutoRefineReviewAt = Date.now(); - if (error instanceof RefineSkippedError) { - // A skipped round is consumed like a reviewer decline, not retained for retry. - this._pendingAutoRefineReview = undefined; - this._turnIntervalAutoRefinePending = false; - this._assistantTurnsSinceAutoRefine = 0; - if (reason === "compact") this._compactAutoRefinePending = false; - } - } finally { - this._autoRefineInProgress = false; - this._scheduleDeferredAutoRefineIfIdle(); - } - } - - _reviewAutoRefine(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise { - if (this._autoRefineReviewer) { - return this._reviewWithCustomReviewer(this._autoRefineReviewer, context, signal); - } - return this._host.review(context, signal); - } - - private async _reviewWithCustomReviewer( - reviewer: AutoRefineReviewer, - context: AutoRefineReviewRequest, - signal?: AbortSignal, - ): Promise { - return reviewer.call(this, context, signal); - } -} +// Compatibility exports; implementation lives with its session owner. +export { + AutoRefinement, + type AutoRefinementHost, + type AutoRefineReviewer, + type AutoRefineReviewRequest, + autoRefineInstructions, +} from "./automatic.js"; diff --git a/packages/coding-agent/src/session/refinement/automatic.ts b/packages/coding-agent/src/session/refinement/automatic.ts new file mode 100644 index 0000000000..ba28d347c3 --- /dev/null +++ b/packages/coding-agent/src/session/refinement/automatic.ts @@ -0,0 +1,365 @@ +import type { Api, Model } from "@earendil-works/pi-ai"; +import type { SettingsManager } from "../../core/settings-manager.js"; +import { RefineSkippedError } from "./execution.js"; +import type { AutoRefineReason, AutoRefineReview, RefinementResult } from "./types.js"; +export interface AutoRefineReviewRequest { + reason: AutoRefineReason; + turnsSinceLastReview: number; +} +export type AutoRefineReviewer = (request: AutoRefineReviewRequest, signal?: AbortSignal) => Promise; + +export function autoRefineInstructions(reason: AutoRefineReason, review: AutoRefineReview): string { + const detail = review.instructions + ? ` +Reviewer instructions: ${review.instructions}` + : ""; + return `Automatic refine review triggered by ${reason}. Only create/update/delete local harness entries if there is clear evidence that should help this session continue. Prefer an empty edits array over speculative or one-off memories. Do not promote anything global unless explicitly requested. Reviewer rationale: ${review.rationale}${detail}`; +} + +export interface AutoRefinementHost { + settingsManager: Pick; + isDisposed(): boolean; + isDisposing(): boolean; + isStreaming(): boolean; + isCompacting(): boolean; + isAllowed(): boolean; + getModel(): Model | undefined; + isContinuationScheduled(): boolean; + cancelContinuation(): void; + refine(options: { instructions?: string }, internal: { trigger: "auto" }): Promise; + runSerialized(options: { instructions?: string }, source: "auto"): Promise; + emitFailure(error: unknown): void; + review(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise; +} + +/** Owns automatic review triggers, cooldowns, cancellation, and scheduled work. */ +export class AutoRefinement { + private _assistantTurnsSinceAutoRefine = 0; + private _lastAutoRefineReviewAt = 0; + private _autoRefineInProgress = false; + private readonly _autoRefineOperations = new Set>(); + private readonly _scheduledAutoRefineTimers = new Set>(); + private _compactAutoRefinePending = false; + private _turnIntervalAutoRefinePending = false; + private _pendingAutoRefineReview: { reason: AutoRefineReason; review: AutoRefineReview } | undefined; + private _autoRefineBranchVersion = 0; + private _autoRefineReviewAbort?: AbortController; + private readonly _autoRefineReviewer?: AutoRefineReviewer; + + constructor( + private readonly _host: AutoRefinementHost, + private readonly _serialized: boolean, + reviewer?: AutoRefineReviewer, + ) { + this._autoRefineReviewer = reviewer; + } + + get branchVersion(): number { + return this._autoRefineBranchVersion; + } + get turnsSinceReview(): number { + return this._assistantTurnsSinceAutoRefine; + } + get lastReviewAt(): number { + return this._lastAutoRefineReviewAt; + } + get hasPendingCompact(): boolean { + return this._compactAutoRefinePending; + } + observeAssistantEnd(): void { + this._assistantTurnsSinceAutoRefine++; + } + resetTurns(): void { + this._assistantTurnsSinceAutoRefine = 0; + } + stampCooldown(): void { + this._lastAutoRefineReviewAt = Date.now(); + } + invalidatePlans(): void { + this._autoRefineBranchVersion++; + } + abortReview(): void { + this._autoRefineReviewAbort?.abort(); + } + discardCompact(): void { + this._compactAutoRefinePending = false; + } + cancelScheduled(): void { + for (const timer of this._scheduledAutoRefineTimers) clearTimeout(timer); + this._scheduledAutoRefineTimers.clear(); + } + pendingOperations(): Promise[] { + return [...this._autoRefineOperations]; + } + + async _runSerializedAutoRefineReview(reason: "compact" | "turn_interval", branchVersion: number): Promise { + const reviewAbort = new AbortController(); + this._autoRefineReviewAbort = reviewAbort; + this._autoRefineInProgress = true; + try { + const review = await this._reviewAutoRefine( + { reason, turnsSinceLastReview: this._assistantTurnsSinceAutoRefine }, + reviewAbort.signal, + ); + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { + return; + } + if (!review.shouldRefine) { + this._lastAutoRefineReviewAt = Date.now(); + this._assistantTurnsSinceAutoRefine = 0; + return; + } + await this._host.runSerialized({ instructions: autoRefineInstructions(reason, review) }, "auto"); + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { + return; + } + this._lastAutoRefineReviewAt = Date.now(); + this._assistantTurnsSinceAutoRefine = 0; + } catch (error) { + if (branchVersion === this._autoRefineBranchVersion) { + this._lastAutoRefineReviewAt = Date.now(); + // An extension skip is an intentional non-round, not a failure. + if (error instanceof RefineSkippedError) { + this._assistantTurnsSinceAutoRefine = 0; + } else { + this._host.emitFailure(error); + } + } + } finally { + if (this._autoRefineReviewAbort === reviewAbort) { + this._autoRefineReviewAbort = undefined; + } + this._autoRefineInProgress = false; + } + } + + _discardPendingAutoRefine(options: { cancelPostCompactionContinue?: boolean } = {}): void { + this._compactAutoRefinePending = false; + this._turnIntervalAutoRefinePending = false; + this._pendingAutoRefineReview = undefined; + if (options.cancelPostCompactionContinue) { + this._host.cancelContinuation(); + } + } + + _scheduleAutoRefineAfterAgentEnd(): void { + if (!this._host.isAllowed()) { + return; + } + if (this._pendingAutoRefineReview) { + this._scheduleAutoRefine(this._pendingAutoRefineReview.reason); + return; + } + if (this._compactAutoRefinePending) { + if (this._host.isContinuationScheduled()) { + return; + } + this._scheduleAutoRefine("compact"); + return; + } + + this._scheduleAutoRefine("turn_interval"); + } + + _scheduleAutoRefineAfterCompaction(willContinueAfterCompaction: boolean): void { + if (!this._host.isAllowed()) { + return; + } + if (this._serialized) { + // Serialized sessions must service compaction-triggered refinement at + // shouldStopAfterTurn (or disposal), never through the interactive path. + this._compactAutoRefinePending = true; + return; + } + if (willContinueAfterCompaction) { + this._compactAutoRefinePending = true; + return; + } + + this._scheduleAutoRefine("compact"); + } + + private _shouldSkipAutoRefineForActiveAgent(): boolean { + return this._host.isStreaming() || this._host.isCompacting(); + } + + private _scheduleDeferredAutoRefineIfIdle(): void { + if (this._autoRefineInProgress || this._shouldSkipAutoRefineForActiveAgent() || this._pendingAutoRefineReview) { + return; + } + if (this._turnIntervalAutoRefinePending) { + this._turnIntervalAutoRefinePending = false; + this._scheduleAutoRefine("turn_interval"); + } + } + + _scheduleAutoRefine(reason: AutoRefineReason, branchVersion = this._autoRefineBranchVersion): void { + const timer = setTimeout(() => { + this._scheduledAutoRefineTimers.delete(timer); + if (branchVersion !== this._autoRefineBranchVersion) { + return; + } + const operation = this._maybeAutoRefine(reason); + this._autoRefineOperations.add(operation); + void operation.finally(() => this._autoRefineOperations.delete(operation)).catch(() => undefined); + }, 0); + this._scheduledAutoRefineTimers.add(timer); + } + + async _maybeAutoRefine(reason: AutoRefineReason): Promise { + if (this._host.isDisposed() || this._host.isDisposing()) { + this._discardPendingAutoRefine(); + return; + } + if (!this._host.isAllowed()) { + this._discardPendingAutoRefine(); + return; + } + + const settings = this._host.settingsManager.getAutoRefineSettings(); + if (!settings.enabled) { + this._discardPendingAutoRefine(); + return; + } + if (this._autoRefineInProgress || this._shouldSkipAutoRefineForActiveAgent()) { + if (reason === "compact") { + this._compactAutoRefinePending = true; + } else { + this._turnIntervalAutoRefinePending = true; + } + return; + } + + const nowMs = Date.now(); + const underCooldown = + this._lastAutoRefineReviewAt > 0 && nowMs - this._lastAutoRefineReviewAt < settings.cooldownMs; + + const pendingReview = this._pendingAutoRefineReview; + if (pendingReview) { + // A failed refine stamps the cooldown; keep the pending review for later. + if (underCooldown) { + return; + } + await this._runApprovedRefine(pendingReview.reason, pendingReview.review); + return; + } + + if (reason === "compact" && !settings.compact) { + this._compactAutoRefinePending = false; + reason = "turn_interval"; + } + if (reason === "turn_interval" && this._assistantTurnsSinceAutoRefine < settings.turnInterval) { + return; + } + if (underCooldown) { + if (reason === "compact") { + this._compactAutoRefinePending = true; + } else { + this._turnIntervalAutoRefinePending = true; + } + return; + } + if (reason === "turn_interval") { + this._turnIntervalAutoRefinePending = false; + } + if (!this._host.getModel()) { + if (reason === "compact") { + this._compactAutoRefinePending = true; + } + return; + } + this._autoRefineInProgress = true; + const turnsSinceLastReview = this._assistantTurnsSinceAutoRefine; + const branchVersion = this._autoRefineBranchVersion; + const reviewAbort = new AbortController(); + this._autoRefineReviewAbort = reviewAbort; + let approvedReview: AutoRefineReview | undefined; + try { + const review = await this._reviewAutoRefine({ reason, turnsSinceLastReview }, reviewAbort.signal); + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._autoRefineBranchVersion) { + return; + } + if (!review.shouldRefine) { + const preserveTurnIntervalReview = + reason === "compact" && this._assistantTurnsSinceAutoRefine >= settings.turnInterval; + if (preserveTurnIntervalReview) { + this._turnIntervalAutoRefinePending = true; + } else { + this._lastAutoRefineReviewAt = nowMs; + this._assistantTurnsSinceAutoRefine = 0; + } + if (reason === "compact") { + this._compactAutoRefinePending = false; + } + return; + } + if (this._shouldSkipAutoRefineForActiveAgent()) { + this._pendingAutoRefineReview = { reason, review }; + return; + } + approvedReview = review; + } catch { + // Failed review: stamp the cooldown so a persistent failure (bad auth, + // unparseable output) doesn't retry a full review on every agent end. + if (branchVersion === this._autoRefineBranchVersion) { + this._lastAutoRefineReviewAt = Date.now(); + } + } finally { + if (this._autoRefineReviewAbort === reviewAbort) { + this._autoRefineReviewAbort = undefined; + } + this._autoRefineInProgress = false; + // When a refine follows, _runApprovedRefine schedules the deferred pass. + if (!approvedReview) { + this._scheduleDeferredAutoRefineIfIdle(); + } + } + if (approvedReview) { + await this._runApprovedRefine(reason, approvedReview); + } + } + + private async _runApprovedRefine(reason: AutoRefineReason, review: AutoRefineReview): Promise { + this._autoRefineInProgress = true; + try { + await this._host.refine({ instructions: autoRefineInstructions(reason, review) }, { trigger: "auto" }); + this._pendingAutoRefineReview = undefined; + this._turnIntervalAutoRefinePending = false; + this._lastAutoRefineReviewAt = Date.now(); + this._assistantTurnsSinceAutoRefine = 0; + if (reason === "compact") { + this._compactAutoRefinePending = false; + } + } catch (error) { + // Auto-refine is opportunistic; manual /refine remains available. + // Stamp the cooldown so a persistently failing refine doesn't retry + // (via a retained pending review) on every agent end. + this._lastAutoRefineReviewAt = Date.now(); + if (error instanceof RefineSkippedError) { + // A skipped round is consumed like a reviewer decline, not retained for retry. + this._pendingAutoRefineReview = undefined; + this._turnIntervalAutoRefinePending = false; + this._assistantTurnsSinceAutoRefine = 0; + if (reason === "compact") this._compactAutoRefinePending = false; + } + } finally { + this._autoRefineInProgress = false; + this._scheduleDeferredAutoRefineIfIdle(); + } + } + + _reviewAutoRefine(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise { + if (this._autoRefineReviewer) { + return this._reviewWithCustomReviewer(this._autoRefineReviewer, context, signal); + } + return this._host.review(context, signal); + } + + private async _reviewWithCustomReviewer( + reviewer: AutoRefineReviewer, + context: AutoRefineReviewRequest, + signal?: AbortSignal, + ): Promise { + return reviewer.call(this, context, signal); + } +} diff --git a/packages/coding-agent/src/session/refinement/controller.ts b/packages/coding-agent/src/session/refinement/controller.ts new file mode 100644 index 0000000000..8130a706e2 --- /dev/null +++ b/packages/coding-agent/src/session/refinement/controller.ts @@ -0,0 +1,928 @@ +import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Api, Model } from "@earendil-works/pi-ai"; +import type { ExtensionRunner } from "../../core/extensions/index.js"; +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; +import type { SessionManager } from "../../core/session-manager.js"; +import type { SettingsManager } from "../../core/settings-manager.js"; +import type { CustomMessage, RefinementSource } from "../context/messages.js"; +import { AutoRefinement, type AutoRefineReviewer, autoRefineInstructions } from "./automatic.js"; +import { RefinementExecution, RefineSkippedError, type SessionRefinementEvent } from "./execution.js"; +import type { HarnessState, RefinementPlan, RefinementResult } from "./types.js"; + +export type { AutoRefineReviewer, AutoRefineReviewRequest } from "./automatic.js"; + +export interface SessionRefinementHost { + sessionManager: Pick< + SessionManager, + "getSessionArtifactDir" | "getEntries" | "appendCustomMessageEntryWithRollback" | "appendCustomEntry" + >; + settingsManager: Pick; + getRetryPolicy(): ProviderRetryPolicy; + getSessionId?(): string; + isDisposed(): boolean; + isDisposing(): boolean; + isStreaming(): boolean; + isCompacting(): boolean; + getDepth(): number; + getRlmSessionDir(): string | undefined; + getModel(): Model | undefined; + getThinkingLevel(): ThinkingLevel; + getMessages(): AgentMessage[]; + getRequiredRequestAuth( + model: Model, + ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; + getExtensionRunner(): Pick; + getEventQueue(): Promise; + getCompactionOperation(): Promise | undefined; + getBranchSummaryOperation(): Promise | undefined; + waitForAgentIdle(): Promise; + dispatchRefine( + options: { instructions?: string; global?: boolean }, + internal: { source: "self" } | { trigger: "auto" }, + ): Promise; + disconnect(): void; + reconnect(): void; + emit(event: SessionRefinementEvent): void; + retainUnpersistedOutcome(message: CustomMessage): void; + notifyCheckpoints(): void; + scheduleInputPump(): void; + isContinuationScheduled(): boolean; + cancelContinuation(): void; +} + +/** Thrown when a session_before_refine extension skips the refinement round. */ +export { RefineSkippedError } from "./execution.js"; + +/** + * Discriminated result from a serialized-mode background planning pass. + * - "plan": review approved and planning succeeded; carry the exact plan, + * options, and abort controller so the boundary can apply directly + * without a second planning request. + * - "skip": reviewer declined; no refine needed. + * - "failure": review or planning threw; boundary should not retry. + */ +export type SerializedBackgroundPlanResult = + | { + status: "plan"; + plan: RefinementPlan; + options: { instructions?: string; rollbackId?: string; global?: boolean }; + abort: AbortController; + branchVersion: number; + source: Exclude; + } + | { status: "skip"; explicit?: boolean } + | { status: "invalidated"; branchVersion: number } + | { + status: "failure"; + explicit: boolean; + options: { instructions?: string; rollbackId?: string; global?: boolean }; + branchVersion: number; + }; + +/** Owns refinement admission, planning/apply barriers, and serialized plan claims. */ +export class SessionRefinement { + private readonly _auto: AutoRefinement; + private readonly _execution: RefinementExecution; + private _refineAbortController?: AbortController; + private readonly _serializedRefine: boolean; + private _refineInFlight?: Promise; + private _refinePlanInFlight?: Promise; + private _serializedPlanInFlight?: Promise; + private _serializedPlanClaim?: Promise; + private _serializedExplicitRefineOptions?: { + instructions?: string; + global?: boolean; + }; + private _pendingRequestedRefine: { instructions?: string; global?: boolean } | undefined; + + constructor( + private readonly _host: SessionRefinementHost, + config: { autoRefineReviewer?: AutoRefineReviewer; serializedRefine?: boolean }, + ) { + this._execution = new RefinementExecution(_host, (abort) => { + if (this._refineAbortController === abort) this._refineAbortController = undefined; + }); + this._auto = new AutoRefinement( + { + settingsManager: _host.settingsManager, + isDisposed: () => _host.isDisposed(), + isDisposing: () => _host.isDisposing(), + isStreaming: () => _host.isStreaming(), + isCompacting: () => _host.isCompacting(), + isAllowed: () => this._autoRefineAllowedForSession(), + getModel: () => _host.getModel(), + isContinuationScheduled: () => _host.isContinuationScheduled(), + cancelContinuation: () => _host.cancelContinuation(), + refine: (options, internal) => _host.dispatchRefine(options, internal), + runSerialized: (options, source) => this._runSerializedRefine(options, source), + emitFailure: (error) => this._emitRefineFailed(error), + review: (context, signal) => this._execution.review(context, signal), + }, + config.serializedRefine ?? false, + config.autoRefineReviewer, + ); + this._serializedRefine = config.serializedRefine ?? false; + } + + requestAbort(): void { + this._pendingRequestedRefine = undefined; + this._auto.invalidatePlans(); + this._auto.abortReview(); + this._refineAbortController?.abort(); + } + observeAssistantEnd(): void { + this._auto.observeAssistantEnd(); + this._maybeStartSerializedBackgroundPlan(); + } + get isApplying(): boolean { + return this._refineInFlight !== undefined; + } + get serialized(): boolean { + return this._serializedRefine; + } + /** Split settlement preserves the caller's existing await boundary. */ + beginAbortedTurnCleanup(): { promise: Promise; finish(): void } | undefined { + this._pendingRequestedRefine = undefined; + const plan = this._serializedPlanInFlight; + if (!plan) return undefined; + this._auto.invalidatePlans(); + this._refineAbortController?.abort(); + return { + promise: plan, + finish: () => { + if (this._serializedPlanInFlight === plan) { + this._serializedPlanInFlight = undefined; + this._serializedExplicitRefineOptions = undefined; + } + }, + }; + } + dispose(): void { + this._auto.abortReview(); + this._refineAbortController?.abort(); + this._auto.cancelScheduled(); + this._serializedPlanInFlight = undefined; + this._serializedExplicitRefineOptions = undefined; + this._pendingRequestedRefine = undefined; + this._auto._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); + this._auto.invalidatePlans(); + } + + async _runSerializedRefineCheckpoint(): Promise { + if (this._host.isDisposed() || this._host.isDisposing()) { + return; + } + + // 1. Await any background plan that was started at message_end + // (either for a pending refine.run or for interval-triggered + // auto-refine). This must be checked BEFORE the pending and + // interval checks because background planning may have consumed + // the pending request at message_end. + const branchVersion = this._auto.branchVersion; + const bgConsumption = await this._consumeSerializedBackgroundPlan(async (bgResult) => { + if (this._host.isDisposed() || this._host.isDisposing()) { + return true; + } + + if (bgResult?.status === "plan") { + if (bgResult.branchVersion !== this._auto.branchVersion) { + if (!this._pendingRequestedRefine) { + this._auto.stampCooldown(); + this._auto.resetTurns(); + return true; + } + } else { + // Apply the EXACT background plan directly via _applyRefine + // (no second _planRefine call). + try { + await this._applySerializedPlan(bgResult); + } catch (error) { + this._emitRefineFailed(error); + } + this._auto.stampCooldown(); + this._auto.resetTurns(); + if (!this._pendingRequestedRefine) { + return true; + } + } + } + + if (bgResult?.status === "skip") { + // Reviewer declined or an extension skipped during background planning. + // Reset exactly once. Never retry the interval review; only fall through for a separate pending refine.run. + if (bgResult.explicit) { + this._emitRefineFailed(new RefineSkippedError("Refinement skipped by extension")); + } + this._auto.stampCooldown(); + this._auto.resetTurns(); + if (!this._pendingRequestedRefine) { + return true; + } + } + + if (bgResult?.status === "failure") { + // Background review or planning failure stamps cooldown without a synchronous retry. + // A separately queued refine.run may still be serviced below. + if (branchVersion === this._auto.branchVersion) { + this._auto.stampCooldown(); + } + // Re-queue an explicit refine.run whose background plan failed, + // but only when branchVersion is still current and no newer + // pending request has arrived since the background plan consumed + // the original one. A newer request retains priority; interval + // failures keep existing no-retry cooldown semantics. + if ( + bgResult.explicit && + bgResult.branchVersion === this._auto.branchVersion && + !this._pendingRequestedRefine + ) { + this._pendingRequestedRefine = bgResult.options; + } + if (!this._pendingRequestedRefine) { + return true; + } + } + + if (bgResult?.status === "invalidated" && !this._pendingRequestedRefine) { + this._auto.stampCooldown(); + this._auto.resetTurns(); + return true; + } + + await this._runSerializedRefineCheckpointAfterBackground(branchVersion); + return true; + }); + if (this._host.isDisposed() || this._host.isDisposing() || bgConsumption !== "none") { + return; + } + await this._runSerializedRefineCheckpointAfterBackground(branchVersion); + } + + private async _runSerializedRefineCheckpointAfterBackground(branchVersion: number): Promise { + // No background result, or a refine.run arrived while the background result was + // in flight. Fall through so an explicit pending request is serviced at this boundary. + + // 2. Agent-callable refine.run requests that were NOT consumed by + // background planning (e.g. interval not reached at message_end, + // or cooldown was active). Service them synchronously. + const pending = this._pendingRequestedRefine; + if (pending) { + this._pendingRequestedRefine = undefined; + try { + await this._runSerializedRefine(pending, "self"); + } catch (error) { + this._emitRefineFailed(error); + } + this._auto.stampCooldown(); + this._auto.resetTurns(); + return; + } + + // 3. Post-compaction auto-refine. Serialized sessions defer the + // compaction trigger to this boundary instead of entering the interactive + // path, which waits for agent idle and can never run inside a tool loop. + if (!this._autoRefineAllowedForSession()) { + this._auto.discardCompact(); + return; + } + const settings = this._host.settingsManager.getAutoRefineSettings(); + if (!settings.enabled) { + this._auto.discardCompact(); + return; + } + if (this._auto.hasPendingCompact) { + if (!settings.compact) { + this._auto.discardCompact(); + } else { + const nowMs = Date.now(); + const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; + if (underCooldown) { + // Preserve the compact trigger for a later boundary, matching the + // interactive path's pending behavior while the cooldown is active. + return; + } + this._auto.discardCompact(); + await this._auto._runSerializedAutoRefineReview("compact", branchVersion); + return; + } + } + + // 4. Interval-triggered auto-refine (no background plan was started). + if (this._auto.turnsSinceReview < settings.turnInterval) { + return; + } + const nowMs = Date.now(); + const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; + if (underCooldown) { + return; + } + await this._auto._runSerializedAutoRefineReview("turn_interval", branchVersion); + } + + private async _consumeSerializedBackgroundPlan( + consume: (result: SerializedBackgroundPlanResult | undefined) => Promise, + ): Promise<"none" | "waited" | "continue" | "stop"> { + if (this._serializedPlanClaim) { + await this._serializedPlanClaim.catch(() => undefined); + return "waited"; + } + const planInFlight = this._serializedPlanInFlight; + if (!planInFlight) { + return "none"; + } + + let releaseClaim: () => void = () => {}; + const claim = new Promise((resolve) => { + releaseClaim = resolve; + }); + this._serializedPlanClaim = claim; + try { + const result = await planInFlight.catch(() => undefined); + if (this._serializedPlanInFlight === planInFlight) { + this._serializedPlanInFlight = undefined; + this._serializedExplicitRefineOptions = undefined; + } + return (await consume(result)) ? "stop" : "continue"; + } finally { + releaseClaim(); + if (this._serializedPlanClaim === claim) { + this._serializedPlanClaim = undefined; + } + } + } + + private async _applySerializedPlan( + bgResult: Extract, + ): Promise { + let resolveApplySettled: () => void = () => {}; + const applySettled = new Promise((resolve) => { + resolveApplySettled = resolve; + }); + this._refineInFlight = applySettled; + try { + await this._execution._applyRefine(bgResult.plan, bgResult.options, bgResult.abort, bgResult.source); + } finally { + resolveApplySettled(); + if (this._refineInFlight === applySettled) { + this._refineInFlight = undefined; + } + this._host.notifyCheckpoints(); + this._host.scheduleInputPump(); + } + } + + private _maybeStartSerializedBackgroundPlan(): void { + if (!this._serializedRefine || this._host.isDisposed() || this._host.isDisposing()) { + return; + } + // Don't start if a plan is already in flight. + if (this._serializedPlanInFlight || this._refineInFlight || this._refinePlanInFlight) { + return; + } + + // Start background planning for a pending agent-callable + // refine.run request, so its plan is ready at the shouldStopAfterTurn + // boundary. The pending request is consumed (cleared) here so the + // boundary doesn't re-plan it. Explicit refine.run skips the review gate. + const pending = this._pendingRequestedRefine; + if (pending) { + this._pendingRequestedRefine = undefined; + this._serializedExplicitRefineOptions = pending; + const refineAbort = new AbortController(); + this._refineAbortController = refineAbort; + const branchVersion = this._auto.branchVersion; + this._serializedPlanInFlight = this._runBackgroundPlan(pending, refineAbort, branchVersion, true); + return; + } + + // Interval-triggered auto-refine background planning. + if (!this._autoRefineAllowedForSession()) { + return; + } + const settings = this._host.settingsManager.getAutoRefineSettings(); + if (!settings.enabled) { + return; + } + if (this._auto.turnsSinceReview < settings.turnInterval) { + return; + } + const nowMs = Date.now(); + const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; + if (underCooldown) { + return; + } + + const refineAbort = new AbortController(); + this._refineAbortController = refineAbort; + const branchVersion = this._auto.branchVersion; + // Pass empty options — _runBackgroundPlan derives instructions from + // the review result for interval-triggered auto-refine. + this._serializedPlanInFlight = this._runBackgroundPlan({}, refineAbort, branchVersion); + } + + private async _runBackgroundPlan( + options: { instructions?: string; rollbackId?: string; global?: boolean }, + refineAbort: AbortController, + branchVersion: number, + skipReview = false, + ): Promise { + try { + let planOptions = options; + if (!skipReview) { + // Interval-triggered: run the review gate first, then derive + // instructions from the review result (not prepopulated). + const review = await this._auto._reviewAutoRefine( + { + reason: "turn_interval", + turnsSinceLastReview: this._auto.turnsSinceReview, + }, + refineAbort.signal, + ); + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { + return { status: "invalidated", branchVersion }; + } + if (!review.shouldRefine) { + return { status: "skip" }; + } + planOptions = { + instructions: autoRefineInstructions("turn_interval", review), + }; + } + // For explicit refine.run (skipReview=true), plan directly with + // the user-provided options — no auto-review gate. + const plan = await this._execution._planRefine( + planOptions, + refineAbort.signal, + skipReview ? "manual" : "auto", + ); + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { + return { status: "invalidated", branchVersion }; + } + return { + status: "plan", + plan, + options: planOptions, + abort: refineAbort, + branchVersion, + source: skipReview ? "self" : "auto", + }; + } catch (error) { + if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { + return { status: "invalidated", branchVersion }; + } + if (error instanceof RefineSkippedError) { + return { status: "skip", explicit: skipReview }; + } + return { + status: "failure", + explicit: skipReview, + options, + branchVersion, + }; + } finally { + if (this._refineAbortController === refineAbort) { + this._refineAbortController = undefined; + } + } + } + + private async _runSerializedRefine( + options: { + instructions?: string; + rollbackId?: string; + global?: boolean; + }, + source: Exclude, + ): Promise { + if (this._host.isDisposed() || this._host.isDisposing()) { + return; + } + // Guard: serialize against concurrent _runSerializedRefine calls. + // _serializedPlanInFlight covers background planning; _refineInFlight + // covers the apply phase. Both must be settled before starting a new + // plan+apply cycle. + while (this._serializedPlanInFlight || this._refineInFlight || this._refinePlanInFlight) { + if (this._serializedPlanInFlight) { + await this._consumeSerializedBackgroundPlan(async () => false); + } else if (this._refineInFlight) { + await this._refineInFlight; + } else { + await this._refinePlanInFlight; + } + } + if (this._host.isDisposed() || this._host.isDisposing()) { + return; + } + + const refineAbort = new AbortController(); + this._refineAbortController = refineAbort; + + const planRun = this._execution._planRefine(options, refineAbort.signal, source === "auto" ? "auto" : "manual"); + const planSettled = planRun.then( + () => undefined, + () => undefined, + ); + this._refinePlanInFlight = planSettled; + let plan: RefinementPlan; + try { + plan = await planRun; + } catch (error) { + if (this._refineAbortController === refineAbort) { + this._refineAbortController = undefined; + } + this._host.scheduleInputPump(); + throw error; + } finally { + if (this._refinePlanInFlight === planSettled) { + this._refinePlanInFlight = undefined; + } + } + + if (this._host.isDisposed() || refineAbort.signal.aborted) { + if (this._refineAbortController === refineAbort) { + this._refineAbortController = undefined; + } + this._host.scheduleInputPump(); + return; + } + + // Do NOT call agent.waitForIdle() — we are at the quiescent boundary + // already (shouldStopAfterTurn). _applyRefine handles disconnect/reconnect internally. + let resolveApplySettled: () => void = () => {}; + const applySettled = new Promise((resolve) => { + resolveApplySettled = resolve; + }); + this._refineInFlight = applySettled; + try { + await this._execution._applyRefine(plan, options, refineAbort, source); + } finally { + resolveApplySettled(); + if (this._refineInFlight === applySettled) { + this._refineInFlight = undefined; + } + this._host.notifyCheckpoints(); + this._host.scheduleInputPump(); + } + } + + handleRefineHostRequest(type: string, payload: Record = {}): Record { + switch (type) { + case "refine.status": { + return { + pending: this._pendingRequestedRefine !== undefined, + in_flight: + this._refineInFlight !== undefined || + this._refinePlanInFlight !== undefined || + this._serializedPlanInFlight !== undefined, + }; + } + case "refine.run": { + const instructions = payload.instructions; + if (instructions !== undefined && typeof instructions !== "string") { + throw new Error("refine.run instructions must be a string when provided"); + } + const globalFlag = payload.global; + if (globalFlag !== undefined && typeof globalFlag !== "boolean") { + throw new Error("refine.run global must be a boolean when provided"); + } + if (!this._host.isStreaming()) { + return { + scheduled: false, + reason: "no active turn; refine can only be requested while a turn is running", + }; + } + const previous = this._pendingRequestedRefine ?? this._serializedExplicitRefineOptions; + this._pendingRequestedRefine = { + instructions: instructions ?? previous?.instructions, + global: globalFlag ?? previous?.global, + }; + // In serialized mode, kick off background planning immediately + // (the primary response ended at message_end, tools are active). + // This lets planning overlap tool execution rather than waiting + // for the shouldStopAfterTurn boundary. + if (this._serializedRefine) { + if (this._serializedPlanInFlight) { + this._auto.invalidatePlans(); + if (this._refineAbortController) { + this._refineAbortController.abort(); + } else { + this._serializedPlanInFlight = Promise.resolve({ + status: "invalidated", + branchVersion: this._auto.branchVersion, + }); + } + } else { + this._maybeStartSerializedBackgroundPlan(); + } + } + return { + scheduled: true, + note: "Refinement runs when the current turn ends; applied edits are appended to your context as a refinement notice and you resume automatically. Continue working normally.", + }; + } + default: + throw new Error(`unknown refine request type "${type}"`); + } + } + + async _drainPendingRefinementForDisposal(): Promise { + this._auto.cancelScheduled(); + await Promise.allSettled(this._auto.pendingOperations()); + this._auto.cancelScheduled(); + // Wait for in-flight refinement (including serialized background plan) to settle. + while (this._refineInFlight || this._refinePlanInFlight || this._serializedPlanInFlight) { + if (this._refineInFlight) { + await this._refineInFlight; + } else if (this._refinePlanInFlight) { + await this._refinePlanInFlight; + } else if (this._serializedPlanInFlight) { + // Await the background plan and apply a ready "plan" result before teardown. + await this._consumeSerializedBackgroundPlan(async (bgResult) => { + if (bgResult?.status === "plan" && bgResult.branchVersion === this._auto.branchVersion) { + try { + await this._applySerializedPlan(bgResult); + } catch (error) { + this._emitRefineFailed(error); + } + // Stamp cooldown and reset counter so the interval + // check below does not trigger a duplicate refine. + this._auto.stampCooldown(); + this._auto.resetTurns(); + } + // Preserve a consumed explicit request when its background plan failed, + // matching the turn-boundary recovery path. The pending drain below + // retries it once before disposal. + if ( + bgResult?.status === "failure" && + bgResult.explicit && + bgResult.branchVersion === this._auto.branchVersion && + !this._pendingRequestedRefine + ) { + this._pendingRequestedRefine = bgResult.options; + } + if (bgResult?.status === "skip" && bgResult.explicit) { + this._emitRefineFailed(new RefineSkippedError("Refinement skipped by extension")); + } + // For "skip" or "failure", stamp cooldown and reset counter + // so the interval check below does not trigger a duplicate + // terminal retry. + if ( + bgResult?.status === "skip" || + bgResult?.status === "failure" || + bgResult?.status === "invalidated" + ) { + this._auto.stampCooldown(); + this._auto.resetTurns(); + } + return false; + }); + } else { + await new Promise((resolve) => setTimeout(resolve, 0)); + } + } + // Drain an agent-callable refine.run request that was scheduled but + // not yet consumed. Use the direct serialized path (no waitForIdle) + // since the agent may still own activeRun at the final agent_end. + if (this._pendingRequestedRefine) { + const pending = this._pendingRequestedRefine; + this._pendingRequestedRefine = undefined; + try { + await this._runSerializedRefine(pending, "self"); + } catch { + // Best-effort drain; refinement errors must not block disposal. + } + // Stamp cooldown and reset counter so the interval check below + // does not trigger a duplicate refine after the explicit drain. + this._auto.stampCooldown(); + this._auto.resetTurns(); + } + // A serialized compaction can finish without another model turn. Drain its + // pending review here so disposal does not silently lose the trigger. + if (this._serializedRefine && this._auto.hasPendingCompact && this._autoRefineAllowedForSession()) { + const compactSettings = this._host.settingsManager.getAutoRefineSettings(); + if (!compactSettings.enabled || !compactSettings.compact) { + this._auto.discardCompact(); + } else { + const nowMs = Date.now(); + const underCooldown = + this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < compactSettings.cooldownMs; + this._auto.discardCompact(); + if (!underCooldown) { + try { + await this._auto._runSerializedAutoRefineReview("compact", this._auto.branchVersion); + } catch { + // Best-effort drain; refinement errors must not block disposal. + } + return; + } + } + } + + // If auto-refine is due but has not started yet, run it now so the + // refinement is persisted before disposal. Use the direct serialized + // path in serialized mode, or _maybeAutoRefine in interactive mode + // (where the agent is idle at this point). + if (this._host.isDisposed() || !this._autoRefineAllowedForSession()) { + return; + } + const settings = this._host.settingsManager.getAutoRefineSettings(); + if (!settings.enabled) { + return; + } + if (this._auto.turnsSinceReview < settings.turnInterval) { + return; + } + const nowMs = Date.now(); + const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; + if (underCooldown) { + return; + } + if (this._serializedRefine) { + await this._runSerializedRefineCheckpoint(); + } else { + await this._auto._maybeAutoRefine("turn_interval"); + } + } + + _autoRefineAllowedForSession(): boolean { + return this._host.getDepth() === 0 && this._execution._localHarnessStateDir() !== undefined; + } + + async _invalidatePendingAutoRefineForBranchChange(): Promise { + this._auto.abortReview(); + this._auto._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); + this._auto.resetTurns(); + // Increment branch version BEFORE aborting/awaiting the serialized plan. + // This invalidates the plan's branchVersion check at the boundary + // so even if the plan completes, the boundary will reject it + // (bgResult.branchVersion !== this._auto.branchVersion). + this._auto.invalidatePlans(); + // Abort the in-flight refine/bplan controller so any pending + // _planRefine or _reviewAutoRefine call settles via signal abort + // rather than hanging forever. + this._refineAbortController?.abort(); + if (this._serializedPlanInFlight) { + await this._consumeSerializedBackgroundPlan(async () => false); + } + while (this._refinePlanInFlight) { + await this._refinePlanInFlight; + } + await this._waitForRefineIdle(); + } + + _emitRefineFailed(error: unknown): void { + this._host.emit({ + type: "refine_failed", + error: error instanceof Error ? error.message : String(error), + }); + } + + _consumePendingRequestedRefine(): boolean { + const pending = this._pendingRequestedRefine; + if (!pending) return false; + this._pendingRequestedRefine = undefined; + void this._host.dispatchRefine(pending, { source: "self" }).catch((error) => this._emitRefineFailed(error)); + return true; + } + + async refine( + options: { + instructions?: string; + rollbackId?: string; + global?: boolean; + } = {}, + internal: { skipAbort?: boolean; trigger?: "manual" | "auto"; source?: RefinementSource } = {}, + ): Promise { + // Queued /refine executes from the session-input pump between turns; + // refine never aborts the agent (planning is backgrounded and the apply + // phase waits for quiescence), so skipAbort only asserts the pump's + // idle invariant instead of changing abort behavior. + if (internal.skipAbort && this._host.isStreaming()) { + throw new Error("Cannot refine without aborting while the agent is running."); + } + // Wait for any existing refine (both planning and application) before + // starting a new run. This serializes concurrent /refine calls so two + // planning phases cannot race into concurrent _applyRefine calls that + // overwrite harness state. + while (this._refineInFlight || this._refinePlanInFlight || this._serializedPlanInFlight) { + if (this._refineInFlight) { + await this._refineInFlight; + } else if (this._refinePlanInFlight) { + await this._refinePlanInFlight; + } else { + // A serialized background plan is in flight (started during an + // active turn at message_end). Wait for planning and for the active + // turn to settle so its normal checkpoint can consume the plan. + const serializedPlanInFlight = this._serializedPlanInFlight; + await serializedPlanInFlight; + if (this._refineInFlight || this._refinePlanInFlight) { + continue; + } + await this._host.waitForAgentIdle(); + // Aborted turns skip shouldStopAfterTurn. Drop their settled plan + // after idle so a later public refine cannot spin on it forever. + if (this._serializedPlanInFlight === serializedPlanInFlight) { + this._serializedPlanInFlight = undefined; + this._serializedExplicitRefineOptions = undefined; + } + } + } + + const refineAbort = new AbortController(); + this._refineAbortController = refineAbort; + + const planRun = this._execution._planRefine(options, refineAbort.signal, internal.trigger ?? "manual"); + const planSettled = planRun.then( + () => undefined, + () => undefined, + ); + this._refinePlanInFlight = planSettled; + let plan: RefinementPlan; + try { + plan = await planRun; + } catch (e) { + if (this._refineAbortController === refineAbort) { + this._refineAbortController = undefined; + } + this._host.scheduleInputPump(); + throw e; + } finally { + if (this._refinePlanInFlight === planSettled) { + this._refinePlanInFlight = undefined; + } + } + + // Block new turns before waiting for the current turn to finish. One shared + // settled promise covers the full transition and apply critical section. + let resolveApplySettled: () => void = () => {}; + const applySettled = new Promise((resolve) => { + resolveApplySettled = resolve; + }); + this._refineInFlight = applySettled; + try { + // Wait for the session to become quiescent before applying. Planning is + // allowed to overlap active user work, but application must not disconnect + // event handling until that work and its queued events have completed. + await this._host.waitForAgentIdle(); + while (true) { + const eventQueue = this._host.getEventQueue(); + const compactionOp = this._host.getCompactionOperation(); + const branchSummaryOp = this._host.getBranchSummaryOperation(); + await Promise.allSettled([ + eventQueue, + ...(compactionOp ? [compactionOp] : []), + ...(branchSummaryOp ? [branchSummaryOp] : []), + ]); + if ( + eventQueue === this._host.getEventQueue() && + compactionOp === this._host.getCompactionOperation() && + branchSummaryOp === this._host.getBranchSummaryOperation() + ) { + break; + } + } + if (this._host.isDisposed() || refineAbort.signal.aborted) { + throw new Error("Refinement cancelled because the session was disposed."); + } + return await this._execution._applyRefine( + plan, + options, + refineAbort, + internal.source ?? (internal.trigger === "auto" ? "auto" : "user"), + ); + } finally { + resolveApplySettled(); + if (this._refineInFlight === applySettled) { + this._refineInFlight = undefined; + } + this._host.notifyCheckpoints(); + this._host.scheduleInputPump(); + } + } + + async _waitForRefineIdle(): Promise { + while (this._refineInFlight) { + await this._refineInFlight; + } + } + + _discardPendingAutoRefine(options: { cancelPostCompactionContinue?: boolean } = {}): void { + this._auto._discardPendingAutoRefine(options); + } + + _scheduleAutoRefineAfterAgentEnd(): void { + this._auto._scheduleAutoRefineAfterAgentEnd(); + } + + _scheduleAutoRefineAfterCompaction(willContinueAfterCompaction: boolean): void { + this._auto._scheduleAutoRefineAfterCompaction(willContinueAfterCompaction); + } + + _localHarnessStateDir(): string | undefined { + return this._execution._localHarnessStateDir(); + } + + _loadMergedHarnessState(): HarnessState { + return this._execution._loadMergedHarnessState(); + } +} diff --git a/packages/coding-agent/src/session/refinement/execution.ts b/packages/coding-agent/src/session/refinement/execution.ts new file mode 100644 index 0000000000..18fd7340bc --- /dev/null +++ b/packages/coding-agent/src/session/refinement/execution.ts @@ -0,0 +1,337 @@ +import { existsSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Api, Model } from "@earendil-works/pi-ai"; +import { formatNoModelSelectedMessage } from "../../core/auth-guidance.js"; +import type { ExtensionRunner, SessionBeforeRefineResult } from "../../core/extensions/index.js"; +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; +import type { SessionManager } from "../../core/session-manager.js"; +import { serializeConversation } from "../context/conversation-text.js"; +import { + type CustomMessage, + convertToLlm, + createRefinementNoticeMessage, + createRefinementOutcomeMessage, + type RefinementSource, +} from "../context/messages.js"; +import type { AutoRefineReviewRequest } from "./automatic.js"; +import { + appendGlobalRefinement, + applyRefinementProposal, + generateRefinementId, + getGlobalHarnessStateDir, + getLocalHarnessStateDir, + getRefinementHistory, + inferRefinementResultScope, + loadGlobalRefinementHistory, + loadHarnessState, + mergeHarnessStates, + mergeRefinementHistory, + normalizeRefinementProposal, + saveHarnessState, +} from "./harness-state.js"; +import { planRefinement, reviewAutoRefine } from "./planning.js"; +import type { AutoRefineReview, HarnessState, RefinementPlan, RefinementResult } from "./types.js"; +export type SessionRefinementEvent = + | { type: "refine_complete"; result: RefinementResult } + | { type: "refine_failed"; error: string } + | { type: "message_start" | "message_end"; message: CustomMessage }; +export interface RefinementExecutionHost { + sessionManager: Pick< + SessionManager, + "getSessionArtifactDir" | "getEntries" | "appendCustomMessageEntryWithRollback" | "appendCustomEntry" + >; + isDisposed(): boolean; + getRlmSessionDir(): string | undefined; + getModel(): Model | undefined; + getThinkingLevel(): ThinkingLevel; + getMessages(): AgentMessage[]; + getRequiredRequestAuth( + model: Model, + ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; + getRetryPolicy(): ProviderRetryPolicy; + getSessionId?(): string; + getExtensionRunner(): Pick; + disconnect(): void; + reconnect(): void; + emit(event: SessionRefinementEvent): void; + retainUnpersistedOutcome(message: CustomMessage): void; +} + +/** Thrown when a session_before_refine extension skips the refinement round. */ +export class RefineSkippedError extends Error {} + +/** Plans against current session dependencies and persists results during the apply barrier. */ +export class RefinementExecution { + constructor( + private readonly _host: RefinementExecutionHost, + private readonly _releaseAbort: (abort: AbortController) => void, + ) {} + _localHarnessStateDir(): string | undefined { + return ( + getLocalHarnessStateDir(this._host.sessionManager.getSessionArtifactDir()) ?? + (this._host.getRlmSessionDir() ? getLocalHarnessStateDir(this._host.getRlmSessionDir()) : undefined) + ); + } + + _loadMergedHarnessState(): HarnessState { + const localHarnessStateDir = this._localHarnessStateDir(); + return mergeHarnessStates( + loadHarnessState(getGlobalHarnessStateDir(), "global"), + localHarnessStateDir ? loadHarnessState(localHarnessStateDir, "local") : undefined, + ); + } + + private _loadRefinementHistory(): RefinementResult[] { + return mergeRefinementHistory( + loadGlobalRefinementHistory(getGlobalHarnessStateDir()), + getRefinementHistory(this._host.sessionManager.getEntries().filter((entry) => entry.type === "custom")), + ); + } + + async _planRefine( + options: { instructions?: string; rollbackId?: string; global?: boolean }, + signal: AbortSignal, + trigger: "manual" | "auto" = "manual", + ): Promise { + if (this._host.isDisposed()) { + throw new Error("Cannot refine a disposed session."); + } + + if (!this._host.getModel()) { + throw new Error(formatNoModelSelectedMessage()); + } + + const model = this._host.getModel()!; + const { apiKey, headers, requestModel } = await this._host.getRequiredRequestAuth(model); + const globalHarnessStateDir = getGlobalHarnessStateDir(); + const localHarnessStateDir = this._localHarnessStateDir(); + const requestedScope = options.global ? "global" : "local"; + if (!options.rollbackId && requestedScope === "local" && !localHarnessStateDir) { + throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); + } + const globalPlanningState = loadHarnessState(globalHarnessStateDir, "global"); + const localPlanningState = localHarnessStateDir ? loadHarnessState(localHarnessStateDir, "local") : undefined; + const planningState = + requestedScope === "global" + ? globalPlanningState + : mergeHarnessStates(globalPlanningState, localPlanningState); + const history = this._loadRefinementHistory(); + const rollbackTarget = options.rollbackId ? history.find((item) => item.id === options.rollbackId) : undefined; + let baselineScope = rollbackTarget + ? (inferRefinementResultScope(rollbackTarget) ?? requestedScope) + : requestedScope; + let baselineHarnessStateDir = baselineScope === "global" ? globalHarnessStateDir : localHarnessStateDir; + if (rollbackTarget?.harnessStatePath) { + baselineHarnessStateDir = dirname(rollbackTarget.harnessStatePath); + baselineScope = resolve(baselineHarnessStateDir) === resolve(globalHarnessStateDir) ? "global" : "local"; + } + if (!baselineHarnessStateDir) { + throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); + } + const baselineState = rollbackTarget + ? loadHarnessState(baselineHarnessStateDir, baselineScope) + : baselineScope === "global" + ? globalPlanningState + : localPlanningState!; + if (!options.rollbackId && this._host.getExtensionRunner().hasHandlers("session_before_refine")) { + const result = (await this._host.getExtensionRunner().emit({ + type: "session_before_refine", + preparation: { + trigger, + instructions: options.instructions, + scope: requestedScope, + planningState, + history, + conversationText: serializeConversation(convertToLlm(this._host.getMessages())).slice(-80_000), + }, + signal, + })) as SessionBeforeRefineResult | undefined; + if (this._host.isDisposed() || signal.aborted) { + throw new Error("Refinement cancelled because the session was disposed."); + } + if (result?.skip) { + throw new RefineSkippedError("Refinement skipped by extension"); + } + if (result?.proposal !== undefined) { + return { + proposal: normalizeRefinementProposal(result.proposal), + id: generateRefinementId(), + baselineState, + }; + } + } + const plan = await planRefinement( + this._host.getMessages(), + planningState, + history, + requestModel ?? model, + apiKey, + { ...options, retry: this._host.getRetryPolicy() }, + headers, + signal, + this._host.getThinkingLevel(), + this._host.getSessionId?.(), + ); + if (this._host.isDisposed() || signal.aborted) { + throw new Error("Refinement cancelled because the session was disposed."); + } + return { ...plan, baselineState }; + } + + async _applyRefine( + plan: RefinementPlan, + options: { instructions?: string; rollbackId?: string; global?: boolean }, + refineAbort: AbortController, + source: RefinementSource, + ): Promise { + if (this._host.isDisposed()) { + throw new Error("Cannot refine a disposed session."); + } + // The caller has already set _refineInFlight and waited for agent idle. + // Disconnect only for the brief apply + save + reconnect critical section. + this._host.disconnect(); + + try { + const globalHarnessStateDir = getGlobalHarnessStateDir(); + const localHarnessStateDir = this._localHarnessStateDir(); + const requestedScope = options.global ? "global" : "local"; + const history = this._loadRefinementHistory(); + const rollbackTarget = options.rollbackId ? history.find((item) => item.id === options.rollbackId) : undefined; + let targetScope = plan.rollbackScope ?? requestedScope; + let targetHarnessStateDir = targetScope === "global" ? globalHarnessStateDir : localHarnessStateDir; + if (targetScope === "local" && rollbackTarget?.harnessStatePath) { + if (!existsSync(rollbackTarget.harnessStatePath)) { + throw new Error( + `Local refinement ${rollbackTarget.id} state file not found: ${rollbackTarget.harnessStatePath}`, + ); + } + targetHarnessStateDir = dirname(rollbackTarget.harnessStatePath); + // Legacy records predate scope fields and default to "local" but may point + // at the global store; honor the recorded path so its entries stay global. + if (resolve(targetHarnessStateDir) === resolve(globalHarnessStateDir)) { + targetScope = "global"; + } + } + if (!targetHarnessStateDir) { + throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); + } + // Re-read the target state immediately before applying so concurrent kernel + // (`rlm.harness`) writes during the LLM pass are not clobbered. + const state = loadHarnessState(targetHarnessStateDir, targetScope); + const proposal = { + ...plan.proposal, + edits: plan.proposal.edits.map((edit) => { + const localPrefix = "local:"; + const globalPrefix = "global:"; + return { + ...edit, + id: edit.id?.startsWith(localPrefix) + ? edit.id.slice(localPrefix.length) + : edit.id?.startsWith(globalPrefix) + ? edit.id.slice(globalPrefix.length) + : edit.id, + }; + }), + }; + if (this._host.isDisposed() || refineAbort.signal.aborted) { + throw new Error("Refinement cancelled because the session was disposed."); + } + const result = applyRefinementProposal(state, proposal, { + id: plan.id, + rollbackOf: plan.rollbackOf, + scope: targetScope, + baselineState: plan.baselineState, + }); + result.harnessStatePath = saveHarnessState(targetHarnessStateDir, state); + if (targetScope === "global") { + appendGlobalRefinement(globalHarnessStateDir, result); + } + let refinementAuditAppendError: { error: unknown } | undefined; + try { + this._host.sessionManager.appendCustomEntry("prime-agent.refinement", result); + } catch (error) { + refinementAuditAppendError = { error }; + } + try { + this._recordRefinementOutcome(result); + } catch (error) { + if (!refinementAuditAppendError) throw error; + } + if (refinementAuditAppendError) throw refinementAuditAppendError.error; + // The prompt stays byte-identical so the provider prefix cache survives; the notice carries the change. + this._recordRefinementNotice(result, source); + try { + this._host.emit({ type: "refine_complete", result }); + } catch { + // Listener failures must not flip a successful refinement into + // a reported failure — the refinement is already persisted. + } + try { + await this._host.getExtensionRunner().emit({ + type: "refine_complete", + id: result.id, + summary: result.summary, + appliedEdits: result.appliedEdits.filter((edit) => edit.applied).length, + scope: result.scope ?? "local", + }); + } catch { + // Extension emit failures must not flip a successful refinement + // into a reported failure — the refinement is already persisted. + } + return result; + } finally { + this._releaseAbort(refineAbort); + if (!this._host.isDisposed()) { + this._host.reconnect(); + } + } + } + + private _recordRefinementOutcome(result: RefinementResult): void { + this._appendDurableRefineMessage(createRefinementOutcomeMessage(result)); + } + + private _recordRefinementNotice(result: RefinementResult, source: RefinementSource): void { + if (!result.appliedEdits.some((edit) => edit.applied)) return; + this._appendDurableRefineMessage(createRefinementNoticeMessage(result, source)); + } + + private _appendDurableRefineMessage(message: CustomMessage): void { + try { + this._host.sessionManager.appendCustomMessageEntryWithRollback( + message.customType, + message.content, + message.display, + message.details, + ); + } catch { + // Not in the session file, so context rebuilds would drop the outcome. + this._host.retainUnpersistedOutcome(message); + } + this._host.getMessages().push(message); + this._host.emit({ type: "message_start", message }); + this._host.emit({ type: "message_end", message }); + } + + async review(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise { + const model = this._host.getModel(); + if (!model) { + return { shouldRefine: false, rationale: "No model selected." }; + } + const { apiKey, headers, requestModel } = await this._host.getRequiredRequestAuth(model); + return reviewAutoRefine( + this._host.getMessages(), + this._loadMergedHarnessState(), + this._loadRefinementHistory(), + requestModel ?? model, + apiKey, + context, + headers, + signal, + this._host.getThinkingLevel(), + this._host.getRetryPolicy(), + this._host.getSessionId?.(), + ); + } +} diff --git a/packages/coding-agent/src/session/refinement/format.ts b/packages/coding-agent/src/session/refinement/format.ts new file mode 100644 index 0000000000..77ce076a13 --- /dev/null +++ b/packages/coding-agent/src/session/refinement/format.ts @@ -0,0 +1,167 @@ +import type { HarnessState, RefinementKind, RefinementResult } from "./types.js"; + +const DEFAULT_OVERVIEW_ENTRY_LIMIT = 6; + +const DEFAULT_OVERVIEW_REFINEMENT_LIMIT = 5; + +const DEFAULT_OVERVIEW_CONTENT_LIMIT = 180; + +function compactText(text: string, maxLength: number): string { + const normalized = text.replace(/\s+/g, " ").trim(); + if (normalized.length <= maxLength) { + return normalized; + } + return `${normalized.slice(0, Math.max(0, maxLength - 3))}...`; +} + +/** Notice body in digest notation: trigger line plus applied edits as `action kind [scope:id] title: content`; rollbacks print via their rollback summaries. */ +export function formatRefinementNoticeBody(result: RefinementResult): string { + const lines = [compactText(result.summary, DEFAULT_OVERVIEW_CONTENT_LIMIT)]; + for (const edit of result.appliedEdits) { + if (!edit.applied) continue; + const entry = edit.after ?? edit.before; + const scope = entry?.scope ?? result.scope ?? "local"; + lines.push( + `- ${edit.action} ${edit.kind} [${scope}:${edit.id}] ${entry?.title ?? edit.id}: ${compactText( + entry?.content ?? "", + DEFAULT_OVERVIEW_CONTENT_LIMIT, + )}`, + ); + } + return lines.join("\n"); +} + +export function formatHarnessStateForPrompt( + state: HarnessState, + options: { + maxEntriesPerKind?: number; + maxRefinements?: number; + maxContentLength?: number; + includeIpythonExamples?: boolean; + includeShellExamples?: boolean; + includeRefineExamples?: boolean; + } = {}, +): string { + const maxEntriesPerKind = options.maxEntriesPerKind ?? DEFAULT_OVERVIEW_ENTRY_LIMIT; + const maxRefinements = options.maxRefinements ?? DEFAULT_OVERVIEW_REFINEMENT_LIMIT; + const maxContentLength = options.maxContentLength ?? DEFAULT_OVERVIEW_CONTENT_LIMIT; + const includeIpythonExamples = options.includeIpythonExamples ?? true; + const includeRefineExamples = options.includeRefineExamples ?? includeIpythonExamples; + const lines = [ + "# Continual Harness State", + "", + "Local continual harness entries belong to this Prime Agent session. Global continual harness entries persist across Prime Agent sessions.", + "The continual harness entries below are compact summaries, not full descriptions. Use them as routing/context hints; inspect or refine the underlying continual harness entry only when detail matters.", + "Default to local continual harness refinement for current task progress, temporary blockers, and session coordination. Use global continual harness refinement only for stable cross-session lessons, durable user preferences, reusable skills/subagents, or explicitly project-qualified facts.", + "Use these continual harness prompt notes, memories, skills, and subagent specs when they are relevant. The base system prompt is immutable; prompt entries below are supplemental notes only.", + "", + includeRefineExamples + ? "When to call `await refine.run()`: after a repeated failure, a reusable tactic emerges, a repeated delegation role should become a subagent spec, a repeated procedure should become a skill, a durable fact/preference should become a memory, a narrow behavioral policy should become a prompt addendum, a user corrects behavior that should persist locally or globally, validation shows a continual harness entry is wrong, or a skill/subagent/memory/prompt note should be created, updated, deleted, or rolled back. Keep `await refine.run()` continual harness edits small and evidence-backed." + : "When to refine the continual harness: after a repeated failure, a reusable tactic emerges, a repeated delegation role should become a subagent spec, a repeated procedure should become a skill, a durable fact/preference should become a memory, a narrow behavioral policy should become a prompt addendum, a user corrects behavior that should persist locally or globally, validation shows a continual harness entry is wrong, or a skill/subagent/memory/prompt note should be created, updated, deleted, or rolled back. Keep continual harness edits small and evidence-backed.", + "", + includeIpythonExamples + ? "Call contract: read each installed Python skill's SKILL.md and call its documented module function in the Python REPL; do not assume a `.run` entrypoint. Use ` ...` in shell when a CLI exists. Continual harness skill entries are Python REPL skills with an explicit Python `reference` and `arguments` contract. Spawn a continual harness subagent spec by composing a concise task prompt and calling `handle = await rlm.spawn('sub-task', name='worker')`; admission returns immediately with `rlm_child_id`, `name`, `session_dir`, and `model`, never the child's answer. Results arrive only through explicit `agent_message` replies or files; children reply with `await agent_message.send(message, receiver_role='parent')`. Use `await rlm.list_subagents()` to recover direct child handles and `await agent_message.send(..., receiver_role='child', receiver_name=handle.name)` for follow-ups. Do not invent wrappers such as `call_skill(...)`, `run_subagent(...)`, or named subagent registries." + : options.includeShellExamples + ? "Call contract: use installed skills as shell commands when available (for example ` ...`). Continual harness entries are routing/context hints only in sessions without the Python REPL; do not use Python `await`, `asyncio`, or `rlm` examples unless the prompt also documents a Python kernel." + : "Call contract: continual harness entries are routing/context hints only in sessions without the Python REPL or shell access; do not use Python `await`, `asyncio`, `rlm`, or shell skill commands unless the prompt also documents those interfaces.", + "", + ]; + + let totalEntries = 0; + for (const kind of Object.keys(state.entries) as RefinementKind[]) { + const entries = Object.values(state.entries[kind]).sort((a, b) => + [a.path, a.title, a.id].join("\0").localeCompare([b.path, b.title, b.id].join("\0")), + ); + totalEntries += entries.length; + // Render subagent specs as a task-shaped roster the model can match against — the + // analogue of Claude Code's agent-type menu — rather than a bare count. In + // REPL sessions, include the native `rlm` invocation hint. + if (kind === "subagent" && entries.length > 0 && includeIpythonExamples) { + lines.push( + `${kind}: ${entries.length} (invoke a spec by turning it into a concise task prompt and spawning with \`await rlm.spawn('', name='')\`; admission returns a child handle, never the answer)`, + ); + } else { + lines.push(`${kind}: ${entries.length}`); + } + for (const entry of entries.slice(0, maxEntriesPerKind)) { + const argumentsText = + entry.kind === "skill" && Object.keys(entry.arguments).length > 0 + ? ` args=${compactText(JSON.stringify(entry.arguments), maxContentLength)}` + : ""; + const referenceText = + entry.kind === "skill" && Object.keys(entry.reference).length > 0 + ? ` ref=${compactText(JSON.stringify(entry.reference), maxContentLength)}` + : ""; + lines.push( + `- [${entry.scope ?? "global"}:${entry.id}] ${entry.title} (${entry.path}, v${entry.version})${referenceText}${argumentsText}: ${compactText( + entry.content, + maxContentLength, + )}`, + ); + } + const overflow = entries.length - Math.min(entries.length, maxEntriesPerKind); + if (overflow > 0) { + lines.push(`- +${overflow} more ${kind} entries`); + } + lines.push(""); + } + + if (totalEntries === 0) { + lines.push("No saved harness entries yet.", ""); + } + + lines.push(`recent refinements: ${state.refinements.length}`); + for (const event of state.refinements.slice(-maxRefinements)) { + const changes = event.changes.length > 0 ? event.changes.join(", ") : "no applied edits"; + const outcome = event.outcome ? `; outcome: ${compactText(event.outcome, maxContentLength)}` : ""; + lines.push(`- [${event.id}] ${compactText(event.trigger, maxContentLength)}: ${changes}${outcome}`); + } + const refinementOverflow = state.refinements.length - Math.min(state.refinements.length, maxRefinements); + if (refinementOverflow > 0) { + lines.push(`- +${refinementOverflow} older refinement events`); + } + + return lines.join("\n").trim(); +} + +export function overviewForPrompt(state: HarnessState): string { + const lines: string[] = []; + for (const kind of Object.keys(state.entries) as RefinementKind[]) { + const entries = Object.values(state.entries[kind]); + lines.push(`${kind}: ${entries.length}`); + for (const entry of entries.slice(0, 40)) { + const content = entry.content.replace(/\s+/g, " ").slice(0, 240); + const argumentsText = + entry.kind === "skill" && Object.keys(entry.arguments).length > 0 + ? ` args=${JSON.stringify(entry.arguments).slice(0, 240)}` + : ""; + const referenceText = + entry.kind === "skill" && Object.keys(entry.reference).length > 0 + ? ` ref=${JSON.stringify(entry.reference).slice(0, 240)}` + : ""; + lines.push( + `- [${entry.scope ?? "global"}:${entry.id}] ${entry.title} (${entry.path}, v${entry.version})${referenceText}${argumentsText}: ${content}`, + ); + } + if (entries.length > 40) { + lines.push(`- +${entries.length - 40} more ${kind} entries`); + } + } + return lines.join("\n"); +} + +export function historyForPrompt(history: RefinementResult[]): string { + if (history.length === 0) { + return "No prior refinement history."; + } + return history + .slice(-20) + .map((item) => { + const edits = item.appliedEdits + .map((edit) => `${edit.applied ? "applied" : "failed"} ${edit.action} ${edit.kind}:${edit.id}`) + .join(", "); + const rollback = item.rollbackOf ? ` rollbackOf=${item.rollbackOf}` : ""; + return `[${item.id}]${rollback} ${item.summary}\n${edits}\nExpected outcome: ${item.expectedOutcome}`; + }) + .join("\n\n"); +} diff --git a/packages/coding-agent/src/session/refinement/harness-state.ts b/packages/coding-agent/src/session/refinement/harness-state.ts new file mode 100644 index 0000000000..30ae7b10bb --- /dev/null +++ b/packages/coding-agent/src/session/refinement/harness-state.ts @@ -0,0 +1,458 @@ +import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync } from "node:fs"; +import { join } from "node:path"; +import { getAgentDir } from "../../config.js"; +import type { CustomEntry } from "../../core/session-manager.js"; +import { realpathIfPresentSync, writeFileAtomicSync } from "../../utils/atomic-file.js"; +import type { + AppliedRefinementEdit, + HarnessEntry, + HarnessScope, + HarnessState, + RefinementAction, + RefinementEdit, + RefinementKind, + RefinementProposal, + RefinementResult, +} from "./types.js"; +import { REFINEMENT_CUSTOM_TYPE } from "./types.js"; + +const HARNESS_STATE_DIR_NAME = "harness"; + +const REFINEMENT_HISTORY_FILE_NAME = "refinements.jsonl"; + +export function now(): string { + return new Date().toISOString(); +} + +function emptyHarnessState(): HarnessState { + return { + schema: 1, + entries: { + prompt: {}, + memory: {}, + skill: {}, + subagent: {}, + }, + refinements: [], + }; +} + +function slug(raw: string, fallback: string): string { + const normalized = raw + .trim() + .toLowerCase() + .replace(/[^a-z0-9]+/g, "_") + .replace(/^_+|_+$/g, "") + .slice(0, 80); + return normalized || fallback; +} + +function cloneEntry(entry: HarnessEntry | undefined): HarnessEntry | undefined { + return entry ? JSON.parse(JSON.stringify(entry)) : undefined; +} + +function objectRecord(value: unknown): Record | undefined { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return undefined; + } + return value as Record; +} + +function normalizeHarnessScope(value: unknown, fallback: HarnessScope): HarnessScope { + return value === "global" || value === "local" ? value : fallback; +} + +export function inferRefinementResultScope(result: RefinementResult): HarnessScope | undefined { + if (result.scope) { + return result.scope; + } + + const scopes = new Set(); + for (const edit of result.appliedEdits) { + const scope = edit.after?.scope ?? edit.before?.scope; + if (scope) { + scopes.add(scope); + } + } + return scopes.size === 1 ? [...scopes][0] : undefined; +} + +function withDefaultRefinementScope(result: RefinementResult, scope: HarnessScope): RefinementResult { + const inferred = inferRefinementResultScope(result); + return { ...result, scope: inferred ?? scope }; +} + +export function getGlobalHarnessStateDir(agentDir: string = getAgentDir()): string { + return join(agentDir, HARNESS_STATE_DIR_NAME); +} + +export function getLocalHarnessStateDir(sessionArtifactDir: string | undefined): string | undefined { + return sessionArtifactDir ? join(sessionArtifactDir, HARNESS_STATE_DIR_NAME) : undefined; +} + +export function getHarnessStatePath(harnessStateDir: string = getGlobalHarnessStateDir()): string { + return join(harnessStateDir, "harness_state.json"); +} + +export function loadHarnessState( + harnessStateDir: string = getGlobalHarnessStateDir(), + scope: HarnessScope = "global", +): HarnessState { + const statePath = getHarnessStatePath(harnessStateDir); + if (!existsSync(statePath)) { + return emptyHarnessState(); + } + let parsed: Partial; + try { + const raw = JSON.parse(readFileSync(statePath, "utf8")); + // loadHarnessState runs on every system-prompt build and before each /refine, so + // a corrupt or unreadable (or non-object) state file must degrade to empty rather + // than throw and break the session. The next saveHarnessState rewrites it cleanly. + if (typeof raw !== "object" || raw === null || Array.isArray(raw)) { + return emptyHarnessState(); + } + parsed = raw as Partial; + } catch { + return emptyHarnessState(); + } + const state = emptyHarnessState(); + state.schema = typeof parsed.schema === "number" ? parsed.schema : 1; + for (const kind of Object.keys(state.entries) as RefinementKind[]) { + const records = parsed.entries?.[kind]; + if (records && typeof records === "object") { + for (const [id, rawEntry] of Object.entries(records)) { + const entry = objectRecord(rawEntry); + if (!entry) continue; + state.entries[kind][id] = { + ...(entry as unknown as HarnessEntry), + scope: normalizeHarnessScope(entry.scope, scope), + reference: objectRecord(entry.reference) ?? {}, + arguments: objectRecord(entry.arguments) ?? {}, + metadata: objectRecord(entry.metadata) ?? {}, + }; + } + } + } + if (Array.isArray(parsed.refinements)) { + state.refinements = parsed.refinements; + } + return state; +} + +export function mergeHarnessStates(globalState: HarnessState, localState?: HarnessState): HarnessState { + const merged = emptyHarnessState(); + merged.schema = Math.max(globalState.schema, localState?.schema ?? 1); + for (const kind of Object.keys(merged.entries) as RefinementKind[]) { + for (const [id, entry] of Object.entries(globalState.entries[kind])) { + const cloned = cloneEntry(entry)!; + merged.entries[kind][id] = { ...cloned, scope: normalizeHarnessScope(cloned.scope, "global") }; + } + for (const [id, entry] of Object.entries(localState?.entries[kind] ?? {})) { + const cloned = cloneEntry(entry)!; + const scopedEntry = { ...cloned, scope: normalizeHarnessScope(cloned.scope, "local") }; + const mergedId = merged.entries[kind][id] ? `${scopedEntry.scope}:${id}` : id; + merged.entries[kind][mergedId] = scopedEntry; + } + } + merged.refinements = [...globalState.refinements, ...(localState?.refinements ?? [])]; + return merged; +} + +export function saveHarnessState(harnessStateDir: string, state: HarnessState): string { + const statePath = getHarnessStatePath(harnessStateDir); + mkdirSync(harnessStateDir, { recursive: true }); + const targetPath = realpathIfPresentSync(statePath); + const mode = existsSync(targetPath) ? statSync(targetPath).mode & 0o777 : 0o600; + writeFileAtomicSync(targetPath, `${JSON.stringify(state, null, 2)}\n`, { mode }); + return statePath; +} + +export function getRefinementHistoryPath(harnessStateDir: string = getGlobalHarnessStateDir()): string { + return join(harnessStateDir, REFINEMENT_HISTORY_FILE_NAME); +} + +function isRefinementResult(data: unknown): data is RefinementResult { + return typeof data === "object" && data !== null && "id" in data && "appliedEdits" in data; +} + +/** + * Append a global-scope refinement to the cross-session history log so it can be + * rolled back from any session. Local-scope refinements are recorded only in the + * session JSONL and roll back via their recorded harnessStatePath. + */ +export function appendGlobalRefinement(harnessStateDir: string, result: RefinementResult): string { + const historyPath = getRefinementHistoryPath(harnessStateDir); + mkdirSync(harnessStateDir, { recursive: true }); + appendFileSync(historyPath, `${JSON.stringify(result)}\n`, "utf8"); + return historyPath; +} + +export function loadGlobalRefinementHistory(harnessStateDir: string = getGlobalHarnessStateDir()): RefinementResult[] { + const historyPath = getRefinementHistoryPath(harnessStateDir); + if (!existsSync(historyPath)) { + return []; + } + const results: RefinementResult[] = []; + for (const line of readFileSync(historyPath, "utf8").split("\n")) { + const trimmed = line.trim(); + if (!trimmed) continue; + try { + const parsed = JSON.parse(trimmed); + if (isRefinementResult(parsed)) { + results.push(withDefaultRefinementScope(parsed, "global")); + } + } catch { + // Skip malformed lines so a single bad append cannot break rollback. + } + } + return results; +} + +/** + * Merge global and session refinement history, de-duplicating by id. Session entries + * win on conflict so a session that is mid-flight still resolves its own latest result. + */ +export function mergeRefinementHistory( + global: readonly RefinementResult[], + session: readonly RefinementResult[], +): RefinementResult[] { + const byId = new Map(); + for (const result of global) { + byId.set(result.id, result); + } + for (const result of session) { + const existing = byId.get(result.id); + byId.set(result.id, result.scope || !existing?.scope ? result : { ...result, scope: existing.scope }); + } + return [...byId.values()]; +} + +/** + * Normalizes an untrusted refinement proposal while preserving invalid edit + * fields for apply-time validation. + */ +export function normalizeRefinementProposal(value: unknown): RefinementProposal { + const record = + typeof value === "object" && value !== null && !Array.isArray(value) ? (value as Record) : {}; + const edits = Array.isArray(record.edits) ? record.edits : []; + return { + summary: typeof record.summary === "string" ? record.summary : "Refined continual harness state", + rationale: typeof record.rationale === "string" ? record.rationale : "", + expectedOutcome: typeof record.expectedOutcome === "string" ? record.expectedOutcome : "", + edits: edits + .filter((edit): edit is Record => typeof edit === "object" && edit !== null) + .map((edit) => ({ + action: edit.action as RefinementAction, + kind: edit.kind as RefinementKind, + id: typeof edit.id === "string" ? edit.id : undefined, + title: typeof edit.title === "string" ? edit.title : undefined, + content: typeof edit.content === "string" ? edit.content : undefined, + path: typeof edit.path === "string" ? edit.path : undefined, + reference: objectRecord(edit.reference), + arguments: objectRecord(edit.arguments), + metadata: + typeof edit.metadata === "object" && edit.metadata !== null && !Array.isArray(edit.metadata) + ? (edit.metadata as Record) + : undefined, + reason: typeof edit.reason === "string" ? edit.reason : undefined, + })), + }; +} + +function validateEdit(edit: RefinementEdit, computedId?: string): string | undefined { + if (!["create", "update", "delete"].includes(edit.action)) { + return `unsupported action ${String(edit.action)}`; + } + if (!["prompt", "memory", "skill", "subagent"].includes(edit.kind)) { + return `unsupported kind ${String(edit.kind)}`; + } + if (edit.kind === "prompt" && (edit.id === "base_system_prompt" || computedId === "base_system_prompt")) { + return "base system prompt is not editable"; + } + if (edit.action !== "create" && !edit.id) { + return `${edit.action} requires id`; + } + if (edit.action !== "delete" && (!edit.title || !edit.content)) { + return `${edit.action} requires title and content`; + } + if (edit.action !== "delete" && edit.kind === "skill" && edit.arguments === undefined) { + return `${edit.action} skill requires arguments`; + } + if (edit.action !== "delete" && edit.kind === "skill") { + const reference = edit.reference; + if (!reference) { + return `${edit.action} skill requires python reference`; + } + if (reference.type !== "python") { + return `${edit.action} skill reference.type must be python`; + } + const hasImport = + (typeof reference.import === "string" && reference.import.length > 0) || + (typeof reference.python_import === "string" && reference.python_import.length > 0); + const hasCallable = + (typeof reference.callable === "string" && reference.callable.length > 0) || + (typeof reference.call_pattern === "string" && reference.call_pattern.length > 0); + if (!hasImport) { + return `${edit.action} skill requires python import`; + } + if (!hasCallable) { + return `${edit.action} skill requires callable or call_pattern`; + } + } + return undefined; +} + +export function applyRefinementProposal( + state: HarnessState, + proposal: RefinementProposal, + options: { id: string; rollbackOf?: string; scope?: HarnessScope; baselineState?: HarnessState }, +): RefinementResult { + const appliedEdits: AppliedRefinementEdit[] = []; + const proposalModifiedKeys = new Set(); + for (const edit of proposal.edits) { + const computedId = edit.id ?? (edit.action === "create" ? slug(edit.title ?? edit.kind, edit.kind) : undefined); + const id = computedId ?? ""; + const validationError = validateEdit(edit, id); + if (validationError) { + appliedEdits.push({ ...edit, id, applied: false, error: validationError }); + continue; + } + + const records = state.entries[edit.kind]; + const before = cloneEntry(records[id]); + const entryKey = `${edit.kind}:${id}`; + const baseline = cloneEntry(options.baselineState?.entries[edit.kind][id]); + if ( + options.baselineState && + !proposalModifiedKeys.has(entryKey) && + JSON.stringify(before) !== JSON.stringify(baseline) + ) { + appliedEdits.push({ + ...edit, + id, + before, + applied: false, + error: "entry changed during refinement planning", + }); + continue; + } + if (edit.action === "delete") { + if (!before) { + appliedEdits.push({ ...edit, id, applied: false, error: "entry not found" }); + continue; + } + delete records[id]; + proposalModifiedKeys.add(entryKey); + appliedEdits.push({ ...edit, id, before, applied: true }); + continue; + } + if (edit.action === "create" && before) { + appliedEdits.push({ ...edit, id, before, applied: false, error: "entry already exists" }); + continue; + } + if (edit.action === "update" && !before) { + appliedEdits.push({ ...edit, id, applied: false, error: "entry not found" }); + continue; + } + + const createdAt = before?.created_at ?? now(); + const version = before ? before.version + 1 : 1; + const after: HarnessEntry = { + id, + kind: edit.kind, + title: edit.title ?? before?.title ?? id, + content: edit.content ?? before?.content ?? "", + path: edit.path ?? before?.path ?? "general", + scope: before?.scope ?? options.scope ?? "local", + reference: edit.reference ?? before?.reference ?? {}, + arguments: edit.arguments ?? before?.arguments ?? {}, + metadata: edit.metadata ?? before?.metadata ?? {}, + source: "refine", + created_at: createdAt, + updated_at: now(), + version, + }; + records[id] = after; + proposalModifiedKeys.add(entryKey); + appliedEdits.push({ ...edit, id, before, after: cloneEntry(after), applied: true }); + } + + const changes = appliedEdits.filter((edit) => edit.applied).map((edit) => `${edit.action} ${edit.kind}:${edit.id}`); + state.refinements.push({ + id: options.id, + trigger: proposal.summary, + changes, + evidence: proposal.rationale, + outcome: proposal.expectedOutcome, + created_at: now(), + }); + + return { + id: options.id, + summary: proposal.summary, + rationale: proposal.rationale, + expectedOutcome: proposal.expectedOutcome, + appliedEdits, + harnessStatePath: "", + rollbackOf: options.rollbackOf, + scope: options.scope, + }; +} + +export function rollbackProposal(target: RefinementResult): RefinementProposal { + const edits: RefinementEdit[] = []; + for (const edit of [...target.appliedEdits].reverse()) { + if (!edit.applied) continue; + if (edit.before) { + edits.push({ + action: edit.after ? "update" : "create", + kind: edit.kind, + id: edit.id, + title: edit.before.title, + content: edit.before.content, + path: edit.before.path, + reference: edit.before.reference, + arguments: edit.before.arguments, + metadata: edit.before.metadata, + reason: `Rollback ${target.id}`, + }); + } else if (edit.after) { + edits.push({ + action: "delete", + kind: edit.kind, + id: edit.id, + reason: `Rollback ${target.id}`, + }); + } + } + return { + summary: `Rollback refinement ${target.id}`, + rationale: `Restores continual harness state snapshots from refinement ${target.id}.`, + expectedOutcome: "Faulty refinement edits are reverted.", + edits, + }; +} + +export function getRefinementHistory(entries: readonly CustomEntry[]): RefinementResult[] { + return entries + .filter((entry) => entry.customType === REFINEMENT_CUSTOM_TYPE) + .map((entry) => entry.data) + .filter((data): data is RefinementResult => { + return typeof data === "object" && data !== null && "id" in data && "appliedEdits" in data; + }); +} + +/** + * Produce a refinement proposal (the LLM pass, or a rollback proposal) without + * mutating any harness state. Separated from {@link applyRefinementProposal} so + * callers can re-read the harness file immediately before applying — the LLM call + * here can take many seconds, during which the kernel or another session may write + * the shared `harness_state.json`. + */ +/** Mint a refinement id in the canonical `refine_` format. */ +export function generateRefinementId(): string { + return `refine_${new Date() + .toISOString() + .replace(/[^0-9]/g, "") + .slice(0, 17)}`; +} diff --git a/packages/coding-agent/src/session/refinement/planning.ts b/packages/coding-agent/src/session/refinement/planning.ts new file mode 100644 index 0000000000..0410d8cf63 --- /dev/null +++ b/packages/coding-agent/src/session/refinement/planning.ts @@ -0,0 +1,430 @@ +import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; +import type { Api, Model } from "@earendil-works/pi-ai"; +import { completeSimple } from "@earendil-works/pi-ai"; +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; +import { completeWithProviderRetry } from "../../core/provider-retry.js"; +import { getAuxiliaryThinkingLevel } from "../../core/thinking-levels.js"; +import { serializeConversation } from "../context/conversation-text.js"; +import { convertToLlm } from "../context/messages.js"; +import { historyForPrompt, overviewForPrompt } from "./format.js"; +import { + applyRefinementProposal, + generateRefinementId, + inferRefinementResultScope, + normalizeRefinementProposal, + rollbackProposal, +} from "./harness-state.js"; +import type { + AutoRefineReview, + AutoRefineReviewContext, + HarnessScope, + HarnessState, + RefinementPlan, + RefinementProposal, + RefinementResult, + RefineOptions, +} from "./types.js"; + +const REFINEMENT_SYSTEM_PROMPT = `You are Prime Agent's /refine continual harness subsystem. + +Your job is to improve the editable continual harness state from the current trajectory. +This is similar in spirit to context compaction, but instead of summarizing the +conversation you emit precise Create, Update, or Delete edits to reusable state. +The continual harness is the persistent, editable set of prompt notes, memories, +skills, and subagent specs that lets Prime Agent improve reusable behavior +outside the token history. +Use "continual harness" for that persistent artifact layer; keep "RLM" for the +runtime, Python REPL kernel, and native call interface that executes those artifacts. + +Continual harness components: +- prompt: supplemental prompt notes only. The base system prompt is immutable and MUST NOT be rewritten. +- memory: durable facts, decisions, failures, preferences, and outcomes. +- skill: installed Python REPL skill. Skill create/update edits MUST include a \`reference\` object with \`{"type":"python"}\`, a Python import, and a callable or call pattern; they also MUST include an \`arguments\` object describing accepted inputs, required fields, defaults, and constraints. Use \`{}\` for \`arguments\` only when the Python callable truly needs no external inputs. Include the RLM-native call form \`await (...)\`. +- subagent: reusable delegation specs, including purpose, instructions, and when to invoke. Include the RLM-native call form: compose a concise task prompt and spawn with \`handle = await rlm.spawn("sub-task", name="worker")\`; admission returns immediately with \`rlm_child_id\`, \`name\`, \`session_dir\`, and \`model\`, never the child's answer. Results arrive only through explicit \`agent_message\` replies or files; children reply with \`await agent_message.send(message, receiver_role="parent")\`. Use \`await rlm.list_subagents()\` to recover direct child handles and \`await agent_message.send(..., receiver_role="child", receiver_name=handle.name)\` for follow-ups. Do not invent wrappers like \`run_subagent(...)\`. + +Scope and persistence policy: +- The default editable continual harness store is local to the current Prime Agent session. Use it for session-specific progress, active task state, current-run coordination notes, temporary blockers, and project facts that should not affect other sessions. +- A caller may explicitly request global refinement. Global edits must be stable cross-session lessons, durable user preferences, reusable skills/subagents, or tool/environment facts that should affect future sessions. +- Entry ids in the harness overview may carry a display-only \`local:\` or \`global:\` prefix. Always use the bare id (no prefix) in edits. +- All edits in one refinement apply only to the requested scope's store. During a local refinement, global entries are read-only context: never propose update or delete edits for them; create a local entry instead when a session-specific override is genuinely needed. +- Project/workspace-specific lessons may be persisted globally only when the title, path, or content explicitly names the project/workspace and the lesson is likely to be reused in future sessions for that project. Prefer local edits when the lesson only belongs in the current conversation. +- Use memory for declarative facts and preferences, skill for repeatable procedures exposed as Python calls, prompt for narrow behavioral policy addendums, and subagent for reusable delegation roles. +- Create or update the smallest relevant component: repeated delegation roles should become subagent specs, repeated procedures should become skills, durable facts/preferences should become memories, and narrow behavioral policies should become prompt addendums. +- When an edit is persisted, include metadata such as \`{"scope":"local"}\` or \`{"scope":"global"}\` when that helps future review understand the intended blast radius. + +Use the trajectory, current continual harness state, and prior refinement history. Prefer +small evidence-backed edits. If prior refinements caused issues, rollback or +replace the faulty editable entries. Never edit source files directly. Output +JSON only with this exact shape: + +{ + "summary": "one sentence", + "rationale": "why these edits are justified by trajectory evidence", + "expectedOutcome": "what should improve and how to validate it", + "edits": [ + { + "action": "create|update|delete", + "kind": "prompt|memory|skill|subagent", + "id": "stable id for update/delete, optional for create", + "title": "required for create/update except delete", + "content": "required for create/update except delete", + "path": "optional grouping path", + "reference": {"type": "python", "import": "package.module", "callable": "function_name", "call_pattern": "await function_name(...)"}, + "arguments": {"name": {"type": "string", "required": true, "description": "accepted input"}}, + "metadata": {}, + "reason": "why this edit is useful" + } + ] +}`; + +const AUTO_REFINE_REVIEW_SYSTEM_PROMPT = `You are Prime Agent's automatic /refine review gate. + +Decide whether this checkpoint should run /refine. Auto /refine writes local continual harness state by default, so approve when the trajectory contains evidence useful to this session's future turns. +Reject one-off noise, unsupported hypotheses, and transient tool outputs. Ask for global refinement only for durable cross-session lessons or explicitly project-qualified lessons likely to be reused in future sessions. + +Return JSON only: +{ + "shouldRefine": true|false, + "rationale": "short reason", + "instructions": "optional concise instructions for /refine if shouldRefine is true" +}`; + +// These caps apply only with reasoning off; thinking and JSON otherwise share the model's output budget. + +const REFINEMENT_MAX_OUTPUT_TOKENS = 32_000; + +const AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS = 4_096; + +const REFINEMENT_CONTEXT_OVERHEAD_TOKENS = 1_024; + +const TRUNCATED_JSON_ERROR = + "the model stopped before completing its JSON object. This usually means the output budget was exhausted; retry with a smaller request."; + +function refinementInputTokenBound(text: string): number { + // One token per UTF-8 byte bounds byte-based tokenizers, including dense or unusual text. + return Buffer.byteLength(text, "utf8"); +} + +function refinementRequest( + model: Model, + systemPrompt: string, + conversationText: string, + buildPrompt: (conversation: string) => string, + outputReserve: number, +): { model: Model; userPrompt: string } { + const systemReserve = refinementInputTokenBound(systemPrompt) + REFINEMENT_CONTEXT_OVERHEAD_TOKENS; + const inputBudget = + model.contextWindow - Math.min(model.maxTokens, outputReserve, Math.floor(model.contextWindow / 2)); + let userPrompt = buildPrompt(conversationText); + if (systemReserve + refinementInputTokenBound(userPrompt) > inputBudget && conversationText.length > 0) { + const promptForLength = (length: number): string => { + let start = conversationText.length - length; + const first = conversationText.charCodeAt(start); + if (first >= 0xdc00 && first <= 0xdfff) start++; + return buildPrompt( + `[Earlier conversation omitted to fit the model context.]\n${conversationText.slice(start)}`, + ); + }; + let low = 0; + let high = conversationText.length; + while (low < high) { + const length = Math.ceil((low + high) / 2); + if (systemReserve + refinementInputTokenBound(promptForLength(length)) <= inputBudget) low = length; + else high = length - 1; + } + userPrompt = promptForLength(low); + } + const maxTokens = Math.min( + model.maxTokens, + model.contextWindow - systemReserve - refinementInputTokenBound(userPrompt), + ); + if (maxTokens <= 0) { + throw new Error( + "Refinement prompt leaves no room for output in the model's context window; retry with a smaller request.", + ); + } + // Bound the request's model ceiling too: some adapters add thinking tokens before clamping to it. + return { model: { ...model, maxTokens }, userPrompt }; +} + +/** + * Whether a JSON candidate ends mid-value: an unterminated string, or unclosed + * objects/arrays. A reply cut off by an exhausted output budget is incomplete in + * this sense, while a complete-but-malformed reply is balanced. Brace slicing can + * also produce a balanced fragment, so callers treat "balanced" as malformed. + */ +function isIncompleteJson(candidate: string): boolean { + let depth = 0; + let inString = false; + let escaped = false; + for (const char of candidate) { + if (escaped) { + escaped = false; + continue; + } + if (inString) { + if (char === "\\") escaped = true; + else if (char === '"') inString = false; + continue; + } + if (char === '"') inString = true; + else if (char === "{" || char === "[") depth++; + else if (char === "}" || char === "]") depth--; + } + return inString || depth > 0; +} + +function parseJsonCandidate(candidate: string): unknown { + try { + return JSON.parse(candidate); + } catch (error) { + // A truncated reply and a malformed one both fail here, and JSON.parse + // describes the fragment rather than the cause. Name the cause instead. + if (isIncompleteJson(candidate)) { + throw new Error(TRUNCATED_JSON_ERROR); + } + throw new Error(`the model did not return valid JSON: ${error instanceof Error ? error.message : String(error)}`); + } +} + +function extractJsonObject(text: string): unknown { + const trimmed = text.trim(); + if (trimmed.startsWith("{") && trimmed.endsWith("}")) { + // A reply truncated after a nested closing brace still looks well-formed + // here, so this path needs the same diagnosis as the slicing fallback. + return parseJsonCandidate(trimmed); + } + const fenced = trimmed.match(/```(?:json)?\s*([\s\S]*?)```/); + if (fenced) { + return parseJsonCandidate(fenced[1].trim()); + } + // Brace slicing recovers JSON wrapped in prose. On a reply truncated inside the + // edits array it slices to an earlier edit's closing brace, so a failure here + // is diagnosed against the original text rather than the balanced fragment. + const start = trimmed.indexOf("{"); + const end = trimmed.lastIndexOf("}"); + if (start !== -1 && end > start) { + try { + return JSON.parse(trimmed.slice(start, end + 1)); + } catch { + return parseJsonCandidate(trimmed.slice(start)); + } + } + if (isIncompleteJson(trimmed)) { + throw new Error(TRUNCATED_JSON_ERROR); + } + throw new Error("Refiner did not return a JSON object"); +} + +function parseProposal(text: string): RefinementProposal { + const value = extractJsonObject(text); + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error("Refiner JSON must be an object"); + } + return normalizeRefinementProposal(value); +} + +export async function planRefinement( + messages: AgentMessage[], + state: HarnessState, + history: RefinementResult[], + model: Model, + apiKey: string, + options: RefineOptions = {}, + headers?: Record, + signal?: AbortSignal, + thinkingLevel?: ThinkingLevel, + sessionId?: string, +): Promise { + const id = generateRefinementId(); + if (options.rollbackId) { + const target = history.find((item) => item.id === options.rollbackId); + if (!target) { + throw new Error(`Refinement ${options.rollbackId} not found`); + } + const fallbackScope: HarnessScope = options.global ? "global" : "local"; + return { + proposal: rollbackProposal(target), + id, + rollbackOf: target.id, + rollbackScope: inferRefinementResultScope(target) ?? fallbackScope, + }; + } + + const conversationText = serializeConversation(convertToLlm(messages)).slice(-80_000); + const scopeInstruction = options.global + ? "Requested refinement scope: global. Only propose stable cross-session continual harness edits, durable user preferences, reusable skills/subagents, or explicitly project-qualified facts that should affect future Prime Agent sessions. Do not persist session-only progress, temporary blockers, or current-run coordination globally." + : "Requested refinement scope: local. Prefer local continual harness edits for current task progress, temporary blockers, current-run coordination, and project facts that are not clearly reusable across Prime Agent sessions. Global entries in the overview are read-only context: do not propose update or delete edits for them; create a local entry instead if an override is needed."; + const buildPrompt = (conversation: string): string => + [ + `\n${overviewForPrompt(state)}\n`, + `\n${historyForPrompt(history)}\n`, + `\n${conversation}\n`, + `\n${scopeInstruction}\n`, + options.instructions ? `\n${options.instructions}\n` : "", + "Return only JSON edits. If no useful edit is justified, return an empty edits array with a rationale.", + ] + .filter(Boolean) + .join("\n\n"); + const reasoning = getAuxiliaryThinkingLevel(model, thinkingLevel); + const { model: requestModel, userPrompt } = refinementRequest( + model, + REFINEMENT_SYSTEM_PROMPT, + conversationText, + buildPrompt, + reasoning === "off" ? REFINEMENT_MAX_OUTPUT_TOKENS : model.maxTokens, + ); + const maxTokens = + reasoning === "off" ? Math.min(requestModel.maxTokens, REFINEMENT_MAX_OUTPUT_TOKENS) : requestModel.maxTokens; + + const response = await completeWithProviderRetry( + () => + completeSimple( + requestModel, + { + systemPrompt: REFINEMENT_SYSTEM_PROMPT, + messages: [{ role: "user", content: [{ type: "text", text: userPrompt }], timestamp: Date.now() }], + }, + { + reasoning, + maxTokens, + signal, + apiKey, + headers, + sessionId, + }, + ), + { policy: options.retry, signal }, + ); + + if (response.stopReason === "error") { + throw new Error(`Refinement failed: ${response.errorMessage || "Unknown error"}`); + } + if (response.stopReason === "length") { + throw new Error(`Refinement failed: ${TRUNCATED_JSON_ERROR}`); + } + + const text = response.content + .filter((content): content is { type: "text"; text: string } => content.type === "text") + .map((content) => content.text) + .join("\n"); + return { proposal: parseProposal(text), id }; +} + +function parseAutoRefineReview(text: string): AutoRefineReview { + const value = extractJsonObject(text); + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error("Auto-refine review JSON must be an object"); + } + const record = value as Record; + return { + shouldRefine: record.shouldRefine === true, + rationale: typeof record.rationale === "string" ? record.rationale : "No rationale provided.", + instructions: typeof record.instructions === "string" ? record.instructions : undefined, + }; +} + +export async function reviewAutoRefine( + messages: AgentMessage[], + state: HarnessState, + history: RefinementResult[], + model: Model, + apiKey: string, + context: AutoRefineReviewContext, + headers?: Record, + signal?: AbortSignal, + thinkingLevel?: ThinkingLevel, + retry?: ProviderRetryPolicy, + sessionId?: string, +): Promise { + const conversationText = serializeConversation(convertToLlm(messages)).slice(-40_000); + const buildPrompt = (conversation: string): string => + [ + ` +${context.reason}; ${context.turnsSinceLastReview} assistant turns since last auto-refine review +`, + ` +${overviewForPrompt(state)} +`, + ` +${historyForPrompt(history)} +`, + ` +${conversation} +`, + "Return shouldRefine=true when the trajectory contains evidence useful to this session's future turns. Prefer local harness edits for current task progress, temporary blockers, and current-run coordination. Ask for global refinement only for durable cross-session lessons or explicitly project-qualified facts likely to be reused in future sessions.", + ].join("\n\n"); + const reasoning = getAuxiliaryThinkingLevel(model, thinkingLevel); + const { model: requestModel, userPrompt } = refinementRequest( + model, + AUTO_REFINE_REVIEW_SYSTEM_PROMPT, + conversationText, + buildPrompt, + reasoning === "off" ? AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS : model.maxTokens, + ); + const maxTokens = + reasoning === "off" + ? Math.min(requestModel.maxTokens, AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS) + : requestModel.maxTokens; + const response = await completeWithProviderRetry( + () => + completeSimple( + requestModel, + { + systemPrompt: AUTO_REFINE_REVIEW_SYSTEM_PROMPT, + messages: [{ role: "user", content: [{ type: "text", text: userPrompt }], timestamp: Date.now() }], + }, + { + reasoning, + maxTokens, + signal, + apiKey, + headers, + sessionId, + }, + ), + { policy: retry, signal }, + ); + if (response.stopReason === "error") { + throw new Error(`Auto-refine review failed: ${response.errorMessage || "Unknown error"}`); + } + if (response.stopReason === "length") { + throw new Error(`Auto-refine review failed: ${TRUNCATED_JSON_ERROR}`); + } + const text = response.content + .filter((content): content is { type: "text"; text: string } => content.type === "text") + .map((content) => content.text) + .join("\n"); + return parseAutoRefineReview(text); +} + +export async function refineHarness( + messages: AgentMessage[], + state: HarnessState, + history: RefinementResult[], + model: Model, + apiKey: string, + options: RefineOptions = {}, + headers?: Record, + signal?: AbortSignal, + thinkingLevel?: ThinkingLevel, + sessionId?: string, +): Promise { + const plan = await planRefinement( + messages, + state, + history, + model, + apiKey, + options, + headers, + signal, + thinkingLevel, + sessionId, + ); + return applyRefinementProposal(state, plan.proposal, { + id: plan.id, + rollbackOf: plan.rollbackOf, + scope: plan.rollbackScope ?? (options.global ? "global" : "local"), + }); +} diff --git a/packages/coding-agent/src/session/refinement/refinement-execution.ts b/packages/coding-agent/src/session/refinement/refinement-execution.ts index 479f59a1ae..5d28b96fa3 100644 --- a/packages/coding-agent/src/session/refinement/refinement-execution.ts +++ b/packages/coding-agent/src/session/refinement/refinement-execution.ts @@ -1,341 +1,7 @@ -import { existsSync } from "node:fs"; -import { dirname, resolve } from "node:path"; -import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { Api, Model } from "@earendil-works/pi-ai"; -import { formatNoModelSelectedMessage } from "../../core/auth-guidance.js"; -import { serializeConversation } from "../../core/compaction/index.js"; -import type { ExtensionRunner, SessionBeforeRefineResult } from "../../core/extensions/index.js"; -import { - type CustomMessage, - convertToLlm, - createRefinementNoticeMessage, - createRefinementOutcomeMessage, - type RefinementSource, -} from "../../core/messages.js"; -import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; -import { - type AutoRefineReview, - appendGlobalRefinement, - applyRefinementProposal, - generateRefinementId, - getGlobalHarnessStateDir, - getLocalHarnessStateDir, - getRefinementHistory, - type HarnessState, - inferRefinementResultScope, - loadGlobalRefinementHistory, - loadHarnessState, - mergeHarnessStates, - mergeRefinementHistory, - normalizeRefinementProposal, - planRefinement, - type RefinementPlan, - type RefinementResult, - reviewAutoRefine, - saveHarnessState, -} from "../../core/refinement/index.js"; -import type { SessionManager } from "../../core/session-manager.js"; -import type { AutoRefineReviewRequest } from "./auto-refinement.js"; -export type SessionRefinementEvent = - | { type: "refine_complete"; result: RefinementResult } - | { type: "refine_failed"; error: string } - | { type: "message_start" | "message_end"; message: CustomMessage }; -export interface RefinementExecutionHost { - sessionManager: Pick< - SessionManager, - "getSessionArtifactDir" | "getEntries" | "appendCustomMessageEntryWithRollback" | "appendCustomEntry" - >; - isDisposed(): boolean; - getRlmSessionDir(): string | undefined; - getModel(): Model | undefined; - getThinkingLevel(): ThinkingLevel; - getMessages(): AgentMessage[]; - getRequiredRequestAuth( - model: Model, - ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; - getRetryPolicy(): ProviderRetryPolicy; - getSessionId?(): string; - getExtensionRunner(): Pick; - disconnect(): void; - reconnect(): void; - emit(event: SessionRefinementEvent): void; - retainUnpersistedOutcome(message: CustomMessage): void; -} - -/** Thrown when a session_before_refine extension skips the refinement round. */ -export class RefineSkippedError extends Error {} - -/** Plans against current session dependencies and persists results during the apply barrier. */ -export class RefinementExecution { - constructor( - private readonly _host: RefinementExecutionHost, - private readonly _releaseAbort: (abort: AbortController) => void, - ) {} - _localHarnessStateDir(): string | undefined { - return ( - getLocalHarnessStateDir(this._host.sessionManager.getSessionArtifactDir()) ?? - (this._host.getRlmSessionDir() ? getLocalHarnessStateDir(this._host.getRlmSessionDir()) : undefined) - ); - } - - _loadMergedHarnessState(): HarnessState { - const localHarnessStateDir = this._localHarnessStateDir(); - return mergeHarnessStates( - loadHarnessState(getGlobalHarnessStateDir(), "global"), - localHarnessStateDir ? loadHarnessState(localHarnessStateDir, "local") : undefined, - ); - } - - private _loadRefinementHistory(): RefinementResult[] { - return mergeRefinementHistory( - loadGlobalRefinementHistory(getGlobalHarnessStateDir()), - getRefinementHistory(this._host.sessionManager.getEntries().filter((entry) => entry.type === "custom")), - ); - } - - async _planRefine( - options: { instructions?: string; rollbackId?: string; global?: boolean }, - signal: AbortSignal, - trigger: "manual" | "auto" = "manual", - ): Promise { - if (this._host.isDisposed()) { - throw new Error("Cannot refine a disposed session."); - } - - if (!this._host.getModel()) { - throw new Error(formatNoModelSelectedMessage()); - } - - const model = this._host.getModel()!; - const { apiKey, headers, requestModel } = await this._host.getRequiredRequestAuth(model); - const globalHarnessStateDir = getGlobalHarnessStateDir(); - const localHarnessStateDir = this._localHarnessStateDir(); - const requestedScope = options.global ? "global" : "local"; - if (!options.rollbackId && requestedScope === "local" && !localHarnessStateDir) { - throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); - } - const globalPlanningState = loadHarnessState(globalHarnessStateDir, "global"); - const localPlanningState = localHarnessStateDir ? loadHarnessState(localHarnessStateDir, "local") : undefined; - const planningState = - requestedScope === "global" - ? globalPlanningState - : mergeHarnessStates(globalPlanningState, localPlanningState); - const history = this._loadRefinementHistory(); - const rollbackTarget = options.rollbackId ? history.find((item) => item.id === options.rollbackId) : undefined; - let baselineScope = rollbackTarget - ? (inferRefinementResultScope(rollbackTarget) ?? requestedScope) - : requestedScope; - let baselineHarnessStateDir = baselineScope === "global" ? globalHarnessStateDir : localHarnessStateDir; - if (rollbackTarget?.harnessStatePath) { - baselineHarnessStateDir = dirname(rollbackTarget.harnessStatePath); - baselineScope = resolve(baselineHarnessStateDir) === resolve(globalHarnessStateDir) ? "global" : "local"; - } - if (!baselineHarnessStateDir) { - throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); - } - const baselineState = rollbackTarget - ? loadHarnessState(baselineHarnessStateDir, baselineScope) - : baselineScope === "global" - ? globalPlanningState - : localPlanningState!; - if (!options.rollbackId && this._host.getExtensionRunner().hasHandlers("session_before_refine")) { - const result = (await this._host.getExtensionRunner().emit({ - type: "session_before_refine", - preparation: { - trigger, - instructions: options.instructions, - scope: requestedScope, - planningState, - history, - conversationText: serializeConversation(convertToLlm(this._host.getMessages())).slice(-80_000), - }, - signal, - })) as SessionBeforeRefineResult | undefined; - if (this._host.isDisposed() || signal.aborted) { - throw new Error("Refinement cancelled because the session was disposed."); - } - if (result?.skip) { - throw new RefineSkippedError("Refinement skipped by extension"); - } - if (result?.proposal !== undefined) { - return { - proposal: normalizeRefinementProposal(result.proposal), - id: generateRefinementId(), - baselineState, - }; - } - } - const plan = await planRefinement( - this._host.getMessages(), - planningState, - history, - requestModel ?? model, - apiKey, - { ...options, retry: this._host.getRetryPolicy() }, - headers, - signal, - this._host.getThinkingLevel(), - this._host.getSessionId?.(), - ); - if (this._host.isDisposed() || signal.aborted) { - throw new Error("Refinement cancelled because the session was disposed."); - } - return { ...plan, baselineState }; - } - - async _applyRefine( - plan: RefinementPlan, - options: { instructions?: string; rollbackId?: string; global?: boolean }, - refineAbort: AbortController, - source: RefinementSource, - ): Promise { - if (this._host.isDisposed()) { - throw new Error("Cannot refine a disposed session."); - } - // The caller has already set _refineInFlight and waited for agent idle. - // Disconnect only for the brief apply + save + reconnect critical section. - this._host.disconnect(); - - try { - const globalHarnessStateDir = getGlobalHarnessStateDir(); - const localHarnessStateDir = this._localHarnessStateDir(); - const requestedScope = options.global ? "global" : "local"; - const history = this._loadRefinementHistory(); - const rollbackTarget = options.rollbackId ? history.find((item) => item.id === options.rollbackId) : undefined; - let targetScope = plan.rollbackScope ?? requestedScope; - let targetHarnessStateDir = targetScope === "global" ? globalHarnessStateDir : localHarnessStateDir; - if (targetScope === "local" && rollbackTarget?.harnessStatePath) { - if (!existsSync(rollbackTarget.harnessStatePath)) { - throw new Error( - `Local refinement ${rollbackTarget.id} state file not found: ${rollbackTarget.harnessStatePath}`, - ); - } - targetHarnessStateDir = dirname(rollbackTarget.harnessStatePath); - // Legacy records predate scope fields and default to "local" but may point - // at the global store; honor the recorded path so its entries stay global. - if (resolve(targetHarnessStateDir) === resolve(globalHarnessStateDir)) { - targetScope = "global"; - } - } - if (!targetHarnessStateDir) { - throw new Error("Local harness refinement requires a persisted session; use global refinement instead."); - } - // Re-read the target state immediately before applying so concurrent kernel - // (`rlm.harness`) writes during the LLM pass are not clobbered. - const state = loadHarnessState(targetHarnessStateDir, targetScope); - const proposal = { - ...plan.proposal, - edits: plan.proposal.edits.map((edit) => { - const localPrefix = "local:"; - const globalPrefix = "global:"; - return { - ...edit, - id: edit.id?.startsWith(localPrefix) - ? edit.id.slice(localPrefix.length) - : edit.id?.startsWith(globalPrefix) - ? edit.id.slice(globalPrefix.length) - : edit.id, - }; - }), - }; - if (this._host.isDisposed() || refineAbort.signal.aborted) { - throw new Error("Refinement cancelled because the session was disposed."); - } - const result = applyRefinementProposal(state, proposal, { - id: plan.id, - rollbackOf: plan.rollbackOf, - scope: targetScope, - baselineState: plan.baselineState, - }); - result.harnessStatePath = saveHarnessState(targetHarnessStateDir, state); - if (targetScope === "global") { - appendGlobalRefinement(globalHarnessStateDir, result); - } - let refinementAuditAppendError: { error: unknown } | undefined; - try { - this._host.sessionManager.appendCustomEntry("prime-agent.refinement", result); - } catch (error) { - refinementAuditAppendError = { error }; - } - try { - this._recordRefinementOutcome(result); - } catch (error) { - if (!refinementAuditAppendError) throw error; - } - if (refinementAuditAppendError) throw refinementAuditAppendError.error; - // The prompt stays byte-identical so the provider prefix cache survives; the notice carries the change. - this._recordRefinementNotice(result, source); - try { - this._host.emit({ type: "refine_complete", result }); - } catch { - // Listener failures must not flip a successful refinement into - // a reported failure — the refinement is already persisted. - } - try { - await this._host.getExtensionRunner().emit({ - type: "refine_complete", - id: result.id, - summary: result.summary, - appliedEdits: result.appliedEdits.filter((edit) => edit.applied).length, - scope: result.scope ?? "local", - }); - } catch { - // Extension emit failures must not flip a successful refinement - // into a reported failure — the refinement is already persisted. - } - return result; - } finally { - this._releaseAbort(refineAbort); - if (!this._host.isDisposed()) { - this._host.reconnect(); - } - } - } - - private _recordRefinementOutcome(result: RefinementResult): void { - this._appendDurableRefineMessage(createRefinementOutcomeMessage(result)); - } - - private _recordRefinementNotice(result: RefinementResult, source: RefinementSource): void { - if (!result.appliedEdits.some((edit) => edit.applied)) return; - this._appendDurableRefineMessage(createRefinementNoticeMessage(result, source)); - } - - private _appendDurableRefineMessage(message: CustomMessage): void { - try { - this._host.sessionManager.appendCustomMessageEntryWithRollback( - message.customType, - message.content, - message.display, - message.details, - ); - } catch { - // Not in the session file, so context rebuilds would drop the outcome. - this._host.retainUnpersistedOutcome(message); - } - this._host.getMessages().push(message); - this._host.emit({ type: "message_start", message }); - this._host.emit({ type: "message_end", message }); - } - - async review(context: AutoRefineReviewRequest, signal?: AbortSignal): Promise { - const model = this._host.getModel(); - if (!model) { - return { shouldRefine: false, rationale: "No model selected." }; - } - const { apiKey, headers, requestModel } = await this._host.getRequiredRequestAuth(model); - return reviewAutoRefine( - this._host.getMessages(), - this._loadMergedHarnessState(), - this._loadRefinementHistory(), - requestModel ?? model, - apiKey, - context, - headers, - signal, - this._host.getThinkingLevel(), - this._host.getRetryPolicy(), - this._host.getSessionId?.(), - ); - } -} +// Compatibility exports; implementation lives with its session owner. +export { + RefinementExecution, + type RefinementExecutionHost, + RefineSkippedError, + type SessionRefinementEvent, +} from "./execution.js"; diff --git a/packages/coding-agent/src/session/refinement/refinement.ts b/packages/coding-agent/src/session/refinement/refinement.ts index 77f75094e0..2c30487f9d 100644 --- a/packages/coding-agent/src/session/refinement/refinement.ts +++ b/packages/coding-agent/src/session/refinement/refinement.ts @@ -1,928 +1,9 @@ -import type { AgentMessage, ThinkingLevel } from "@earendil-works/pi-agent-core"; -import type { Api, Model } from "@earendil-works/pi-ai"; -import type { ExtensionRunner } from "../../core/extensions/index.js"; -import type { CustomMessage, RefinementSource } from "../../core/messages.js"; -import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; -import type { HarnessState, RefinementPlan, RefinementResult } from "../../core/refinement/index.js"; -import type { SessionManager } from "../../core/session-manager.js"; -import type { SettingsManager } from "../../core/settings-manager.js"; -import { AutoRefinement, type AutoRefineReviewer, autoRefineInstructions } from "./auto-refinement.js"; -import { RefinementExecution, RefineSkippedError, type SessionRefinementEvent } from "./refinement-execution.js"; - -export type { AutoRefineReviewer, AutoRefineReviewRequest } from "./auto-refinement.js"; - -export interface SessionRefinementHost { - sessionManager: Pick< - SessionManager, - "getSessionArtifactDir" | "getEntries" | "appendCustomMessageEntryWithRollback" | "appendCustomEntry" - >; - settingsManager: Pick; - getRetryPolicy(): ProviderRetryPolicy; - getSessionId?(): string; - isDisposed(): boolean; - isDisposing(): boolean; - isStreaming(): boolean; - isCompacting(): boolean; - getDepth(): number; - getRlmSessionDir(): string | undefined; - getModel(): Model | undefined; - getThinkingLevel(): ThinkingLevel; - getMessages(): AgentMessage[]; - getRequiredRequestAuth( - model: Model, - ): Promise<{ apiKey: string; headers?: Record; requestModel?: Model }>; - getExtensionRunner(): Pick; - getEventQueue(): Promise; - getCompactionOperation(): Promise | undefined; - getBranchSummaryOperation(): Promise | undefined; - waitForAgentIdle(): Promise; - dispatchRefine( - options: { instructions?: string; global?: boolean }, - internal: { source: "self" } | { trigger: "auto" }, - ): Promise; - disconnect(): void; - reconnect(): void; - emit(event: SessionRefinementEvent): void; - retainUnpersistedOutcome(message: CustomMessage): void; - notifyCheckpoints(): void; - scheduleInputPump(): void; - isContinuationScheduled(): boolean; - cancelContinuation(): void; -} - -/** Thrown when a session_before_refine extension skips the refinement round. */ -export { RefineSkippedError } from "./refinement-execution.js"; - -/** - * Discriminated result from a serialized-mode background planning pass. - * - "plan": review approved and planning succeeded; carry the exact plan, - * options, and abort controller so the boundary can apply directly - * without a second planning request. - * - "skip": reviewer declined; no refine needed. - * - "failure": review or planning threw; boundary should not retry. - */ -export type SerializedBackgroundPlanResult = - | { - status: "plan"; - plan: RefinementPlan; - options: { instructions?: string; rollbackId?: string; global?: boolean }; - abort: AbortController; - branchVersion: number; - source: Exclude; - } - | { status: "skip"; explicit?: boolean } - | { status: "invalidated"; branchVersion: number } - | { - status: "failure"; - explicit: boolean; - options: { instructions?: string; rollbackId?: string; global?: boolean }; - branchVersion: number; - }; - -/** Owns refinement admission, planning/apply barriers, and serialized plan claims. */ -export class SessionRefinement { - private readonly _auto: AutoRefinement; - private readonly _execution: RefinementExecution; - private _refineAbortController?: AbortController; - private readonly _serializedRefine: boolean; - private _refineInFlight?: Promise; - private _refinePlanInFlight?: Promise; - private _serializedPlanInFlight?: Promise; - private _serializedPlanClaim?: Promise; - private _serializedExplicitRefineOptions?: { - instructions?: string; - global?: boolean; - }; - private _pendingRequestedRefine: { instructions?: string; global?: boolean } | undefined; - - constructor( - private readonly _host: SessionRefinementHost, - config: { autoRefineReviewer?: AutoRefineReviewer; serializedRefine?: boolean }, - ) { - this._execution = new RefinementExecution(_host, (abort) => { - if (this._refineAbortController === abort) this._refineAbortController = undefined; - }); - this._auto = new AutoRefinement( - { - settingsManager: _host.settingsManager, - isDisposed: () => _host.isDisposed(), - isDisposing: () => _host.isDisposing(), - isStreaming: () => _host.isStreaming(), - isCompacting: () => _host.isCompacting(), - isAllowed: () => this._autoRefineAllowedForSession(), - getModel: () => _host.getModel(), - isContinuationScheduled: () => _host.isContinuationScheduled(), - cancelContinuation: () => _host.cancelContinuation(), - refine: (options, internal) => _host.dispatchRefine(options, internal), - runSerialized: (options, source) => this._runSerializedRefine(options, source), - emitFailure: (error) => this._emitRefineFailed(error), - review: (context, signal) => this._execution.review(context, signal), - }, - config.serializedRefine ?? false, - config.autoRefineReviewer, - ); - this._serializedRefine = config.serializedRefine ?? false; - } - - requestAbort(): void { - this._pendingRequestedRefine = undefined; - this._auto.invalidatePlans(); - this._auto.abortReview(); - this._refineAbortController?.abort(); - } - observeAssistantEnd(): void { - this._auto.observeAssistantEnd(); - this._maybeStartSerializedBackgroundPlan(); - } - get isApplying(): boolean { - return this._refineInFlight !== undefined; - } - get serialized(): boolean { - return this._serializedRefine; - } - /** Split settlement preserves the caller's existing await boundary. */ - beginAbortedTurnCleanup(): { promise: Promise; finish(): void } | undefined { - this._pendingRequestedRefine = undefined; - const plan = this._serializedPlanInFlight; - if (!plan) return undefined; - this._auto.invalidatePlans(); - this._refineAbortController?.abort(); - return { - promise: plan, - finish: () => { - if (this._serializedPlanInFlight === plan) { - this._serializedPlanInFlight = undefined; - this._serializedExplicitRefineOptions = undefined; - } - }, - }; - } - dispose(): void { - this._auto.abortReview(); - this._refineAbortController?.abort(); - this._auto.cancelScheduled(); - this._serializedPlanInFlight = undefined; - this._serializedExplicitRefineOptions = undefined; - this._pendingRequestedRefine = undefined; - this._auto._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); - this._auto.invalidatePlans(); - } - - async _runSerializedRefineCheckpoint(): Promise { - if (this._host.isDisposed() || this._host.isDisposing()) { - return; - } - - // 1. Await any background plan that was started at message_end - // (either for a pending refine.run or for interval-triggered - // auto-refine). This must be checked BEFORE the pending and - // interval checks because background planning may have consumed - // the pending request at message_end. - const branchVersion = this._auto.branchVersion; - const bgConsumption = await this._consumeSerializedBackgroundPlan(async (bgResult) => { - if (this._host.isDisposed() || this._host.isDisposing()) { - return true; - } - - if (bgResult?.status === "plan") { - if (bgResult.branchVersion !== this._auto.branchVersion) { - if (!this._pendingRequestedRefine) { - this._auto.stampCooldown(); - this._auto.resetTurns(); - return true; - } - } else { - // Apply the EXACT background plan directly via _applyRefine - // (no second _planRefine call). - try { - await this._applySerializedPlan(bgResult); - } catch (error) { - this._emitRefineFailed(error); - } - this._auto.stampCooldown(); - this._auto.resetTurns(); - if (!this._pendingRequestedRefine) { - return true; - } - } - } - - if (bgResult?.status === "skip") { - // Reviewer declined or an extension skipped during background planning. - // Reset exactly once. Never retry the interval review; only fall through for a separate pending refine.run. - if (bgResult.explicit) { - this._emitRefineFailed(new RefineSkippedError("Refinement skipped by extension")); - } - this._auto.stampCooldown(); - this._auto.resetTurns(); - if (!this._pendingRequestedRefine) { - return true; - } - } - - if (bgResult?.status === "failure") { - // Background review or planning failure stamps cooldown without a synchronous retry. - // A separately queued refine.run may still be serviced below. - if (branchVersion === this._auto.branchVersion) { - this._auto.stampCooldown(); - } - // Re-queue an explicit refine.run whose background plan failed, - // but only when branchVersion is still current and no newer - // pending request has arrived since the background plan consumed - // the original one. A newer request retains priority; interval - // failures keep existing no-retry cooldown semantics. - if ( - bgResult.explicit && - bgResult.branchVersion === this._auto.branchVersion && - !this._pendingRequestedRefine - ) { - this._pendingRequestedRefine = bgResult.options; - } - if (!this._pendingRequestedRefine) { - return true; - } - } - - if (bgResult?.status === "invalidated" && !this._pendingRequestedRefine) { - this._auto.stampCooldown(); - this._auto.resetTurns(); - return true; - } - - await this._runSerializedRefineCheckpointAfterBackground(branchVersion); - return true; - }); - if (this._host.isDisposed() || this._host.isDisposing() || bgConsumption !== "none") { - return; - } - await this._runSerializedRefineCheckpointAfterBackground(branchVersion); - } - - private async _runSerializedRefineCheckpointAfterBackground(branchVersion: number): Promise { - // No background result, or a refine.run arrived while the background result was - // in flight. Fall through so an explicit pending request is serviced at this boundary. - - // 2. Agent-callable refine.run requests that were NOT consumed by - // background planning (e.g. interval not reached at message_end, - // or cooldown was active). Service them synchronously. - const pending = this._pendingRequestedRefine; - if (pending) { - this._pendingRequestedRefine = undefined; - try { - await this._runSerializedRefine(pending, "self"); - } catch (error) { - this._emitRefineFailed(error); - } - this._auto.stampCooldown(); - this._auto.resetTurns(); - return; - } - - // 3. Post-compaction auto-refine. Serialized sessions defer the - // compaction trigger to this boundary instead of entering the interactive - // path, which waits for agent idle and can never run inside a tool loop. - if (!this._autoRefineAllowedForSession()) { - this._auto.discardCompact(); - return; - } - const settings = this._host.settingsManager.getAutoRefineSettings(); - if (!settings.enabled) { - this._auto.discardCompact(); - return; - } - if (this._auto.hasPendingCompact) { - if (!settings.compact) { - this._auto.discardCompact(); - } else { - const nowMs = Date.now(); - const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; - if (underCooldown) { - // Preserve the compact trigger for a later boundary, matching the - // interactive path's pending behavior while the cooldown is active. - return; - } - this._auto.discardCompact(); - await this._auto._runSerializedAutoRefineReview("compact", branchVersion); - return; - } - } - - // 4. Interval-triggered auto-refine (no background plan was started). - if (this._auto.turnsSinceReview < settings.turnInterval) { - return; - } - const nowMs = Date.now(); - const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; - if (underCooldown) { - return; - } - await this._auto._runSerializedAutoRefineReview("turn_interval", branchVersion); - } - - private async _consumeSerializedBackgroundPlan( - consume: (result: SerializedBackgroundPlanResult | undefined) => Promise, - ): Promise<"none" | "waited" | "continue" | "stop"> { - if (this._serializedPlanClaim) { - await this._serializedPlanClaim.catch(() => undefined); - return "waited"; - } - const planInFlight = this._serializedPlanInFlight; - if (!planInFlight) { - return "none"; - } - - let releaseClaim: () => void = () => {}; - const claim = new Promise((resolve) => { - releaseClaim = resolve; - }); - this._serializedPlanClaim = claim; - try { - const result = await planInFlight.catch(() => undefined); - if (this._serializedPlanInFlight === planInFlight) { - this._serializedPlanInFlight = undefined; - this._serializedExplicitRefineOptions = undefined; - } - return (await consume(result)) ? "stop" : "continue"; - } finally { - releaseClaim(); - if (this._serializedPlanClaim === claim) { - this._serializedPlanClaim = undefined; - } - } - } - - private async _applySerializedPlan( - bgResult: Extract, - ): Promise { - let resolveApplySettled: () => void = () => {}; - const applySettled = new Promise((resolve) => { - resolveApplySettled = resolve; - }); - this._refineInFlight = applySettled; - try { - await this._execution._applyRefine(bgResult.plan, bgResult.options, bgResult.abort, bgResult.source); - } finally { - resolveApplySettled(); - if (this._refineInFlight === applySettled) { - this._refineInFlight = undefined; - } - this._host.notifyCheckpoints(); - this._host.scheduleInputPump(); - } - } - - private _maybeStartSerializedBackgroundPlan(): void { - if (!this._serializedRefine || this._host.isDisposed() || this._host.isDisposing()) { - return; - } - // Don't start if a plan is already in flight. - if (this._serializedPlanInFlight || this._refineInFlight || this._refinePlanInFlight) { - return; - } - - // Start background planning for a pending agent-callable - // refine.run request, so its plan is ready at the shouldStopAfterTurn - // boundary. The pending request is consumed (cleared) here so the - // boundary doesn't re-plan it. Explicit refine.run skips the review gate. - const pending = this._pendingRequestedRefine; - if (pending) { - this._pendingRequestedRefine = undefined; - this._serializedExplicitRefineOptions = pending; - const refineAbort = new AbortController(); - this._refineAbortController = refineAbort; - const branchVersion = this._auto.branchVersion; - this._serializedPlanInFlight = this._runBackgroundPlan(pending, refineAbort, branchVersion, true); - return; - } - - // Interval-triggered auto-refine background planning. - if (!this._autoRefineAllowedForSession()) { - return; - } - const settings = this._host.settingsManager.getAutoRefineSettings(); - if (!settings.enabled) { - return; - } - if (this._auto.turnsSinceReview < settings.turnInterval) { - return; - } - const nowMs = Date.now(); - const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; - if (underCooldown) { - return; - } - - const refineAbort = new AbortController(); - this._refineAbortController = refineAbort; - const branchVersion = this._auto.branchVersion; - // Pass empty options — _runBackgroundPlan derives instructions from - // the review result for interval-triggered auto-refine. - this._serializedPlanInFlight = this._runBackgroundPlan({}, refineAbort, branchVersion); - } - - private async _runBackgroundPlan( - options: { instructions?: string; rollbackId?: string; global?: boolean }, - refineAbort: AbortController, - branchVersion: number, - skipReview = false, - ): Promise { - try { - let planOptions = options; - if (!skipReview) { - // Interval-triggered: run the review gate first, then derive - // instructions from the review result (not prepopulated). - const review = await this._auto._reviewAutoRefine( - { - reason: "turn_interval", - turnsSinceLastReview: this._auto.turnsSinceReview, - }, - refineAbort.signal, - ); - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { - return { status: "invalidated", branchVersion }; - } - if (!review.shouldRefine) { - return { status: "skip" }; - } - planOptions = { - instructions: autoRefineInstructions("turn_interval", review), - }; - } - // For explicit refine.run (skipReview=true), plan directly with - // the user-provided options — no auto-review gate. - const plan = await this._execution._planRefine( - planOptions, - refineAbort.signal, - skipReview ? "manual" : "auto", - ); - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { - return { status: "invalidated", branchVersion }; - } - return { - status: "plan", - plan, - options: planOptions, - abort: refineAbort, - branchVersion, - source: skipReview ? "self" : "auto", - }; - } catch (error) { - if (this._host.isDisposed() || this._host.isDisposing() || branchVersion !== this._auto.branchVersion) { - return { status: "invalidated", branchVersion }; - } - if (error instanceof RefineSkippedError) { - return { status: "skip", explicit: skipReview }; - } - return { - status: "failure", - explicit: skipReview, - options, - branchVersion, - }; - } finally { - if (this._refineAbortController === refineAbort) { - this._refineAbortController = undefined; - } - } - } - - private async _runSerializedRefine( - options: { - instructions?: string; - rollbackId?: string; - global?: boolean; - }, - source: Exclude, - ): Promise { - if (this._host.isDisposed() || this._host.isDisposing()) { - return; - } - // Guard: serialize against concurrent _runSerializedRefine calls. - // _serializedPlanInFlight covers background planning; _refineInFlight - // covers the apply phase. Both must be settled before starting a new - // plan+apply cycle. - while (this._serializedPlanInFlight || this._refineInFlight || this._refinePlanInFlight) { - if (this._serializedPlanInFlight) { - await this._consumeSerializedBackgroundPlan(async () => false); - } else if (this._refineInFlight) { - await this._refineInFlight; - } else { - await this._refinePlanInFlight; - } - } - if (this._host.isDisposed() || this._host.isDisposing()) { - return; - } - - const refineAbort = new AbortController(); - this._refineAbortController = refineAbort; - - const planRun = this._execution._planRefine(options, refineAbort.signal, source === "auto" ? "auto" : "manual"); - const planSettled = planRun.then( - () => undefined, - () => undefined, - ); - this._refinePlanInFlight = planSettled; - let plan: RefinementPlan; - try { - plan = await planRun; - } catch (error) { - if (this._refineAbortController === refineAbort) { - this._refineAbortController = undefined; - } - this._host.scheduleInputPump(); - throw error; - } finally { - if (this._refinePlanInFlight === planSettled) { - this._refinePlanInFlight = undefined; - } - } - - if (this._host.isDisposed() || refineAbort.signal.aborted) { - if (this._refineAbortController === refineAbort) { - this._refineAbortController = undefined; - } - this._host.scheduleInputPump(); - return; - } - - // Do NOT call agent.waitForIdle() — we are at the quiescent boundary - // already (shouldStopAfterTurn). _applyRefine handles disconnect/reconnect internally. - let resolveApplySettled: () => void = () => {}; - const applySettled = new Promise((resolve) => { - resolveApplySettled = resolve; - }); - this._refineInFlight = applySettled; - try { - await this._execution._applyRefine(plan, options, refineAbort, source); - } finally { - resolveApplySettled(); - if (this._refineInFlight === applySettled) { - this._refineInFlight = undefined; - } - this._host.notifyCheckpoints(); - this._host.scheduleInputPump(); - } - } - - handleRefineHostRequest(type: string, payload: Record = {}): Record { - switch (type) { - case "refine.status": { - return { - pending: this._pendingRequestedRefine !== undefined, - in_flight: - this._refineInFlight !== undefined || - this._refinePlanInFlight !== undefined || - this._serializedPlanInFlight !== undefined, - }; - } - case "refine.run": { - const instructions = payload.instructions; - if (instructions !== undefined && typeof instructions !== "string") { - throw new Error("refine.run instructions must be a string when provided"); - } - const globalFlag = payload.global; - if (globalFlag !== undefined && typeof globalFlag !== "boolean") { - throw new Error("refine.run global must be a boolean when provided"); - } - if (!this._host.isStreaming()) { - return { - scheduled: false, - reason: "no active turn; refine can only be requested while a turn is running", - }; - } - const previous = this._pendingRequestedRefine ?? this._serializedExplicitRefineOptions; - this._pendingRequestedRefine = { - instructions: instructions ?? previous?.instructions, - global: globalFlag ?? previous?.global, - }; - // In serialized mode, kick off background planning immediately - // (the primary response ended at message_end, tools are active). - // This lets planning overlap tool execution rather than waiting - // for the shouldStopAfterTurn boundary. - if (this._serializedRefine) { - if (this._serializedPlanInFlight) { - this._auto.invalidatePlans(); - if (this._refineAbortController) { - this._refineAbortController.abort(); - } else { - this._serializedPlanInFlight = Promise.resolve({ - status: "invalidated", - branchVersion: this._auto.branchVersion, - }); - } - } else { - this._maybeStartSerializedBackgroundPlan(); - } - } - return { - scheduled: true, - note: "Refinement runs when the current turn ends; applied edits are appended to your context as a refinement notice and you resume automatically. Continue working normally.", - }; - } - default: - throw new Error(`unknown refine request type "${type}"`); - } - } - - async _drainPendingRefinementForDisposal(): Promise { - this._auto.cancelScheduled(); - await Promise.allSettled(this._auto.pendingOperations()); - this._auto.cancelScheduled(); - // Wait for in-flight refinement (including serialized background plan) to settle. - while (this._refineInFlight || this._refinePlanInFlight || this._serializedPlanInFlight) { - if (this._refineInFlight) { - await this._refineInFlight; - } else if (this._refinePlanInFlight) { - await this._refinePlanInFlight; - } else if (this._serializedPlanInFlight) { - // Await the background plan and apply a ready "plan" result before teardown. - await this._consumeSerializedBackgroundPlan(async (bgResult) => { - if (bgResult?.status === "plan" && bgResult.branchVersion === this._auto.branchVersion) { - try { - await this._applySerializedPlan(bgResult); - } catch (error) { - this._emitRefineFailed(error); - } - // Stamp cooldown and reset counter so the interval - // check below does not trigger a duplicate refine. - this._auto.stampCooldown(); - this._auto.resetTurns(); - } - // Preserve a consumed explicit request when its background plan failed, - // matching the turn-boundary recovery path. The pending drain below - // retries it once before disposal. - if ( - bgResult?.status === "failure" && - bgResult.explicit && - bgResult.branchVersion === this._auto.branchVersion && - !this._pendingRequestedRefine - ) { - this._pendingRequestedRefine = bgResult.options; - } - if (bgResult?.status === "skip" && bgResult.explicit) { - this._emitRefineFailed(new RefineSkippedError("Refinement skipped by extension")); - } - // For "skip" or "failure", stamp cooldown and reset counter - // so the interval check below does not trigger a duplicate - // terminal retry. - if ( - bgResult?.status === "skip" || - bgResult?.status === "failure" || - bgResult?.status === "invalidated" - ) { - this._auto.stampCooldown(); - this._auto.resetTurns(); - } - return false; - }); - } else { - await new Promise((resolve) => setTimeout(resolve, 0)); - } - } - // Drain an agent-callable refine.run request that was scheduled but - // not yet consumed. Use the direct serialized path (no waitForIdle) - // since the agent may still own activeRun at the final agent_end. - if (this._pendingRequestedRefine) { - const pending = this._pendingRequestedRefine; - this._pendingRequestedRefine = undefined; - try { - await this._runSerializedRefine(pending, "self"); - } catch { - // Best-effort drain; refinement errors must not block disposal. - } - // Stamp cooldown and reset counter so the interval check below - // does not trigger a duplicate refine after the explicit drain. - this._auto.stampCooldown(); - this._auto.resetTurns(); - } - // A serialized compaction can finish without another model turn. Drain its - // pending review here so disposal does not silently lose the trigger. - if (this._serializedRefine && this._auto.hasPendingCompact && this._autoRefineAllowedForSession()) { - const compactSettings = this._host.settingsManager.getAutoRefineSettings(); - if (!compactSettings.enabled || !compactSettings.compact) { - this._auto.discardCompact(); - } else { - const nowMs = Date.now(); - const underCooldown = - this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < compactSettings.cooldownMs; - this._auto.discardCompact(); - if (!underCooldown) { - try { - await this._auto._runSerializedAutoRefineReview("compact", this._auto.branchVersion); - } catch { - // Best-effort drain; refinement errors must not block disposal. - } - return; - } - } - } - - // If auto-refine is due but has not started yet, run it now so the - // refinement is persisted before disposal. Use the direct serialized - // path in serialized mode, or _maybeAutoRefine in interactive mode - // (where the agent is idle at this point). - if (this._host.isDisposed() || !this._autoRefineAllowedForSession()) { - return; - } - const settings = this._host.settingsManager.getAutoRefineSettings(); - if (!settings.enabled) { - return; - } - if (this._auto.turnsSinceReview < settings.turnInterval) { - return; - } - const nowMs = Date.now(); - const underCooldown = this._auto.lastReviewAt > 0 && nowMs - this._auto.lastReviewAt < settings.cooldownMs; - if (underCooldown) { - return; - } - if (this._serializedRefine) { - await this._runSerializedRefineCheckpoint(); - } else { - await this._auto._maybeAutoRefine("turn_interval"); - } - } - - _autoRefineAllowedForSession(): boolean { - return this._host.getDepth() === 0 && this._execution._localHarnessStateDir() !== undefined; - } - - async _invalidatePendingAutoRefineForBranchChange(): Promise { - this._auto.abortReview(); - this._auto._discardPendingAutoRefine({ cancelPostCompactionContinue: true }); - this._auto.resetTurns(); - // Increment branch version BEFORE aborting/awaiting the serialized plan. - // This invalidates the plan's branchVersion check at the boundary - // so even if the plan completes, the boundary will reject it - // (bgResult.branchVersion !== this._auto.branchVersion). - this._auto.invalidatePlans(); - // Abort the in-flight refine/bplan controller so any pending - // _planRefine or _reviewAutoRefine call settles via signal abort - // rather than hanging forever. - this._refineAbortController?.abort(); - if (this._serializedPlanInFlight) { - await this._consumeSerializedBackgroundPlan(async () => false); - } - while (this._refinePlanInFlight) { - await this._refinePlanInFlight; - } - await this._waitForRefineIdle(); - } - - _emitRefineFailed(error: unknown): void { - this._host.emit({ - type: "refine_failed", - error: error instanceof Error ? error.message : String(error), - }); - } - - _consumePendingRequestedRefine(): boolean { - const pending = this._pendingRequestedRefine; - if (!pending) return false; - this._pendingRequestedRefine = undefined; - void this._host.dispatchRefine(pending, { source: "self" }).catch((error) => this._emitRefineFailed(error)); - return true; - } - - async refine( - options: { - instructions?: string; - rollbackId?: string; - global?: boolean; - } = {}, - internal: { skipAbort?: boolean; trigger?: "manual" | "auto"; source?: RefinementSource } = {}, - ): Promise { - // Queued /refine executes from the session-input pump between turns; - // refine never aborts the agent (planning is backgrounded and the apply - // phase waits for quiescence), so skipAbort only asserts the pump's - // idle invariant instead of changing abort behavior. - if (internal.skipAbort && this._host.isStreaming()) { - throw new Error("Cannot refine without aborting while the agent is running."); - } - // Wait for any existing refine (both planning and application) before - // starting a new run. This serializes concurrent /refine calls so two - // planning phases cannot race into concurrent _applyRefine calls that - // overwrite harness state. - while (this._refineInFlight || this._refinePlanInFlight || this._serializedPlanInFlight) { - if (this._refineInFlight) { - await this._refineInFlight; - } else if (this._refinePlanInFlight) { - await this._refinePlanInFlight; - } else { - // A serialized background plan is in flight (started during an - // active turn at message_end). Wait for planning and for the active - // turn to settle so its normal checkpoint can consume the plan. - const serializedPlanInFlight = this._serializedPlanInFlight; - await serializedPlanInFlight; - if (this._refineInFlight || this._refinePlanInFlight) { - continue; - } - await this._host.waitForAgentIdle(); - // Aborted turns skip shouldStopAfterTurn. Drop their settled plan - // after idle so a later public refine cannot spin on it forever. - if (this._serializedPlanInFlight === serializedPlanInFlight) { - this._serializedPlanInFlight = undefined; - this._serializedExplicitRefineOptions = undefined; - } - } - } - - const refineAbort = new AbortController(); - this._refineAbortController = refineAbort; - - const planRun = this._execution._planRefine(options, refineAbort.signal, internal.trigger ?? "manual"); - const planSettled = planRun.then( - () => undefined, - () => undefined, - ); - this._refinePlanInFlight = planSettled; - let plan: RefinementPlan; - try { - plan = await planRun; - } catch (e) { - if (this._refineAbortController === refineAbort) { - this._refineAbortController = undefined; - } - this._host.scheduleInputPump(); - throw e; - } finally { - if (this._refinePlanInFlight === planSettled) { - this._refinePlanInFlight = undefined; - } - } - - // Block new turns before waiting for the current turn to finish. One shared - // settled promise covers the full transition and apply critical section. - let resolveApplySettled: () => void = () => {}; - const applySettled = new Promise((resolve) => { - resolveApplySettled = resolve; - }); - this._refineInFlight = applySettled; - try { - // Wait for the session to become quiescent before applying. Planning is - // allowed to overlap active user work, but application must not disconnect - // event handling until that work and its queued events have completed. - await this._host.waitForAgentIdle(); - while (true) { - const eventQueue = this._host.getEventQueue(); - const compactionOp = this._host.getCompactionOperation(); - const branchSummaryOp = this._host.getBranchSummaryOperation(); - await Promise.allSettled([ - eventQueue, - ...(compactionOp ? [compactionOp] : []), - ...(branchSummaryOp ? [branchSummaryOp] : []), - ]); - if ( - eventQueue === this._host.getEventQueue() && - compactionOp === this._host.getCompactionOperation() && - branchSummaryOp === this._host.getBranchSummaryOperation() - ) { - break; - } - } - if (this._host.isDisposed() || refineAbort.signal.aborted) { - throw new Error("Refinement cancelled because the session was disposed."); - } - return await this._execution._applyRefine( - plan, - options, - refineAbort, - internal.source ?? (internal.trigger === "auto" ? "auto" : "user"), - ); - } finally { - resolveApplySettled(); - if (this._refineInFlight === applySettled) { - this._refineInFlight = undefined; - } - this._host.notifyCheckpoints(); - this._host.scheduleInputPump(); - } - } - - async _waitForRefineIdle(): Promise { - while (this._refineInFlight) { - await this._refineInFlight; - } - } - - _discardPendingAutoRefine(options: { cancelPostCompactionContinue?: boolean } = {}): void { - this._auto._discardPendingAutoRefine(options); - } - - _scheduleAutoRefineAfterAgentEnd(): void { - this._auto._scheduleAutoRefineAfterAgentEnd(); - } - - _scheduleAutoRefineAfterCompaction(willContinueAfterCompaction: boolean): void { - this._auto._scheduleAutoRefineAfterCompaction(willContinueAfterCompaction); - } - - _localHarnessStateDir(): string | undefined { - return this._execution._localHarnessStateDir(); - } - - _loadMergedHarnessState(): HarnessState { - return this._execution._loadMergedHarnessState(); - } -} +// Compatibility exports; implementation lives with its session owner. +export { + type AutoRefineReviewer, + type AutoRefineReviewRequest, + RefineSkippedError, + type SerializedBackgroundPlanResult, + SessionRefinement, + type SessionRefinementHost, +} from "./controller.js"; diff --git a/packages/coding-agent/src/session/refinement/types.ts b/packages/coding-agent/src/session/refinement/types.ts new file mode 100644 index 0000000000..0ce5b13928 --- /dev/null +++ b/packages/coding-agent/src/session/refinement/types.ts @@ -0,0 +1,110 @@ +import type { ProviderRetryPolicy } from "../../core/provider-retry.js"; + +export const REFINEMENT_CUSTOM_TYPE = "prime-agent.refinement"; + +export const REFINE_SKILL_NAME = "refine"; + +export type RefinementKind = "prompt" | "memory" | "skill" | "subagent"; + +export type RefinementAction = "create" | "update" | "delete"; + +export type HarnessScope = "local" | "global"; + +export interface HarnessEntry { + id: string; + kind: RefinementKind; + title: string; + content: string; + path: string; + scope?: HarnessScope; + reference: Record; + arguments: Record; + metadata: Record; + source: string; + created_at: string; + updated_at: string; + version: number; +} + +export interface HarnessRefinementEvent { + id: string; + trigger: string; + changes: string[]; + evidence: string; + outcome: string; + created_at: string; +} + +export interface HarnessState { + schema: number; + entries: Record>; + refinements: HarnessRefinementEvent[]; +} + +export interface RefinementEdit { + action: RefinementAction; + kind: RefinementKind; + id?: string; + title?: string; + content?: string; + path?: string; + reference?: Record; + arguments?: Record; + metadata?: Record; + reason?: string; +} + +export interface RefinementProposal { + summary: string; + rationale: string; + edits: RefinementEdit[]; + expectedOutcome: string; +} + +export interface AppliedRefinementEdit extends RefinementEdit { + id: string; + before?: HarnessEntry; + after?: HarnessEntry; + applied: boolean; + error?: string; +} + +export interface RefinementResult { + id: string; + summary: string; + rationale: string; + expectedOutcome: string; + appliedEdits: AppliedRefinementEdit[]; + harnessStatePath: string; + rollbackOf?: string; + scope?: HarnessScope; +} + +export interface RefineOptions { + instructions?: string; + rollbackId?: string; + global?: boolean; + retry?: ProviderRetryPolicy; +} + +export type AutoRefineReason = "turn_interval" | "compact"; + +export interface AutoRefineReviewContext { + reason: AutoRefineReason; + turnsSinceLastReview: number; +} + +export interface AutoRefineReview { + shouldRefine: boolean; + rationale: string; + instructions?: string; +} + +export interface RefinementPlan { + proposal: RefinementProposal; + id: string; + rollbackOf?: string; + rollbackScope?: HarnessScope; + /** Target-scope state captured before planning, used to reject conflicting edits at apply time. */ + baselineState?: HarnessState; +} diff --git a/packages/coding-agent/src/session/tools/bash.ts b/packages/coding-agent/src/session/tools/bash.ts index 0252ab093b..3b20abd714 100644 --- a/packages/coding-agent/src/session/tools/bash.ts +++ b/packages/coding-agent/src/session/tools/bash.ts @@ -1,7 +1,7 @@ import { type BashResult, executeBashWithOperations } from "../../core/bash-executor.js"; import type { UserBashEvent, UserBashEventResult } from "../../core/extensions/types.js"; -import type { BashExecutionMessage } from "../../core/messages.js"; import { type BashOperations, createLocalBashOperations } from "../../core/tools/bash.js"; +import type { BashExecutionMessage } from "../context/messages.js"; export interface ExecuteBashOptions { excludeFromContext?: boolean; diff --git a/packages/coding-agent/src/session/tools/tools.ts b/packages/coding-agent/src/session/tools/tools.ts index 96fc146d1e..b1590870e3 100644 --- a/packages/coding-agent/src/session/tools/tools.ts +++ b/packages/coding-agent/src/session/tools/tools.ts @@ -10,10 +10,10 @@ import type { McpManager } from "../../core/mcp/mcp-manager.js"; import type { ResourceLoader } from "../../core/resource-loader.js"; import type { Skill } from "../../core/skills.js"; import { createSyntheticSourceInfo, type SourceInfo } from "../../core/source-info.js"; -import { type BuildSystemPromptOptions, buildSystemPrompt } from "../../core/system-prompt.js"; import { acpMcpToolNames, createAcpMcpToolDefinitions } from "../../core/tools/acp-mcp.js"; import type { IpythonKernelProvisioner } from "../../core/tools/ipython.js"; import { createToolDefinitionFromAgentTool } from "../../core/tools/tool-definition-wrapper.js"; +import { type BuildSystemPromptOptions, buildSystemPrompt } from "../context/system-prompt.js"; interface ToolDefinitionEntry { definition: ToolDefinition; diff --git a/packages/coding-agent/src/session/turns/autonomous-continuation.ts b/packages/coding-agent/src/session/turns/autonomous-continuation.ts index 6915ceec8d..4fe0c53cd1 100644 --- a/packages/coding-agent/src/session/turns/autonomous-continuation.ts +++ b/packages/coding-agent/src/session/turns/autonomous-continuation.ts @@ -1,387 +1 @@ -import type { Agent, AgentMessage } from "@earendil-works/pi-agent-core"; -import type { AssistantMessage, Usage } from "@earendil-works/pi-ai"; -import type { AgentSessionEvent } from "../../core/agent-session.js"; -import { - type AgentAutonomousConfig, - type AgentAutonomousStatus, - type AutonomousRuntimeState, - addAutonomousContinuation, - addAutonomousUsage, - autonomousStatus, - createAutonomousRuntimeState, - isUnlimitedAutonomousLimit, - nextAutonomousContinuation, - refreshAutonomousQualityGates, - setAutonomousEnabled, - setAutonomousLimits, - UNLIMITED_AUTONOMOUS_LIMIT, -} from "../../core/autonomous.js"; -import type { CustomMessage } from "../../core/messages.js"; -import { parseCommandArgs } from "../../core/prompt-templates.js"; -import type { SessionManager } from "../../core/session-manager.js"; -import { parseSessionSlashCommand } from "../../core/slash-commands.js"; -import type { SessionCompaction } from "../compaction/compaction.js"; -import type { SessionInputAdmission } from "../input/input-admission.js"; -import { createPreparedTurnAction, primaryDeliveryRecord, type QueuedSessionAction } from "../prepared-actions.js"; -import type { SessionContinuation } from "./continuation.js"; - -type AutonomousSlashCommand = { kind: "status" } | { kind: "on"; config?: AgentAutonomousConfig } | { kind: "off" }; -type AutonomousRuntimeSnapshot = Pick< - AutonomousRuntimeState, - "continuationsUsed" | "gateAttempts" | "lastGateFailure" | "lastGateFailureSnapshot" ->; - -const AUTONOMOUS_STATUS_NUMBER_FORMAT = new Intl.NumberFormat("en-US"); - -const AUTONOMOUS_BUDGET_USAGE = - "Usage: /autonomous [status|off] or /autonomous on [--max-continuations ] [--max-turns ] [--max-tokens ] [--timeout-ms ] [--gate ] [--gate-retries ] [--gate-timeout-ms ]"; - -// `/autonomous` budget flags mirror the `--autonomous-*` CLI options. The CLI -// spelling (`--autonomous-max-continuations`) is accepted as an alias so the -// exact CLI budget flags also work from the slash command. -const AUTONOMOUS_BUDGET_FLAGS: ReadonlySet = new Set([ - "max-continuations", - "max-turns", - "max-tokens", - "timeout-ms", - "gate", - "gate-retries", - "gate-timeout-ms", -]); - -function parseAutonomousBudgetInt(flag: string, value: string, allowUnlimited = false): number { - if (allowUnlimited && value.toLowerCase() === "unlimited") { - return UNLIMITED_AUTONOMOUS_LIMIT; - } - // Commas and underscores are accepted as digit separators (100,000,000). - const digits = value.replace(/[,_]/g, ""); - if (!/^[1-9]\d*$/.test(digits)) { - throw new Error( - `--${flag} must be a positive integer${allowUnlimited ? ' or "unlimited"' : ""}. ${AUTONOMOUS_BUDGET_USAGE}`, - ); - } - return Number(digits); -} - -function parseAutonomousBudgetOptions(tokens: string[]): AgentAutonomousConfig { - const config: AgentAutonomousConfig = {}; - const gateCommands: string[] = []; - for (let i = 0; i < tokens.length; i++) { - const token = tokens[i]!; - if (!token.startsWith("--")) { - throw new Error(`Unexpected autonomous argument: ${token}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - const equalsIndex = token.indexOf("="); - const rawFlag = equalsIndex === -1 ? token : token.slice(0, equalsIndex); - const inlineValue = equalsIndex === -1 ? undefined : token.slice(equalsIndex + 1); - const flag = rawFlag.startsWith("--autonomous-") ? rawFlag.slice("--autonomous-".length) : rawFlag.slice(2); - if (!AUTONOMOUS_BUDGET_FLAGS.has(flag)) { - throw new Error(`Unknown autonomous budget flag: ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - let value = inlineValue; - if (value === undefined) { - const next = tokens[i + 1]; - if (next === undefined || next.startsWith("--")) { - throw new Error(`Missing value for ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - value = next; - i++; - } - if (value === "") { - throw new Error(`Missing value for ${rawFlag}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - switch (flag) { - case "gate": - gateCommands.push(value); - break; - case "gate-retries": - config.gates = config.gates ?? {}; - config.gates.maxRetries = parseAutonomousBudgetInt(flag, value); - break; - case "gate-timeout-ms": - config.gates = config.gates ?? {}; - config.gates.timeoutMs = parseAutonomousBudgetInt(flag, value); - break; - case "max-continuations": - config.maxContinuations = parseAutonomousBudgetInt(flag, value, true); - break; - case "max-turns": - config.maxTurns = parseAutonomousBudgetInt(flag, value, true); - break; - case "max-tokens": - config.maxTokens = parseAutonomousBudgetInt(flag, value, true); - break; - case "timeout-ms": - config.timeoutMs = parseAutonomousBudgetInt(flag, value, true); - break; - } - } - if (gateCommands.length > 0) { - config.gates = { ...config.gates, commands: gateCommands }; - } - // Named budget flags define the whole budget: any limit the user did not - // name stops cutting the run short. With no budget flags at all, the - // configured or default limits still apply. - if ( - config.maxContinuations !== undefined || - config.maxTurns !== undefined || - config.maxTokens !== undefined || - config.timeoutMs !== undefined - ) { - config.maxContinuations ??= UNLIMITED_AUTONOMOUS_LIMIT; - config.maxTurns ??= UNLIMITED_AUTONOMOUS_LIMIT; - config.maxTokens ??= UNLIMITED_AUTONOMOUS_LIMIT; - config.timeoutMs ??= UNLIMITED_AUTONOMOUS_LIMIT; - } - return config; -} - -export interface SessionAutonomousContinuationHost { - getStatus(): AgentAutonomousStatus; - getCwd(): string; - getAgent(): Pick; - getStore(): Pick; - emit(event: AgentSessionEvent): void; - getContinuation(): Pick; - getArrivalEpoch(): number; - admit: SessionInputAdmission["admitSessionInput"]; - cancelActions(predicate: (action: QueuedSessionAction) => boolean, error: Error): QueuedSessionAction[]; - emitQueueUpdate(): void; - getCompaction(): Pick; - getUnfinishedActionCount(): number; - cancelContinuation(): void; -} -export class SessionAutonomousContinuation { - private readonly state: AutonomousRuntimeState; - private suppressionDepth = 0; - private readonly suppressedMessages = new WeakSet(); - private readonly thresholdContinuations = new WeakMap(); - private readonly snapshots = new WeakMap(); - private pendingThresholdMessages: AgentMessage[] = []; - constructor( - config: AgentAutonomousConfig | undefined, - private readonly host: SessionAutonomousContinuationHost, - ) { - this.state = createAutonomousRuntimeState(config, { cwd: host.getCwd() }); - } - forgetSnapshot(message: AgentMessage): void { - this.snapshots.delete(message); - } - - takePendingThresholdMessages(): AgentMessage[] { - return this.pendingThresholdMessages.splice(0); - } - - recordUsage(usage: Usage): void { - addAutonomousUsage(this.state, usage); - } - isSuppressed(messages: AgentMessage[]): boolean { - return this.suppressionDepth > 0 || messages.some((message) => this.suppressedMessages.has(message)); - } - next(message: AssistantMessage, signal?: AbortSignal): Promise { - return nextAutonomousContinuation(this.state, message, { cwd: this.host.getCwd(), signal }); - } - - parseAutonomousSlashCommand(text: string): AutonomousSlashCommand | undefined { - const command = parseSessionSlashCommand(text); - if (command?.name !== "autonomous") return undefined; - const tokens = parseCommandArgs(command.args); - if (tokens.length === 0 || tokens[0]!.toLowerCase() === "status") { - if (tokens.length > 1) { - throw new Error(`Unexpected autonomous argument: ${tokens[1]}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - return { kind: "status" }; - } - const subcommand = tokens[0]!.toLowerCase(); - if (subcommand === "on" || subcommand === "enable" || subcommand === "enabled") { - return { kind: "on", config: parseAutonomousBudgetOptions(tokens.slice(1)) }; - } - if (subcommand === "off" || subcommand === "disable" || subcommand === "disabled") { - if (tokens.length > 1) { - throw new Error(`Unexpected autonomous argument: ${tokens[1]}. ${AUTONOMOUS_BUDGET_USAGE}`); - } - return { kind: "off" }; - } - throw new Error(AUTONOMOUS_BUDGET_USAGE); - } - - formatAutonomousStatus(): string { - const status = this.host.getStatus(); - const state = status.enabled ? "on" : "off"; - const elapsedSeconds = status.startedAt ? Math.round((Date.now() - status.startedAt) / 1000) : 0; - const gateSummary = - status.gates.commands.length > 0 ? status.gates.commands.map((command) => `"${command}"`).join(", ") : "none"; - const formatCount = (value: number): string => - isUnlimitedAutonomousLimit(value) ? "unlimited" : AUTONOMOUS_STATUS_NUMBER_FORMAT.format(value); - const timeBudget = isUnlimitedAutonomousLimit(status.limits.timeoutMs) - ? "unlimited" - : `${AUTONOMOUS_STATUS_NUMBER_FORMAT.format(Math.round(status.limits.timeoutMs / 1000))}s`; - return `[autonomous-status: ${state}]\n\nContinuations: ${formatCount(status.continuationsUsed)}/${formatCount(status.limits.maxContinuations)}. Turns: ${formatCount(status.turnsUsed)}/${formatCount(status.limits.maxTurns)}. Tokens: ${formatCount(status.tokensUsed)}/${formatCount(status.limits.maxTokens)}. Time: ${elapsedSeconds}s/${timeBudget}. Gates: ${gateSummary}.`; - } - - emitAutonomousStatus(): void { - const message = { - role: "custom" as const, - customType: "autonomous_status", - content: this.formatAutonomousStatus(), - display: true, - details: this.host.getStatus(), - timestamp: Date.now(), - } satisfies CustomMessage; - this.host.getAgent().state.messages.push(message); - this.host - .getStore() - .appendCustomMessageEntry(message.customType, message.content, message.display, message.details); - this.host.emit({ type: "message_start", message }); - this.host.emit({ type: "message_end", message }); - } - - async handleAutonomousSlashCommand(text: string): Promise { - const command = this.parseAutonomousSlashCommand(text); - if (!command) { - return false; - } - if (command.kind === "on") { - setAutonomousEnabled(this.state, true, { cwd: this.host.getCwd() }); - setAutonomousLimits(this.state, command.config); - } else if (command.kind === "off") { - setAutonomousEnabled(this.state, false); - this.clearQueuedAutonomousContinuations(); - } - this.emitAutonomousStatus(); - return true; - } - - snapshotAutonomousRuntimeState(): AutonomousRuntimeSnapshot { - return { - continuationsUsed: this.state.continuationsUsed, - gateAttempts: { ...this.state.gateAttempts }, - lastGateFailure: this.state.lastGateFailure ? { ...this.state.lastGateFailure } : undefined, - lastGateFailureSnapshot: this.state.lastGateFailureSnapshot - ? { ...this.state.lastGateFailureSnapshot } - : undefined, - }; - } - - restoreAutonomousRuntimeSnapshot(snapshot: AutonomousRuntimeSnapshot): void { - this.state.continuationsUsed = snapshot.continuationsUsed; - this.state.gateAttempts = { ...snapshot.gateAttempts }; - this.state.lastGateFailure = snapshot.lastGateFailure ? { ...snapshot.lastGateFailure } : undefined; - this.state.lastGateFailureSnapshot = snapshot.lastGateFailureSnapshot - ? { ...snapshot.lastGateFailureSnapshot } - : undefined; - } - - async queueAutonomousContinuationForThresholdCompaction( - message: AssistantMessage, - ): Promise { - const queuedMessage = this.thresholdContinuations.get(message); - if (queuedMessage && this.host.getContinuation().messages.includes(queuedMessage)) { - return queuedMessage; - } - const snapshot = this.snapshotAutonomousRuntimeState(); - const arrivalEpoch = this.host.getArrivalEpoch(); - const autonomousMessage = await nextAutonomousContinuation(this.state, message, { - cwd: this.host.getCwd(), - signal: this.host.getAgent().signal, - }); - if (!autonomousMessage) { - return undefined; - } - if (this.host.getArrivalEpoch() !== arrivalEpoch) { - this.restoreAutonomousRuntimeSnapshot(snapshot); - return undefined; - } - this.thresholdContinuations.set(message, autonomousMessage); - this.snapshots.set(autonomousMessage, snapshot); - this.host.getContinuation().track(autonomousMessage); - this.pendingThresholdMessages.push(autonomousMessage); - const text = - typeof autonomousMessage.content === "string" - ? autonomousMessage.content - : autonomousMessage.content.map((block) => (block.type === "text" ? block.text : "")).join("\n"); - this.host.admit( - createPreparedTurnAction("followUp", text, undefined, { - message: autonomousMessage, - }), - ); - return autonomousMessage; - } - - clearQueuedAutonomousContinuations( - options: { restoreAutonomousState?: boolean; messages?: AgentMessage[] } = {}, - ): void { - const requestedMessages = options.messages ?? [...this.host.getContinuation().messages]; - const requestedMessageSet = new Set(requestedMessages); - const queuedMessages = this.host.getContinuation().messages.filter((message) => requestedMessageSet.has(message)); - if (queuedMessages.length === 0) { - return; - } - const queuedMessageSet = new Set(queuedMessages); - this.host.getContinuation().remove(queuedMessageSet); - this.host.getAgent().removeQueuedMessages((message) => queuedMessageSet.has(message)); - this.host.cancelActions( - (action) => action.payload.kind === "turn" && queuedMessageSet.has(primaryDeliveryRecord(action).message), - new Error("Queued autonomous continuation was cleared before delivery."), - ); - this.host.emitQueueUpdate(); - if (options.restoreAutonomousState) { - for (const queuedMessage of queuedMessages) { - const snapshot = this.snapshots.get(queuedMessage); - if (snapshot) { - this.restoreAutonomousRuntimeSnapshot(snapshot); - break; - } - } - } - for (const queuedMessage of queuedMessages) { - this.snapshots.delete(queuedMessage); - } - this.pendingThresholdMessages = this.pendingThresholdMessages.filter((message) => !queuedMessageSet.has(message)); - if (options.messages === undefined) { - this.host.getCompaction().resetContinuation(); - } - if (!this.host.getAgent().hasQueuedMessages() && this.host.getUnfinishedActionCount() === 0) { - this.host.cancelContinuation(); - } - } - - clearQueuedAutonomousContinuationsAfterSkippedThresholdCompaction( - shouldContinueAfterThreshold: boolean, - queuedMessages: AgentMessage[], - ): void { - if (shouldContinueAfterThreshold) { - this.clearQueuedAutonomousContinuations({ - restoreAutonomousState: true, - messages: queuedMessages, - }); - } - } - - getAutonomousStatus(): AgentAutonomousStatus { - return autonomousStatus(this.state); - } - - recordHostAutonomousContinuation(): void { - addAutonomousContinuation(this.state); - } - - async refreshAutonomousGates(): Promise { - await refreshAutonomousQualityGates(this.state, { - cwd: this.host.getCwd(), - }); - } - - async runWithAutonomousContinuationSuppressed(fn: () => Promise): Promise { - this.suppressionDepth++; - try { - return await fn(); - } finally { - this.suppressionDepth--; - } - } - - markAutonomousContinuationSuppressed(message: AgentMessage): void { - this.suppressedMessages.add(message); - } -} +export { SessionAutonomousContinuation, type SessionAutonomousContinuationHost } from "../autonomy/continuation.js"; diff --git a/packages/coding-agent/src/session/turns/command-execution.ts b/packages/coding-agent/src/session/turns/command-execution.ts index 399507811c..80550f59fd 100644 --- a/packages/coding-agent/src/session/turns/command-execution.ts +++ b/packages/coding-agent/src/session/turns/command-execution.ts @@ -1,26 +1,26 @@ import type { Agent } from "@earendil-works/pi-agent-core"; import type { ImageContent } from "@earendil-works/pi-ai"; -import type { AgentSessionEvent } from "../../core/agent-session.js"; -import type { CompactionResult } from "../../core/compaction/index.js"; +import type { SessionManager } from "../../core/session-manager.js"; +import { parseRefineCommandOptions, type SessionSlashCommand } from "../../core/slash-commands.js"; +import type { AgentSessionEvent } from "../agent-session.js"; +import { CompactionSkippedError } from "../compaction/execution.js"; +import type { CompactionResult } from "../compaction/types.js"; import { type CustomMessage, createSessionSlashCommandMessage, createSessionSlashCommandResultMessage, -} from "../../core/messages.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; +} from "../context/messages.js"; +import type { GoalState } from "../goals/contracts.js"; import { type ActionStore, canSelectSessionAction, type RuntimeActivity, transitionSessionAction, -} from "../../core/session-action-store.js"; -import type { SessionManager } from "../../core/session-manager.js"; -import { parseRefineCommandOptions, type SessionSlashCommand } from "../../core/slash-commands.js"; -import { CompactionSkippedError } from "../compaction/compaction-execution.js"; -import type { GoalState } from "../goals/contracts.js"; +} from "../input/action-store.js"; import type { SessionCommitFence, SessionCommitLease } from "../input/commit-fence.js"; -import type { QueuedSessionAction } from "../prepared-actions.js"; -import type { SessionRefinement } from "../refinement/refinement.js"; +import type { QueuedSessionAction } from "../input/prepared-actions.js"; +import type { SessionRefinement } from "../refinement/controller.js"; +import type { RefinementResult } from "../refinement/types.js"; function asError(error: unknown): Error { return error instanceof Error ? error : new Error(String(error)); diff --git a/packages/coding-agent/src/session/turns/events.ts b/packages/coding-agent/src/session/turns/events.ts index 89dd609eca..f86145a74a 100644 --- a/packages/coding-agent/src/session/turns/events.ts +++ b/packages/coding-agent/src/session/turns/events.ts @@ -14,18 +14,14 @@ import type { TurnStartEvent, } from "../../core/extensions/index.js"; import type { KernelSentAgentMessage } from "../../core/kernel/index.js"; -import type { RefinementResult } from "../../core/refinement/index.js"; -import { - type ActionStore, - type SessionActionSnapshot, - transitionSessionAction, -} from "../../core/session-action-store.js"; import type { SessionManager } from "../../core/session-manager.js"; import type { RlmChildAgentSnapshot } from "../children/child-types.js"; -import type { SessionCompaction, SessionCompactionEvent } from "../compaction/compaction.js"; +import type { SessionCompaction, SessionCompactionEvent } from "../compaction/controller.js"; import type { GoalState } from "../goals/contracts.js"; -import { primaryDeliveryRecord, type QueuedSessionAction } from "../prepared-actions.js"; -import type { SessionRefinement } from "../refinement/refinement.js"; +import { type ActionStore, type SessionActionSnapshot, transitionSessionAction } from "../input/action-store.js"; +import { primaryDeliveryRecord, type QueuedSessionAction } from "../input/prepared-actions.js"; +import type { SessionRefinement } from "../refinement/controller.js"; +import type { RefinementResult } from "../refinement/types.js"; import type { SessionBashEvent } from "../tools/bash.js"; import type { SessionRetry, SessionRetryEvent } from "./retry.js"; diff --git a/packages/coding-agent/src/session/turns/turn-execution.ts b/packages/coding-agent/src/session/turns/turn-execution.ts index d5e0bcaf82..87a85eeab8 100644 --- a/packages/coding-agent/src/session/turns/turn-execution.ts +++ b/packages/coding-agent/src/session/turns/turn-execution.ts @@ -1,8 +1,8 @@ import type { Agent, AgentMessage } from "@earendil-works/pi-agent-core"; import type { ExtensionRunner } from "../../core/extensions/index.js"; -import { type CustomMessage, createHarnessDigestMessage, HARNESS_DIGEST_CUSTOM_TYPE } from "../../core/messages.js"; -import { type SessionAction, transitionSessionAction } from "../../core/session-action-store.js"; -import type { BuildSystemPromptOptions } from "../../core/system-prompt.js"; +import { type CustomMessage, createHarnessDigestMessage, HARNESS_DIGEST_CUSTOM_TYPE } from "../context/messages.js"; +import type { BuildSystemPromptOptions } from "../context/system-prompt.js"; +import { type SessionAction, transitionSessionAction } from "../input/action-store.js"; import type { SessionCommitFence, SessionCommitLease } from "../input/commit-fence.js"; import { createDeliveryRecord, @@ -11,7 +11,7 @@ import { type PreparedTurnPayload, primaryDeliveryRecord, type QueuedSessionAction, -} from "../prepared-actions.js"; +} from "../input/prepared-actions.js"; import type { TurnPreparer } from "./turn-preparation.js"; export interface SessionTurnExecutionHost { diff --git a/packages/coding-agent/src/session/turns/turn-policy.ts b/packages/coding-agent/src/session/turns/turn-policy.ts index 0f75c2a756..8b3bccb077 100644 --- a/packages/coding-agent/src/session/turns/turn-policy.ts +++ b/packages/coding-agent/src/session/turns/turn-policy.ts @@ -4,13 +4,13 @@ import type { ShouldStopAfterTurnContext, } from "@earendil-works/pi-agent-core"; import type { Api, AssistantMessage, Model } from "@earendil-works/pi-ai"; -import { shouldCompact } from "../../core/compaction/index.js"; import { getLatestCompactionEntry, type SessionManager } from "../../core/session-manager.js"; import type { SettingsManager } from "../../core/settings-manager.js"; -import type { SessionCompaction } from "../compaction/compaction.js"; +import type { SessionAutonomousContinuation } from "../autonomy/continuation.js"; +import type { SessionCompaction } from "../compaction/controller.js"; +import { shouldCompact } from "../compaction/summary.js"; import type { GoalController } from "../goals/controller.js"; -import type { SessionRefinement } from "../refinement/refinement.js"; -import type { SessionAutonomousContinuation } from "./autonomous-continuation.js"; +import type { SessionRefinement } from "../refinement/controller.js"; export interface SessionTurnPolicyHost { steeringStopPending(): boolean; diff --git a/packages/coding-agent/test/session/context-compatibility.test.ts b/packages/coding-agent/test/session/context-compatibility.test.ts new file mode 100644 index 0000000000..162df6d0fe --- /dev/null +++ b/packages/coding-agent/test/session/context-compatibility.test.ts @@ -0,0 +1,274 @@ +import { describe, expect, it } from "vitest"; +import * as CoreCompactionBranchSummarization from "../../src/core/compaction/branch-summarization.js"; +import * as CoreCompactionCompaction from "../../src/core/compaction/compaction.js"; +import * as CoreCompactionIndex from "../../src/core/compaction/index.js"; +import * as CoreCompactionUtils from "../../src/core/compaction/utils.js"; +import * as CoreContextTree from "../../src/core/context-tree.js"; +import * as CoreMessages from "../../src/core/messages.js"; +import * as CorePromptsIndex from "../../src/core/prompts/index.js"; +import * as CorePromptsRlm from "../../src/core/prompts/rlm.js"; +import * as CoreRefinementIndex from "../../src/core/refinement/index.js"; +import * as CoreRefinementRefinement from "../../src/core/refinement/refinement.js"; +import * as CoreSessionStats from "../../src/core/session-stats.js"; +import * as CoreSystemPrompt from "../../src/core/system-prompt.js"; +import * as CoreUsage from "../../src/core/usage.js"; +import * as SessionCompactionCompaction from "../../src/session/compaction/compaction.js"; +import * as SessionCompactionCompactionExecution from "../../src/session/compaction/compaction-execution.js"; +import * as SessionCompactionController from "../../src/session/compaction/controller.js"; +import * as SessionCompactionExecution from "../../src/session/compaction/execution.js"; +import * as SessionCompactionSummary from "../../src/session/compaction/summary.js"; +import * as SessionCompactionTypes from "../../src/session/compaction/types.js"; +import * as SessionContextBranchSummary from "../../src/session/context/branch-summary.js"; +import * as SessionContextContextTree from "../../src/session/context/context-tree.js"; +import * as SessionContextConversationText from "../../src/session/context/conversation-text.js"; +import * as SessionContextFileTracking from "../../src/session/context/file-tracking.js"; +import * as SessionContextMessages from "../../src/session/context/messages.js"; +import * as SessionContextPromptsIndex from "../../src/session/context/prompts/index.js"; +import * as SessionContextPromptsRlm from "../../src/session/context/prompts/rlm.js"; +import * as SessionContextSystemPrompt from "../../src/session/context/system-prompt.js"; +import * as SessionContextTokenEstimate from "../../src/session/context/token-estimate.js"; +import * as SessionContextUsage from "../../src/session/context/usage.js"; +import * as SessionRefinementAutoRefinement from "../../src/session/refinement/auto-refinement.js"; +import * as SessionRefinementAutomatic from "../../src/session/refinement/automatic.js"; +import * as SessionRefinementController from "../../src/session/refinement/controller.js"; +import * as SessionRefinementExecution from "../../src/session/refinement/execution.js"; +import * as SessionRefinementFormat from "../../src/session/refinement/format.js"; +import * as SessionRefinementHarnessState from "../../src/session/refinement/harness-state.js"; +import * as SessionRefinementPlanning from "../../src/session/refinement/planning.js"; +import * as SessionRefinementRefinement from "../../src/session/refinement/refinement.js"; +import * as SessionRefinementRefinementExecution from "../../src/session/refinement/refinement-execution.js"; +import * as SessionRefinementTypes from "../../src/session/refinement/types.js"; + +function assertExports(actual: Record, expected: Record): void { + expect(Object.keys(actual).sort()).toEqual(Object.keys(expected).sort()); + for (const [name, value] of Object.entries(expected)) { + expect(actual[name], name).toBe(value); + } +} + +describe("context ownership compatibility exports", () => { + it("preserves core/compaction/compaction.ts", () => { + assertExports(CoreCompactionCompaction, { + COMPACT_SKILL_NAME: SessionCompactionTypes.COMPACT_SKILL_NAME, + DEFAULT_COMPACTION_SETTINGS: SessionCompactionTypes.DEFAULT_COMPACTION_SETTINGS, + calculateContextTokens: SessionContextTokenEstimate.calculateContextTokens, + getLastAssistantUsage: SessionContextTokenEstimate.getLastAssistantUsage, + estimateContextTokens: SessionContextTokenEstimate.estimateContextTokens, + shouldCompact: SessionCompactionSummary.shouldCompact, + estimateTokens: SessionContextTokenEstimate.estimateTokens, + findTurnStartIndex: SessionCompactionSummary.findTurnStartIndex, + findCutPoint: SessionCompactionSummary.findCutPoint, + buildSummarizationPrompt: SessionCompactionSummary.buildSummarizationPrompt, + generateSummary: SessionCompactionSummary.generateSummary, + prepareCompaction: SessionCompactionSummary.prepareCompaction, + compact: SessionCompactionSummary.compact, + }); + }); + it("preserves core/compaction/utils.ts", () => { + assertExports(CoreCompactionUtils, { + createFileOps: SessionContextFileTracking.createFileOps, + extractFileOpsFromMessage: SessionContextFileTracking.extractFileOpsFromMessage, + computeFileLists: SessionContextFileTracking.computeFileLists, + formatFileOperations: SessionContextFileTracking.formatFileOperations, + serializeConversation: SessionContextConversationText.serializeConversation, + SUMMARIZATION_SYSTEM_PROMPT: SessionContextConversationText.SUMMARIZATION_SYSTEM_PROMPT, + }); + }); + it("preserves core/refinement/refinement.ts", () => { + assertExports(CoreRefinementRefinement, { + REFINEMENT_CUSTOM_TYPE: SessionRefinementTypes.REFINEMENT_CUSTOM_TYPE, + REFINE_SKILL_NAME: SessionRefinementTypes.REFINE_SKILL_NAME, + inferRefinementResultScope: SessionRefinementHarnessState.inferRefinementResultScope, + getGlobalHarnessStateDir: SessionRefinementHarnessState.getGlobalHarnessStateDir, + getLocalHarnessStateDir: SessionRefinementHarnessState.getLocalHarnessStateDir, + getHarnessStatePath: SessionRefinementHarnessState.getHarnessStatePath, + loadHarnessState: SessionRefinementHarnessState.loadHarnessState, + mergeHarnessStates: SessionRefinementHarnessState.mergeHarnessStates, + saveHarnessState: SessionRefinementHarnessState.saveHarnessState, + getRefinementHistoryPath: SessionRefinementHarnessState.getRefinementHistoryPath, + appendGlobalRefinement: SessionRefinementHarnessState.appendGlobalRefinement, + loadGlobalRefinementHistory: SessionRefinementHarnessState.loadGlobalRefinementHistory, + mergeRefinementHistory: SessionRefinementHarnessState.mergeRefinementHistory, + formatRefinementNoticeBody: SessionRefinementFormat.formatRefinementNoticeBody, + formatHarnessStateForPrompt: SessionRefinementFormat.formatHarnessStateForPrompt, + normalizeRefinementProposal: SessionRefinementHarnessState.normalizeRefinementProposal, + applyRefinementProposal: SessionRefinementHarnessState.applyRefinementProposal, + getRefinementHistory: SessionRefinementHarnessState.getRefinementHistory, + generateRefinementId: SessionRefinementHarnessState.generateRefinementId, + planRefinement: SessionRefinementPlanning.planRefinement, + reviewAutoRefine: SessionRefinementPlanning.reviewAutoRefine, + refineHarness: SessionRefinementPlanning.refineHarness, + }); + }); + it("preserves core/compaction/branch-summarization.ts", () => { + assertExports(CoreCompactionBranchSummarization, { + collectEntriesForBranchSummary: SessionContextBranchSummary.collectEntriesForBranchSummary, + prepareBranchEntries: SessionContextBranchSummary.prepareBranchEntries, + generateBranchSummary: SessionContextBranchSummary.generateBranchSummary, + }); + }); + it("preserves core/messages.ts", () => { + assertExports(CoreMessages, { + COMPACTION_SUMMARY_PREFIX: SessionContextMessages.COMPACTION_SUMMARY_PREFIX, + COMPACTION_SUMMARY_SUFFIX: SessionContextMessages.COMPACTION_SUMMARY_SUFFIX, + BRANCH_SUMMARY_PREFIX: SessionContextMessages.BRANCH_SUMMARY_PREFIX, + BRANCH_SUMMARY_SUFFIX: SessionContextMessages.BRANCH_SUMMARY_SUFFIX, + HEARTBEAT_PROMPT_CUSTOM_TYPE: SessionContextMessages.HEARTBEAT_PROMPT_CUSTOM_TYPE, + HEARTBEAT_PROMPT_PREVIEW_LABEL: SessionContextMessages.HEARTBEAT_PROMPT_PREVIEW_LABEL, + IPYTHON_STATE_RESTORED_CUSTOM_TYPE: SessionContextMessages.IPYTHON_STATE_RESTORED_CUSTOM_TYPE, + SESSION_SLASH_COMMAND_CUSTOM_TYPE: SessionContextMessages.SESSION_SLASH_COMMAND_CUSTOM_TYPE, + SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE: SessionContextMessages.SESSION_SLASH_COMMAND_RESULT_CUSTOM_TYPE, + COMPACTION_OUTCOME_CUSTOM_TYPE: SessionContextMessages.COMPACTION_OUTCOME_CUSTOM_TYPE, + REFINEMENT_OUTCOME_CUSTOM_TYPE: SessionContextMessages.REFINEMENT_OUTCOME_CUSTOM_TYPE, + REFINEMENT_NOTICE_CUSTOM_TYPE: SessionContextMessages.REFINEMENT_NOTICE_CUSTOM_TYPE, + HARNESS_DIGEST_CUSTOM_TYPE: SessionContextMessages.HARNESS_DIGEST_CUSTOM_TYPE, + RLM_CHILD_FAILURE_CUSTOM_TYPE: SessionContextMessages.RLM_CHILD_FAILURE_CUSTOM_TYPE, + RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE: SessionContextMessages.RLM_CHILD_TERMINAL_NOTICE_CUSTOM_TYPE, + ASYNC_BASH_COMPLETION_CUSTOM_TYPE: SessionContextMessages.ASYNC_BASH_COMPLETION_CUSTOM_TYPE, + ASYNC_BASH_COMPLETION_PREVIEW_LABEL: SessionContextMessages.ASYNC_BASH_COMPLETION_PREVIEW_LABEL, + sanitizeMessageHeaderValue: SessionContextMessages.sanitizeMessageHeaderValue, + HARNESS_DIGEST_PREFIX: SessionContextMessages.HARNESS_DIGEST_PREFIX, + HARNESS_DIGEST_SUFFIX: SessionContextMessages.HARNESS_DIGEST_SUFFIX, + createHarnessDigestMessage: SessionContextMessages.createHarnessDigestMessage, + createAsyncBashCompletionMessage: SessionContextMessages.createAsyncBashCompletionMessage, + createRlmChildFailureMessage: SessionContextMessages.createRlmChildFailureMessage, + createRlmChildTerminalNoticeMessage: SessionContextMessages.createRlmChildTerminalNoticeMessage, + bashOutputToText: SessionContextMessages.bashOutputToText, + bashExecutionToText: SessionContextMessages.bashExecutionToText, + createBranchSummaryMessage: SessionContextMessages.createBranchSummaryMessage, + createCompactionSummaryMessage: SessionContextMessages.createCompactionSummaryMessage, + createCustomMessage: SessionContextMessages.createCustomMessage, + createSessionSlashCommandMessage: SessionContextMessages.createSessionSlashCommandMessage, + createSessionSlashCommandResultMessage: SessionContextMessages.createSessionSlashCommandResultMessage, + createCompactionOutcomeMessage: SessionContextMessages.createCompactionOutcomeMessage, + createRefinementOutcomeMessage: SessionContextMessages.createRefinementOutcomeMessage, + createRefinementNoticeMessage: SessionContextMessages.createRefinementNoticeMessage, + isSessionSlashCommand: SessionContextMessages.isSessionSlashCommand, + isSessionSlashCommandMessage: SessionContextMessages.isSessionSlashCommandMessage, + isSessionSlashCommandResultMessage: SessionContextMessages.isSessionSlashCommandResultMessage, + isCompactionOutcomeMessage: SessionContextMessages.isCompactionOutcomeMessage, + isRefinementOutcomeMessage: SessionContextMessages.isRefinementOutcomeMessage, + createHeartbeatPromptMessage: SessionContextMessages.createHeartbeatPromptMessage, + convertToLlm: SessionContextMessages.convertToLlm, + }); + }); + it("preserves core/context-tree.ts", () => { + assertExports(CoreContextTree, { + computeOwnAndTotalUsage: SessionContextContextTree.computeOwnAndTotalUsage, + loadContextTreeChildFromDisk: SessionContextContextTree.loadContextTreeChildFromDisk, + loadContextTreeChildrenFromDisk: SessionContextContextTree.loadContextTreeChildrenFromDisk, + }); + }); + it("preserves core/usage.ts", () => { + assertExports(CoreUsage, { + sessionUsageSummaryFrom: SessionContextUsage.sessionUsageSummaryFrom, + emptyUsage: SessionContextUsage.emptyUsage, + addAssistantUsage: SessionContextUsage.addAssistantUsage, + subtractAssistantUsage: SessionContextUsage.subtractAssistantUsage, + cloneUsage: SessionContextUsage.cloneUsage, + }); + }); + it("preserves core/session-stats.ts", () => { + assertExports(CoreSessionStats, {}); + }); + it("preserves core/system-prompt.ts", () => { + assertExports(CoreSystemPrompt, { + buildSystemPrompt: SessionContextSystemPrompt.buildSystemPrompt, + }); + }); + it("preserves core/prompts/index.ts", () => { + assertExports(CorePromptsIndex, { + buildChildAgentDoctrine: SessionContextPromptsIndex.buildChildAgentDoctrine, + buildRlmPrompt: SessionContextPromptsIndex.buildRlmPrompt, + buildSubagentGuidance: SessionContextPromptsIndex.buildSubagentGuidance, + }); + }); + it("preserves core/prompts/rlm.ts", () => { + assertExports(CorePromptsRlm, { + buildChildAgentDoctrine: SessionContextPromptsRlm.buildChildAgentDoctrine, + buildRlmPrompt: SessionContextPromptsRlm.buildRlmPrompt, + buildSubagentGuidance: SessionContextPromptsRlm.buildSubagentGuidance, + }); + }); + it("preserves session/compaction/compaction.ts", () => { + assertExports(SessionCompactionCompaction, { + SessionCompaction: SessionCompactionController.SessionCompaction, + }); + }); + it("preserves session/compaction/compaction-execution.ts", () => { + assertExports(SessionCompactionCompactionExecution, { + CompactionSkippedError: SessionCompactionExecution.CompactionSkippedError, + performSessionCompaction: SessionCompactionExecution.performSessionCompaction, + }); + }); + it("preserves session/refinement/refinement.ts", () => { + assertExports(SessionRefinementRefinement, { + SessionRefinement: SessionRefinementController.SessionRefinement, + RefineSkippedError: SessionRefinementController.RefineSkippedError, + }); + }); + it("preserves session/refinement/auto-refinement.ts", () => { + assertExports(SessionRefinementAutoRefinement, { + autoRefineInstructions: SessionRefinementAutomatic.autoRefineInstructions, + AutoRefinement: SessionRefinementAutomatic.AutoRefinement, + }); + }); + it("preserves session/refinement/refinement-execution.ts", () => { + assertExports(SessionRefinementRefinementExecution, { + RefineSkippedError: SessionRefinementExecution.RefineSkippedError, + RefinementExecution: SessionRefinementExecution.RefinementExecution, + }); + }); + it("preserves core/compaction/index.ts", () => { + assertExports(CoreCompactionIndex, { + COMPACT_SKILL_NAME: SessionCompactionTypes.COMPACT_SKILL_NAME, + DEFAULT_COMPACTION_SETTINGS: SessionCompactionTypes.DEFAULT_COMPACTION_SETTINGS, + calculateContextTokens: SessionContextTokenEstimate.calculateContextTokens, + getLastAssistantUsage: SessionContextTokenEstimate.getLastAssistantUsage, + estimateContextTokens: SessionContextTokenEstimate.estimateContextTokens, + shouldCompact: SessionCompactionSummary.shouldCompact, + estimateTokens: SessionContextTokenEstimate.estimateTokens, + findTurnStartIndex: SessionCompactionSummary.findTurnStartIndex, + findCutPoint: SessionCompactionSummary.findCutPoint, + buildSummarizationPrompt: SessionCompactionSummary.buildSummarizationPrompt, + generateSummary: SessionCompactionSummary.generateSummary, + prepareCompaction: SessionCompactionSummary.prepareCompaction, + compact: SessionCompactionSummary.compact, + createFileOps: SessionContextFileTracking.createFileOps, + extractFileOpsFromMessage: SessionContextFileTracking.extractFileOpsFromMessage, + computeFileLists: SessionContextFileTracking.computeFileLists, + formatFileOperations: SessionContextFileTracking.formatFileOperations, + serializeConversation: SessionContextConversationText.serializeConversation, + SUMMARIZATION_SYSTEM_PROMPT: SessionContextConversationText.SUMMARIZATION_SYSTEM_PROMPT, + collectEntriesForBranchSummary: SessionContextBranchSummary.collectEntriesForBranchSummary, + prepareBranchEntries: SessionContextBranchSummary.prepareBranchEntries, + generateBranchSummary: SessionContextBranchSummary.generateBranchSummary, + }); + }); + it("preserves core/refinement/index.ts", () => { + assertExports(CoreRefinementIndex, { + REFINEMENT_CUSTOM_TYPE: SessionRefinementTypes.REFINEMENT_CUSTOM_TYPE, + REFINE_SKILL_NAME: SessionRefinementTypes.REFINE_SKILL_NAME, + inferRefinementResultScope: SessionRefinementHarnessState.inferRefinementResultScope, + getGlobalHarnessStateDir: SessionRefinementHarnessState.getGlobalHarnessStateDir, + getLocalHarnessStateDir: SessionRefinementHarnessState.getLocalHarnessStateDir, + getHarnessStatePath: SessionRefinementHarnessState.getHarnessStatePath, + loadHarnessState: SessionRefinementHarnessState.loadHarnessState, + mergeHarnessStates: SessionRefinementHarnessState.mergeHarnessStates, + saveHarnessState: SessionRefinementHarnessState.saveHarnessState, + getRefinementHistoryPath: SessionRefinementHarnessState.getRefinementHistoryPath, + appendGlobalRefinement: SessionRefinementHarnessState.appendGlobalRefinement, + loadGlobalRefinementHistory: SessionRefinementHarnessState.loadGlobalRefinementHistory, + mergeRefinementHistory: SessionRefinementHarnessState.mergeRefinementHistory, + formatRefinementNoticeBody: SessionRefinementFormat.formatRefinementNoticeBody, + formatHarnessStateForPrompt: SessionRefinementFormat.formatHarnessStateForPrompt, + normalizeRefinementProposal: SessionRefinementHarnessState.normalizeRefinementProposal, + applyRefinementProposal: SessionRefinementHarnessState.applyRefinementProposal, + getRefinementHistory: SessionRefinementHarnessState.getRefinementHistory, + generateRefinementId: SessionRefinementHarnessState.generateRefinementId, + planRefinement: SessionRefinementPlanning.planRefinement, + reviewAutoRefine: SessionRefinementPlanning.reviewAutoRefine, + refineHarness: SessionRefinementPlanning.refineHarness, + }); + }); +}); diff --git a/packages/coding-agent/test/session/input-ownership.test.ts b/packages/coding-agent/test/session/input-ownership.test.ts new file mode 100644 index 0000000000..461f2ffcfd --- /dev/null +++ b/packages/coding-agent/test/session/input-ownership.test.ts @@ -0,0 +1,44 @@ +import { describe, expect, it } from "vitest"; +import * as legacySession from "../../src/core/agent-session.js"; +import * as legacyAutonomy from "../../src/core/autonomous.js"; +import * as legacyAdmission from "../../src/core/prompt-admission.js"; +import * as legacyRlm from "../../src/core/rlm-runtime.js"; +import * as legacyActions from "../../src/core/session-action-store.js"; +import * as residency from "../../src/modes/daemon/workers/residency-policy.js"; +import * as session from "../../src/session/agent-session.js"; +import * as autonomy from "../../src/session/autonomy/autonomous.js"; +import * as autonomousContinuation from "../../src/session/autonomy/continuation.js"; +import * as childRequests from "../../src/session/children/host-requests.js"; +import * as spawnOptions from "../../src/session/children/spawn-options.js"; +import * as actions from "../../src/session/input/action-store.js"; +import * as bashRequests from "../../src/session/input/bash-host-requests.js"; +import * as prepared from "../../src/session/input/prepared-actions.js"; +import * as admission from "../../src/session/input/prompt-admission.js"; +import * as modelSearch from "../../src/session/models/model-search.js"; +import * as legacyPrepared from "../../src/session/prepared-actions.js"; +import * as legacyAutonomousContinuation from "../../src/session/turns/autonomous-continuation.js"; + +describe("session ownership compatibility", () => { + it.each([ + ["session facade", legacySession, session], + ["autonomy", legacyAutonomy, autonomy], + ["autonomous continuation", legacyAutonomousContinuation, autonomousContinuation], + ["prompt admission", legacyAdmission, admission], + ["prepared actions", legacyPrepared, prepared], + ["actions and daemon residency", legacyActions, { ...actions, ...residency }], + ["RLM request adapters", legacyRlm, { ...childRequests, ...spawnOptions, ...bashRequests, ...modelSearch }], + ])("retains identical runtime exports from the former %s path", (_name, legacy, canonical) => { + expect(Object.keys(legacy).sort()).toEqual(Object.keys(canonical).sort()); + for (const [name, value] of Object.entries(legacy)) { + expect(value, name).toBe((canonical as Record)[name]); + } + }); + + it("shares cancellation class identity across paths", async () => { + const abort = new AbortController(); + abort.abort(); + await expect(admission.waitForPromptAdmission(Promise.resolve(), abort.signal)).rejects.toBeInstanceOf( + legacyAdmission.PromptAdmissionCancelledError, + ); + }); +}); diff --git a/packages/coding-agent/test/session/rlm-host-requests.test.ts b/packages/coding-agent/test/session/rlm-host-requests.test.ts new file mode 100644 index 0000000000..04846f3055 --- /dev/null +++ b/packages/coding-agent/test/session/rlm-host-requests.test.ts @@ -0,0 +1,130 @@ +import { getModel } from "@earendil-works/pi-ai"; +import { describe, expect, it, vi } from "vitest"; +import { + createRlmDeleteSubagentHostHandler, + createRlmRunHostHandler, +} from "../../src/session/children/host-requests.js"; +import type { RlmRunRequest, RlmSubagentRegistryEntry } from "../../src/session/children/runtime-contracts.js"; +import { + createAsyncBashCompletionHostHandler, + createAsyncBashConsumedHostHandler, +} from "../../src/session/input/bash-host-requests.js"; +import { + createRlmFindModelsHostHandler, + DEFAULT_RLM_MODEL_SEARCH_LIMIT, + findRlmModelMatches, + MAX_RLM_MODEL_SEARCH_LIMIT, +} from "../../src/session/models/model-search.js"; + +describe("child host request ownership", () => { + it("retains validation, kwargs identity, spawning source, and result identity", async () => { + const result = { rlm_child_id: "child-1" }; + const run = vi.fn(async (_request: RlmRunRequest) => result); + const handler = createRlmRunHostHandler(run); + await expect(handler({ prompt: null })).rejects.toThrow("rlm.spawn prompt must be a string"); + expect(run).not.toHaveBeenCalled(); + const kwargs = { name: "worker", model: "provider/model" }; + expect(await handler({ prompt: "work", kwargs, cellSourceCode: "spawn()" })).toBe(result); + expect(run).toHaveBeenLastCalledWith({ prompt: "work", kwargs, cellSourceCode: "spawn()" }); + expect(run.mock.calls[0]?.[0]?.kwargs).toBe(kwargs); + await handler({ prompt: "", kwargs: [], cellSourceCode: 123 }); + expect(run).toHaveBeenLastCalledWith({ prompt: "", kwargs: {}, cellSourceCode: undefined }); + }); + + it("retains optional deletion outcomes and validates before dispatch", async () => { + const subagent: RlmSubagentRegistryEntry = { + rlm_child_id: "child", + active_session_id: null, + session_id: "saved", + session_name: "worker", + session_dir: "/tmp/worker", + status: "completed", + }; + const remove = vi.fn(async (_target: string) => ({ subagent })); + const handler = createRlmDeleteSubagentHostHandler(remove); + await expect(handler({ target: " " })).rejects.toThrow("rlm.delete_subagent target must be a non-empty string"); + expect(remove).not.toHaveBeenCalled(); + expect(await handler({ target: " worker " })).toEqual({ subagent }); + expect(remove).toHaveBeenCalledExactlyOnceWith("worker"); + expect( + await createRlmDeleteSubagentHostHandler(async () => ({ subagent, outcome: "skipped_running" }))({ + target: "worker", + }), + ).toEqual({ subagent, outcome: "skipped_running" }); + }); +}); + +describe("bash input host request ownership", () => { + it.each([ + [{ pid: 0, command: "echo", exitCode: 0 }, "pid"], + [{ pid: 1.5, command: "echo", exitCode: 0 }, "pid"], + [{ pid: 1, command: "", exitCode: 0 }, "command"], + [{ pid: 1, command: "echo", exitCode: 0.5 }, "exitCode"], + ])("rejects invalid completion data before notifying input", async (payload, field) => { + const notify = vi.fn(); + await expect(createAsyncBashCompletionHostHandler(notify)(payload)).rejects.toThrow(`bash.completed ${field}`); + expect(notify).not.toHaveBeenCalled(); + }); + + it("awaits completion and propagates consumption callback failures", async () => { + let finish!: () => void; + const complete = vi.fn( + () => + new Promise((resolve) => { + finish = resolve; + }), + ); + const payload = { pid: 12, command: "echo", exitCode: -1 }; + let settled = false; + const pending = Promise.resolve(createAsyncBashCompletionHostHandler(complete)(payload)).then((result) => { + settled = true; + return result; + }); + await Promise.resolve(); + expect(settled).toBe(false); + finish(); + await expect(pending).resolves.toEqual({}); + expect(complete).toHaveBeenCalledExactlyOnceWith(payload); + const failure = new Error("withdraw failed"); + const consume = vi.fn(async () => { + throw failure; + }); + const handler = createAsyncBashConsumedHostHandler(consume); + await expect(handler({ pid: -1, command: "echo" })).rejects.toThrow("bash.consumed pid"); + await expect(handler({ pid: 12, command: "" })).rejects.toThrow("bash.consumed command"); + await expect(handler(payload)).rejects.toBe(failure); + expect(consume).toHaveBeenCalledExactlyOnceWith({ pid: 12, command: "echo" }); + }); +}); + +describe("model search ownership", () => { + it("retains bounds, defaults, query identity, and result identity", async () => { + const models = [{ provider: "p", id: "m", name: "model", selector: "p/m" }]; + const search = vi.fn(() => ({ models })); + const handler = createRlmFindModelsHostHandler(search); + await expect(handler({ query: 1 })).rejects.toThrow("query must be a string"); + for (const limit of [0, 1.5, MAX_RLM_MODEL_SEARCH_LIMIT + 1, null]) + await expect(handler({ query: "", limit })).rejects.toThrow("limit must be an integer"); + expect(search).not.toHaveBeenCalled(); + expect((await handler({ query: " Model " })).models).toBe(models); + expect(search).toHaveBeenCalledExactlyOnceWith(" Model ", DEFAULT_RLM_MODEL_SEARCH_LIMIT); + }); + + it("ranks normalized exact, prefix, and partial matches and resolves ties by selector", () => { + const base = getModel("anthropic", "claude-sonnet-4-5"); + const models = [ + { ...base, provider: "z", id: "prefix-code", name: "Prefix" }, + { ...base, provider: "p", id: "code-next", name: "Next" }, + { ...base, provider: "p", id: "code", name: "Exact" }, + { ...base, provider: "a", id: "prefix-code", name: "Prefix" }, + ]; + expect(findRlmModelMatches(" C-O_D E ", models, 4).map((m) => m.selector)).toEqual([ + "p/code", + "p/code-next", + "a/prefix-code", + "z/prefix-code", + ]); + expect(findRlmModelMatches("code", models, 1).map((m) => m.selector)).toEqual(["p/code"]); + expect(findRlmModelMatches("missing", models, 8)).toEqual([]); + }); +}); diff --git a/packages/coding-agent/test/suite/regressions/2151-wait-for-idle-spin.test.ts b/packages/coding-agent/test/suite/regressions/2151-wait-for-idle-spin.test.ts new file mode 100644 index 0000000000..803dc9dd76 --- /dev/null +++ b/packages/coding-agent/test/suite/regressions/2151-wait-for-idle-spin.test.ts @@ -0,0 +1,48 @@ +import { setImmediate as yieldToEventLoop } from "node:timers/promises"; +import { fauxAssistantMessage } from "@earendil-works/pi-ai"; +import { expect, it } from "vitest"; +import { createHarness, getAssistantTexts } from "../harness.js"; + +it("parks idle waits while bash blocks the input pump and drains later prompts in order", async () => { + const harness = await createHarness(); + let releaseBash = () => {}; + const bashGate = new Promise((resolve) => { + releaseBash = resolve; + }); + const session = harness.session as unknown as { _scheduleSessionInputPump(): void }; + const schedule = session._scheduleSessionInputPump.bind(session); + const bash = harness.session.executeBash("held bash", undefined, { + transient: true, + operations: { + exec: async () => { + await bashGate; + return { exitCode: 0 }; + }, + }, + }); + try { + expect(harness.session.isBashRunning).toBe(true); + harness.setResponses([fauxAssistantMessage("first done"), fauxAssistantMessage("second done")]); + const first = harness.session.prompt("first"); + await yieldToEventLoop(); + let schedules = 0; + session._scheduleSessionInputPump = () => { + schedules++; + // Release even under the regression so microtask starvation cannot hang the test runner. + if (schedules === 200) releaseBash(); + schedule(); + }; + const idle = harness.session.waitForIdle(); + await yieldToEventLoop(); + const second = harness.session.prompt("second"); + releaseBash(); + await Promise.all([idle, bash, first, second]); + expect(schedules).toBeLessThan(200); + expect(getAssistantTexts(harness)).toEqual(["first done", "second done"]); + } finally { + session._scheduleSessionInputPump = schedule; + releaseBash(); + await bash; + harness.cleanup(); + } +});