Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/user-guide/en/token-saving/tokenless/QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,12 +120,26 @@ anolisa adapter scan
| Qoder | `anolisa adapter enable tokenless qoder` |
| Claude Code | `anolisa adapter enable tokenless claude-code` |
| Codex | `anolisa adapter enable tokenless codex` |
| DeepSeek Harness (dsh) | `anolisa adapter enable tokenless dsh --profile <profile>` |
| OpenCode | Lifecycle script (see below) |
| Qwen Code | `anolisa adapter enable tokenless qwencode` |

Restart the Agent CLI or IDE after setting it up. OpenClaw also requires
`openclaw gateway restart`; if its security check rejects the plugin, follow
the [OpenClaw integration instructions](framework-integration.md#2-enable-one-adapter).
For DeepSeek Harness, `<profile>` is required and must match the name used by
`dsh --profile <profile>`; restart that profile after enabling the bundle.
To enable more than one profile, repeat `--profile` in the same command:

```bash
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
```

Every later enable or re-enable replaces the entire recorded profile set.
Include every profile that should retain Tokenless each time.

OpenCode is not registered with `anolisa adapter enable` in this release; use
the bundled lifecycle script described in the
[OpenCode integration instructions](framework-integration.md#opencode).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Python framework package that application developers install and register explic
| Qoder | `qoder` | Hard-disabled | Emits rewritten shell input | Emits `additionalContext` | Attempted after response compression | — |
| Claude Code | `claude-code` | Hard-disabled | Replaces Bash input | Replaces output on 2.1.121 or later; otherwise passes through | Used only when the replacement can remain text | — |
| Codex | `codex` | Hard-disabled | Replaces supported shell input | Keeps the original and adds analysis or a compressed alternative | Used to build that alternative | — |
| DeepSeek Harness | `dsh` | — | — | Replaces an accepted single-text JSON result when the replacement is smaller | — | — |
| OpenCode | `opencode` | Hard-disabled | Replaces Bash input | Replaces tool output | Attempted after response compression | ✅ |
| Qwen Code | `qwencode` | Hard-disabled | Emits rewritten shell input | Emits `additionalContext` | Attempted after response compression | ✅ |

Expand All @@ -42,6 +43,83 @@ The shared response hook, OpenClaw, and Hermes skip inputs shorter than 200 char

Claude Code requires version 2.1.121 or later for `updatedToolOutput`. On older or unknown versions, response compression is disabled to avoid duplicating the original. Structured tool outputs preserve their host schema and do not switch to textual TOON; JSON carried as a string can use TOON when it is smaller.

### DeepSeek Harness native processing

The DSH bundle requires Node.js 22 or later and a compatible DSH profile. Pass
all desired profile names in the same enable command, then start DSH with one
of those names:

```bash
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
dsh --profile web
```

`--profile` is required and repeatable. Each enable or re-enable treats its
arguments as the complete desired profile set. It removes the bundle from any
profile recorded by the prior receipt but omitted from the new command, so
always include every profile that should retain Tokenless. ANOLISA records the
selected profiles and their resolved DSH home in the adapter receipt, so later
status, disable, and re-enable operations continue to address the same profile
tree.

The plugin runs on DSH's `tools/post-execute` waterfall. It attempts
`tokenless compress-response` only for a successful result containing one text
block whose text is a JSON object or array. It replaces the content only when
the CLI returns valid JSON that is strictly shorter. Multiple blocks, images,
plain text, invalid JSON, errored results, Code Mode child executions, and the
default content-retrieval tools are not compressed. A missing, failing, or
timed-out CLI also preserves the original content. This native path does not
run the TOON second stage and has no pre-spawn minimum-size gate.

Add an override for the installed row to
`$DSH_HOME/profiles/<profile>/cordis.patch.yml`, then restart that DSH profile:

```yaml
- id: anolisa-tokenless
config:
responseCompressionEnabled: true
timeoutMs: 5000
maxBuffer: 4194304
noStash: false
```

Later DSH patch layers replace the row's complete `config` value. The plugin
supplies defaults for omitted keys, so the override may contain only the keys
that need to differ.

| Option | Default | Behavior |
|--------|---------|----------|
| `responseCompressionEnabled` | `true` | Enables response compression. Setting it to `false` does not disable environment-error attribution. |
| `tokenlessBin` | `$TOKENLESS_BIN`, then `tokenless` | Selects the Tokenless CLI executable. A non-empty plugin value takes precedence over the environment variable. |
| `skipTools` | Content-retrieval set below | Skips compression for matching tool names. A configured array replaces the default set; an empty array skips none. Attribution remains active. |
| `shellTools` | Shell/process set below | Selects shell thresholds and the tools whose structured `value` may be interpreted for failure attribution. A configured array replaces the default set. |
| `truncateStringsAt` | Shell `65536`; other `1048576` | Overrides the maximum retained string length for every tool class. Only a positive integer is accepted. |
| `truncateArraysAt` | Shell `128`; other `65536` | Overrides the maximum retained array length for every tool class. Only a positive integer is accepted. |
| `maxDepth` | Shell `8`; other `32` | Overrides maximum JSON depth for every tool class. Only a positive integer is accepted. |
| `timeoutMs` | `3000` | Bounds one Tokenless child process in milliseconds. Only a positive integer is accepted. |
| `maxBuffer` | `2097152` | Bounds captured child-process output in bytes. Only a positive integer is accepted. |
| `agentId` | `dsh` | Sets the `--agent-id` recorded by Tokenless statistics. |
| `noStash` | `false` | Passes `--no-stash` when `true`; dropped array items are otherwise eligible for Stash storage. |

The default `skipTools` set is `Read`, `read`, `read_file`, `read_many_files`,
`Glob`, `glob`, `search_file`, `list_directory`, `list_dir`, `Grep`, `grep`,
`grep_code`, `grep_search`, `search_files`, `Lsp`, `lsp`, `NotebookRead`,
`notebook_read`, and `notebookread`.

The default `shellTools` set is `Bash`, `bash`, `Shell`, `shell`, `exec`,
`terminal`, `run_shell_command`, `run_in_terminal`, `get_terminal_output`,
`execute_command`, and `process`.

Raw DSH failures marked with `isError` may receive dependency, permission,
path, network, or package attribution for any tool. Structured output is
classified only for `shellTools`. Attribution is independent of compression,
so it remains active when compression is disabled, skipped, or produces no
smaller result. When a later waterfall listener replaces the canonical
`value`, Tokenless classifies that replacement and does not carry attribution
from the superseded result.

## Manage adapters with anolisa (recommended)

These commands require an ANOLISA component record. If Tokenless was installed
Expand Down Expand Up @@ -83,9 +161,19 @@ anolisa adapter enable tokenless qoder
anolisa adapter enable tokenless claude-code
anolisa adapter enable tokenless codex
anolisa adapter enable tokenless qwencode
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
```

Enable only Agent products that you use. When enabling more than one, run and verify each command separately.
Enable only Agent products that you use. Run and verify each product's command
separately. For DSH, include every desired profile in its single enable
command.

DeepSeek Harness is profile-scoped and therefore requires at least one
`--profile`. Each name must match one passed to `dsh --profile <profile>`; the
generic command without a profile is rejected. A later enable or re-enable
must repeat every profile that should remain registered.

OpenCode uses its bundled install script under
[Manual integration after npm installation](#manual-integration-after-npm-installation).
Expand Down Expand Up @@ -196,6 +284,14 @@ The marketplace plugin takes effect after restarting Claude Code. The install sc

The plugin loads in a new Codex session. Close the old session and start a new one before verifying statistics. Its PostToolUse hook is additive: use statistics as candidate-compression telemetry, not as proof that the original Codex tool output left the prompt.

### DeepSeek Harness

The native bundle loads when the selected DSH profile starts. After enabling
or changing its profile patch, restart `dsh --profile <profile>`, run a tool
that returns compressible JSON, and inspect `tokenless stats list`. Disable the
adapter with `anolisa adapter disable tokenless dsh`; the receipt already
records the profile names, so disable does not accept another `--profile`.

### OpenCode

OpenCode discovers global local plugins at startup. Use the bundled Tokenless lifecycle script described above, restart OpenCode after installation or removal, then run a tool call and inspect `tokenless stats list`. The script resolves the configuration directory from `TOKENLESS_OPENCODE_CONFIG_DIR`, then `OPENCODE_CONFIG_DIR`, then `XDG_CONFIG_HOME/opencode`, and finally `~/.config/opencode`. Installation creates only `plugins/tokenless.js` as a managed symlink and refuses to replace an unrelated file at that path.
Expand Down
17 changes: 16 additions & 1 deletion docs/user-guide/zh/token-saving/tokenless/QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,27 @@ anolisa adapter scan
| Qoder | `anolisa adapter enable tokenless qoder` |
| Claude Code | `anolisa adapter enable tokenless claude-code` |
| Codex | `anolisa adapter enable tokenless codex` |
| DeepSeek Harness(dsh) | `anolisa adapter enable tokenless dsh --profile <profile>` |
| OpenCode | 生命周期脚本(见下文) |
| Qwen Code | `anolisa adapter enable tokenless qwencode` |

接入后重启对应的 Agent CLI 或 IDE。OpenClaw 还需要运行
`openclaw gateway restart`;如果安全检查拒绝 Plugin,请按照
[OpenClaw 接入说明](framework-integration.md#2-启用一个-adapter)处理。本版本尚未将
[OpenClaw 接入说明](framework-integration.md#2-启用一个-adapter)处理。
DeepSeek Harness 必须提供 `<profile>`,并与 `dsh --profile <profile>` 使用的名称
保持一致。启用 Bundle 后应重启这个 profile。需要启用多个 profile 时,应在同一条
命令中重复传入 `--profile`。

```bash
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
```

后续每次 enable 或 re-enable 都会替换 receipt 记录的完整 profile 集合。每次都要
列出需要继续使用 Tokenless 的全部 profile。

本版本尚未将
OpenCode 注册到 `anolisa adapter enable`;请使用
[OpenCode 接入说明](framework-integration.md#opencode)中的随附生命周期脚本。

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Python 框架包。
| Qoder | `qoder` | 已硬关闭 | 输出改写后的 Shell 输入 | 输出 `additionalContext` | 在响应压缩后尝试 | — |
| Claude Code | `claude-code` | 已硬关闭 | 替换 Bash 输入 | 2.1.121 及以上替换输出;否则透传 | 仅在替换结果可保持文本时使用 | — |
| Codex | `codex` | 已硬关闭 | 替换受支持的 Shell 输入 | 保留原文,追加分析或压缩备选内容 | 用于生成该备选内容 | — |
| DeepSeek Harness | `dsh` | 未注册 | 未注册 | 只在结果更小时替换已接受的单文本块 JSON 结果 | 未注册 | 未注册 |
| OpenCode | `opencode` | 已硬关闭 | 替换 Bash 输入 | 替换工具输出 | 在响应压缩后尝试 | ✅ |
| Qwen Code | `qwencode` | 已硬关闭 | 输出改写后的 Shell 输入 | 输出 `additionalContext` | 在响应压缩后尝试 | ✅ |

Expand All @@ -42,6 +43,74 @@ OpenCode 当前使用下文说明的随附生命周期脚本,本版本尚未

Claude Code 需要 2.1.121 或更高版本才能使用 `updatedToolOutput`。版本更旧或无法确定时,响应压缩会关闭,以免重复注入原文。结构化工具输出会保留宿主 Schema,不会转换成文本 TOON;以字符串承载的 JSON 在 TOON 更小时可以使用 TOON。

### DeepSeek Harness 原生处理路径

DSH Bundle 要求 Node.js 22 或更高版本,并需要兼容的 DSH profile。应在同一条
enable 命令中列出全部目标 profile,随后使用其中一个名称启动 DSH。

```bash
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
dsh --profile web
```

`--profile` 是必填且可重复的参数。每次 enable 或 re-enable 都会把本次参数视为
完整目标集合。旧 receipt 中已有但新命令没有列出的 profile 会卸载 Bundle,因此
每次都要列出需要继续使用 Tokenless 的全部 profile。ANOLISA 会把选择的 profile
和解析后的 DSH home 写入 adapter receipt。后续 status、disable 和 re-enable 会
继续操作同一棵 profile 目录树。

Plugin 在 DSH 的 `tools/post-execute` waterfall 上运行。只有成功结果包含一个文本块,
且文本是 JSON object 或 array 时,才会尝试执行 `tokenless compress-response`。
CLI 返回更短的合法 JSON 后才会替换内容。多文本块、图片、普通文本、非法 JSON、
错误结果、Code Mode 子调用和默认内容读取类工具不参与压缩。CLI 缺失、失败或
超时也会保留原始内容。当前原生路径不执行 TOON 第二阶段,也没有启动子进程前的
最小尺寸门控。

在 `$DSH_HOME/profiles/<profile>/cordis.patch.yml` 中覆盖安装后的 row,然后重启
对应的 DSH profile。

```yaml
- id: anolisa-tokenless
config:
responseCompressionEnabled: true
timeoutMs: 5000
maxBuffer: 4194304
noStash: false
```

后续 DSH patch layer 会替换该 row 的完整 `config` 值。Plugin 会为省略的 key 提供
默认值,因此只需写出准备修改的 key。

| 配置项 | 默认值 | 行为 |
|--------|--------|------|
| `responseCompressionEnabled` | `true` | 控制响应压缩。设为 `false` 后,环境错误归因仍保持启用。 |
| `tokenlessBin` | `$TOKENLESS_BIN`,随后使用 `tokenless` | 选择 Tokenless CLI 可执行文件。非空 Plugin 配置优先于环境变量。 |
| `skipTools` | 下文列出的内容读取类集合 | 跳过匹配工具的压缩。配置数组会替换默认集合,空数组表示不跳过任何工具。错误归因仍保持启用。 |
| `shellTools` | 下文列出的 Shell 和 process 集合 | 选择 Shell 阈值,也决定哪些工具的结构化 `value` 可以用于失败归因。配置数组会替换默认集合。 |
| `truncateStringsAt` | Shell 为 `65536`,其他工具为 `1048576` | 覆盖全部工具类别的字符串保留上限。只接受正整数。 |
| `truncateArraysAt` | Shell 为 `128`,其他工具为 `65536` | 覆盖全部工具类别的数组保留上限。只接受正整数。 |
| `maxDepth` | Shell 为 `8`,其他工具为 `32` | 覆盖全部工具类别的 JSON 最大深度。只接受正整数。 |
| `timeoutMs` | `3000` | 限制一次 Tokenless 子进程的运行时间,单位为毫秒。只接受正整数。 |
| `maxBuffer` | `2097152` | 限制捕获的子进程输出,单位为 byte。只接受正整数。 |
| `agentId` | `dsh` | 设置 Tokenless 统计记录中的 `--agent-id`。 |
| `noStash` | `false` | 设为 `true` 时传入 `--no-stash`。默认允许把删除的数组项写入 Stash。 |

默认 `skipTools` 集合包括 `Read`、`read`、`read_file`、`read_many_files`、`Glob`、
`glob`、`search_file`、`list_directory`、`list_dir`、`Grep`、`grep`、`grep_code`、
`grep_search`、`search_files`、`Lsp`、`lsp`、`NotebookRead`、`notebook_read` 和
`notebookread`。

默认 `shellTools` 集合包括 `Bash`、`bash`、`Shell`、`shell`、`exec`、`terminal`、
`run_shell_command`、`run_in_terminal`、`get_terminal_output`、`execute_command` 和
`process`。

DSH 使用 `isError` 标记的原始失败可以为任何工具追加依赖、权限、路径、网络或包
错误归因。结构化输出只会为 `shellTools` 分类。归因独立于压缩,关闭或跳过压缩、
压缩没有得到更短结果时仍会生效。后续 waterfall listener 替换 canonical `value`
后,Tokenless 会按替换值重新分类,不会沿用已经被替换结果的旧归因。

## 通过 anolisa 管理(推荐)

这些命令需要 ANOLISA 组件记录。如果 Tokenless 是通过 YUM 直接安装的,
Expand Down Expand Up @@ -82,9 +151,17 @@ anolisa adapter enable tokenless qoder
anolisa adapter enable tokenless claude-code
anolisa adapter enable tokenless codex
anolisa adapter enable tokenless qwencode
anolisa adapter enable tokenless dsh \
--profile web \
--profile headless
```

只需启用实际使用的 Agent 产品。启用多个产品时,应逐个执行并分别验证。
只需启用实际使用的 Agent 产品。多个产品应分别执行并验证各自的命令。DSH 的全部
目标 profile 应写在同一条 enable 命令中。

DeepSeek Harness 按 profile 管理,因此必须至少提供一个 `--profile`。每个名称应与
`dsh --profile <profile>` 使用的名称一致,不带 profile 的通用命令会被拒绝。
后续 enable 或 re-enable 必须再次列出需要保留的全部 profile。

OpenCode 应使用 [npm 安装后的手动接入](#npm-安装后的手动接入)中的随附安装脚本。

Expand Down Expand Up @@ -193,6 +270,13 @@ Marketplace Plugin 在 Claude Code 重启后生效,也可以按照安装脚本

Plugin 在新的 Codex 会话中加载。关闭旧会话并重新启动后验证统计。它的 PostToolUse Hook 是追加型的:统计只能作为压缩候选遥测,不能证明原始 Codex 工具结果已离开 Prompt。

### DeepSeek Harness

原生 Bundle 会在选定的 DSH profile 启动时加载。启用 Bundle 或修改 profile patch
后,重启 `dsh --profile <profile>`,运行一个返回可压缩 JSON 的工具,再检查
`tokenless stats list`。禁用命令是 `anolisa adapter disable tokenless dsh`。
receipt 已经记录 profile 名称,因此 disable 不再接受 `--profile`。

### OpenCode

OpenCode 启动时会自动加载配置目录下的 Plugin。使用上述 Tokenless 生命周期脚本
Expand Down
14 changes: 14 additions & 0 deletions src/tokenless/.anolisa/component.toml.in
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,20 @@ detect = { binary = "openclaw" }
[adapters.bundle]
entry = "openclaw.plugin.json"

[[adapters]]
framework = "dsh"
adapter_type = "plugin"
plugin_id = "anolisa-tokenless"
source = "adapters/dsh"
dest = "{datadir}/adapters/{component}/dsh/"
detect = { binary = "dsh" }

[adapters.bundle]
entry = "package.json"

[adapters.compat]
framework_version = ">=0.1.0-rc.2 <0.2.0"

[[adapters]]
framework = "hermes"
adapter_type = "plugin"
Expand Down
1 change: 1 addition & 0 deletions src/tokenless/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ third_party/rtk/
adapters/tokenless/manifest.json
adapters/tokenless/openclaw/package.json
adapters/tokenless/openclaw/openclaw.plugin.json
adapters/tokenless/dsh/package.json
adapters/tokenless/hermes/plugin.yaml
adapters/tokenless/qoder/.qoder-plugin/plugin.json
adapters/tokenless/claude-code/.claude-plugin/plugin.json
Expand Down
Loading
Loading