Skip to content

perf(pi): reduce DeepSeek input overhead and detect local context budgets - #399

Merged
Ed1s0nZ merged 2 commits into
AIPentest:mainfrom
Sycun:codex/automatic-token-efficiency
Nov 13, 2026
Merged

Ed1s0nZ merged 2 commits into
AIPentest:mainfrom
Sycun:codex/automatic-token-efficiency

Conversation

@Sycun

@Sycun Sycun commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

DeepSeek conversations resend builtin tool schemas and formatted tool results on each model turn. This PR applies automatic input-saving defaults and selects local context budgets from backend metadata when channels are saved or tested.

  • Use native Pi tool discovery for DeepSeek catalogs with at least 16 tools. Keep six catalog entries, core tools and explicitly pinned tools visible, and stabilize MCP tool order across runs.
  • Compact valid JSON tool results while preserving string contents, numeric lexemes and original session records. With an authorized local reader, save large results as full evidence and return a bounded preview. Preserve the full response if writing the evidence fails.
  • Send DeepSeek output limits through max_tokens and include provider-reported reasoning tokens in the usage breakdown without adding them again to total usage.
  • Detect Ollama, llama.cpp, LM Studio, vLLM and oMLX. Prefer oMLX model/alias status windows and fall back to its model list when status access is unavailable. Apply local budgets after successful identification, preserve tighter configured windows and clear automatic classification when the model or endpoint changes.
  • Apply optimization in the background. Retain configuration-file overrides and show detected context information in the existing channel editor.

The local metadata and budget plumbing is included because current main lacks these prerequisites. This overlaps the context-handling work in #367 and is adapted to the current Pi runtime.

Validation on the upstream-based branch passed.

  • npm run build and npm test in runtime/pi — 15 tests.
  • 19 focused frontend tests covering settings, credentials and usage.
  • go test -race ./internal/agentruntime ./internal/multiagent ./internal/config ./internal/modelmeta ./internal/tooloutput ./internal/handler ./internal/app.
  • YAML context-setting round-trip regression, go run ./cmd/build-runtime, bash scripts/check-pi-only.sh, go build ./cmd/server and git diff --check.

In a deterministic fixture, first-request tool schemas decreased from 13,798 to 3,417 bytes. A 10,070-byte tool result became a 4,387-byte preview, and a subsequent read recovered the omitted middle evidence. These measurements describe fixture payload size. Live provider billing, latency and task-success impact remain unmeasured. Discovery and evidence readback can add model turns.

中文摘要

默认自动精简 DeepSeek 输入,并根据 Ollama、llama.cpp、LM Studio、vLLM 和 oMLX 元数据应用本地上下文预算。保留完整证据与原始会话记录,优化策略在后台应用。真实模型的费用和任务效果仍需对照验证。

@Ed1s0nZ
Ed1s0nZ merged commit 01ffb00 into AIPentest:main Nov 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants