diff --git a/config.example.yaml b/config.example.yaml index c1bc6fdbd..2998a74a0 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -120,6 +120,8 @@ ai: # HITL 审计仅在未显式配置 provider/base_url/api_key 时继承主渠道 headers； # 独立审计身份如需请求头，请在 hitl.audit_model.headers 中单独配置。 # 空 headers 与未配置等价；所有自定义头值在设置接口中均按敏感值脱敏。 + context_window: 0 # 0=未知；保存通道时按服务元数据自动检测 + context_profile: auto # 默认自动优化；识别本地服务后保存为 local_auto；custom 可使用手动运行时参数 # Pi 路径模型推理：DeepSeek/OpenAI 为 thinking / reasoning_effort；Claude 4.6+ 为 adaptive + output_config.effort（仅显式配置 effort 时下发）；3.7 为 enabled+budget_tokens:10000（文档示例），effort 不映射，自定义预算用 extra_request_fields reasoning: mode: on # auto | on | off；off：OpenAI/Claude 不附加推理字段，DeepSeek 发送 thinking.type=disabled（其默认开启思考） diff --git a/docs/en-US/configuration.md b/docs/en-US/configuration.md index a7fbd79a3..bd4ae6ded 100644 --- a/docs/en-US/configuration.md +++ b/docs/en-US/configuration.md @@ -72,10 +72,22 @@ ai: | `base_url/api_key/model` | Required. Base URL usually includes a version path such as `/v1`. | | `max_total_tokens` | Shared context budget for compression, attack-chain generation, multi-agent summaries, and similar paths. | | `max_completion_tokens` | Per-response output cap; default is used when empty. | +| `context_window` | Runtime context window, or 0 when unknown. Channel saves can fill the detected value and retain a smaller configured window. | +| `context_profile` | Background policy, default `auto`. Recognized local backends use `local_auto`. Configuration files also accept manual `local`, explicit `economy` savings, and `custom` runtime settings. | | `reasoning` | Default reasoning fields for the channel. Gateway support varies; try `mode: off` first when a provider rejects requests. | The chat page reads saved channels into the “AI Channel” selector. A non-empty request `aiChannelId` selects a channel for that run/session without sending API credentials through the prompt path. Empty `aiChannelId` follows `ai.default_channel`. +### Automatic input savings + +DeepSeek endpoints, explicit DeepSeek reasoning profiles and model names containing DeepSeek enable input savings automatically. With at least 16 tools, discovery declares six catalog entries, core tools and explicitly pinned tools. JSON tool results lose formatting whitespace. When an authorized local reader is available, results over 8,000 bytes become previews backed by the full output. Provider-reported reasoning tokens are shown separately within completion usage. + +Channel saves and connection tests probe metadata from Ollama, llama.cpp, LM Studio, vLLM and oMLX. oMLX is identified by `owned_by: omlx`. Its matching model or alias in `/v1/models/status` supplies the preferred window, with the model list as a fallback when status access is unavailable. Generic compatible responses and private IP addresses alone do not select local optimization. Failed probes allow the channel to be saved. + +The local input budget is capped at 60% of the window and 131,072 tokens, with output room reserved. An unknown window uses 32,768 tokens. Local policies tighten tool discovery and result limits, and use the `max_tokens` output field. New channels reporting only an architecture maximum retain an unknown runtime window. Changing a model or endpoint clears the previous automatic classification. These policies are applied in the background. + +Compare complete tasks using input, cache hits, output and call counts to measure savings. Tool discovery and evidence readback may add calls. Explicit configuration-file overrides remain supported. + ### Disabling reasoning by provider and model Use `reasoning.mode: off` (or **Reasoning → Off** in chat) to disable reasoning. The request format depends on the channel: diff --git a/docs/en-US/token-usage-monitoring.md b/docs/en-US/token-usage-monitoring.md index da319a823..7b76df7cd 100644 --- a/docs/en-US/token-usage-monitoring.md +++ b/docs/en-US/token-usage-monitoring.md @@ -2,8 +2,16 @@ Pi reports usage; Go accumulates completed model turns and emits `agent_usage_summary` into the existing database and timeline. Local statistics are not a provider billing statement and may not cover every auxiliary HTTP request. -Prompt tokens are Pi input + cacheRead + cacheWrite. Completion tokens are output, and total is prompt + completion. Cache details are already included and must not be added again. Model-call counts reflect observed turns. Missing reasoning/cache details mean unreported data, not proven zero use. +Prompt tokens are Pi input + cacheRead + cacheWrite. Completion tokens are output, and total is prompt + completion. Cache details are already included and must not be added again. The usage view shows provider-reported reasoning tokens as a subset of completion tokens. Model-call counts reflect observed turns. Missing reasoning/cache details mean unreported data, not proven zero use. Durable assessments retain their own conservative budget ledger. The dashboard and Settings usage view retain period, model and project filters. APIs are `GET /api/usage/tokens` and `GET /api/conversations/:id/token-usage`. Existing usage rows and historical timeline backfill remain compatible. Streaming updates carry deltas; complete messages are committed at message end. Implementation: `internal/multiagent/pi_runner.go` and `internal/database/model_token_usage.go`. See [Pi AI](https://github.com/earendil-works/pi/tree/v1.5.2/packages/ai). + +## DeepSeek + +In the channel’s reasoning settings, use `off` for simple Q&A and format conversions or pair `auto` with `low` to keep reasoning on at a lower effort. The official API defaults to `high` when effort is unspecified and maps `medium` to `high`. The channel sends its per-request output limit as `max_tokens`, shared by reasoning and the visible answer. Compare output tokens, reasoning tokens and the cache hit rate for the same tasks at different effort levels. [DeepSeek Thinking Mode](https://api-docs.deepseek.com/guides/thinking_mode/) · [Chat Completions](https://api-docs.deepseek.com/api/create-chat-completion/) + +The official API automatically caches reusable request prefixes. Pi orders MCP tool definitions by name so the same catalog introduces fewer prefix changes across requests. The provider reports the actual cache hits. [DeepSeek Context Caching](https://api-docs.deepseek.com/guides/kv_cache/) + +Input savings activate in the background for the official DeepSeek endpoint, explicit `deepseek` / `deepseek_compat` reasoning profiles, and model names containing `deepseek` (case-insensitive). At 16 tools, native discovery keeps six catalog entries plus core tools and explicit always-visible tools available. Other builtins load on demand. With authorized file access, results over 8,000 bytes become head/tail previews backed by the full output. Tighter configured limits take precedence. Valid JSON tool results lose formatting whitespace, including when replaying older sessions. Strings and numeric lexemes retain their original text. Existing configuration-file overrides remain supported. Compare complete tasks, including discovery/readback calls and evidence quality, to measure actual savings. diff --git a/docs/zh-CN/configuration.md b/docs/zh-CN/configuration.md index a44d118c3..070b62946 100644 --- a/docs/zh-CN/configuration.md +++ b/docs/zh-CN/configuration.md @@ -73,8 +73,20 @@ ai: | `base_url/api_key/model` | 必填。Base URL 通常需要包含版本路径，如 OpenAI/兼容网关的 `/v1`。 | | `max_total_tokens` | 上下文压缩、攻击链构建、多代理摘要等共用的总预算。 | | `max_completion_tokens` | 单次模型输出上限；未填时使用默认值。 | +| `context_window` | 运行时上下文窗口，0 表示未知。保存通道时可自动填入服务报告的窗口，已填写的更小窗口优先。 | +| `context_profile` | 后台策略，默认 `auto`。识别本地服务后记录 `local_auto`。配置文件支持手动 `local`、显式节省 `economy` 和使用运行时参数的 `custom`。 | | `reasoning` | 该通道的默认推理扩展字段。不同网关支持差异较大，异常时先尝试 `mode: off`。 | +### 自动输入优化 + +DeepSeek 官方地址、明确的 DeepSeek 推理线路或包含 DeepSeek 的模型名会自动启用输入节省。达到 16 个工具时使用按需发现，默认常驻前 6 个工具、核心工具及显式常驻清单。JSON 回包精简格式空白。有授权的本地读取工具时，超过 8,000 字节的结果保存全文并返回可回读的预览。推理 Token 单列统计且计入输出总量。 + +保存通道和测试连接会探测 Ollama、llama.cpp、LM Studio、vLLM 和 oMLX 的元数据。oMLX 通过 `owned_by: omlx` 识别，优先读取 `/v1/models/status` 的匹配模型或别名窗口，权限不足时回退到模型列表。普通兼容响应和内网 IP 本身不足以触发本地策略。探测失败可继续保存。 + +本地输入预算取窗口的 60%，上限 131,072 Token，并预留输出空间。窗口未知时使用 32,768 的保守窗口。工具发现和回包上限按本地窗口收紧，输出字段使用 `max_tokens`。模型只报告架构上限时，新通道保留窗口未知状态。切换模型或服务会清除旧的自动判定。页面在后台应用这些策略。 + +验证成本时，应比较完整任务的输入、缓存命中、输出及调用次数。工具搜索和分段回读可能增加调用。已有配置文件的显式覆盖继续生效。 + ### 按提供商和模型关闭推理 使用 `reasoning.mode: off`，或在对话中选择“推理 → 关闭”。实际请求参数按通道区分： diff --git a/docs/zh-CN/token-usage-monitoring.md b/docs/zh-CN/token-usage-monitoring.md index b42952331..c6a5fd406 100644 --- a/docs/zh-CN/token-usage-monitoring.md +++ b/docs/zh-CN/token-usage-monitoring.md @@ -6,8 +6,10 @@ - 输入总量由 Pi 的 input、cacheRead、cacheWrite 相加；输出使用 output。 - 总量为输入与输出之和；缓存读取和缓存写入已经包含在输入中，不能重复叠加。 +- 供应商单独报告的推理 Token 已包含在输出中，面板单列展示。 - 模型调用次数按已观察到的轮次统计，重试会产生新的轮次。 - 未报告的细分不能推断为没有发生。推理 Token 等没有独立字段时，不根据文本长度伪造账单计数。 +- 深度评估使用其持久预算账本；供应商未报告用量时采用原有保守计费策略。 ## 查看 @@ -16,3 +18,11 @@ 已有模型用量表保持兼容。旧时间线用量事件可作为历史记录补齐；新记录仅由原生 Pi 事件生成。原生流式消息只传输增量，完整消息在结束时提交，避免重复累计与逐块重复发送整份内容。 实现：`internal/multiagent/pi_runner.go`、`internal/database/model_token_usage.go`、前端用量页面。基础模型 usage 语义见 [Pi AI](https://github.com/earendil-works/pi/tree/v1.5.2/packages/ai)。 + +## DeepSeek + +在 AI 通道的推理设置中，可用 `off` 处理简单问答和格式转换，或用 `auto` 配合 `low` 保留低强度推理。官方接口未指定强度时默认使用 `high`，`medium` 也映射到 `high`。通道的「单次输出上限」通过 `max_tokens` 发送，推理和正文共用额度。对照面板里的输出、推理 Token 和缓存命中率，可比较相同任务在不同强度下的用量。[DeepSeek 推理模式](https://api-docs.deepseek.com/guides/thinking_mode/) · [Chat Completions](https://api-docs.deepseek.com/api/create-chat-completion/) + +官方接口自动缓存可复用的请求前缀。Pi 按名称固定 MCP 工具定义的顺序，以减少相同工具目录在请求之间的前缀变化。缓存命中以供应商返回的用量为准。[DeepSeek 上下文缓存](https://api-docs.deepseek.com/guides/kv_cache/) + +后台默认根据官方 DeepSeek 地址、DeepSeek 线路或模型名自动启用按需工具发现、JSON 回包精简与可回读的大结果预览。用户配置模型后即可使用。验证节省效果时，对照完整任务的输入、输出、缓存命中和调用次数，并检查任务产物与证据完整性。具体阈值见[配置说明](configuration.md#deepseek-输入优化)。 diff --git a/internal/agentruntime/types.go b/internal/agentruntime/types.go index 4989d2890..54022c4ae 100644 --- a/internal/agentruntime/types.go +++ b/internal/agentruntime/types.go @@ -9,18 +9,19 @@ import ( const ProtocolVersion = 1 type Model struct { - Provider string `json:"provider"` - API string `json:"api,omitempty"` - ID string `json:"id"` - BaseURL string `json:"baseUrl"` - APIKey string `json:"apiKey"` - Headers map[string]string `json:"headers,omitempty"` - ContextWindow int `json:"contextWindow"` - MaxTokens int `json:"maxTokens"` - Thinking string `json:"thinking,omitempty"` - ExtraFields map[string]any `json:"extraFields,omitempty"` - Compat map[string]any `json:"compat,omitempty"` - InputBudget int `json:"inputBudget,omitempty"` + Provider string `json:"provider"` + API string `json:"api,omitempty"` + ID string `json:"id"` + BaseURL string `json:"baseUrl"` + APIKey string `json:"apiKey"` + Headers map[string]string `json:"headers,omitempty"` + ContextWindow int `json:"contextWindow"` + MaxTokens int `json:"maxTokens"` + Thinking string `json:"thinking,omitempty"` + ExtraFields map[string]any `json:"extraFields,omitempty"` + Compat map[string]any `json:"compat,omitempty"` + InputBudget int `json:"inputBudget,omitempty"` + CompactToolJSON bool `json:"compactToolJSON,omitempty"` } type Tool struct { diff --git a/internal/app/app.go b/internal/app/app.go index 77dd0f916..4a9046c46 100644 --- a/internal/app/app.go +++ b/internal/app/app.go @@ -1158,6 +1158,7 @@ func setupRoutes( protected.PUT("/config", configHandler.UpdateConfig) protected.POST("/config/apply", configHandler.ApplyConfig) protected.POST("/config/test-openai", configHandler.TestOpenAI) + protected.POST("/config/detect-context", configHandler.DetectContext) protected.POST("/config/test-typesafe", configHandler.TestTypeSafe) protected.POST("/config/test-vision", configHandler.TestVision) protected.POST("/config/list-models", configHandler.ListModels) diff --git a/internal/config/config.go b/internal/config/config.go index b0e593f61..42db0478b 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -728,7 +728,12 @@ type OpenAIConfig struct { // Headers 追加或覆盖发给模型网关的 HTTP 请求头，值支持 ${ENV_VAR} 展开。 // 用于兼容要求私有鉴权头、版本头或组织头的第三方 OpenAI 兼容供应商。 Headers map[string]string `yaml:"headers,omitempty" json:"headers,omitempty"` - // Reasoning 控制 Agent ChatModel 的 thinking / reasoning_effort / output_config 等（单/多代理路径生效）。 + // ContextWindow 是模型原生上下文窗口（token）；0 表示未知。用于校验/压缩 max_total_tokens。 + ContextWindow int `yaml:"context_window,omitempty" json:"context_window,omitempty"` + // ContextProfile selects auto, economy, custom or local; local_auto records + // a local backend identified through the model metadata probe. + ContextProfile string `yaml:"context_profile,omitempty" json:"context_profile,omitempty"` + // Reasoning 控制 Agent ChatModel 的 thinking / reasoning_effort / output_config 等（Agent 单/多代理路径生效）。 Reasoning OpenAIReasoningConfig `yaml:"reasoning,omitempty" json:"reasoning,omitempty"` } @@ -748,6 +753,8 @@ type AIChannelConfig struct { MaxTotalTokens int `yaml:"max_total_tokens,omitempty" json:"max_total_tokens,omitempty"` MaxCompletionTokens int `yaml:"max_completion_tokens,omitempty" json:"max_completion_tokens,omitempty"` Headers map[string]string `yaml:"headers,omitempty" json:"headers,omitempty"` + ContextWindow int `yaml:"context_window,omitempty" json:"context_window,omitempty"` + ContextProfile string `yaml:"context_profile,omitempty" json:"context_profile,omitempty"` Reasoning OpenAIReasoningConfig `yaml:"reasoning,omitempty" json:"reasoning,omitempty"` } @@ -764,6 +771,8 @@ func (c AIChannelConfig) ToOpenAIConfig() OpenAIConfig { MaxTotalTokens: c.MaxTotalTokens, MaxCompletionTokens: c.MaxCompletionTokens, Headers: ExpandHeaderMap(c.Headers), + ContextWindow: c.ContextWindow, + ContextProfile: NormalizeContextProfile(c.ContextProfile), Reasoning: c.Reasoning, } } @@ -781,6 +790,8 @@ func AIChannelFromOpenAI(id, name string, oa OpenAIConfig) AIChannelConfig { MaxTotalTokens: oa.MaxTotalTokens, MaxCompletionTokens: oa.MaxCompletionTokens, Headers: ExpandHeaderMap(oa.Headers), + ContextWindow: oa.ContextWindow, + ContextProfile: oa.ContextProfile, Reasoning: oa.Reasoning, } } @@ -862,10 +873,14 @@ func (c *Config) ApplyDefaultAIChannel() { } func (c OpenAIConfig) MaxCompletionTokensEffective() int { - if c.MaxCompletionTokens > 0 { - return c.MaxCompletionTokens + out := c.MaxCompletionTokens + if out <= 0 { + out = DefaultMaxCompletionTokens } - return DefaultMaxCompletionTokens + if window := c.EffectiveContextWindow(); window > 0 && out > window/2 { + out = window / 2 + } + return out } // IsDeepSeekEndpointOrModel reports whether the channel targets DeepSeek's @@ -1561,6 +1576,11 @@ func Load(path string) (*Config, error) { if err := validateOpenAIOutputLimits(cfg.OpenAI); err != nil { return nil, err } + for id, ch := range cfg.AI.Channels { + if err := ValidateContextSettings("ai.channels."+id, ch.ContextWindow, ch.ContextProfile); err != nil { + return nil, err + } + } // 如果配置了工具目录，从目录加载工具配置 if cfg.Security.ToolsDir != "" { inlineTools := append([]ToolConfig(nil), cfg.Security.Tools...) @@ -1627,6 +1647,22 @@ func validateOpenAIOutputLimits(openAI OpenAIConfig) error { if openAI.MaxCompletionTokens < 0 { return fmt.Errorf("openai.max_completion_tokens 必须为正数") } + return ValidateContextSettings("openai", openAI.ContextWindow, openAI.ContextProfile) +} + +// ValidateContextSettings 校验主通道与 ai.channels 共用的上下文配置。 +func ValidateContextSettings(prefix string, window int, profile string) error { + prefix = strings.TrimSpace(prefix) + if prefix == "" { + prefix = "openai" + } + if window < 0 { + return fmt.Errorf("%s.context_window 不能为负数", prefix) + } + if raw := strings.TrimSpace(profile); raw != "" && + !strings.EqualFold(raw, "auto") && NormalizeContextProfile(raw) == "" { + return fmt.Errorf("%s.context_profile 只支持 auto、economy、custom、local 或 local_auto", prefix) + } return nil } diff --git a/internal/config/context_profile.go b/internal/config/context_profile.go new file mode 100644 index 000000000..2ebd5df35 --- /dev/null +++ b/internal/config/context_profile.go @@ -0,0 +1,253 @@ +package config + +import "strings" + +const ( + // DefaultMaxTotalTokens 是未显式配置时的通用上下文预算。 + DefaultMaxTotalTokens = 120000 + // DefaultLocalContextWindow 是 local 档位下未提供 context_window 时的保守窗口。 + DefaultLocalContextWindow = 32768 + // LocalContextBudgetPercent 是 local 档位把原生窗口折算成提示词预算的比例。 + LocalContextBudgetPercent = 60 + // LocalContextBudgetCap 防止 256K/1M 窗口把预算放大到超出本地模型有效上下文。 + LocalContextBudgetCap = 131072 + // MinLocalContextWindow 以下不建议运行带工具的 agent。 + MinLocalContextWindow = 16384 + + ContextWarningLocalWindowUnknown = "local_context_window_unknown" + ContextWarningLocalWindowSmall = "local_context_window_small" + ContextWarningMaxTotalClamped = "max_total_tokens_clamped" + ContextWarningMaxCompletionClamped = "max_completion_tokens_clamped" +) + +// NormalizeContextProfile returns an explicit profile, or empty for auto. +func NormalizeContextProfile(raw string) string { + switch profile := strings.ToLower(strings.TrimSpace(raw)); profile { + case "local", "local_auto", "economy", "custom": + return profile + } + return "" +} + +// IsEconomyContextProfile enables input savings for DeepSeek by default. An +// explicit custom/local profile takes precedence, including on older channels. +func (c OpenAIConfig) IsEconomyContextProfile() bool { + switch NormalizeContextProfile(c.ContextProfile) { + case "economy": + return true + case "custom", "local", "local_auto": + return false + } + profile := c.Reasoning.ProfileEffective() + // Model-name detection only selects input savings. The gateway's wire + // protocol and reasoning parameters still follow its provider profile. + return c.IsDeepSeekOfficialEndpoint() || profile == "deepseek" || profile == "deepseek_compat" || + strings.Contains(strings.ToLower(c.Model), "deepseek") +} + +func (c OpenAIConfig) IsLocalContextProfile() bool { + profile := NormalizeContextProfile(c.ContextProfile) + return profile == "local" || profile == "local_auto" +} + +// OutputLimitField 返回输出上限字段名。本地端点普遍只认旧字段 max_tokens。 +func (c OpenAIConfig) OutputLimitField() string { + if c.IsLocalContextProfile() { + return "max_tokens" + } + return "max_completion_tokens" +} + +// EffectiveContextWindow 返回用于预算计算的原生窗口；local 档位缺省时按保守值处理。 +func (c OpenAIConfig) EffectiveContextWindow() int { + if c.ContextWindow > 0 { + return c.ContextWindow + } + if c.IsLocalContextProfile() { + return DefaultLocalContextWindow + } + return 0 +} + +// EffectiveMaxTotalTokens 把 max_total_tokens 收紧到模型窗口内。 +// 通用档位保留输出预留；local 档位再按 Effective Context 经验折到窗口的 60%，并限制在 128K 以内。 +func (c OpenAIConfig) EffectiveMaxTotalTokens() int { + configured := c.MaxTotalTokens + if configured <= 0 { + configured = DefaultMaxTotalTokens + } + window := c.EffectiveContextWindow() + if window <= 0 { + return configured + } + + reserve := c.MaxCompletionTokensEffective() + if reserve > window/2 { + reserve = window / 2 + } + hard := window - reserve + if hard < 4096 { + hard = window * 85 / 100 + } + if hard < 1024 { + hard = 1024 + } + + budget := configured + if budget > hard { + budget = hard + } + if c.IsLocalContextProfile() { + local := window * LocalContextBudgetPercent / 100 + if local > LocalContextBudgetCap { + local = LocalContextBudgetCap + } + if local < 8192 && window >= 8192 { + local = 8192 + } + if local > hard { + local = hard + } + if budget > local { + budget = local + } + } + if budget < 1024 { + budget = 1024 + } + if window > 0 && budget > window { + budget = window + } + return budget +} + +// ContextBudgetWarnings 返回可本地化的告警码；不直接拼自然语言，便于 UI 多语言与日志检索。 +func (c OpenAIConfig) ContextBudgetWarnings() []string { + var out []string + if c.IsLocalContextProfile() { + if c.ContextWindow <= 0 { + out = append(out, ContextWarningLocalWindowUnknown) + } else if c.ContextWindow < MinLocalContextWindow { + out = append(out, ContextWarningLocalWindowSmall) + } + } + configured := c.MaxTotalTokens + if configured <= 0 { + configured = DefaultMaxTotalTokens + } + if c.EffectiveContextWindow() > 0 && configured > c.EffectiveMaxTotalTokens() { + out = append(out, ContextWarningMaxTotalClamped) + } + if c.EffectiveContextWindow() > 0 && c.MaxCompletionTokens > c.MaxCompletionTokensEffective() { + out = append(out, ContextWarningMaxCompletionClamped) + } + return out +} + +// EffectiveRuntimeForChannel applies input-saving or local-model limits to a +// runtime copy. Explicit custom profiles use the configured runtime settings. +func EffectiveRuntimeForChannel(mw AgentTuningConfig, oa OpenAIConfig) AgentTuningConfig { + if oa.IsEconomyContextProfile() { + out := mw + out.ToolSearchEnable = true + out.ToolSearchMinTools = tightenInt(mw.ToolSearchMinTools, 16) + out.ToolSearchAlwaysVisible = tightenInt(mw.ToolSearchAlwaysVisible, 6) + includeBuiltins := false + out.ToolSearchIncludeBuiltins = &includeBuiltins + out.ReductionEnable = true + out.ReductionMaxLengthForTrunc = tightenInt(mw.ReductionMaxLengthForTruncEffective(), 8000) + return out + } + if !oa.IsLocalContextProfile() { + return mw + } + window := oa.EffectiveContextWindow() + out := mw + + out.ReductionMaxLengthForTrunc = tightenInt( + mw.ReductionMaxLengthForTruncEffective(), + localToolTruncBytes(window), + ) + out.ReductionEnable = true + + out.ToolSearchEnable = true + if mw.ToolSearchMinTools <= 0 || mw.ToolSearchMinTools > 8 { + out.ToolSearchMinTools = 8 + } + if mw.ToolSearchAlwaysVisible <= 0 || mw.ToolSearchAlwaysVisible > 8 { + out.ToolSearchAlwaysVisible = 8 + } + out.ToolSearchAlwaysVisibleTools = trimLocalAlwaysVisibleTools(mw.ToolSearchAlwaysVisibleTools) + includeBuiltins := false + out.ToolSearchIncludeBuiltins = &includeBuiltins + + return out +} + +func tightenInt(current, ceil int) int { + if ceil <= 0 { + return current + } + if current <= 0 || current > ceil { + return ceil + } + return current +} + +func localToolTruncBytes(window int) int { + bytes := window / 6 + if bytes < 8192 { + bytes = 8192 + } + if bytes > 32768 { + bytes = 32768 + } + if window > 0 && bytes > window { + bytes = window + } + return bytes +} + +// localCoreToolNames 是本地小窗口下值得常驻的工具；其余交给 tool_search 按需解锁。 +var localCoreToolNames = map[string]struct{}{ + "read_file": {}, + "glob": {}, + "grep": {}, + "write_file": {}, + "edit_file": {}, + "execute": {}, + "exec": {}, + "task": {}, + "transfer_to_agent": {}, + "exit": {}, + "write_todos": {}, + "skill": {}, + "tool_search": {}, + "record_vulnerability": {}, + "get_tool_execution": {}, + "wait_tool_execution": {}, + "analyze_image": {}, + "search_knowledge_base": {}, + "http-framework-test": {}, +} + +func trimLocalAlwaysVisibleTools(in []string) []string { + if len(in) == 0 { + return nil + } + out := make([]string, 0, len(localCoreToolNames)) + seen := make(map[string]struct{}, len(in)) + for _, raw := range in { + name := strings.TrimSpace(raw) + key := strings.ToLower(name) + if _, ok := localCoreToolNames[key]; !ok { + continue + } + if _, dup := seen[key]; dup { + continue + } + seen[key] = struct{}{} + out = append(out, name) + } + return out +} diff --git a/internal/config/context_profile_test.go b/internal/config/context_profile_test.go new file mode 100644 index 000000000..aed342043 --- /dev/null +++ b/internal/config/context_profile_test.go @@ -0,0 +1,260 @@ +package config + +import ( + "reflect" + "testing" +) + +func TestEffectiveMaxTotalTokensNeverExceedsWindow(t *testing.T) { + t.Parallel() + for _, window := range []int{512, 1024, 2048, 4096, 8192} { + oa := OpenAIConfig{ContextWindow: window, ContextProfile: "local", MaxTotalTokens: 120000} + if got := oa.EffectiveMaxTotalTokens(); got > window { + t.Fatalf("window %d: effective budget %d exceeds window", window, got) + } + } +} + +func TestOutputLimitField(t *testing.T) { + t.Parallel() + if got := (OpenAIConfig{}).OutputLimitField(); got != "max_completion_tokens" { + t.Fatalf("auto output field = %q", got) + } + if got := (OpenAIConfig{ContextProfile: "local"}).OutputLimitField(); got != "max_tokens" { + t.Fatalf("local output field = %q", got) + } +} + +func TestEffectiveMaxTotalTokens(t *testing.T) { + t.Parallel() + cases := []struct { + name string + oa OpenAIConfig + want int + }{ + { + name: "unknown window keeps configured", + oa: OpenAIConfig{MaxTotalTokens: 64000}, + want: 64000, + }, + { + name: "unknown window default 120k", + oa: OpenAIConfig{}, + want: 120000, + }, + { + name: "remote window reserves output", + oa: OpenAIConfig{ContextWindow: 131072, MaxTotalTokens: 120000}, + want: 114688, + }, + { + name: "remote explicit smaller budget wins", + oa: OpenAIConfig{ContextWindow: 131072, MaxTotalTokens: 64000}, + want: 64000, + }, + { + name: "local 128k uses 60 percent", + oa: OpenAIConfig{ContextWindow: 131072, ContextProfile: "local", MaxTotalTokens: 120000}, + want: 78643, + }, + { + name: "local explicit smaller budget wins", + oa: OpenAIConfig{ContextWindow: 131072, ContextProfile: "local", MaxTotalTokens: 20000}, + want: 20000, + }, + { + name: "local 32k reserves half for output", + oa: OpenAIConfig{ContextWindow: 32768, ContextProfile: "local", MaxTotalTokens: 120000}, + want: 16384, + }, + { + name: "local 32k keeps configured output", + oa: OpenAIConfig{ContextWindow: 32768, ContextProfile: "local", MaxTotalTokens: 120000, MaxCompletionTokens: 4096}, + want: 19660, + }, + { + name: "local missing window assumes 32k", + oa: OpenAIConfig{ContextProfile: "local", MaxTotalTokens: 120000}, + want: 16384, + }, + { + name: "local 256k caps budget", + oa: OpenAIConfig{ContextWindow: 262144, ContextProfile: "local", MaxTotalTokens: 200000}, + want: 131072, + }, + { + name: "local 8k stays within window", + oa: OpenAIConfig{ContextWindow: 8192, ContextProfile: "local", MaxTotalTokens: 120000}, + want: 4096, + }, + } + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + if got := tc.oa.EffectiveMaxTotalTokens(); got != tc.want { + t.Fatalf("EffectiveMaxTotalTokens() = %d, want %d", got, tc.want) + } + }) + } +} + +func TestMaxCompletionTokensEffectiveClampsToWindow(t *testing.T) { + t.Parallel() + cases := []struct { + name string + oa OpenAIConfig + want int + }{ + {name: "no window uses configured", oa: OpenAIConfig{MaxCompletionTokens: 32768}, want: 32768}, + {name: "no window uses default", oa: OpenAIConfig{}, want: DefaultMaxCompletionTokens}, + {name: "32k window halves output", oa: OpenAIConfig{ContextWindow: 32768, MaxCompletionTokens: 32768}, want: 16384}, + {name: "local missing window assumes 32k", oa: OpenAIConfig{ContextProfile: "local", MaxCompletionTokens: 32768}, want: 16384}, + {name: "128k window keeps output", oa: OpenAIConfig{ContextWindow: 131072, MaxCompletionTokens: 16384}, want: 16384}, + } + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + if got := tc.oa.MaxCompletionTokensEffective(); got != tc.want { + t.Fatalf("MaxCompletionTokensEffective() = %d, want %d", got, tc.want) + } + }) + } +} + +func TestContextBudgetWarnings(t *testing.T) { + t.Parallel() + cases := []struct { + name string + oa OpenAIConfig + want []string + }{ + { + name: "no window no warning", + oa: OpenAIConfig{MaxTotalTokens: 120000}, + }, + { + name: "local missing window warns twice", + oa: OpenAIConfig{ContextProfile: "local", MaxTotalTokens: 120000}, + want: []string{ContextWarningLocalWindowUnknown, ContextWarningMaxTotalClamped}, + }, + { + name: "local tiny window warns", + oa: OpenAIConfig{ContextProfile: "local", ContextWindow: 8192}, + want: []string{ContextWarningLocalWindowSmall, ContextWarningMaxTotalClamped}, + }, + { + name: "remote oversized budget clamps", + oa: OpenAIConfig{ContextWindow: 32768, MaxTotalTokens: 64000}, + want: []string{ContextWarningMaxTotalClamped}, + }, + { + name: "oversized output cap warns", + oa: OpenAIConfig{ContextWindow: 32768, MaxCompletionTokens: 32768}, + want: []string{ContextWarningMaxTotalClamped, ContextWarningMaxCompletionClamped}, + }, + } + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + if got := tc.oa.ContextBudgetWarnings(); !reflect.DeepEqual(got, tc.want) { + t.Fatalf("ContextBudgetWarnings() = %#v, want %#v", got, tc.want) + } + }) + } +} + +func TestLocalRuntimeBudgetsPreserveTighterSettings(t *testing.T) { + oa := OpenAIConfig{ContextWindow: 131072, ContextProfile: "local"} + original := AgentTuningConfig{ReductionMaxLengthForTrunc: 100000, ToolSearchAlwaysVisible: 12} + got := EffectiveRuntimeForChannel(original, oa) + if !got.ReductionEnable || got.ReductionMaxLengthForTrunc != 21845 || !got.ToolSearchEnable || got.ToolSearchMinTools != 8 || got.ToolSearchAlwaysVisible != 8 || got.ToolSearchIncludeBuiltinsEffective() { + t.Fatalf("unexpected local tuning: %+v", got) + } + if original.ReductionEnable || original.ToolSearchEnable { + t.Fatal("runtime tuning mutated the saved config") + } + got.ReductionMaxLengthForTrunc, got.ToolSearchMinTools, got.ToolSearchAlwaysVisible = 2048, 4, 2 + if again := EffectiveRuntimeForChannel(got, oa); !reflect.DeepEqual(got, again) { + t.Fatal("tighter settings were changed") + } +} + +func TestEffectiveAgentMiddlewareForChannelAutoUnchanged(t *testing.T) { + t.Parallel() + mw := AgentTuningConfig{ + SummarizationTriggerRatio: 0.8, + ToolSearchAlwaysVisibleTools: []string{"read_file", "webshell_exec"}, + } + if got := EffectiveRuntimeForChannel(mw, OpenAIConfig{}); !reflect.DeepEqual(got, mw) { + t.Fatalf("auto profile changed middleware: %+v", got) + } +} + +func TestEconomyContextProfileSelectionAndLimits(t *testing.T) { + for _, tc := range []struct { + name string + oa OpenAIConfig + want bool + }{ + {"official default", OpenAIConfig{BaseURL: "https://api.deepseek.com/v1"}, true}, + {"gateway explicit profile", OpenAIConfig{Reasoning: OpenAIReasoningConfig{Profile: "deepseek_compat"}}, true}, + {"gateway model name", OpenAIConfig{Model: "deepseek-flash", BaseURL: "https://gateway.example/v1"}, true}, + {"namespaced model", OpenAIConfig{Model: "deepseek-ai/DeepSeek-V4", Reasoning: OpenAIReasoningConfig{Profile: "openai_compat"}}, true}, + {"other model", OpenAIConfig{Model: "gpt-example", BaseURL: "https://gateway.example/v1"}, false}, + {"gateway custom opt out", OpenAIConfig{Model: "deepseek-flash", ContextProfile: "custom"}, false}, + {"lookalike host", OpenAIConfig{BaseURL: "https://api.deepseek.com.example/v1"}, false}, + {"custom opt out", OpenAIConfig{BaseURL: "https://api.deepseek.com", ContextProfile: "custom"}, false}, + {"local opt in", OpenAIConfig{BaseURL: "https://api.deepseek.com", ContextProfile: "local"}, false}, + {"detected local DeepSeek", OpenAIConfig{Model: "deepseek-flash", ContextProfile: "local_auto"}, false}, + {"economy opt in", OpenAIConfig{ContextProfile: " ECONOMY "}, true}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := tc.oa.IsEconomyContextProfile(); got != tc.want { + t.Fatalf("economy=%v, want %v", got, tc.want) + } + }) + } + includeBuiltins := true + mw := AgentTuningConfig{ToolSearchMinTools: 20, ToolSearchAlwaysVisible: 12, ToolSearchIncludeBuiltins: &includeBuiltins, + ToolSearchAlwaysVisibleTools: []string{"record_vulnerability"}, ReductionMaxLengthForTrunc: 12000} + oa := OpenAIConfig{BaseURL: "https://api.deepseek.com/v1", MaxTotalTokens: 120000, MaxCompletionTokens: 32768, + Reasoning: OpenAIReasoningConfig{Mode: "on", Effort: "high"}} + out := EffectiveRuntimeForChannel(mw, oa) + if !out.ToolSearchEnable || out.ToolSearchMinTools != 16 || out.ToolSearchAlwaysVisible != 6 || out.ToolSearchIncludeBuiltinsEffective() || !out.ReductionEnable || out.ReductionMaxLengthForTrunc != 8000 { + t.Fatalf("unexpected economy runtime: %+v", out) + } + if !reflect.DeepEqual(out.ToolSearchAlwaysVisibleTools, mw.ToolSearchAlwaysVisibleTools) || !*mw.ToolSearchIncludeBuiltins || mw.ToolSearchEnable { + t.Fatal("explicit visible tools or original config modified") + } + if again := EffectiveRuntimeForChannel(out, oa); !reflect.DeepEqual(out, again) { + t.Fatal("economy profile is not idempotent") + } + out.ToolSearchMinTools, out.ToolSearchAlwaysVisible, out.ReductionMaxLengthForTrunc = 4, 2, 2048 + if again := EffectiveRuntimeForChannel(out, oa); !reflect.DeepEqual(out, again) { + t.Fatal("economy profile loosened tighter limits") + } + for _, profile := range []string{"auto", "economy", "custom", "local", "local_auto"} { + if err := ValidateContextSettings("channel", 0, profile); err != nil { + t.Fatal(err) + } + } + oa.ContextProfile = "custom" + if got := EffectiveRuntimeForChannel(mw, oa); !reflect.DeepEqual(got, mw) { + t.Fatal("custom profile did not preserve runtime settings") + } +} + +func TestDetectedLocalProfileUsesLocalBudgets(t *testing.T) { + manual := OpenAIConfig{ContextProfile: "local", ContextWindow: 65536, MaxCompletionTokens: 8192} + detected := manual + detected.ContextProfile = "local_auto" + if !detected.IsLocalContextProfile() || detected.EffectiveMaxTotalTokens() != manual.EffectiveMaxTotalTokens() || detected.OutputLimitField() != "max_tokens" { + t.Fatalf("detected backend lost local limits: %+v", detected) + } + if !reflect.DeepEqual(EffectiveRuntimeForChannel(AgentTuningConfig{}, detected), EffectiveRuntimeForChannel(AgentTuningConfig{}, manual)) { + t.Fatal("detected backend lost local runtime tuning") + } +} diff --git a/internal/handler/config.go b/internal/handler/config.go index 8ab978d49..b21b662c7 100644 --- a/internal/handler/config.go +++ b/internal/handler/config.go @@ -18,8 +18,10 @@ import ( "cyberstrike-ai/internal/config" "cyberstrike-ai/internal/database" "cyberstrike-ai/internal/knowledge" + "cyberstrike-ai/internal/llm" "cyberstrike-ai/internal/mcp" "cyberstrike-ai/internal/mcp/builtin" + "cyberstrike-ai/internal/modelmeta" "cyberstrike-ai/internal/openai" "cyberstrike-ai/internal/security" "cyberstrike-ai/internal/toolguard" @@ -349,6 +351,11 @@ func (h *ConfigHandler) GetConfig(c *gin.Context) { if load, err := agents.LoadMarkdownAgentsDir(agentsDir); err == nil { subAgentCount = len(agents.MergeYAMLAndMarkdown(h.config.MultiAgent.SubAgents, load.SubAgents)) } + effectiveRuntime := config.EffectiveRuntimeForChannel(h.config.MultiAgent.Runtime, h.config.OpenAI) + builtinAlwaysVisible := builtin.GetAllBuiltinTools() + if !effectiveRuntime.ToolSearchIncludeBuiltinsEffective() { + builtinAlwaysVisible = []string{builtin.ToolGetToolExecution, builtin.ToolWaitToolExecution, builtin.ToolCancelToolExecution} + } multiPub := config.MultiAgentPublic{ Enabled: h.config.MultiAgent.Enabled, RobotDefaultAgentMode: config.NormalizeRobotAgentMode(h.config.MultiAgent), @@ -363,7 +370,7 @@ func (h *ConfigHandler) GetConfig(c *gin.Context) { ToolSearchAlwaysVisibleTools: append([]string(nil), h.config.MultiAgent.Runtime.ToolSearchAlwaysVisibleTools...), ToolSearchAlwaysVisibleEffectiveTools: mergeToolNameLists( h.config.MultiAgent.Runtime.ToolSearchAlwaysVisibleTools, - builtin.GetAllBuiltinTools(), + builtinAlwaysVisible, ), } @@ -802,6 +809,21 @@ func (h *ConfigHandler) UpdateConfig(c *gin.Context) { c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) return } + if req.OpenAI != nil { + if err := config.ValidateContextSettings("openai", req.OpenAI.ContextWindow, req.OpenAI.ContextProfile); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + } + if req.AI != nil { + for id, ch := range req.AI.Channels { + if err := config.ValidateContextSettings("ai.channels."+id, ch.ContextWindow, ch.ContextProfile); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + } + } + if req.AgentRuntime != nil { updated := h.config.AgentRuntime applyAgentRuntimeUpdate(&updated, req.AgentRuntime) @@ -1247,13 +1269,139 @@ func (h *ConfigHandler) UpdateConfig(c *gin.Context) { // TestOpenAIRequest 测试OpenAI连接请求 type TestOpenAIRequest struct { - CredentialScope string `json:"credential_scope,omitempty"` - ChannelID string `json:"channel_id,omitempty"` - Provider string `json:"provider"` - BaseURL string `json:"base_url"` - APIKey string `json:"api_key"` - Model string `json:"model"` - Headers map[string]string `json:"headers,omitempty"` + CredentialScope string `json:"credential_scope,omitempty"` + ChannelID string `json:"channel_id,omitempty"` + Provider string `json:"provider"` + BaseURL string `json:"base_url"` + APIKey string `json:"api_key"` + Model string `json:"model"` + Headers map[string]string `json:"headers,omitempty"` + MaxTotalTokens int `json:"max_total_tokens,omitempty"` + MaxCompletionTokens int `json:"max_completion_tokens,omitempty"` + ContextProfile string `json:"context_profile,omitempty"` +} + +// DetectContextRequest probes model metadata without issuing a completion. +type DetectContextRequest = TestOpenAIRequest + +func probeModelContext(ctx context.Context, provider, baseURL, apiKey, model string, headers map[string]string) modelmeta.ContextInfo { + if llm.IsClaudeProvider(provider) { + return modelmeta.ContextInfo{} + } + probeCtx, cancel := context.WithTimeout(ctx, 8*time.Second) + defer cancel() + return modelmeta.DetectContext(probeCtx, baseURL, model, apiKey, &http.Client{ + Timeout: 5 * time.Second, + Transport: contextProbeTransport{headers: headers}, + CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }, + }) +} + +func addModelContextToResponse(resp gin.H, info modelmeta.ContextInfo, maxTotalTokens, maxCompletionTokens int, contextProfile string) { + if info.Backend != "" { + resp["context_backend"] = info.Backend + } + if info.ContextWindow > 0 { + resp["context_window"] = info.ContextWindow + } + if info.Source != "" { + resp["context_source"] = info.Source + } + if info.Loaded { + resp["context_loaded"] = true + } + if info.RuntimeSlots > 0 { + resp["context_slots"] = info.RuntimeSlots + } + if info.RuntimeUnknown { + resp["context_runtime_unknown"] = true + } + profile := config.NormalizeContextProfile(contextProfile) + if profile == "" || profile == "local_auto" { + if info.IsLocalBackend() { + profile = "local_auto" + } else if info.Source != "" { + profile = "" + } + } + resp["context_profile"] = profile + oa := config.OpenAIConfig{ + ContextWindow: info.ContextWindow, + ContextProfile: profile, + MaxTotalTokens: maxTotalTokens, + MaxCompletionTokens: maxCompletionTokens, + } + if profile == "local_auto" && info.RuntimeUnknown { + oa.ContextWindow = 0 + } + if info.ContextWindow > 0 || profile != "" { + resp["effective_max_total_tokens"] = oa.EffectiveMaxTotalTokens() + } + warnings := append([]string(nil), info.Warnings...) + if info.ContextWindow > 0 && maxTotalTokens > info.ContextWindow { + warnings = append(warnings, modelmeta.WarningBudgetExceedsWindow) + } + warnings = append(warnings, oa.ContextBudgetWarnings()...) + if len(warnings) > 0 { + resp["context_warnings"] = dedupeContextWarnings(warnings) + } +} + +func dedupeContextWarnings(warnings []string) []string { + out := make([]string, 0, len(warnings)) + seen := make(map[string]struct{}, len(warnings)) + for _, code := range warnings { + code = strings.TrimSpace(code) + if code == "" { + continue + } + if _, ok := seen[code]; ok { + continue + } + seen[code] = struct{}{} + out = append(out, code) + } + return out +} + +// DetectContext probes local inference metadata (context window, runtime source). +func (h *ConfigHandler) DetectContext(c *gin.Context) { + var req DetectContextRequest + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": "无效的请求参数: " + err.Error()}) + return + } + resolvedKey, resolveErr := h.resolveProbeSecret(req.APIKey, req.ChannelID, req.BaseURL, req.CredentialScope) + if resolveErr != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": resolveErr.Error()}) + return + } + req.APIKey = resolvedKey + resolvedHeaders, headerErr := h.resolveProbeHeaders(req) + if headerErr != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": headerErr.Error()}) + return + } + if llm.IsClaudeProvider(req.Provider) { + c.JSON(http.StatusOK, gin.H{ + "success": false, + "supported": false, + "error": "Claude (Anthropic Messages API) 不提供上下文窗口探测，请手动填写", + }) + return + } + if strings.TrimSpace(req.Model) == "" { + c.JSON(http.StatusBadRequest, gin.H{"error": "模型不能为空"}) + return + } + baseURL := strings.TrimSuffix(strings.TrimSpace(req.BaseURL), "/") + if baseURL == "" { + baseURL = "https://api.openai.com/v1" + } + info := probeModelContext(c.Request.Context(), req.Provider, baseURL, req.APIKey, req.Model, resolvedHeaders) + resp := gin.H{"success": true, "model": req.Model} + addModelContextToResponse(resp, info, req.MaxTotalTokens, req.MaxCompletionTokens, req.ContextProfile) + c.JSON(http.StatusOK, resp) } // TestOpenAI 测试OpenAI API连接是否可用 @@ -1293,13 +1441,18 @@ func (h *ConfigHandler) TestOpenAI(c *gin.Context) { } } - // 构造一个最小的 chat completion 请求 + // Identify local servers before the completion so the first connection + // test already uses the output-limit field supported by that backend. + metadata := gin.H{} + addModelContextToResponse(metadata, probeModelContext(c.Request.Context(), req.Provider, baseURL, req.APIKey, req.Model, resolvedHeaders), req.MaxTotalTokens, req.MaxCompletionTokens, req.ContextProfile) + detectedProfile, _ := metadata["context_profile"].(string) + maxTokensField := (config.OpenAIConfig{ContextProfile: detectedProfile}).OutputLimitField() payload := map[string]interface{}{ "model": req.Model, "messages": []map[string]string{ {"role": "user", "content": "Hi"}, }, - "max_completion_tokens": 5, + maxTokensField: 5, } // OpenAI-compatible 通道使用内部客户端；Claude 通道在下方直接使用 Agent agenticclaude。 @@ -1366,11 +1519,10 @@ func (h *ConfigHandler) TestOpenAI(c *gin.Context) { return } - c.JSON(http.StatusOK, gin.H{ - "success": true, - "model": chatResp.Model, - "latency_ms": latency.Milliseconds(), - }) + metadata["success"] = true + metadata["model"] = chatResp.Model + metadata["latency_ms"] = latency.Milliseconds() + c.JSON(http.StatusOK, metadata) } // TestTypeSafeRequest 测试 TypeSafe / Jev 连接。 @@ -1569,7 +1721,7 @@ func (h *ConfigHandler) TestVision(c *gin.Context) { "messages": []map[string]string{ {"role": "user", "content": "Hi"}, }, - "max_completion_tokens": 5, + oa.OutputLimitField(): 5, } tmpCfg := &config.OpenAIConfig{ @@ -2129,6 +2281,16 @@ func updateOpenAIConfig(doc *yaml.Node, cfg config.OpenAIConfig) { if cfg.MaxTotalTokens > 0 { setIntInMap(openaiNode, "max_total_tokens", cfg.MaxTotalTokens) } + if cfg.ContextWindow > 0 { + setIntInMap(openaiNode, "context_window", cfg.ContextWindow) + } else { + removeKeyFromMap(openaiNode, "context_window") + } + if profile := config.NormalizeContextProfile(cfg.ContextProfile); profile != "" { + setStringInMap(openaiNode, "context_profile", profile) + } else { + removeKeyFromMap(openaiNode, "context_profile") + } rn := ensureMap(openaiNode, "reasoning") if strings.TrimSpace(cfg.Reasoning.Mode) != "" { setStringInMap(rn, "mode", cfg.Reasoning.Mode) @@ -2189,6 +2351,16 @@ func updateAIConfig(doc *yaml.Node, cfg config.AIConfig) { if ch.MaxCompletionTokens > 0 { setIntInMap(channelNode, "max_completion_tokens", ch.MaxCompletionTokens) } + if ch.ContextWindow > 0 { + setIntInMap(channelNode, "context_window", ch.ContextWindow) + } else { + removeKeyFromMap(channelNode, "context_window") + } + if profile := config.NormalizeContextProfile(ch.ContextProfile); profile != "" { + setStringInMap(channelNode, "context_profile", profile) + } else { + removeKeyFromMap(channelNode, "context_profile") + } rn := ensureMap(channelNode, "reasoning") if strings.TrimSpace(ch.Reasoning.Mode) != "" { setStringInMap(rn, "mode", ch.Reasoning.Mode) diff --git a/internal/handler/config_context_test.go b/internal/handler/config_context_test.go new file mode 100644 index 000000000..79fb0d6d4 --- /dev/null +++ b/internal/handler/config_context_test.go @@ -0,0 +1,193 @@ +package handler + +import ( + "context" + "net/http" + "net/http/httptest" + "testing" + + "cyberstrike-ai/internal/config" + "cyberstrike-ai/internal/modelmeta" + + "github.com/gin-gonic/gin" + "gopkg.in/yaml.v3" +) + +func TestDetectedContextSettingsRoundTripThroughSavedYAML(t *testing.T) { + var doc yaml.Node + if err := yaml.Unmarshal([]byte("ai:\n default_channel: local\n channels:\n local:\n model: fixture\n context_profile: custom\n context_window: 131072\n"), &doc); err != nil { + t.Fatal(err) + } + for _, profile := range []string{"local_auto", "custom", ""} { + ch := config.AIChannelConfig{Model: "fixture", ContextWindow: 32768, ContextProfile: profile} + updateAIConfig(&doc, config.AIConfig{DefaultChannel: "local", Channels: map[string]config.AIChannelConfig{"local": ch}}) + raw, err := yaml.Marshal(&doc) + if err != nil { + t.Fatal(err) + } + var saved config.Config + if err := yaml.Unmarshal(raw, &saved); err != nil { + t.Fatal(err) + } + oa := saved.AI.Channels["local"].ToOpenAIConfig() + if oa.ContextWindow != 32768 || oa.ContextProfile != profile || oa.Model != "fixture" { + t.Fatalf("context settings lost on save: %+v", oa) + } + } +} + +func TestAddModelContextToResponseLocalProfilePreview(t *testing.T) { + t.Parallel() + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{ + Backend: modelmeta.BackendOllama, + ContextWindow: 131072, + Source: "ollama_ps", + Loaded: true, + RuntimeSlots: 2, + }, 120000, 0, "local") + + if got := resp["context_window"]; got != 131072 { + t.Fatalf("context_window = %#v, want 131072", got) + } + if got := resp["context_slots"]; got != 2 { + t.Fatalf("context_slots = %#v, want 2", got) + } + if got := resp["effective_max_total_tokens"]; got != 78643 { + t.Fatalf("effective_max_total_tokens = %#v, want 78643", got) + } + warnings, ok := resp["context_warnings"].([]string) + if !ok { + t.Fatalf("context_warnings type = %T", resp["context_warnings"]) + } + if !containsWarning(warnings, "max_total_tokens_clamped") { + t.Fatalf("missing clamp warning: %#v", warnings) + } + if countWarning(warnings, "max_total_tokens_clamped") != 1 { + t.Fatalf("clamp warning duplicated: %#v", warnings) + } +} + +func TestAddModelContextToResponseLocalUnknownWindowAssumes32K(t *testing.T) { + t.Parallel() + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{}, 120000, 0, "local") + if got := resp["effective_max_total_tokens"]; got != 16384 { + t.Fatalf("effective_max_total_tokens = %#v, want 16384", got) + } + warnings := resp["context_warnings"].([]string) + if !containsWarning(warnings, "local_context_window_unknown") { + t.Fatalf("missing unknown-window warning: %#v", warnings) + } +} + +func TestAddModelContextToResponseHonorsOutputCap(t *testing.T) { + t.Parallel() + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{ContextWindow: 32768}, 120000, 4096, "local") + if got := resp["effective_max_total_tokens"]; got != 19660 { + t.Fatalf("effective_max_total_tokens = %#v, want 19660", got) + } +} + +func TestAddModelContextToResponseAutoProfileKeepsNoPreviewWithoutWindow(t *testing.T) { + t.Parallel() + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{}, 120000, 0, "auto") + if _, ok := resp["effective_max_total_tokens"]; ok { + t.Fatalf("auto profile without window should not preview a budget: %#v", resp) + } +} + +func TestDetectedBackendAutomaticallySelectsLocalProfile(t *testing.T) { + for _, backend := range []string{modelmeta.BackendOllama, modelmeta.BackendLlamaCpp, modelmeta.BackendLMStudio, modelmeta.BackendOMLX, modelmeta.BackendVLLM} { + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{Backend: backend, ContextWindow: 65536}, 120000, 8192, "auto") + if resp["context_profile"] != "local_auto" || resp["effective_max_total_tokens"] != 39321 { + t.Fatalf("%s: %+v", backend, resp) + } + } + for _, profile := range []string{"custom", "economy", "local"} { + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{Backend: modelmeta.BackendOMLX, ContextWindow: 65536}, 120000, 8192, profile) + if resp["context_profile"] != profile { + t.Fatalf("explicit profile changed: %+v", resp) + } + } + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{Backend: modelmeta.BackendOpenAICompatible, ContextWindow: 131072, Source: "openai_models_context_length"}, 120000, 8192, "local_auto") + if resp["context_profile"] != "" { + t.Fatalf("generic gateway was treated as local: %+v", resp) + } + resp = gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{Backend: modelmeta.BackendOllama, ContextWindow: 262144, RuntimeUnknown: true}, 120000, 8192, "auto") + if resp["effective_max_total_tokens"] != 19660 { + t.Fatalf("architecture limit treated as runtime limit: %+v", resp) + } +} + +func TestAddModelContextToResponseDedupesWarnings(t *testing.T) { + t.Parallel() + resp := gin.H{} + addModelContextToResponse(resp, modelmeta.ContextInfo{ + ContextWindow: 32768, + Warnings: []string{modelmeta.WarningContextUnknown, modelmeta.WarningContextUnknown}, + }, 64000, 0, "auto") + warnings := resp["context_warnings"].([]string) + if countWarning(warnings, modelmeta.WarningContextUnknown) != 1 { + t.Fatalf("warnings not deduped: %#v", warnings) + } + if !containsWarning(warnings, modelmeta.WarningBudgetExceedsWindow) { + t.Fatalf("missing budget-exceeds-window warning: %#v", warnings) + } +} + +func containsWarning(warnings []string, want string) bool { + for _, code := range warnings { + if code == want { + return true + } + } + return false +} + +func countWarning(warnings []string, want string) int { + count := 0 + for _, code := range warnings { + if code == want { + count++ + } + } + return count +} + +func TestContextProbePreservesCustomHeaders(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Header.Get("X-Gateway-Key") != "test-gateway" || r.Header.Get("X-Session") != "connect-test" { + t.Errorf("context probe omitted resolved gateway/session headers") + w.WriteHeader(http.StatusUnauthorized) + return + } + w.Header().Set("Content-Type", "application/json") + if r.URL.Path == "/props" { + w.Write([]byte(`{"default_generation_settings":{"n_ctx":32768}}`)) + return + } + w.WriteHeader(http.StatusNotFound) + })) + defer server.Close() + info := probeModelContext(context.Background(), "openai", server.URL+"/v1", "test-key", "test-model", map[string]string{"X-Gateway-Key": "test-gateway", "X-Session": "%session_id%"}) + if info.ContextWindow != 32768 { + t.Fatalf("context window = %d", info.ContextWindow) + } +} + +func TestContextProbeDoesNotRedirectCredentials(t *testing.T) { + destination := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + t.Error("context probe followed credential-bearing redirect") + })) + defer destination.Close() + source := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, destination.URL, http.StatusFound) })) + defer source.Close() + probeModelContext(context.Background(), "openai", source.URL+"/v1", "test-key", "test-model", map[string]string{"X-Gateway-Key": "test-gateway"}) +} diff --git a/internal/handler/config_probe_headers.go b/internal/handler/config_probe_headers.go index 1b7f8e284..b4b9810fd 100644 --- a/internal/handler/config_probe_headers.go +++ b/internal/handler/config_probe_headers.go @@ -3,6 +3,7 @@ package handler import ( "cyberstrike-ai/internal/config" "fmt" + "net/http" "strings" ) @@ -92,3 +93,18 @@ func probeHeaderURL(provider, raw string) string { } return raw } + +// Metadata probes use the same resolved headers as connection tests. Redirects +// are disabled by the caller so provider credentials cannot change origin. +type contextProbeTransport struct{ headers map[string]string } + +func (t contextProbeTransport) RoundTrip(req *http.Request) (*http.Response, error) { + copy := req.Clone(req.Context()) + copy.Header = req.Header.Clone() + for key, value := range t.headers { + if strings.TrimSpace(value) != "" { + copy.Header.Set(key, strings.ReplaceAll(value, "%session_id%", "connect-test")) + } + } + return http.DefaultTransport.RoundTrip(copy) +} diff --git a/internal/handler/config_testopenai_headers_test.go b/internal/handler/config_testopenai_headers_test.go index fc4423dd1..6b69cb162 100644 --- a/internal/handler/config_testopenai_headers_test.go +++ b/internal/handler/config_testopenai_headers_test.go @@ -13,6 +13,43 @@ import ( "cyberstrike-ai/internal/config" ) +func TestFirstOMLXConnectionProbeUsesDetectedLocalOutputLimit(t *testing.T) { + var completions int + mux := http.NewServeMux() + mux.HandleFunc("/v1/models", func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"data":[{"id":"local-model","owned_by":"omlx","max_model_len":65536}]}`)) + }) + mux.HandleFunc("/v1/chat/completions", func(w http.ResponseWriter, r *http.Request) { + completions++ + var payload map[string]any + if err := json.NewDecoder(r.Body).Decode(&payload); err != nil { + t.Error(err) + } + if payload["max_tokens"] != float64(5) || payload["max_completion_tokens"] != nil { + t.Errorf("first probe missed local output cap: %+v", payload) + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"id":"fixture","model":"local-model","choices":[{"message":{"role":"assistant","content":"Hi"}}]}`)) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + h := &ConfigHandler{config: &config.Config{}, logger: zap.NewNop()} + body, _ := json.Marshal(map[string]any{"provider": "openai_compatible", "base_url": srv.URL + "/v1", "api_key": "fixture", "model": "local-model"}) + w := httptest.NewRecorder() + c, _ := gin.CreateTestContext(w) + c.Request = httptest.NewRequest(http.MethodPost, "/api/config/test-openai", strings.NewReader(string(body))) + c.Request.Header.Set("Content-Type", "application/json") + h.TestOpenAI(c) + var result map[string]any + if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil { + t.Fatal(err) + } + if result["success"] != true || result["context_backend"] != "omlx" || result["context_profile"] != "local_auto" || completions != 1 { + t.Fatalf("unexpected probe result: %s, completions=%d", w.Body.String(), completions) + } +} + // TestTestOpenAIForwardsCustomHeaders 校验「测试连接」与真实对话一样会带上渠道自定义请求头： // 静态头原样发送，%session_id% 用探测值填充，Authorization 仍按 api_key 生成。 func TestTestOpenAIForwardsCustomHeaders(t *testing.T) { diff --git a/internal/modelmeta/context.go b/internal/modelmeta/context.go new file mode 100644 index 000000000..746c8fd08 --- /dev/null +++ b/internal/modelmeta/context.go @@ -0,0 +1,504 @@ +// Package modelmeta probes local inference servers for runtime model metadata. +// It is intentionally best-effort: callers treat an empty ContextWindow as "unknown". +package modelmeta + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/url" + "strconv" + "strings" + "time" +) + +const ( + BackendOllama = "ollama" + BackendLlamaCpp = "llamacpp" + BackendLMStudio = "lmstudio" + BackendOMLX = "omlx" + BackendVLLM = "vllm" + BackendOpenAICompatible = "openai_compatible" + + WarningOllamaV1IgnoresNumCtx = "ollama_v1_ignores_num_ctx" + WarningOllamaRuntimeUnknown = "ollama_runtime_context_unknown" + WarningRuntimeUnknown = "context_runtime_unknown" + WarningLlamaCppParallelSlots = "llamacpp_parallel_slots" + WarningContextUnknown = "context_window_unknown" + WarningBudgetExceedsWindow = "configured_budget_exceeds_context_window" + probeTimeout = 2 * time.Second +) + +type ContextInfo struct { + Backend string `json:"backend,omitempty"` + ContextWindow int `json:"context_window,omitempty"` + Source string `json:"context_source,omitempty"` + Loaded bool `json:"context_loaded,omitempty"` + RuntimeSlots int `json:"context_slots,omitempty"` + RuntimeUnknown bool `json:"context_runtime_unknown,omitempty"` + Warnings []string `json:"context_warnings,omitempty"` +} + +// IsLocalBackend requires a recognizable inference-server response. +func (info ContextInfo) IsLocalBackend() bool { + switch info.Backend { + case BackendOllama, BackendLlamaCpp, BackendLMStudio, BackendOMLX, BackendVLLM: + return true + default: + return false + } +} + +// DetectContext probes the endpoint in local-backend-first order and returns the +// most authoritative context window it can find. It never returns an error: a +// failed probe is represented by an empty ContextWindow plus a warning. +func DetectContext(ctx context.Context, baseURL, model, apiKey string, client *http.Client) ContextInfo { + if ctx == nil { + ctx = context.Background() + } + if client == nil { + client = &http.Client{Timeout: 5 * time.Second} + } + parsed, err := parseBaseURL(baseURL) + if err != nil { + return ContextInfo{Warnings: []string{WarningContextUnknown}} + } + base := strings.TrimSuffix(parsed.String(), "/") + model = strings.TrimSpace(model) + + for _, root := range probeRoots(parsed) { + if info, ok := detectOllama(ctx, client, root, base, model, apiKey); ok { + return info + } + if info, ok := detectLlamaCpp(ctx, client, root, model, apiKey); ok { + return info + } + if info, ok := detectLMStudio(ctx, client, root, model, apiKey); ok { + return info + } + } + return detectOpenAICompatible(ctx, client, base, model, apiKey) +} + +// probeRoots returns candidate server roots, most specific first: the base URL +// without its version suffix (covers reverse proxies such as /ollama/v1), then +// the scheme+host origin. Duplicates are removed. +func probeRoots(parsed *url.URL) []string { + roots := make([]string, 0, 2) + add := func(root string) { + root = strings.TrimSuffix(strings.TrimSpace(root), "/") + if root == "" { + return + } + for _, existing := range roots { + if existing == root { + return + } + } + roots = append(roots, root) + } + path := strings.TrimSuffix(parsed.Path, "/") + path = strings.TrimSuffix(path, "/v1") + prefix := parsed.Scheme + "://" + parsed.Host + path + add(prefix) + add(parsed.Scheme + "://" + parsed.Host) + return roots +} + +func parseBaseURL(raw string) (*url.URL, error) { + raw = strings.TrimSpace(raw) + if raw == "" { + raw = "https://api.openai.com/v1" + } + if !strings.Contains(raw, "://") { + raw = "http://" + raw + } + u, err := url.Parse(raw) + if err != nil || u.Host == "" { + return nil, err + } + return u, nil +} + +func detectOllama(ctx context.Context, client *http.Client, root, base, model, apiKey string) (ContextInfo, bool) { + var version struct { + Version string `json:"version"` + } + if status, err := probeJSON(ctx, client, http.MethodGet, root+"/api/version", apiKey, nil, &version); err != nil || status != http.StatusOK || strings.TrimSpace(version.Version) == "" { + return ContextInfo{}, false + } + + info := ContextInfo{Backend: BackendOllama} + if strings.HasSuffix(strings.TrimRight(base, "/"), "/v1") { + info.Warnings = append(info.Warnings, WarningOllamaV1IgnoresNumCtx) + } + + if n, loaded := ollamaRuntimeContext(ctx, client, root, model, apiKey); n > 0 { + info.ContextWindow = n + info.Source = "ollama_ps" + info.Loaded = loaded + return info, true + } + + show := struct { + Parameters string `json:"parameters"` + ModelInfo map[string]json.RawMessage `json:"model_info"` + }{} + body := map[string]string{"model": model} + if status, err := probeJSON(ctx, client, http.MethodPost, root+"/api/show", apiKey, body, &show); err == nil && status == http.StatusOK { + if n := ollamaNumCtx(show.Parameters); n > 0 { + info.ContextWindow = n + info.Source = "ollama_show_num_ctx" + return info, true + } + if n := ollamaModelContext(show.ModelInfo); n > 0 { + info.ContextWindow = n + info.Source = "ollama_show_model_max" + info.RuntimeUnknown = true + info.Warnings = append(info.Warnings, WarningOllamaRuntimeUnknown) + return info, true + } + } + info.RuntimeUnknown = true + info.Warnings = append(info.Warnings, WarningOllamaRuntimeUnknown, WarningContextUnknown) + return info, true +} + +func ollamaRuntimeContext(ctx context.Context, client *http.Client, root, model, apiKey string) (int, bool) { + var ps struct { + Models []struct { + Name string `json:"name"` + Model string `json:"model"` + ContextLength int `json:"context_length"` + } `json:"models"` + } + if status, err := probeJSON(ctx, client, http.MethodGet, root+"/api/ps", apiKey, nil, &ps); err != nil || status != http.StatusOK { + return 0, false + } + best := 0 + for _, m := range ps.Models { + if m.ContextLength <= 0 || !modelMatches(m.Name, model) && !modelMatches(m.Model, model) { + continue + } + if m.ContextLength > best { + best = m.ContextLength + } + } + if best == 0 && len(ps.Models) == 1 && ps.Models[0].ContextLength > 0 && strings.TrimSpace(model) == "" { + best = ps.Models[0].ContextLength + } + return best, best > 0 +} + +func ollamaNumCtx(parameters string) int { + for _, line := range strings.Split(parameters, "\n") { + fields := strings.Fields(line) + if len(fields) < 2 || !strings.EqualFold(fields[0], "num_ctx") { + continue + } + if n, err := strconv.Atoi(fields[1]); err == nil && n > 0 { + return n + } + } + return 0 +} + +func ollamaModelContext(modelInfo map[string]json.RawMessage) int { + arch := "" + if raw, ok := modelInfo["general.architecture"]; ok { + _ = json.Unmarshal(raw, &arch) + } + if arch != "" { + if raw, ok := modelInfo[arch+".context_length"]; ok { + if n := rawInt(raw); n > 0 { + return n + } + } + } + best := 0 + for key, raw := range modelInfo { + if !strings.HasSuffix(key, ".context_length") { + continue + } + if n := rawInt(raw); n > best { + best = n + } + } + return best +} + +func detectLlamaCpp(ctx context.Context, client *http.Client, root, model, apiKey string) (ContextInfo, bool) { + var props struct { + NCtx int `json:"n_ctx"` + DefaultGenerationSettings struct { + NCtx int `json:"n_ctx"` + Params struct { + NCtx int `json:"n_ctx"` + } `json:"params"` + } `json:"default_generation_settings"` + TotalSlots int `json:"total_slots"` + } + status, err := probeJSON(ctx, client, http.MethodGet, root+"/props", apiKey, nil, &props) + if err != nil || status != http.StatusOK { + return ContextInfo{}, false + } + n := props.DefaultGenerationSettings.Params.NCtx + if n <= 0 { + n = props.DefaultGenerationSettings.NCtx + } + if n <= 0 { + n = props.NCtx + } + if n <= 0 && props.TotalSlots <= 0 { + return ContextInfo{}, false + } + info := ContextInfo{Backend: BackendLlamaCpp, Loaded: true} + if props.TotalSlots > 0 { + info.RuntimeSlots = props.TotalSlots + } + if props.TotalSlots > 1 { + // n_ctx is the shared pool; per-slot capacity depends on the KV mode + // (--parallel historically divides it, unified KV may not). + info.Warnings = append(info.Warnings, WarningLlamaCppParallelSlots) + } + if n > 0 { + info.ContextWindow = n + info.Source = "llamacpp_props" + } else { + info.Warnings = append(info.Warnings, WarningContextUnknown) + } + return info, true +} + +type lmStudioModel struct { + ID string `json:"id"` + Type string `json:"type"` + State string `json:"state"` + MaxContextLength int `json:"max_context_length"` + LoadedContextLength int `json:"loaded_context_length"` + LoadedInstances []lmStudioInstance `json:"loaded_instances"` +} + +type lmStudioInstance struct { + ID string `json:"id"` + ContextLength int `json:"context_length"` + Config struct { + ContextLength int `json:"context_length"` + } `json:"config"` +} + +func detectLMStudio(ctx context.Context, client *http.Client, root, model, apiKey string) (ContextInfo, bool) { + var list struct { + Data []lmStudioModel `json:"data"` + Models []lmStudioModel `json:"models"` + } + paths := []string{"/api/v0/models", "/api/v1/models"} + for _, path := range paths { + status, err := probeJSON(ctx, client, http.MethodGet, root+path, apiKey, nil, &list) + if err != nil || status != http.StatusOK { + continue + } + models := list.Data + if len(models) == 0 { + models = list.Models + } + chosen := pickLMModel(models, model) + if chosen == nil || (chosen.Type == "" && chosen.State == "" && chosen.LoadedContextLength <= 0 && len(chosen.LoadedInstances) == 0) { + continue + } + info := ContextInfo{Backend: BackendLMStudio} + if chosen.LoadedContextLength > 0 { + info.ContextWindow = chosen.LoadedContextLength + info.Source = "lmstudio_loaded_context" + info.Loaded = true + return info, true + } + for _, inst := range chosen.LoadedInstances { + if inst.ContextLength > 0 { + info.ContextWindow = inst.ContextLength + info.Source = "lmstudio_loaded_instance" + info.Loaded = true + return info, true + } + if inst.Config.ContextLength > 0 { + info.ContextWindow = inst.Config.ContextLength + info.Source = "lmstudio_loaded_instance_config" + info.Loaded = true + return info, true + } + } + if chosen.MaxContextLength > 0 { + info.ContextWindow = chosen.MaxContextLength + info.Source = "lmstudio_model_max" + info.RuntimeUnknown = true + info.Warnings = append(info.Warnings, WarningRuntimeUnknown) + return info, true + } + info.Warnings = append(info.Warnings, WarningContextUnknown) + return info, true + } + return ContextInfo{}, false +} + +func pickLMModel(models []lmStudioModel, wanted string) *lmStudioModel { + for i := range models { + if modelMatches(models[i].ID, wanted) { + return &models[i] + } + } + if len(models) == 1 { + return &models[0] + } + return nil +} + +func detectOpenAICompatible(ctx context.Context, client *http.Client, base, model, apiKey string) ContextInfo { + var list struct { + Data []struct { + ID string `json:"id"` + OwnedBy string `json:"owned_by"` + MaxModelLen int `json:"max_model_len"` + ContextLength json.RawMessage `json:"context_length"` + MaxContextLength int `json:"max_context_length"` + } `json:"data"` + } + status, err := probeJSON(ctx, client, http.MethodGet, strings.TrimRight(base, "/")+"/models", apiKey, nil, &list) + if err != nil || status != http.StatusOK { + return ContextInfo{Backend: BackendOpenAICompatible, Warnings: []string{WarningContextUnknown}} + } + info := ContextInfo{Backend: BackendOpenAICompatible} + chosen := -1 + for i := range list.Data { + if modelMatches(list.Data[i].ID, model) { + chosen = i + break + } + } + if chosen < 0 && len(list.Data) == 1 { + chosen = 0 + } + if chosen >= 0 { + m := list.Data[chosen] + // A compatible model list alone does not identify a local runtime. + switch strings.ToLower(strings.TrimSpace(m.OwnedBy)) { + case BackendOMLX, BackendVLLM: + info.Backend = strings.ToLower(strings.TrimSpace(m.OwnedBy)) + } + switch { + case m.MaxModelLen > 0: + info.ContextWindow = m.MaxModelLen + info.Source = "openai_models_max_model_len" + case rawInt(m.ContextLength) > 0: + info.ContextWindow = rawInt(m.ContextLength) + info.Source = "openai_models_context_length" + case m.MaxContextLength > 0: + info.ContextWindow = m.MaxContextLength + info.Source = "openai_models_max_context_length" + } + if info.Backend == BackendOMLX { + if runtime, ok := detectOMLXRuntime(ctx, client, base, m.ID, apiKey); ok { + return runtime + } + if info.ContextWindow > 0 { + info.Source = strings.Replace(info.Source, "openai_", "omlx_", 1) + } + } + if info.ContextWindow > 0 { + info.Loaded = info.Backend != BackendOMLX + return info + } + } + info.Warnings = append(info.Warnings, WarningContextUnknown) + return info +} + +func detectOMLXRuntime(ctx context.Context, client *http.Client, base, model, apiKey string) (ContextInfo, bool) { + var status struct { + Models []struct { + ID string `json:"id"` + Alias string `json:"model_alias"` + MaxContextWindow int `json:"max_context_window"` + Loaded bool `json:"loaded"` + } `json:"models"` + } + code, err := probeJSON(ctx, client, http.MethodGet, strings.TrimRight(base, "/")+"/models/status", apiKey, nil, &status) + if err != nil || code != http.StatusOK { + return ContextInfo{}, false + } + for _, m := range status.Models { + if (modelMatches(m.ID, model) || modelMatches(m.Alias, model)) && m.MaxContextWindow > 0 { + return ContextInfo{Backend: BackendOMLX, ContextWindow: m.MaxContextWindow, Source: "omlx_models_status", Loaded: m.Loaded}, true + } + } + return ContextInfo{}, false +} + +func probeJSON(ctx context.Context, client *http.Client, method, target, apiKey string, body interface{}, out interface{}) (int, error) { + reqCtx, cancel := context.WithTimeout(ctx, probeTimeout) + defer cancel() + var reader io.Reader + if body != nil { + raw, err := json.Marshal(body) + if err != nil { + return 0, err + } + reader = strings.NewReader(string(raw)) + } + req, err := http.NewRequestWithContext(reqCtx, method, target, reader) + if err != nil { + return 0, err + } + if body != nil { + req.Header.Set("Content-Type", "application/json") + } + if strings.TrimSpace(apiKey) != "" { + req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(apiKey)) + } + resp, err := client.Do(req) + if err != nil { + return 0, err + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20)) + if err != nil { + return resp.StatusCode, err + } + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return resp.StatusCode, nil + } + if out == nil || len(raw) == 0 { + return resp.StatusCode, nil + } + if err := json.Unmarshal(raw, out); err != nil { + return resp.StatusCode, err + } + return resp.StatusCode, nil +} + +func rawInt(raw json.RawMessage) int { + if len(raw) == 0 { + return 0 + } + var n int + if err := json.Unmarshal(raw, &n); err == nil { + return n + } + var s string + if err := json.Unmarshal(raw, &s); err == nil { + n, _ = strconv.Atoi(strings.TrimSpace(s)) + } + return n +} + +func modelMatches(candidate, wanted string) bool { + c := strings.ToLower(strings.TrimSpace(candidate)) + w := strings.ToLower(strings.TrimSpace(wanted)) + if c == "" || w == "" { + return false + } + if c == w || strings.TrimSuffix(c, ":latest") == strings.TrimSuffix(w, ":latest") { + return true + } + return strings.HasSuffix(c, "/"+w) || strings.HasSuffix(w, "/"+c) +} diff --git a/internal/modelmeta/context_test.go b/internal/modelmeta/context_test.go new file mode 100644 index 000000000..3a598523c --- /dev/null +++ b/internal/modelmeta/context_test.go @@ -0,0 +1,362 @@ +package modelmeta + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "net/url" + "reflect" + "testing" +) + +func TestDetectContextOllamaLoadedRuntime(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/api/version", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]string{"version": "0.12.0"}) + }) + mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "models": []map[string]interface{}{ + {"name": "qwen3:8b", "model": "qwen3:8b", "context_length": 32768}, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "qwen3:8b", "", srv.Client()) + if info.Backend != BackendOllama || info.ContextWindow != 32768 || info.Source != "ollama_ps" || !info.Loaded { + t.Fatalf("unexpected info: %+v", info) + } + if !contains(info.Warnings, WarningOllamaV1IgnoresNumCtx) { + t.Fatalf("missing ollama v1 warning: %+v", info.Warnings) + } +} + +func TestDetectContextOllamaShowNumCtx(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/api/version", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]string{"version": "0.12.0"}) + }) + mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{"models": []interface{}{}}) + }) + mux.HandleFunc("/api/show", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "parameters": "num_ctx 16384\nstop \"<|endoftext|>\"", + "model_info": map[string]interface{}{ + "general.architecture": "qwen3", + "qwen3.context_length": 131072, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "qwen3:8b", "", srv.Client()) + if info.ContextWindow != 16384 || info.Source != "ollama_show_num_ctx" || info.RuntimeUnknown { + t.Fatalf("unexpected info: %+v", info) + } +} + +func TestDetectContextOllamaShowModelMax(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/api/version", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]string{"version": "0.12.0"}) + }) + mux.HandleFunc("/api/ps", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{"models": []interface{}{}}) + }) + mux.HandleFunc("/api/show", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "parameters": "", + "model_info": map[string]interface{}{ + "general.architecture": "qwen3", + "qwen3.context_length": 131072, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "qwen3:8b", "", srv.Client()) + if info.ContextWindow != 131072 || info.Source != "ollama_show_model_max" || !info.RuntimeUnknown { + t.Fatalf("unexpected info: %+v", info) + } + if !contains(info.Warnings, WarningOllamaRuntimeUnknown) { + t.Fatalf("missing runtime-unknown warning: %+v", info.Warnings) + } +} + +func TestDetectContextLlamaCppParallelSlotsWarns(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/props", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "default_generation_settings": map[string]interface{}{"n_ctx": 32768}, + "total_slots": 2, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "local-model", "", srv.Client()) + if info.ContextWindow != 32768 || info.RuntimeSlots != 2 { + t.Fatalf("unexpected info: %+v", info) + } + if !contains(info.Warnings, WarningLlamaCppParallelSlots) { + t.Fatalf("missing parallel-slots warning: %+v", info.Warnings) + } +} + +func TestDetectContextLlamaCppSingleSlotNoWarning(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/props", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "default_generation_settings": map[string]interface{}{"n_ctx": 32768}, + "total_slots": 1, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "local-model", "", srv.Client()) + if info.RuntimeSlots != 1 { + t.Fatalf("slots = %d, want 1", info.RuntimeSlots) + } + if contains(info.Warnings, WarningLlamaCppParallelSlots) { + t.Fatalf("unexpected parallel-slots warning: %+v", info.Warnings) + } +} + +func TestDetectContextLlamaCpp(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/props", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "default_generation_settings": map[string]interface{}{ + "n_ctx": 8192, + "params": map[string]interface{}{"n_ctx": 8192}, + }, + "total_slots": 1, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "local-model", "", srv.Client()) + if info.Backend != BackendLlamaCpp || info.ContextWindow != 8192 || info.Source != "llamacpp_props" || !info.Loaded { + t.Fatalf("unexpected info: %+v", info) + } +} + +func TestDetectContextLMStudioLoadedInstance(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/api/v0/models", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "data": []map[string]interface{}{ + { + "id": "qwen/qwen3-8b", + "type": "llm", + "state": "loaded", + "max_context_length": 131072, + "loaded_instances": []map[string]interface{}{ + {"id": "qwen/qwen3-8b", "config": map[string]interface{}{"context_length": 16384}}, + }, + }, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "qwen/qwen3-8b", "", srv.Client()) + if info.Backend != BackendLMStudio || info.ContextWindow != 16384 || info.Source != "lmstudio_loaded_instance_config" || !info.Loaded { + t.Fatalf("unexpected info: %+v", info) + } +} + +func TestDetectContextVLLMModels(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "data": []map[string]interface{}{ + {"id": "Qwen/Qwen3-8B", "max_model_len": 65536}, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "Qwen/Qwen3-8B", "", srv.Client()) + if info.Backend != BackendOpenAICompatible || info.ContextWindow != 65536 || info.Source != "openai_models_max_model_len" { + t.Fatalf("unexpected info: %+v", info) + } +} + +func TestDetectContextOMLXAndVLLMIdentity(t *testing.T) { + for _, tc := range []struct { + name, owner, backend, source string + status, window int + }{ + {"omlx runtime alias", "omlx", BackendOMLX, "omlx_models_status", 200, 32768}, + {"omlx inference key", "omlx", BackendOMLX, "omlx_models_max_model_len", 403, 262144}, + {"vllm", "vllm", BackendVLLM, "openai_models_max_model_len", 404, 262144}, + {"gateway", "gateway", BackendOpenAICompatible, "openai_models_max_model_len", 404, 262144}, + } { + t.Run(tc.name, func(t *testing.T) { + statusCalls := 0 + mux := http.NewServeMux() + mux.HandleFunc("/edge/v1/models", func(w http.ResponseWriter, r *http.Request) { + if r.Header.Get("Authorization") != "Bearer fixture" { + t.Error("missing credential") + } + _ = json.NewEncoder(w).Encode(map[string]any{"data": []map[string]any{{"id": "alias", "owned_by": tc.owner, "max_model_len": 262144}}}) + }) + mux.HandleFunc("/edge/v1/models/status", func(w http.ResponseWriter, r *http.Request) { + statusCalls++ + if r.Method != http.MethodGet { + t.Error("probe must be read-only") + } + w.WriteHeader(tc.status) + _ = json.NewEncoder(w).Encode(map[string]any{"models": []map[string]any{{"id": "mlx-community/model", "model_alias": "alias", "max_context_window": 32768, "loaded": true}}}) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + info := DetectContext(context.Background(), srv.URL+"/edge/v1", "alias", "fixture", srv.Client()) + if info.Backend != tc.backend || info.ContextWindow != tc.window || info.Source != tc.source { + t.Fatalf("unexpected metadata: %+v", info) + } + if (statusCalls > 0) != (tc.owner == "omlx") || info.IsLocalBackend() != (tc.owner != "gateway") { + t.Fatalf("backend identity mismatch: %+v, status probes=%d", info, statusCalls) + } + }) + } +} + +func TestGenericGatewaySuccessResponsesDoNotIdentifyLocalBackend(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]any{"data": []map[string]any{{"id": "generic", "context_length": 65536}}}) + })) + defer srv.Close() + info := DetectContext(context.Background(), srv.URL+"/v1", "generic", "", srv.Client()) + if info.IsLocalBackend() || info.Backend != BackendOpenAICompatible || info.ContextWindow != 65536 { + t.Fatalf("generic JSON success misidentified: %+v", info) + } +} + +func TestDetectContextOpenAISingleModelFallback(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "data": []map[string]interface{}{ + {"id": "Qwen/Qwen3-8B", "max_model_len": 65536}, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "local-alias", "", srv.Client()) + if info.ContextWindow != 65536 { + t.Fatalf("single-model fallback failed: %+v", info) + } +} + +func TestDetectContextUnknown(t *testing.T) { + t.Parallel() + srv := httptest.NewServer(http.NotFoundHandler()) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/v1", "x", "", srv.Client()) + if info.ContextWindow != 0 || !contains(info.Warnings, WarningContextUnknown) { + t.Fatalf("unexpected info: %+v", info) + } +} + +func TestDetectContextOllamaBehindPrefix(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/ollama/api/version", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]string{"version": "0.12.0"}) + }) + mux.HandleFunc("/ollama/api/ps", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{ + "models": []map[string]interface{}{ + {"name": "qwen3:8b", "context_length": 65536}, + }, + }) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/ollama/v1", "qwen3:8b", "", srv.Client()) + if info.Backend != BackendOllama || info.ContextWindow != 65536 || info.Source != "ollama_ps" { + t.Fatalf("unexpected info behind prefix: %+v", info) + } +} + +func TestDetectContextLlamaCppBehindPrefix(t *testing.T) { + t.Parallel() + mux := http.NewServeMux() + mux.HandleFunc("/llama/props", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]interface{}{"n_ctx": 16384}) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + info := DetectContext(context.Background(), srv.URL+"/llama/v1", "local-model", "", srv.Client()) + if info.Backend != BackendLlamaCpp || info.ContextWindow != 16384 { + t.Fatalf("unexpected info behind prefix: %+v", info) + } +} + +func TestProbeRootsPrefersPrefix(t *testing.T) { + t.Parallel() + parsed, err := url.Parse("https://example.com/ollama/v1/") + if err != nil { + t.Fatal(err) + } + got := probeRoots(parsed) + want := []string{"https://example.com/ollama", "https://example.com"} + if !reflect.DeepEqual(got, want) { + t.Fatalf("probeRoots() = %#v, want %#v", got, want) + } +} + +func TestModelMatches(t *testing.T) { + t.Parallel() + cases := []struct { + candidate string + wanted string + want bool + }{ + {"qwen3:8b", "qwen3:8b", true}, + {"qwen3:latest", "qwen3", true}, + {"qwen/qwen3-8b", "qwen3-8b", true}, + {"Qwen/Qwen3-8B", "qwen/qwen3-8b", true}, + {"other", "qwen3", false}, + } + for _, tc := range cases { + if got := modelMatches(tc.candidate, tc.wanted); got != tc.want { + t.Fatalf("modelMatches(%q,%q) = %v, want %v", tc.candidate, tc.wanted, got, tc.want) + } + } +} + +func contains(items []string, want string) bool { + for _, item := range items { + if item == want { + return true + } + } + return false +} diff --git a/internal/multiagent/pi_integration_test.go b/internal/multiagent/pi_integration_test.go index b97586834..c560c9e96 100644 --- a/internal/multiagent/pi_integration_test.go +++ b/internal/multiagent/pi_integration_test.go @@ -9,6 +9,8 @@ import ( "os" "os/exec" "path/filepath" + "regexp" + "slices" "strings" "sync" "sync/atomic" @@ -16,9 +18,11 @@ import ( "time" "cyberstrike-ai/internal/agent" + "cyberstrike-ai/internal/authctx" "cyberstrike-ai/internal/config" "cyberstrike-ai/internal/database" "cyberstrike-ai/internal/mcp" + "cyberstrike-ai/internal/mcp/builtin" "go.uber.org/zap" ) @@ -130,7 +134,7 @@ func TestPiDeleteTurnExcludesDeletedContentFromNextModelRequest(t *testing.T) { } } -func piTestConfig(t *testing.T) *config.Config { +func piTestWorkerPath(t *testing.T) string { t.Helper() if _, err := exec.LookPath("node"); err != nil { t.Skip("Node not installed") @@ -142,6 +146,12 @@ func piTestConfig(t *testing.T) *config.Config { if _, err = os.Stat(worker); err != nil { t.Skip("build runtime/pi before SDK integration tests") } + return worker +} + +func piTestConfig(t *testing.T) *config.Config { + t.Helper() + worker := piTestWorkerPath(t) cfg := &config.Config{AgentRuntime: config.AgentRuntimeConfig{Engine: "pi", WorkerPath: worker, SessionDir: t.TempDir()}, OpenAI: config.OpenAIConfig{Provider: "openai", APIKey: "fixture-key", Model: "fixture", MaxTotalTokens: 32000, MaxCompletionTokens: 1024}} cfg.Agent.MaxIterations = 8 cfg.MultiAgent.Skills.Disable = true @@ -166,6 +176,192 @@ func piReply(w http.ResponseWriter, tool, id string, args map[string]any, text s fmt.Fprint(w, "data: [DONE]\n\n") } +func TestPiDeepSeekOutputBudgetAndUsage(t *testing.T) { + for _, tc := range []struct { + profile, mode, thinking, effort string + reasoning int + }{ + {profile: "deepseek", mode: "auto", thinking: "enabled", effort: "low", reasoning: 6}, + {profile: "deepseek_compat", mode: "off", thinking: "disabled"}, + } { + t.Run(tc.profile, func(t *testing.T) { + cfg := piTestConfig(t) + cfg.OpenAI.Model = "deepseek-flash" + cfg.OpenAI.Reasoning = config.OpenAIReasoningConfig{Profile: tc.profile, Mode: tc.mode, Effort: "low"} + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + var body struct { + MaxTokens int `json:"max_tokens"` + MaxCompletionTokens *int `json:"max_completion_tokens"` + Effort string `json:"reasoning_effort"` + Thinking struct { + Type string `json:"type"` + } `json:"thinking"` + StreamOptions struct { + IncludeUsage bool `json:"include_usage"` + } `json:"stream_options"` + Tools []struct { + Function struct { + Name string `json:"name"` + } `json:"function"` + } `json:"tools"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + t.Error(err) + http.Error(w, "invalid fixture request", http.StatusBadRequest) + return + } + if body.MaxTokens != 1024 || body.MaxCompletionTokens != nil || !body.StreamOptions.IncludeUsage { + t.Errorf("unexpected budget or usage options: %+v", body) + } + if body.Thinking.Type != tc.thinking || body.Effort != tc.effort { + t.Errorf("unexpected thinking options: %+v", body) + } + var names []string + for _, tool := range body.Tools { + names = append(names, tool.Function.Name) + } + if !slices.Equal(names, []string{"echo", "secret"}) { + t.Errorf("unexpected tool catalog: %v", names) + } + w.Header().Set("Content-Type", "text/event-stream") + fmt.Fprint(w, `data: {"id":"fixture","object":"chat.completion.chunk","model":"deepseek-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"ok"},"finish_reason":"stop"}]}`+"\n\n") + fmt.Fprintf(w, `data: {"id":"fixture","object":"chat.completion.chunk","model":"deepseek-flash","choices":[],"usage":{"prompt_tokens":100,"prompt_cache_hit_tokens":75,"prompt_cache_miss_tokens":25,"completion_tokens":20,"completion_tokens_details":{"reasoning_tokens":%d},"total_tokens":120}}`+"\n\n", tc.reasoning) + fmt.Fprint(w, "data: [DONE]\n\n") + })) + defer server.Close() + cfg.OpenAI.BaseURL = server.URL + "/v1" + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + var usage map[string]any + result, err := RunSingleAgent(ctx, cfg, &cfg.MultiAgent, piTestAgent(cfg, nil), nil, zap.NewNop(), "", "", "Say ok", nil, nil, func(kind, _ string, data interface{}) { + if kind == "agent_usage_summary" { + usage = data.(map[string]any) + } + }, nil, "") + if err != nil || result == nil || result.Response != "ok" || result.RuntimeStatus != "completed" { + t.Fatalf("result=%+v err=%v", result, err) + } + for field, want := range map[string]int{ + "modelCalls": 1, "promptTokens": 100, "completionTokens": 20, + "cachedTokens": 75, "cacheWriteTokens": 0, "reasoningTokens": tc.reasoning, "totalTokens": 120, + } { + if usage[field] != want { + t.Errorf("%s=%v, want %d (usage=%v)", field, usage[field], want, usage) + } + } + }) + } +} + +func TestPiDeepSeekEconomyDiscoversToolsAndReadsFullEvidence(t *testing.T) { + // Skip the parent too: its final assertion compares measurements from both subtests. + piTestWorkerPath(t) + full := strings.Repeat("fixture output line with details\n", 150) + "MIDDLE_EVIDENCE\n" + strings.Repeat("trailing output line with details\n", 150) + "end" + var baselineBytes, optimizedBytes int + for _, profile := range []string{"custom", "auto"} { + t.Run(profile, func(t *testing.T) { + cfg := piTestConfig(t) + cfg.OpenAI.Reasoning.Profile = "deepseek" + cfg.OpenAI.ContextProfile = profile + cfg.MultiAgent.Skills.Disable = false + cfg.MultiAgent.Runtime.ToolSearchEnable = true + cfg.MultiAgent.Runtime.ReductionRootDir = t.TempDir() + var calls int + var originalPath string + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { + var body struct { + Tools json.RawMessage `json:"tools"` + Messages []struct { + Role string + Content any + } `json:"messages"` + } + if err := json.NewDecoder(req.Body).Decode(&body); err != nil { + t.Error(err) + return + } + calls++ + if profile == "custom" { + baselineBytes = len(body.Tools) + piReply(w, "", "", nil, "baseline") + return + } + switch calls { + case 1: + optimizedBytes = len(body.Tools) + if strings.Contains(string(body.Tools), `"name":"record_vulnerability"`) { + t.Error("unused builtin schema was exposed before discovery") + } + piReply(w, "tool_search", "discover", map[string]any{"query": "record_vulnerability", "limit": 1}, "") + case 2: + if !strings.Contains(string(body.Tools), `"name":"record_vulnerability"`) { + t.Error("discovered tool is unavailable") + } + if !strings.Contains(string(body.Tools), `"name":"read_file"`) { + t.Error("read_file was lost during tool discovery") + } + piReply(w, "record_vulnerability", "record", map[string]any{"text": "fixture"}, "") + case 3: + preview, _ := body.Messages[len(body.Messages)-1].Content.(string) + match := regexp.MustCompile(`Full output saved to: ([^\n]+)`).FindStringSubmatch(preview) + if len(match) != 2 || len(preview) > 8000 || strings.Contains(preview, "MIDDLE_EVIDENCE") { + t.Errorf("expected recoverable preview, got %d bytes", len(preview)) + piReply(w, "", "", nil, "failed preview") + return + } + originalPath = match[1] + t.Logf("tool result bytes: full=%d preview=%d", len(full)+1, len(preview)) + piReply(w, "read_file", "read", map[string]any{"path": originalPath, "offset": 151, "limit": 1}, "") + case 4: + if got := body.Messages[len(body.Messages)-1].Content; got != "MIDDLE_EVIDENCE" { + t.Errorf("evidence readback=%q", got) + } + piReply(w, "", "", nil, "verified middle evidence") + default: + t.Error("unexpected retry") + piReply(w, "", "", nil, "unexpected") + } + })) + defer server.Close() + cfg.OpenAI.BaseURL = server.URL + "/v1" + srv := mcp.NewServer(zap.NewNop()) + srv.ConfigureToolResultSpillRoot(t.TempDir()) + srv.SetToolAuthorizer(func(context.Context, string, map[string]interface{}) error { return nil }) + var executed int + for _, name := range builtin.GetAllBuiltinTools() { + srv.RegisterTool(mcp.Tool{Name: name, Description: "Fixture operation " + name, + InputSchema: piToolSchema(map[string]any{"text": map[string]any{"type": "string", "description": "Fixture evidence text"}}, "text")}, + func(context.Context, map[string]interface{}) (*mcp.ToolResult, error) { + executed++ + return &mcp.ToolResult{Content: []mcp.Content{{Type: "text", Text: full}}}, nil + }) + } + ag := agent.NewAgent(&cfg.OpenAI, &cfg.Agent, srv, nil, zap.NewNop(), 8) + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + ctx = authctx.WithPrincipal(ctx, authctx.NewPrincipal("fixture", "fixture", "all", map[string]bool{"agent:local-execute": true})) + res, err := RunSingleAgent(ctx, cfg, &cfg.MultiAgent, ag, nil, zap.NewNop(), "economy-fixture", "", "Read the evidence", nil, nil, nil, nil, "") + if err != nil || res == nil || res.RuntimeStatus != "completed" { + t.Fatalf("result=%+v error=%v", res, err) + } + if profile == "auto" { + if calls != 4 || executed != 1 || res.Response != "verified middle evidence" { + t.Fatalf("calls=%d executed=%d result=%+v", calls, executed, res) + } + b, err := os.ReadFile(originalPath) + // The host adds a newline after each MCP text block. + if err != nil || string(b) != full+"\n" { + t.Fatalf("full evidence changed: err=%v got=%d want=%d", err, len(b), len(full)+1) + } + } + }) + } + if optimizedBytes >= baselineBytes/2 { + t.Errorf("tool schema savings too small: custom=%d auto=%d", baselineBytes, optimizedBytes) + } + t.Logf("first-request tool schema bytes: custom=%d auto=%d", baselineBytes, optimizedBytes) +} + func piTestAgent(cfg *config.Config, handler mcp.ToolHandler) *agent.Agent { srv := mcp.NewServer(zap.NewNop()) schema := map[string]any{"type": "object", "properties": map[string]any{"text": map[string]any{"type": "string"}}, "required": []string{"text"}} diff --git a/internal/multiagent/pi_model.go b/internal/multiagent/pi_model.go index e1b737caf..7b73d53c4 100644 --- a/internal/multiagent/pi_model.go +++ b/internal/multiagent/pi_model.go @@ -30,19 +30,25 @@ func piModelConfig(oa config.OpenAIConfig, intent *reasoning.ClientIntent, conve } headers[key] = strings.ReplaceAll(value, "%session_id%", conversationID) } - budget := oa.MaxTotalTokens - if budget <= 0 { - budget = oa.MaxCompletionTokensEffective() + budget := oa.EffectiveMaxTotalTokens() + window := oa.EffectiveContextWindow() + if window <= 0 { + window = budget + oa.MaxCompletionTokensEffective() } - window := budget + oa.MaxCompletionTokensEffective() extra := reasoning.ProviderExtraFields(&oa, intent) if api == "openai-responses" { extra = oa.Reasoning.ExtraRequestFields } compat := map[string]any{} - if oa.Reasoning.ProfileEffective() == "deepseek_compat" || (oa.Reasoning.ProfileEffective() == "auto" && oa.IsDeepSeekEndpointOrModel()) { + if oa.IsLocalContextProfile() { + compat["maxTokensField"] = "max_tokens" + compat["supportsStore"] = false + } + profile := oa.Reasoning.ProfileEffective() + if api == "openai-completions" && (profile == "deepseek" || profile == "deepseek_compat" || (profile == "auto" && oa.IsDeepSeekEndpointOrModel())) { compat["thinkingFormat"] = "deepseek" compat["requiresReasoningContentOnAssistantMessages"] = true + compat["maxTokensField"] = "max_tokens" } - return agentruntime.Model{Provider: provider, API: api, ID: oa.Model, BaseURL: base, APIKey: oa.APIKey, Headers: headers, ContextWindow: window, InputBudget: budget, MaxTokens: oa.MaxCompletionTokensEffective(), Thinking: thinking, ExtraFields: extra, Compat: compat} + return agentruntime.Model{Provider: provider, API: api, ID: oa.Model, BaseURL: base, APIKey: oa.APIKey, Headers: headers, ContextWindow: window, InputBudget: budget, MaxTokens: oa.MaxCompletionTokensEffective(), Thinking: thinking, ExtraFields: extra, Compat: compat, CompactToolJSON: oa.IsEconomyContextProfile()} } diff --git a/internal/multiagent/pi_runner.go b/internal/multiagent/pi_runner.go index de02e60e6..3e0b5330c 100644 --- a/internal/multiagent/pi_runner.go +++ b/internal/multiagent/pi_runner.go @@ -6,6 +6,7 @@ import ( "encoding/json" "fmt" "path/filepath" + "slices" "strings" "sync" "sync/atomic" @@ -224,6 +225,11 @@ func runPiMulti(ctx context.Context, cfg *config.Config, ma *config.MultiAgentCo func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { defs := r.ag.ToolsForRole(r.roleTools) + // MCP catalogs are backed by maps. Keep the tool prefix and the first N + // always-visible tools stable across runs with the same catalog. + slices.SortFunc(defs, func(a, b agent.Tool) int { + return strings.Compare(a.Function.Name, b.Function.Name) + }) localDefs, localNames, skillPrompt, err := r.piLocalCatalog(ctx) if err != nil { return nil, err @@ -301,6 +307,9 @@ func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { if visibleCount <= 0 { visibleCount = 12 } + if oa.IsLocalContextProfile() && visibleCount > 6 { + visibleCount = 6 + } for i, t := range tools { if i >= visibleCount { break @@ -397,13 +406,22 @@ func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { result.IsError = true } if r.cfg.MultiAgent.Runtime.ReductionEnable { - result.Content = tooloutput.BoundWithSpill(result.Content, r.cfg.MultiAgent.Runtime.ReductionMaxLengthForTruncEffective(), tooloutput.SpillOpts{RootDir: r.cfg.MultiAgent.Runtime.ReductionRootDir, ConversationID: r.conversationID, ExecutionID: result.ExecutionID}) + limit := r.cfg.MultiAgent.Runtime.ReductionMaxLengthForTruncEffective() + opts := tooloutput.SpillOpts{RootDir: r.cfg.MultiAgent.Runtime.ReductionRootDir, ConversationID: r.conversationID, ExecutionID: result.ExecutionID} + if oa.IsEconomyContextProfile() { + // Automatic previews require an authorized way to read the full evidence. + if localNames["read_file"] { + result.Content = tooloutput.ReduceForModel(result.Content, limit, opts) + } + } else { + result.Content = tooloutput.BoundWithSpill(result.Content, limit, opts) + } } progress("tool_result", result.Content, map[string]any{"toolName": call.Name, "toolCallId": call.ID, "executionId": result.ExecutionID, "result": result.Content, "success": !result.IsError, "isError": result.IsError, "agentFacing": true}) return result, err } var text, thought string - var modelCalls, promptTokens, completionTokens, cacheRead, cacheWrite int + var modelCalls, promptTokens, completionTokens, cacheRead, cacheWrite, reasoningTokens int reasonID := func() string { return fmt.Sprintf("%s-reasoning-%d", runID, iteration.Load()) } emit := func(e agentruntime.Event) { if e.Type == "compaction_end" { @@ -439,6 +457,7 @@ func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { Output int `json:"output"` CacheRead int `json:"cacheRead"` CacheWrite int `json:"cacheWrite"` + Reasoning int `json:"reasoning"` } `json:"usage"` } `json:"message"` } @@ -448,6 +467,7 @@ func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { completionTokens += d.Message.Usage.Output cacheRead += d.Message.Usage.CacheRead cacheWrite += d.Message.Usage.CacheWrite + reasoningTokens += d.Message.Usage.Reasoning } } @@ -505,7 +525,11 @@ func (r *piRun) run(ctx context.Context, message string) (*RunResult, error) { progress("reasoning_chain_stream_end", thought, map[string]any{"streamId": reasonID()}) } if modelCalls > 0 { - progress("agent_usage_summary", "Pi token usage summary", map[string]any{"model": oa.Model, "modelCalls": modelCalls, "promptTokens": promptTokens, "completionTokens": completionTokens, "totalTokens": promptTokens + completionTokens, "cachedTokens": cacheRead, "cacheWriteTokens": cacheWrite}) + progress("agent_usage_summary", "Pi token usage summary", map[string]any{ + "model": oa.Model, "modelCalls": modelCalls, + "promptTokens": promptTokens, "completionTokens": completionTokens, "totalTokens": promptTokens + completionTokens, + "cachedTokens": cacheRead, "cacheWriteTokens": cacheWrite, "reasoningTokens": reasoningTokens, + }) } if res == nil { return nil, runErr diff --git a/internal/multiagent/runtime_dispatch.go b/internal/multiagent/runtime_dispatch.go index 70633a634..e56f2b992 100644 --- a/internal/multiagent/runtime_dispatch.go +++ b/internal/multiagent/runtime_dispatch.go @@ -30,6 +30,9 @@ func RunSingleAgent( if err := appCfg.AgentRuntime.Validate(); err != nil { return nil, err } + copyConfig := *appCfg + copyConfig.MultiAgent.Runtime = config.EffectiveRuntimeForChannel(appCfg.MultiAgent.Runtime, appCfg.OpenAI) + appCfg = ©Config return runPiSingle(ctx, appCfg, ag, db, logger, conversationID, projectID, userMessage, history, roleTools, progress, reasoningClient, systemPromptExtra) } return nil, fmt.Errorf("agent configuration is nil") @@ -56,6 +59,9 @@ func RunMultiAgent( if err := appCfg.AgentRuntime.Validate(); err != nil { return nil, err } + copyConfig := *appCfg + copyConfig.MultiAgent.Runtime = config.EffectiveRuntimeForChannel(appCfg.MultiAgent.Runtime, appCfg.OpenAI) + appCfg = ©Config return runPiMulti(ctx, appCfg, ma, ag, db, logger, conversationID, projectID, userMessage, history, roleTools, progress, agentsMarkdownDir, orchestrationOverride, reasoningClient, systemPromptExtra) } return nil, fmt.Errorf("agent configuration is nil") diff --git a/internal/tooloutput/economy_test.go b/internal/tooloutput/economy_test.go new file mode 100644 index 000000000..72d8a0572 --- /dev/null +++ b/internal/tooloutput/economy_test.go @@ -0,0 +1,38 @@ +package tooloutput + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestReduceForModelRetainsEvidence(t *testing.T) { + jsonText := "{\n \"id\": 9007199254740993,\n \"value\": \"keep spaces\\n\",\n \"id\": 2\n}" + if got := ReduceForModel(jsonText, 8000, SpillOpts{}); got != `{"id":9007199254740993,"value":"keep spaces\n","id":2}` { + t.Fatalf("JSON value or numeric precision changed: %s", got) + } + full := strings.Repeat("first line\n", 1000) + "middle evidence\n" + strings.Repeat("last line\n", 1000) + root := t.TempDir() + opts := SpillOpts{RootDir: root, ConversationID: "fixture", ExecutionID: "result"} + preview := ReduceForModel(full, 8000, opts) + if len(preview) >= len(full) || len(preview) > 8000 || !strings.Contains(preview, "read_file") { + t.Fatalf("invalid preview: %d bytes", len(preview)) + } + b, err := os.ReadFile(filepath.Join(SessionRoot(root, "", "fixture"), "trunc", "result")) + if err != nil || string(b) != full { + t.Fatalf("original evidence lost: %v", err) + } + ReduceForModel(preview, 1024, opts) + b, err = os.ReadFile(filepath.Join(SessionRoot(root, "", "fixture"), "trunc", "result")) + if err != nil || string(b) != full { + t.Fatal("reducing an existing preview overwrote full evidence") + } + blocked := filepath.Join(root, "file") + if err := os.WriteFile(blocked, []byte("file"), 0600); err != nil { + t.Fatal(err) + } + if got := ReduceForModel(full, 8000, SpillOpts{RootDir: blocked}); got != full { + t.Fatal("failed spill lost evidence") + } +} diff --git a/internal/tooloutput/spill.go b/internal/tooloutput/spill.go index 1352e219f..e5f0ca722 100644 --- a/internal/tooloutput/spill.go +++ b/internal/tooloutput/spill.go @@ -4,6 +4,8 @@ package tooloutput import ( + "bytes" + "encoding/json" "fmt" "os" "path/filepath" @@ -14,6 +16,30 @@ import ( "github.com/google/uuid" ) +// ReduceForModel first removes JSON formatting whitespace. If the result still +// exceeds the limit, retain the exact original evidence and return a preview. +// A failed spill keeps the content available to the model. +func ReduceForModel(full string, maxBytes int, opts SpillOpts) string { + // The MCP storage guard may already have saved this execution under the + // same path. Spilling its notice again would overwrite the original file. + if strings.HasPrefix(strings.TrimSpace(full), "") { + return full + } + compact := full + var buf bytes.Buffer + if json.Compact(&buf, []byte(full)) == nil { + compact = buf.String() + } + if maxBytes <= 0 || len(compact) <= maxBytes { + return compact + } + path, err := WriteTruncFile(opts, full) + if err != nil { + return compact + } + return FormatPersistedOutput(full, path, maxBytes) +} + const ( defaultRootDir = "tmp/reduction" readFileHint = "read_file" diff --git a/runtime/pi/src/tool-output.ts b/runtime/pi/src/tool-output.ts new file mode 100644 index 000000000..b4f47ca22 --- /dev/null +++ b/runtime/pi/src/tool-output.ts @@ -0,0 +1,13 @@ +// Compact only valid JSON containers. Retain the original numeric lexemes, +// key order, duplicate keys and string escapes instead of reserializing parsed +// values. Stored session messages and evidence keep their original text. +export function compactToolJSON(text: string): string { + const start = text.trimStart()[0]; + if (start !== "{" && start !== "[") return text; + try { + JSON.parse(text); + } catch { + return text; + } + return text.replace(/"(?:[^"\\]|\\.)*"|\s+/g, part => part.startsWith('"') ? part : ""); +} diff --git a/runtime/pi/src/worker.ts b/runtime/pi/src/worker.ts index 357d60edf..e1d66cd43 100644 --- a/runtime/pi/src/worker.ts +++ b/runtime/pi/src/worker.ts @@ -9,6 +9,7 @@ import { } from "@earendil-works/pi-coding-agent"; import { PROTOCOL_VERSION, RecordReader, send } from "./protocol.js"; import { installModelFetchPolicy } from "./network.js"; +import { compactToolJSON } from "./tool-output.js"; installModelFetchPolicy(); @@ -18,6 +19,7 @@ interface ModelConfig { contextWindow: number; maxTokens: number; thinking?: string; extraFields?: Record; compat?: Record; inputBudget?: number; + compactToolJSON?: boolean; } interface RunRequest { generateOnly?: boolean; @@ -201,20 +203,29 @@ async function run(req: RunRequest): Promise { modelConfigs.set(provider, config); } const streamSimple = runtime.streamSimple.bind(runtime); - runtime.streamSimple = (m, context, options) => streamSimple(m, - m.api === "openai-completions" ? { ...context, messages: pairToolResults(context.messages) } : context, { - ...options, - transport: "sse", - onPayload: async (original, currentModel) => { - const transformed = (await options?.onPayload?.(original, currentModel)) ?? original; - if (!transformed || typeof transformed !== "object") return transformed; - const payload = { ...transformed, ...modelConfigs.get(currentModel.provider)?.extraFields } as Record; - for (const key of ["model", "messages", "tools", "stream"]) { - if (key in transformed) payload[key] = (transformed as Record)[key]; - } - return payload; - }, - }); + runtime.streamSimple = (m, context, options) => { + let messages = context.messages; + if (modelConfigs.get(m.provider)?.compactToolJSON) { + messages = messages.map(message => message.role === "toolResult" ? { + ...message, + content: message.content.map(block => block.type === "text" ? { ...block, text: compactToolJSON(block.text) } : block), + } : message); + } + if (m.api === "openai-completions") messages = pairToolResults(messages); + return streamSimple(m, { ...context, messages }, { + ...options, + transport: "sse", + onPayload: async (original, currentModel) => { + const transformed = (await options?.onPayload?.(original, currentModel)) ?? original; + if (!transformed || typeof transformed !== "object") return transformed; + const payload = { ...transformed, ...modelConfigs.get(currentModel.provider)?.extraFields } as Record; + for (const key of ["model", "messages", "tools", "stream"]) { + if (key in transformed) payload[key] = (transformed as Record)[key]; + } + return payload; + }, + }); + }; let model = physical[0]; if (physical.length > 1) { runtime.registerVirtualModel({ provider: "cyberstrike-router", id: "auto", name: "CyberStrikeAI channels", @@ -263,7 +274,7 @@ async function run(req: RunRequest): Promise { } repairInterruptedCalls(manager); const searchEnabled = req.toolSearch === true && names.length >= (req.toolSearchMinTools || 20) && !names.includes("tool_search"); - const always = new Set(["task", "exit", "transfer_to_agent", "write_todos", "TaskCreate", "TaskUpdate", "TaskGet", "TaskList", "skill", "read_file", "execute", "wait_tool_execution", "get_tool_execution", ...(req.alwaysVisibleTools ?? [])]); + const always = new Set(["task", "exit", "transfer_to_agent", "write_todos", "TaskCreate", "TaskUpdate", "TaskGet", "TaskList", "skill", "read_file", "execute", "wait_tool_execution", "get_tool_execution", "cancel_tool_execution", ...(req.alwaysVisibleTools ?? [])]); const loader = resources(req.systemPrompt + (searchEnabled ? "\nDiscover additional task tools with tool_search. Search results enable matching tools for subsequent calls." : "")); if (searchEnabled) { const extensions = new DefaultResourceLoader({ cwd: scratch, agentDir: scratch, settingsManager: SettingsManager.inMemory(), diff --git a/runtime/pi/test/tool-output.test.mjs b/runtime/pi/test/tool-output.test.mjs new file mode 100644 index 000000000..3ab445850 --- /dev/null +++ b/runtime/pi/test/tool-output.test.mjs @@ -0,0 +1,11 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { compactToolJSON } from "../dist/tool-output.js"; + +test("tool JSON compaction preserves evidence, numeric lexemes and duplicate keys", () => { + const input = '{\n "id": 9007199254740993, "id": 2, "small": 1e-99,\n "text": "a b\\n\\t\\\"quoted\\\"", "nested": [ 1, 2 ]\n}'; + assert.equal(compactToolJSON(input), '{"id":9007199254740993,"id":2,"small":1e-99,"text":"a b\\n\\t\\\"quoted\\\"","nested":[1,2]}'); + for (const text of ['{ broken json }', ' ordinary text\n', '"a string"', '[output] partial', '']) { + assert.equal(compactToolJSON(text), text); + } +}); diff --git a/runtime/pi/test/worker.test.mjs b/runtime/pi/test/worker.test.mjs index 276d9bbe0..4baf70939 100644 --- a/runtime/pi/test/worker.test.mjs +++ b/runtime/pi/test/worker.test.mjs @@ -251,6 +251,39 @@ test("native tool search loads a deferred host tool without exposing the full ca assert.equal(hostCalls, 1); }); +test("economy compacts persisted tool JSON while preserving reasoning and original session evidence", { timeout: 30000 }, async t => { + const raw = '{\n "id": 9007199254740993,\n "evidence": "keep spaces\\n"\n}'; + const fixture = await startFixture(t, (_, res, count) => { + if (count === 1) sse(res, { role: "assistant", reasoning_content: "inspect fixture", tool_calls: [{ index: 0, id: "inspect", type: "function", function: { name: "echo", arguments: '{"text":"fixture"}' } }] }, "tool_calls"); + else sse(res, { role: "assistant", content: "evidence inspected" }, "stop"); + }); + const dir = await mkdtemp(join(tmpdir(), "csai-pi-economy-")); + t.after(() => rm(dir, { recursive: true, force: true })); + const req = request(fixture.url, dir); + req.tools.push(...Array.from({ length: 30 }, (_, i) => ({ name: "old_unused_" + i, description: "Unused fixture operation", parameters: { type: "object", properties: {} } }))); + req.model.thinking = "high"; + req.model.compat = { thinkingFormat: "deepseek", requiresReasoningContentOnAssistantMessages: true }; + req.model.extraFields = { thinking: { type: "enabled" }, reasoning_effort: "high" }; + const first = await invoke(t, req, (r, send) => { + if (r.type === "tool_call") send({ type: "tool_result", id: r.call.id, result: { content: raw } }); + }); + assert.equal(first.result.status, "completed", first.result.error); + assert.equal(fixture.requests[1].messages.find(m => m.role === "tool").content, raw); + + req.model.compactToolJSON = true; + const resumed = await invoke(t, { ...req, toolSearch: true, alwaysVisibleTools: ["echo"], runId: "economy-resume", message: "Continue" }); + assert.equal(resumed.result.status, "completed", resumed.result.error); + const body = fixture.requests[2]; + assert.equal(fixture.requests[0].tools.length, 31); + assert.deepEqual(body.tools.map(t => t.function.name).sort(), ["echo", "tool_search"]); + assert.equal(body.messages.find(m => m.role === "tool").content, '{"id":9007199254740993,"evidence":"keep spaces\\n"}'); + assert.ok(body.messages.some(m => m.reasoning_content === "inspect fixture")); + assert.equal(body.reasoning_effort, "high"); + assert.equal(resumed.result.history.find(m => m.role === "tool").content, raw); + const entries = (await readFile(first.result.sessionFile, "utf8")).trim().split("\n").map(line => JSON.parse(line)); + assert.ok(entries.some(e => e.message?.role === "toolResult" && e.message.content.some(b => b.text === raw))); +}); + test("provider custom credentials never cross a redirect origin", { timeout: 30000 }, async t => { let targetCalls = 0; const target = createServer((_, res) => { targetCalls++; res.end("unexpected"); }); diff --git a/web/static/i18n/en-US.json b/web/static/i18n/en-US.json index 4e3ba1a87..3ba7ba44f 100644 --- a/web/static/i18n/en-US.json +++ b/web/static/i18n/en-US.json @@ -3091,6 +3091,23 @@ "maxCompletionTokens": "Max Output Tokens", "maxCompletionTokensPlaceholder": "32768", "maxCompletionTokensHint": "Maximum tokens for a single model response. Default: 32768", + "contextWindow": "Model Context Window (tokens)", + "contextWindowPlaceholder": "131072", + "contextWindowHint": "Native model window used to validate and clamp the budget. For local models, match the server num_ctx/ctx-size.", + "detectContext": "Auto-detect", + "detectContextFillRequired": "Fill in Base URL and model first", + "detectedContext": "Detected {tokens} tokens", + "effectiveBudget": "Effective budget {tokens} tokens", + "contextWarnWindowUnknown": "No window set for the local profile; assuming 32768", + "contextWarnWindowSmall": "Window below 16384; tool-using agents may be unreliable", + "contextWarnBudgetClamped": "Max context budget was clamped to the model window", + "contextWarnCompletionClamped": "Max output tokens were clamped to the model window", + "contextWarnBudgetExceedsWindow": "Configured budget exceeds the model window; it will be clamped at runtime", + "contextWarnOllamaV1": "Ollama's /v1 endpoint ignores num_ctx; set OLLAMA_CONTEXT_LENGTH or a Modelfile parameter", + "contextWarnOllamaRuntime": "Ollama may cap the window below the model maximum based on VRAM; check ollama ps after loading", + "contextWarnRuntimeUnknown": "Runtime window could not be confirmed and may be below the model maximum", + "contextWarnLlamaCppSlots": "llama.cpp is running multiple parallel slots; the per-request window may be smaller than n_ctx. Check --parallel and the KV mode", + "contextWarnUnknown": "No context window detected; enter it manually", "openaiReasoningTitle": "Reasoning settings", "openaiReasoningHint": "Default reasoning settings for this AI channel; chat Session settings can override them.", "openaiReasoningProfile": "Wire profile", diff --git a/web/static/i18n/ru-RU.json b/web/static/i18n/ru-RU.json index d754136e0..8332f79ee 100644 --- a/web/static/i18n/ru-RU.json +++ b/web/static/i18n/ru-RU.json @@ -3104,6 +3104,23 @@ "maxCompletionTokens": "Макс. токенов ответа", "maxCompletionTokensPlaceholder": "32768", "maxCompletionTokensHint": "Максимум токенов одного ответа модели. По умолчанию: 32768", + "contextWindow": "Окно контекста модели (токены)", + "contextWindowPlaceholder": "131072", + "contextWindowHint": "Собственное окно модели для проверки и ограничения бюджета. Для локальных моделей укажите серверный num_ctx/ctx-size.", + "detectContext": "Автоопределение", + "detectContextFillRequired": "Сначала заполните Base URL и модель", + "detectedContext": "Обнаружено {tokens} токенов", + "effectiveBudget": "Эффективный бюджет {tokens} токенов", + "contextWarnWindowUnknown": "Для локального профиля не задано окно; используется 32768", + "contextWarnWindowSmall": "Окно меньше 16384; агент с инструментами может работать нестабильно", + "contextWarnBudgetClamped": "Бюджет контекста ограничен размером окна модели", + "contextWarnCompletionClamped": "Максимум выходных токенов ограничен размером окна модели", + "contextWarnBudgetExceedsWindow": "Бюджет превышает окно модели и будет ограничен во время работы", + "contextWarnOllamaV1": "Эндпоинт Ollama /v1 игнорирует num_ctx; задайте OLLAMA_CONTEXT_LENGTH или параметр Modelfile", + "contextWarnOllamaRuntime": "Ollama может уменьшить окно ниже максимума модели в зависимости от VRAM; проверьте ollama ps", + "contextWarnRuntimeUnknown": "Не удалось подтвердить рабочее окно; оно может быть меньше максимума модели", + "contextWarnLlamaCppSlots": "llama.cpp запущен с несколькими параллельными слотами; окно одного запроса может быть меньше n_ctx. Проверьте --parallel и режим KV", + "contextWarnUnknown": "Окно контекста не обнаружено; укажите его вручную", "openaiReasoningTitle": "Настройки рассуждения", "openaiReasoningHint": "Настройки рассуждения этого канала ИИ по умолчанию; настройки сессии чата могут их переопределить.", "openaiReasoningProfile": "Профиль передачи", diff --git a/web/static/i18n/zh-CN.json b/web/static/i18n/zh-CN.json index e0a556496..4df133ad2 100644 --- a/web/static/i18n/zh-CN.json +++ b/web/static/i18n/zh-CN.json @@ -3079,6 +3079,23 @@ "maxCompletionTokens": "最大输出 Token 数", "maxCompletionTokensPlaceholder": "32768", "maxCompletionTokensHint": "单次模型回复的输出上限，默认 32768", + "contextWindow": "模型上下文窗口（Token）", + "contextWindowPlaceholder": "131072", + "contextWindowHint": "模型原生窗口；用于校验并收紧预算。本地模型请按服务端 num_ctx/ctx-size 填写。", + "detectContext": "自动检测", + "detectContextFillRequired": "请先填写 Base URL 和模型", + "detectedContext": "检测到 {tokens} tokens", + "effectiveBudget": "有效预算 {tokens} tokens", + "contextWarnWindowUnknown": "本地档位未填写窗口，按 32768 估算", + "contextWarnWindowSmall": "窗口小于 16384，带工具的 agent 可能无法稳定运行", + "contextWarnBudgetClamped": "最大上下文预算已按模型窗口收紧", + "contextWarnCompletionClamped": "最大输出 Token 数已按模型窗口收紧", + "contextWarnBudgetExceedsWindow": "填写的预算超过模型窗口，运行时会自动收紧", + "contextWarnOllamaV1": "Ollama 的 /v1 接口忽略 num_ctx，请在服务端设置 OLLAMA_CONTEXT_LENGTH 或 Modelfile", + "contextWarnOllamaRuntime": "Ollama 可能按显存把窗口压到模型上限以下，建议启动后查看 ollama ps", + "contextWarnRuntimeUnknown": "无法确认运行时窗口，可能小于模型上限", + "contextWarnLlamaCppSlots": "llama.cpp 配置了多个并行槽位，单请求可用窗口可能小于 n_ctx；请按 --parallel/KV 模式确认", + "contextWarnUnknown": "未检测到上下文窗口，请手动填写", "openaiReasoningTitle": "推理设置", "openaiReasoningHint": "作为该 AI 通道的默认推理设置；对话页「会话设置」可覆盖。", "openaiReasoningProfile": "线路 profile", diff --git a/web/static/js/settings-pi-resilience.test.cjs b/web/static/js/settings-pi-resilience.test.cjs index ad5f1ecf9..8780efddb 100644 --- a/web/static/js/settings-pi-resilience.test.cjs +++ b/web/static/js/settings-pi-resilience.test.cjs @@ -2,6 +2,7 @@ const test = require('node:test'); const assert = require('node:assert/strict'); const fs = require('node:fs'); const path = require('node:path'); +const vm = require('node:vm'); const root = process.cwd(); const settings = fs.readFileSync(path.join(root, 'web/static/js/settings.js'), 'utf8'); @@ -66,3 +67,94 @@ test('切换或新增 AI 通道时会刷新推理线路下拉显示', () => { assert.match(settings, /syncSettingsCustomSelect\(profileEl\);/); assert.match(settings, /reasoning:\s*\{\s*mode:\s*'auto',\s*effort:\s*'',\s*profile:\s*'auto'/); }); + +test('隐藏上下文档位后保存和探测仍保留已有配置', async () => { + assert.doesNotMatch(template, /openai-context-profile|settingsBasic\.contextProfile/); + const formCode = settings.slice(settings.indexOf('function normalizeAIChannelId('), settings.indexOf('function writeAIChannelToMainForm(')); + const probeCode = settings.slice(settings.indexOf('async function detectAIContext('), settings.indexOf('// 保存工具配置（独立函数')); + const fields = { + 'openai-provider': 'openai_compatible', + 'openai-base-url': 'https://gateway.example/v1', + 'openai-api-key': 'fixture-key', + 'openai-model': 'deepseek-flash' + }; + for (const profile of [undefined, 'local', 'local_auto', 'economy', 'custom']) { + const requests = []; + const context = { + URL, + document: { getElementById: id => Object.hasOwn(fields, id) ? { value: fields[id] } : null }, + currentConfig: { ai: { channels: { main: { context_profile: profile, provider: fields['openai-provider'], base_url: fields['openai-base-url'], model: fields['openai-model'] } } } }, + selectedAIChannelId: 'main', + aiChannelProbeResults: {}, + window: {}, + settingsT: (_, fallback) => fallback, + syncSelectedAIChannelUI() {}, + updateAIChannelSelectOption() {}, + setConnectionTestResult() {}, + applyDetectedContext() {}, + apiFetch: async (url, options) => { + requests.push(JSON.parse(options.body)); + return { json: async () => ({ success: true, context_window: 131072 }) }; + } + }; + vm.createContext(context); + vm.runInContext(formCode + '\n' + probeCode, context); + const saved = context.readAIChannelFromMainForm('main'); + assert.equal(saved.context_profile, profile); + await context.detectAIContext(); + await context.testOpenAIConnection(); + assert.equal(requests.length, 2); + for (const request of requests) assert.equal(request.context_profile, profile || ''); + } +}); + +test('保存通道自动识别本地服务，切换服务清除自动判定', async () => { + const formCode = settings.slice(settings.indexOf('function normalizeAIChannelId('), settings.indexOf('function writeAIChannelToMainForm(')); + const detectionCode = settings.slice(settings.indexOf('async function detectChannelContextForSave('), settings.indexOf('async function persistAIConfigOnlyToServer(')); + const fields = { 'openai-provider': 'openai_compatible', 'openai-base-url': 'http://127.0.0.1:8000/v1', 'openai-model': 'local-model' }; + let probes = 0; + let result = { success: true, context_profile: 'local_auto', context_window: 65536 }; + const context = { + URL, AbortSignal, + document: { getElementById: id => Object.hasOwn(fields, id) ? { value: fields[id] } : null }, + currentConfig: { ai: { channels: {} } }, + apiFetch: async () => { + probes++; + return { ok: true, json: async () => result }; + } + }; + vm.createContext(context); + vm.runInContext(formCode + '\n' + detectionCode, context); + const channel = context.readAIChannelFromMainForm('main'); + await context.detectChannelContextForSave('main', channel); + assert.equal(channel.context_profile, 'local_auto'); + assert.equal(channel.context_window, 65536); + assert.equal(probes, 1); + context.currentConfig.ai.channels.main = channel; + fields['openai-base-url'] = 'https://gateway.example/v1'; + const changed = context.readAIChannelFromMainForm('main'); + assert.equal(changed.context_profile, ''); + + for (const profile of ['local', 'custom', 'economy']) { + const manual = { ...channel, context_profile: profile }; + await context.detectChannelContextForSave('main', manual); + assert.equal(manual.context_profile, profile); + } + assert.equal(probes, 1); + const manualWindow = { ...changed, context_window: 16384 }; + await context.detectChannelContextForSave('main', manualWindow); + assert.equal(manualWindow.context_window, 16384); + const oversizedWindow = { ...changed, context_window: 131072 }; + await context.detectChannelContextForSave('main', oversizedWindow); + assert.equal(oversizedWindow.context_window, 65536); + result = { success: true, context_profile: 'local_auto', context_window: 262144, context_runtime_unknown: true }; + const unknownRuntime = { ...changed, context_window: 0 }; + await context.detectChannelContextForSave('main', unknownRuntime); + assert.equal(unknownRuntime.context_profile, 'local_auto'); + assert.equal(unknownRuntime.context_window, 0); + + context.apiFetch = async () => { throw new Error('offline'); }; + const offline = { ...changed }; + await context.detectChannelContextForSave('main', offline); + assert.equal(offline.context_profile, ''); +}); diff --git a/web/static/js/settings.js b/web/static/js/settings.js index 90b5ebc39..e5fa60e1c 100644 --- a/web/static/js/settings.js +++ b/web/static/js/settings.js @@ -2025,6 +2025,7 @@ async function applySettings() { currentConfig.ai = ensureAIConfigShape(currentConfig); const activeChannelId = normalizeAIChannelId(selectedAIChannelId || currentConfig.ai.default_channel || 'default'); currentConfig.ai.channels[activeChannelId] = readAIChannelFromMainForm(activeChannelId); + await detectChannelContextForSave(activeChannelId, currentConfig.ai.channels[activeChannelId]); currentConfig.ai.default_channel = activeChannelId; currentConfig.ai = normalizeAIConfigProviderProfiles(currentConfig.ai); renderAIChannelSelect(); @@ -2614,6 +2615,8 @@ function ensureAIConfigShape(cfg) { model: oa.model || '', max_total_tokens: oa.max_total_tokens || 120000, max_completion_tokens: oa.max_completion_tokens || 0, + context_window: oa.context_window || 0, + context_profile: oa.context_profile || '', reasoning: oa.reasoning || {} }; } @@ -2648,7 +2651,7 @@ function formatCustomHeaders(headers) { function readAIChannelFromMainForm(id) { const prev = currentConfig?.ai?.channels?.[id] || {}; const maxCompletionTokens = parseInt(document.getElementById('openai-max-completion-tokens')?.value, 10) || 32768; - return normalizeAIChannelProviderProfile({ + const channel = normalizeAIChannelProviderProfile({ ...prev, name: (document.getElementById('ai-channel-name')?.value || '').trim() || prev.name || id, provider: document.getElementById('openai-provider')?.value || 'openai', @@ -2658,6 +2661,7 @@ function readAIChannelFromMainForm(id) { max_total_tokens: parseInt(document.getElementById('openai-max-total-tokens')?.value, 10) || 120000, max_completion_tokens: maxCompletionTokens, headers: parseCustomHeadersInput(), + context_window: parseInt(document.getElementById('openai-context-window')?.value, 10) || 0, reasoning: { ...(prev.reasoning || {}), mode: document.getElementById('openai-reasoning-mode')?.value || 'auto', @@ -2666,6 +2670,10 @@ function readAIChannelFromMainForm(id) { allow_client_reasoning: document.getElementById('openai-reasoning-allow-client')?.checked !== false } }); + if (channel.context_profile === 'local_auto' && ['provider', 'base_url', 'model'].some(key => channel[key] !== prev[key])) { + channel.context_profile = ''; + } + return channel; } function writeAIChannelToMainForm(id) { @@ -2692,6 +2700,8 @@ function writeAIChannelToMainForm(id) { if (maxCompletionTokensEl) maxCompletionTokensEl.value = ch.max_completion_tokens || 32768; const customHeadersEl = document.getElementById('openai-custom-headers'); if (customHeadersEl) customHeadersEl.value = formatCustomHeaders(ch.headers); + const contextWindowEl = document.getElementById('openai-context-window'); + if (contextWindowEl) contextWindowEl.value = ch.context_window || ''; const r = ch.reasoning || {}; const modeEl = document.getElementById('openai-reasoning-mode'); if (modeEl) { @@ -2797,7 +2807,10 @@ function bindAIChannelEditorPreviewSync() { 'openai-api-key', 'openai-base-url', 'openai-model', - 'openai-custom-headers' + 'openai-custom-headers', + 'openai-max-total-tokens', + 'openai-max-completion-tokens', + 'openai-context-window' ]; ids.forEach((fieldId) => { const el = document.getElementById(fieldId); @@ -3026,6 +3039,7 @@ async function persistAIChannelsToServer(successMessage, options = {}) { renderAIChannelSelect(); showAIChannelSaveHint(settingsT('settingsBasic.aiChannelSaving', '正在保存通道...'), true); try { + await detectChannelContextForSave(id, channelPayload); const shouldMergeLatest = options.mergeLatest !== false; const latestResponse = shouldMergeLatest ? await apiFetch('/api/config') : null; if (latestResponse && latestResponse.ok) { @@ -3065,6 +3079,36 @@ async function persistAIChannelsToServer(successMessage, options = {}) { } } +async function detectChannelContextForSave(id, channel) { + if (!['', 'auto', 'local_auto'].includes(channel.context_profile || '') || ['claude', 'anthropic'].includes(channel.provider)) return; + // These public services already have a known policy. Unknown endpoints + // are identified by metadata, including local services behind proxies. + if (['api.deepseek.com', 'api.openai.com', 'api.anthropic.com'].includes(aiChannelBaseURLHost(channel.base_url))) return; + try { + const response = await apiFetch('/api/config/detect-context', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + signal: AbortSignal.timeout(9000), + body: JSON.stringify({ ...channel, channel_id: id }) + }); + const result = await response.json(); + if (response.ok && result.success) applyChannelContextDetection(channel, result); + } catch (_) { + // Metadata discovery is best-effort. Saving a working channel is valid + // when the server is offline or does not publish runtime metadata. + } +} + +function applyChannelContextDetection(channel, result) { + if (!channel || !result) return; + if (['', 'auto', 'local_auto'].includes(channel.context_profile || '') && ['', 'local_auto'].includes(result.context_profile)) { + channel.context_profile = result.context_profile; + } + if (result.context_window > 0 && !result.context_runtime_unknown && + (!channel.context_window || (channel.context_profile === 'local_auto' && channel.context_window > result.context_window))) { + channel.context_window = result.context_window; + } +} + async function persistAIConfigOnlyToServer(successMessage) { if (typeof requirePermission === 'function' && !requirePermission('config:write')) return false; if (!currentConfig) return false; @@ -3145,6 +3189,8 @@ function createAIChannelFromForm() { model: '', max_total_tokens: 120000, max_completion_tokens: 32768, + context_window: 0, + context_profile: '', reasoning: { mode: 'auto', effort: '', profile: 'auto', allow_client_reasoning: true } }; selectedAIChannelId = id; @@ -3240,14 +3286,19 @@ async function probeSelectedAIChannels() { base_url: ch.base_url || '', api_key: ch.api_key || '', model: ch.model || '', - headers: ch.headers || {} + headers: ch.headers || {}, + max_total_tokens: parseInt(ch.max_total_tokens, 10) || 0, + max_completion_tokens: parseInt(ch.max_completion_tokens, 10) || 0, + context_profile: ch.context_profile || '' }) }); const result = await response.json().catch(() => ({})); if (response.ok && result.success) { + applyChannelContextDetection(ch, result); okCount += 1; const latency = result.latency_ms ? ` ${result.latency_ms}ms` : ''; - aiChannelProbeResults[id] = { status: 'ready', message: settingsT('settingsBasic.aiChannelReadyWithLatency', '可用{latency}').replace('{latency}', latency) }; + const context = result.context_window ? ` · ${result.context_window} tokens` : ''; + aiChannelProbeResults[id] = { status: 'ready', message: settingsT('settingsBasic.aiChannelReadyWithLatency', '可用{latency}').replace('{latency}', latency) + context }; } else { aiChannelProbeResults[id] = { status: 'failed', message: formatConnectionTestError(result.error || settingsT('settingsBasic.testFailed', '连接失败')).message }; } @@ -3309,6 +3360,7 @@ if (typeof window !== 'undefined') { window.copyAIChannelFromForm = copyAIChannelFromForm; window.deleteSelectedAIChannel = deleteSelectedAIChannel; window.probeSelectedAIChannels = probeSelectedAIChannels; + window.detectAIContext = detectAIContext; window.deleteCheckedAIChannels = deleteCheckedAIChannels; } @@ -3903,6 +3955,120 @@ function showConnectionTestFailure(resultEl, errorText) { } // 测试OpenAI连接 +function contextWarningText(code) { + const messages = { + local_context_window_unknown: settingsT('settingsBasic.contextWarnWindowUnknown', '本地档位未填写窗口，按 32768 估算'), + local_context_window_small: settingsT('settingsBasic.contextWarnWindowSmall', '窗口小于 16384，带工具的 agent 可能无法稳定运行'), + max_total_tokens_clamped: settingsT('settingsBasic.contextWarnBudgetClamped', '最大上下文预算已按模型窗口收紧'), + max_completion_tokens_clamped: settingsT('settingsBasic.contextWarnCompletionClamped', '最大输出 Token 数已按模型窗口收紧'), + configured_budget_exceeds_context_window: settingsT('settingsBasic.contextWarnBudgetExceedsWindow', '填写的预算超过模型窗口，运行时会自动收紧'), + ollama_v1_ignores_num_ctx: settingsT('settingsBasic.contextWarnOllamaV1', 'Ollama 的 /v1 接口忽略 num_ctx，请在服务端设置 OLLAMA_CONTEXT_LENGTH 或 Modelfile'), + ollama_runtime_context_unknown: settingsT('settingsBasic.contextWarnOllamaRuntime', 'Ollama 可能按显存把窗口压到模型上限以下，建议启动后查看 ollama ps'), + context_runtime_unknown: settingsT('settingsBasic.contextWarnRuntimeUnknown', '无法确认运行时窗口，可能小于模型上限'), + llamacpp_parallel_slots: settingsT('settingsBasic.contextWarnLlamaCppSlots', 'llama.cpp 配置了多个并行槽位，单请求可用窗口可能小于 n_ctx；请按 --parallel/KV 模式确认'), + context_window_unknown: settingsT('settingsBasic.contextWarnUnknown', '未检测到上下文窗口，请手动填写') + }; + return messages[code] || code; +} + +function formatDetectedContext(result) { + if (!result || !result.context_window) return ''; + const parts = [settingsT('settingsBasic.detectedContext', '检测到 {tokens} tokens').replace('{tokens}', String(result.context_window))]; + if (result.effective_max_total_tokens) { + parts.push(settingsT('settingsBasic.effectiveBudget', '有效预算 {tokens} tokens').replace('{tokens}', String(result.effective_max_total_tokens))); + } + if (result.context_backend) parts.push(result.context_backend); + if (result.context_source) parts.push(result.context_source); + if (Array.isArray(result.context_warnings)) { + result.context_warnings.forEach((code) => parts.push(contextWarningText(code))); + } + return parts.join(' · '); +} + +function applyDetectedContext(result, force) { + if (!result) return; + const channel = currentConfig?.ai?.channels?.[selectedAIChannelId]; + applyChannelContextDetection(channel, result); + if (!result.context_window) return; + const input = document.getElementById('openai-context-window'); + // Explicit auto-detect overwrites the field so the effective-budget preview + // matches what will be saved; a connection test only fills an empty field. + if (input && !result.context_runtime_unknown && (force || !String(input.value || '').trim() || + (channel?.context_profile === 'local_auto' && Number(input.value) > result.context_window))) { + input.value = String(result.context_window); + } + const resultEl = document.getElementById('detect-context-result'); + if (resultEl) { + resultEl.textContent = formatDetectedContext(result); + resultEl.className = 'form-inline-result is-success'; + } +} + +async function detectAIContext() { + const btn = document.getElementById('detect-context-btn'); + const resultEl = document.getElementById('detect-context-result'); + const provider = document.getElementById('openai-provider')?.value || 'openai_compatible'; + const baseUrl = document.getElementById('openai-base-url')?.value.trim() || ''; + const apiKey = document.getElementById('openai-api-key')?.value.trim() || ''; + const model = document.getElementById('openai-model')?.value.trim() || ''; + const channelId = normalizeAIChannelId(selectedAIChannelId || currentConfig?.ai?.default_channel || 'default'); + const isCurrentRequest = () => selectedAIChannelId === channelId && + document.getElementById('openai-base-url')?.value.trim() === baseUrl && + document.getElementById('openai-model')?.value.trim() === model; + if (!baseUrl || !model) { + if (resultEl) { + resultEl.textContent = settingsT('settingsBasic.detectContextFillRequired', '请先填写 Base URL 和模型'); + resultEl.className = 'form-inline-result is-error'; + } + return; + } + if (btn) { + btn.style.pointerEvents = 'none'; + btn.style.opacity = '0.5'; + } + if (resultEl) { + resultEl.textContent = settingsT('settingsBasic.testing', '测试中...'); + resultEl.className = 'form-inline-result is-muted'; + } + try { + const response = await apiFetch('/api/config/detect-context', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + provider: provider, + channel_id: channelId, + base_url: baseUrl, + api_key: apiKey, + model: model, + headers: parseCustomHeadersInput(), + max_total_tokens: parseInt(document.getElementById('openai-max-total-tokens')?.value, 10) || 0, + max_completion_tokens: parseInt(document.getElementById('openai-max-completion-tokens')?.value, 10) || 0, + context_profile: currentConfig?.ai?.channels?.[channelId]?.context_profile || '' + }) + }); + const result = await response.json().catch(() => ({})); + if (!isCurrentRequest()) return; + if (result.success) applyDetectedContext(result, true); + if (!result.success || !result.context_window) { + const message = result.error || contextWarningText('context_window_unknown'); + if (resultEl) { + resultEl.textContent = formatConnectionTestError(message).message; + resultEl.className = 'form-inline-result is-error'; + } + } + } catch (error) { + if (resultEl) { + resultEl.textContent = formatConnectionTestError(error.message || '测试出错').message; + resultEl.className = 'form-inline-result is-error'; + } + } finally { + if (btn) { + btn.style.pointerEvents = ''; + btn.style.opacity = ''; + } + } +} + async function testOpenAIConnection() { const btn = document.getElementById('test-openai-btn'); const resultEl = document.getElementById('test-openai-result'); @@ -3912,7 +4078,9 @@ async function testOpenAIConnection() { const apiKey = document.getElementById('openai-api-key').value.trim(); const model = document.getElementById('openai-model').value.trim(); const channelId = normalizeAIChannelId(selectedAIChannelId || currentConfig?.ai?.default_channel || 'default'); - const isTestingSameChannel = () => normalizeAIChannelId(selectedAIChannelId || currentConfig?.ai?.default_channel || 'default') === channelId; + const isTestingSameChannel = () => normalizeAIChannelId(selectedAIChannelId || currentConfig?.ai?.default_channel || 'default') === channelId && + document.getElementById('openai-base-url')?.value.trim() === baseUrl && + document.getElementById('openai-model')?.value.trim() === model; if (!baseUrl || !apiKey || !model) { const message = typeof window.t === 'function' ? window.t('settingsBasic.testFillRequired') : '请先填写 Base URL、API Key 和模型'; @@ -3945,7 +4113,10 @@ async function testOpenAIConnection() { base_url: baseUrl, api_key: apiKey, model: model, - headers: currentConfig?.ai?.channels?.[channelId]?.headers || {} + headers: currentConfig?.ai?.channels?.[channelId]?.headers || {}, + max_total_tokens: parseInt(document.getElementById('openai-max-total-tokens')?.value, 10) || 0, + max_completion_tokens: parseInt(document.getElementById('openai-max-completion-tokens')?.value, 10) || 0, + context_profile: currentConfig?.ai?.channels?.[channelId]?.context_profile || '' }) }); @@ -3954,9 +4125,11 @@ async function testOpenAIConnection() { if (result.success) { const latency = result.latency_ms ? ` (${result.latency_ms}ms)` : ''; const modelInfo = result.model ? ` [${result.model}]` : ''; - const message = (typeof window.t === 'function' ? window.t('settingsBasic.testSuccess') : '连接成功') + modelInfo + latency; + const contextInfo = result.context_window ? ` · ${result.context_window} tokens` : ''; + const message = (typeof window.t === 'function' ? window.t('settingsBasic.testSuccess') : '连接成功') + modelInfo + latency + contextInfo; aiChannelProbeResults[channelId] = { status: 'ready', message }; if (isTestingSameChannel()) { + applyDetectedContext(result); setConnectionTestResult(resultEl, 'is-success', message); } } else { diff --git a/web/templates/index.html b/web/templates/index.html index 427d1d6e6..fe0c7f3ca 100644 --- a/web/templates/index.html +++ b/web/templates/index.html @@ -3766,6 +3766,15 @@
额度设置
单次模型回复的输出上限，默认 16384 +
+ +
+ + 自动检测 +
+ 模型原生窗口；用于校验并收紧预算。本地模型请按服务端 num_ctx/ctx-size 填写。 + +
@@ -7034,7 +7043,7 @@

- +