Skip to content

Commit 4690583

Browse files
shuv1337shuv1337
authored
revert: remove PR anomalyco#4773 subagent restrictions feature (#105)
* revert: remove PR anomalyco#4773 subagent restrictions feature This feature is not going to be merged upstream in its current form. Removes: - subagents field from Agent schema and built-in agents - subagents config option - filterSubagents function and runtime validation - Subagent filtering from prompt tool resolution - Subagent filtering from TUI autocomplete - subagents-filter.test.ts test file - Subagents documentation section from agents.mdx The SDK types will be regenerated automatically on the next build. * chore: format code * docs: update fork README to remove PR anomalyco#4773 and refresh date * feat: add ghostty-opentui dependency for terminal ANSI rendering * feat: force color output in bash tool for ANSI rendering * feat: add live token tracking during streaming responses * fix: use correct subagent session ID for click navigation * feat: add search, token display, and bash ANSI viewer to TUI - Add Ctrl+F search with match highlighting and navigation - Add toggle tokens command with IN/OUT display - Add full-screen bash output viewer with ANSI color support - Integrate ghostty-terminal component for terminal rendering * docs: add implementation plans for restored PR features * ci: retry tests * [WIP] Update subagent restrictions based on PR feedback (#106) * Initial plan * fix: replace strikethrough with ANSI highlighting for search results
1 parent 5630efc commit 4690583

20 files changed

Lines changed: 1108 additions & 362 deletions

‎CONTEXT/PLAN-4709-restore-live-token-usage-2025-12-09.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Plan: Restore Live Token Usage Feature (PR #4709)
2+
3+
**Date:** 2025-12-09
4+
**Related PR:** https://github.com/sst/opencode/pull/4709
5+
**Status:** IMPLEMENTED - Feature restored 2025-12-10
6+
7+
## Overview
8+
9+
This plan documents the restoration of the "Live Token Usage During Streaming" feature that was originally added in PR #4709. The feature provides:
10+
11+
- Real-time token tracking while streaming responses
12+
- `IN/OUT` format display for input/output tokens
13+
- Reasoning token display for "thinking" models
14+
- Toggle tokens command in TUI
15+
16+
## Current State Analysis
17+
18+
### What's Working
19+
20+
| Component | File | Status |
21+
| ------------------------------ | --------------------------------------------- | -------------------------- |
22+
| Token utility functions | `packages/opencode/src/util/token.ts` | **EXISTS** |
23+
| Message schema fields | `packages/opencode/src/session/message-v2.ts` | **EXISTS** |
24+
| Token calculation in prompt.ts | `packages/opencode/src/session/prompt.ts` | **EXISTS** |
25+
| Subtask completion fix | `packages/opencode/src/session/prompt.ts` | **EXISTS** (bug was fixed) |
26+
27+
### What's Missing
28+
29+
| Component | File | Status |
30+
| ------------------------ | ------------------------------------------------------------ | ----------- |
31+
| Streaming token updates | `packages/opencode/src/session/processor.ts` | **MISSING** |
32+
| `showTokens` state | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
33+
| `contextLimit` memo | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
34+
| IN/OUT token display | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
35+
| Reasoning token display | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
36+
| "Toggle tokens" command | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
37+
| User message token count | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
38+
39+
### Backend Token Utility (Exists)
40+
41+
```typescript
42+
// packages/opencode/src/util/token.ts
43+
Token.estimate(input: string) // Character-based estimation
44+
Token.toCharCount(tokenEstimate: number) // Convert tokens to chars
45+
Token.toTokenEstimate(charCount: number) // Convert chars to tokens
46+
Token.calculateToolResultTokens(parts) // Estimate tool result size
47+
```
48+
49+
### Message Schema Fields (Exist)
50+
51+
```typescript
52+
// UserMessage
53+
sentEstimate: z.number().optional()
54+
contextEstimate: z.number().optional()
55+
56+
// AssistantMessage
57+
outputEstimate: z.number().optional()
58+
reasoningEstimate: z.number().optional()
59+
contextEstimate: z.number().optional()
60+
sentEstimate: z.number().optional()
61+
```
62+
63+
## Technical Approach
64+
65+
### Token Estimation Logic
66+
67+
- Simple estimation based on character count using `CHARS_PER_TOKEN` constant
68+
- `calculateToolResultTokens` estimates size of tool inputs, outputs, and errors
69+
- Estimates prefixed with `~` to indicate they are approximate
70+
71+
### Display Format
72+
73+
- `IN X↓` - Input/context tokens (sent to model)
74+
- `OUT Y↑` - Output tokens (generated by model)
75+
- `~X think` - Reasoning tokens for thinking models
76+
- Context percentage: `X% of limit`
77+
78+
## Implementation Tasks
79+
80+
### Phase 1: Add Streaming Token Updates to Processor
81+
82+
- [x] Import `Token` module in `packages/opencode/src/session/processor.ts`
83+
- [x] Add `reasoningTotal` and `textTotal` character accumulators at processor creation
84+
- [x] Update `reasoning-delta` handler to calculate and store `reasoningEstimate`:
85+
```typescript
86+
case "reasoning-delta":
87+
reasoningTotal += value.text.length
88+
input.assistantMessage.reasoningEstimate = Token.toTokenEstimate(reasoningTotal)
89+
await Session.updateMessage(input.assistantMessage)
90+
```
91+
- [x] Update `text-delta` handler to calculate and store `outputEstimate`:
92+
```typescript
93+
case "text-delta":
94+
textTotal += value.text.length
95+
input.assistantMessage.outputEstimate = Token.toTokenEstimate(textTotal)
96+
await Session.updateMessage(input.assistantMessage)
97+
```
98+
- [ ] Update `finish-step` to emit final `contextEstimate` from usage data
99+
100+
### Phase 2: Add Token Display State
101+
102+
- [x] Add `showTokens` signal to session component:
103+
```typescript
104+
const [showTokens, setShowTokens] = createSignal(kv.get("show_tokens", false))
105+
```
106+
- [ ] Add `contextLimit` memo that gets limit from current model/provider
107+
- [x] Add to context provider: `showTokens: () => boolean`
108+
109+
### Phase 3: Add Toggle Tokens Command
110+
111+
- [x] Add "Toggle tokens" command to `command.register()` array:
112+
```typescript
113+
{
114+
title: showTokens() ? "Hide tokens" : "Show tokens",
115+
value: "session.toggle.tokens",
116+
category: "Session",
117+
onSelect: (dialog) => {
118+
setShowTokens((prev) => {
119+
const next = !prev
120+
kv.set("show_tokens", next)
121+
return next
122+
})
123+
dialog.clear()
124+
},
125+
}
126+
```
127+
128+
### Phase 4: Update AssistantMessage Component
129+
130+
- [x] Add token calculation logic:
131+
```typescript
132+
const inputTokens = createMemo(() => {
133+
const sent = props.message.sentEstimate ?? 0
134+
const context = props.message.contextEstimate ?? 0
135+
return sent + context
136+
})
137+
const outputTokens = createMemo(() => props.message.tokens?.output ?? props.message.outputEstimate ?? 0)
138+
const reasoningTokens = createMemo(() => props.message.tokens?.reasoning ?? props.message.reasoningEstimate ?? 0)
139+
```
140+
- [x] Add conditional token display:
141+
```tsx
142+
<Show when={showTokens()}>
143+
<text fg={theme.textMuted}>
144+
IN {inputTokens().toLocaleString()}↓ OUT {outputTokens().toLocaleString()}↑
145+
<Show when={reasoningTokens()}> ~{reasoningTokens().toLocaleString()} think</Show>
146+
</text>
147+
</Show>
148+
```
149+
150+
### Phase 5: Update UserMessage Component
151+
152+
- [x] Add individual token count display when `showTokens()` is true:
153+
```tsx
154+
<Show when={showTokens() && props.message.sentEstimate}>
155+
<text fg={theme.textMuted}>~{props.message.sentEstimate?.toLocaleString()} tokens</text>
156+
</Show>
157+
```
158+
159+
## Code References
160+
161+
### Internal Files
162+
163+
- `packages/opencode/src/util/token.ts` - Token utility functions (exists)
164+
- `packages/opencode/src/session/message-v2.ts:308-309` - User message estimate fields (exists)
165+
- `packages/opencode/src/session/message-v2.ts:369-372` - Assistant message estimate fields (exists)
166+
- `packages/opencode/src/session/processor.ts:82-88` - reasoning-delta handler (needs update)
167+
- `packages/opencode/src/session/processor.ts:305-315` - text-delta handler (needs update)
168+
- `packages/opencode/src/session/processor.ts:251-270` - finish-step handler (needs update)
169+
- `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx:1097-1162` - AssistantMessage component
170+
- `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx:1001-1095` - UserMessage component
171+
- `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx:241-777` - Command registration
172+
173+
### External References
174+
175+
- Original PR: https://github.com/sst/opencode/pull/4709
176+
177+
## Estimated Changes
178+
179+
| File | Lines Added | Lines Modified |
180+
| -------------- | ----------- | -------------- |
181+
| `processor.ts` | ~15 | ~10 |
182+
| `index.tsx` | ~50 | ~15 |
183+
| **Total** | ~65 | ~25 |
184+
185+
## Validation Criteria
186+
187+
- [x] Token estimates display during streaming (before final usage available)
188+
- [x] `IN X↓` shows input/context tokens accurately
189+
- [x] `OUT Y↑` shows output tokens, updating in real-time during generation
190+
- [x] Reasoning tokens display for models that support thinking (e.g., Claude)
191+
- [x] "Toggle tokens" command appears in command palette
192+
- [x] Toggle persists via KV store across sessions
193+
- [x] User messages show estimated token count when toggle enabled
194+
- [x] Estimates use `~` prefix to indicate approximation
195+
- [x] Final token counts from API replace estimates when available
196+
197+
## Dependencies
198+
199+
None - all required utilities and schema fields already exist.
200+
201+
## Risks & Considerations
202+
203+
1. **Estimation Accuracy**: Character-based estimation is approximate. Actual tokenization varies by model. Consider this acceptable for UX purposes.
204+
205+
2. **Performance**: Updating message on every delta may cause performance issues. Consider throttling updates (e.g., every 100ms or 100 chars).
206+
207+
3. **Context Limit**: Different models have different context limits. Need to properly fetch limit from provider/model configuration.
208+
209+
4. **Subtask Bug Status**: The regression bug mentioned in PR discussions (missing `updatePart` call after `taskTool.execute`) was previously fixed and the fix is still present. No action needed.

‎CONTEXT/PLAN-4791-restore-bash-viewer-ansi-2025-12-09.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# Plan: Restore Bash Tool Expansion & ANSI Output Feature (PR #4791)
2+
3+
**Date:** 2025-12-09
4+
**Related PR:** https://github.com/sst/opencode/pull/4791
5+
**Status:** IMPLEMENTED - Feature restored 2025-12-10
6+
7+
## Overview
8+
9+
This plan documents the restoration of the "Bash Tool Expansion & Colored ANSI Output" feature that was originally added in PR #4791. The feature provides:
10+
11+
- Full-screen viewer for bash command outputs
12+
- ANSI color rendering for terminal output
13+
- Output truncation in chat with "Click to view full output" button
14+
- Forced color output from CLI tools
15+
16+
## Current State Analysis
17+
18+
### What's Missing
19+
20+
| Component | File | Status |
21+
| ------------------------------ | -------------------------------------------------------------- | ----------- |
22+
| `ghostty-opentui` dependency | `packages/opencode/package.json` | **MISSING** |
23+
| `ptyToText` import | `packages/opencode/src/tool/bash.ts` | **MISSING** |
24+
| `FORCE_COLOR` env vars | `packages/opencode/src/tool/bash.ts` | **MISSING** |
25+
| `bashOutput` signal | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
26+
| `showBashOutput` context | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
27+
| Full-screen bash viewer | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
28+
| `ghostty-terminal` component | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
29+
| Keyboard navigation for viewer | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
30+
| Bash tool truncated preview | `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx` | **MISSING** |
31+
| `initialValue` prop | `packages/opencode/src/cli/cmd/tui/component/prompt/index.tsx` | **MISSING** |
32+
| `text` getter on PromptRef | `packages/opencode/src/cli/cmd/tui/component/prompt/index.tsx` | **MISSING** |
33+
34+
### Current Bash Tool Rendering
35+
36+
The current bash tool simply strips ANSI codes and displays plain text:
37+
38+
```typescript
39+
const output = createMemo(() => stripAnsi(props.metadata.output?.trim() ?? ""))
40+
// ...
41+
<text fg={theme.text}>{output()}</text>
42+
```
43+
44+
## Technical Approach
45+
46+
### ANSI Color Rendering
47+
48+
- Use `ghostty-opentui` package for terminal rendering
49+
- `GhosttyTerminalRenderable` component renders ANSI codes properly
50+
- `ptyToText()` processes raw PTY output
51+
52+
### Environment Variables for Color Output
53+
54+
Force CLI tools to produce colored output even when not in TTY:
55+
56+
```typescript
57+
env: {
58+
FORCE_COLOR: "3",
59+
CLICOLOR: "1",
60+
CLICOLOR_FORCE: "1",
61+
TERM: "xterm-256color",
62+
TERM_PROGRAM: "bash-tool",
63+
PY_COLORS: "1",
64+
ANSICON: "1",
65+
// ... more
66+
}
67+
```
68+
69+
### Full-Screen Viewer
70+
71+
- Toggle between chat view and full-screen bash viewer
72+
- Keyboard navigation: ESC to close, Page Up/Down, Home/End for scrolling
73+
- Preserve prompt text when switching views
74+
75+
## Implementation Tasks
76+
77+
### Phase 1: Add Dependencies
78+
79+
- [x] Add `ghostty-opentui` to `packages/opencode/package.json`
80+
```json
81+
"ghostty-opentui": "1.3.6"
82+
```
83+
- [x] Run `bun install` to update lockfile
84+
85+
### Phase 2: Update Bash Tool
86+
87+
- [x] Add import to `packages/opencode/src/tool/bash.ts`:
88+
```typescript
89+
import { ptyToText } from "ghostty-opentui"
90+
```
91+
- [x] Update spawn environment variables (around line 225):
92+
```typescript
93+
env: {
94+
...process.env,
95+
FORCE_COLOR: "3",
96+
CLICOLOR: "1",
97+
CLICOLOR_FORCE: "1",
98+
TERM: "xterm-256color",
99+
TERM_PROGRAM: "bash-tool",
100+
PY_COLORS: "1",
101+
ANSICON: "1",
102+
NO_COLOR: undefined,
103+
}
104+
```
105+
- [x] Wrap output with `ptyToText()` before returning
106+
107+
### Phase 3: Update Session Index
108+
109+
- [x] Add `BashOutputView` type:
110+
```typescript
111+
type BashOutputView = {
112+
command: string
113+
output: () => string
114+
}
115+
```
116+
- [x] Add `bashOutput` signal: `createSignal<BashOutputView | undefined>(undefined)`
117+
- [x] Add `showBashOutput` function to context
118+
- [x] Register `ghostty-terminal` component with opentui
119+
- [x] Add keyboard handlers for viewer navigation (ESC, PageUp/Down, Home/End)
120+
- [x] Add conditional rendering that switches between scrollbox and bash viewer
121+
- [x] Add `promptDraft` signal for preserving prompt text
122+
123+
### Phase 4: Update Bash Tool Renderer
124+
125+
- [x] Update the bash tool registration (around line 1382-1404):
126+
- [x] Use `<ghostty-terminal>` for output preview
127+
- [x] Limit preview to 20 lines
128+
- [x] Add "Click to see full output" button when output exceeds limit
129+
- [x] Wire click handler to `showBashOutput`
130+
131+
### Phase 5: Update Prompt Component
132+
133+
- [ ] Add `initialValue` prop to `PromptProps` type (line 29-36) - Not needed for basic implementation
134+
- [ ] Add `text` getter to `PromptRef` type (line 38-45) - Not needed for basic implementation
135+
- [ ] Handle `initialValue` in `onMount` to restore prompt text - Not needed for basic implementation
136+
137+
## Code References
138+
139+
### Internal Files
140+
141+
- `packages/opencode/package.json` - Add ghostty-opentui dependency
142+
- `packages/opencode/src/tool/bash.ts:225-233` - spawn() call, needs env vars
143+
- `packages/opencode/src/tool/bash.ts:350-358` - return statement, needs ptyToText
144+
- `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx:80-89` - Context definition
145+
- `packages/opencode/src/cli/cmd/tui/routes/session/index.tsx:1382-1404` - Bash tool renderer
146+
- `packages/opencode/src/cli/cmd/tui/component/prompt/index.tsx:29-45` - PromptProps/PromptRef types
147+
148+
### External References
149+
150+
- Original PR: https://github.com/sst/opencode/pull/4791
151+
- ghostty-opentui package: https://www.npmjs.com/package/ghostty-opentui
152+
153+
## Estimated Changes
154+
155+
| File | Lines Added | Lines Modified |
156+
| ------------------ | ----------- | -------------- |
157+
| `package.json` | 1 | 0 |
158+
| `bash.ts` | 15 | 5 |
159+
| `index.tsx` | ~150 | ~30 |
160+
| `prompt/index.tsx` | 10 | 5 |
161+
| **Total** | ~176 | ~40 |
162+
163+
## Validation Criteria
164+
165+
- [x] `bun install` succeeds with new dependency
166+
- [x] CLI tools produce colored output (test with `ls --color`, `git status`)
167+
- [x] Bash output in chat shows ANSI colors (not raw escape codes)
168+
- [x] Long outputs are truncated to 20 lines in chat preview
169+
- [x] "Click to see full output" button appears for truncated outputs
170+
- [x] Clicking opens full-screen bash viewer
171+
- [x] Full-screen viewer shows complete output with colors
172+
- [x] ESC key closes full-screen viewer
173+
- [x] Page Up/Down, Home/End work in viewer
174+
- [ ] Prompt text is preserved when opening/closing viewer - Minor, can be addressed later
175+
176+
## Dependencies
177+
178+
- `ghostty-opentui` npm package (needs to be added)
179+
180+
## Risks & Considerations
181+
182+
1. **Package Compatibility**: The `ghostty-opentui` package may have been updated since PR #4791. Check for any API changes.
183+
184+
2. **Performance**: Rendering ANSI codes in the TUI may impact performance for very large outputs. The 20-line preview helps mitigate this.
185+
186+
3. **Interactive Commands**: This feature does NOT support interactive commands (like `top` or `vim`). It's strictly for static output rendering.
187+
188+
4. **Platform Differences**: Color forcing env vars may behave differently on Windows vs Unix. Test on multiple platforms.

0 commit comments

Comments
 (0)