feat(scout-agent): add conversation compaction module #106
+1,605
−10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds conversation compaction support to prevent context overflow errors. Features: - Token counting using ai-tokenizer with model-specific accuracy - Compaction tool (compact_conversation) for summarizing conversations - Warning message generation when approaching token limits - Context length error detection for various providers - Emergency compaction for when compaction request exceeds context Exports: - countConversationTokens - Count tokens in ModelMessage[] - shouldCompact - Check if compaction is needed - findCompactionSummary - Find existing compaction in messages - applyCompaction - Replace pre-compaction messages with summary - createCompactionTool - Create the compact_conversation tool - createCompactionWarningMessage - Generate warning message - isContextLengthError - Detect context overflow errors - calculateEmergencyCompactionConfig - Plan emergency compaction - createEmergencyCompactionMessage - Generate emergency request - prepareEmergencyCompactionMessages - Split messages for emergency Includes 21 tests covering all functionality.
This commit: - Imports applyCompaction and createCompactionTool from ./compaction - Applies compaction to messages before processing in buildStreamTextParams - Logs when compaction is applied (showing message count reduction) - Adds the compaction tool to the tools object so the model can call it - Uses compacted messages for Slack metadata detection and model conversion - Fixes isolated declarations issue in createCompactionTool by adding return type - Fixes test assertion for execute method on Tool type
…eded This commit adds: - New compaction option in BuildStreamTextParamsOptions to configure: - warningThreshold: token count that triggers compaction warning (default: 80% of max) - maxTokenThreshold: maximum tokens for context (default: 100k) - modelName: model name for token counting - Set to false to disable compaction features - Token counting using ai-tokenizer after message conversion - Automatic injection of compaction warning message when threshold exceeded - Logging for token thresholds and warning injection - Tests for: - Compaction tool is included by default - Existing compaction summaries are applied - Warning message is injected when threshold exceeded - Compaction can be disabled with compaction: false - Custom thresholds are respected Note: core.test.ts tests may not run locally due to bun/HTTPParser incompatibility, but work in CI.
maxTokenThreshold was only used for display in the warning message. Simplified to just use warningThreshold for both triggering and display.
…njected The compact_conversation tool is now only available when the token threshold is exceeded and the warning message is injected. This keeps the tool list clean when compaction is not needed. Updated tests to verify: - Tool is NOT available when under threshold - Tool IS available when warning is injected - Tool is NOT available when compaction is disabled
hugodutka
force-pushed
the
feat/conversation-compaction
branch
6 times, most recently
from
f3d4ce8 to
909fa77
Compare
hugodutka
force-pushed
the
feat/conversation-compaction
branch
from
909fa77 to
c02246f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds conversation compaction support to the scout-agent package to prevent context overflow errors during long conversations.
Problem
During long conversations, the model can run out of context window space and error out. There was no mechanism in scout-agent to handle this gracefully.
Solution
Adds a compaction module that provides:
Token Counting
ai-tokenizerfor accurate, model-specific token countingCompaction Tool
compact_conversationtool that accepts detailed summaries