feat: inbox YAML archiving + nudge cooldown to reduce redundant token cost#170
Open
Rusty-Alucard wants to merge 2 commits into
Open
feat: inbox YAML archiving + nudge cooldown to reduce redundant token cost#170Rusty-Alucard wants to merge 2 commits into
Rusty-Alucard wants to merge 2 commits into
Conversation
… cost
Problem (Idiot Index reasoning):
- Before: inbox_watcher reads the full inbox YAML on every cycle
- Inbox accumulates hundreds of read messages → 50KB+ per read
- Minimum necessary: only unread message bytes (typically <1KB)
- Idiot Index ≈ 50–134× overread per cycle
Changes:
inbox archiving (INBOX_ARCHIVE_ENABLED=1, INBOX_ARCHIVE_KEEP_READ=20):
- archive_read_messages() moves excess read messages to queue/inbox/archive/{agent}.yaml
- Keeps at most 20 read messages in the active inbox YAML
- Safety: unread messages are NEVER archived (SKIP_HAS_UNREAD guard)
- Append-only archive file preserves full history
nudge absolute cooldown (NUDGE_MIN_INTERVAL_SEC=30):
- Prevents nudge spam when follow-up messages arrive in rapid succession
- Guards before the per-count cooldown check
- Phase 3 /clear uses send_cli_command directly — not affected
metrics accuracy (unread_bytes):
- get_unread_info() now returns unread_bytes (actual bytes of unread content)
- update_metrics() uses unread_bytes instead of full file size
- Token estimates now reflect actual processing cost, not file bloat
Safety measures preserved:
- Zero message loss: unread guard before any archive operation
- Deadlock prevention: stale busy safety net unchanged
- Final escalation (/clear path) unchanged
- All 7 resilience invariants maintained
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`flock` is not available on macOS by default. Added `command -v flock` guard with mkdir-based fallback (same pattern as inbox_watcher.sh lines 312/416/518). Linux behavior is unchanged. Fixes macOS Unit Tests CI failure in PR yohey-w#170.
Contributor
Author
|
Also fixes pre-existing macOS incompatibility in watcher_supervisor.sh: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
This PR addresses a token efficiency problem identified via Idiot Index reasoning:
Before this change,
inbox_watcher.shreads the full inbox YAML on every processing cycle. As agents accumulate message history, the inbox grows to 50KB+ — but the minimum necessary information per cycle is only the unread message bytes (typically <1KB). This produces an Idiot Index of 50–134×.Changes
Inbox YAML Archiving (
INBOX_ARCHIVE_ENABLED=1,INBOX_ARCHIVE_KEEP_READ=20)archive_read_messages()function moves excess read messages toqueue/inbox/archive/{agent}.yamlINBOX_ARCHIVE_KEEP_READ(default: 20) read messages in the active inbox YAMLINBOX_ARCHIVE_ENABLED=0to disableNudge Absolute Cooldown (
NUDGE_MIN_INTERVAL_SEC=30)/clearescalation usessend_cli_commanddirectly and is not affected by this throttleMetrics Accuracy (unread bytes)
get_unread_info()now returnsunread_bytes: the actual byte size of unread message contentupdate_metrics()usesunread_bytesinstead of the full inbox file sizeBefore / After
Actual after-values depend on archive trigger (>20 read messages). Estimates based on current inbox sizes with 41/38 messages respectively.
Safety
All existing resilience measures are preserved:
SKIP_HAS_UNREAD) prevents archiving when any unread message exists/clearpath, Phase 3) unchangedarchive_read_messages()uses the sameacquire_inbox_lock/release_inbox_lockpattern as existing codeTesting
tests/unit/test_send_wakeup.batspass with no new failures__INBOX_WATCHER_TESTING__=1mode unchanged (archive function loads correctly)🤖 Generated with Claude Code