Skip to content

fix: deduplicate SQLite session writes via _last_flushed_db_idx#870

Open
teyrebaz33 wants to merge 1 commit intoNousResearch:mainfrom
teyrebaz33:fix/860-session-duplicate-messages
Open

fix: deduplicate SQLite session writes via _last_flushed_db_idx#870
teyrebaz33 wants to merge 1 commit intoNousResearch:mainfrom
teyrebaz33:fix/860-session-duplicate-messages

Conversation

@teyrebaz33
Copy link
Contributor

Closes #860

Problem

_flush_messages_to_session_db() was called from 15+ exit paths in run_conversation(), each time re-flushing all messages from len(conversation_history). Combined with _log_msg_to_db() and the gateway's append_to_transcript(), messages were written 3–4x to SQLite — causing proportionally inflated token usage on every API call.

Fix (Option A from issue)

Track _last_flushed_db_idx on the agent instance. Each flush call starts from the last written index instead of from len(conversation_history), so only truly new messages are written.

start_idx = getattr(self, '_last_flushed_db_idx', len(conversation_history) if conversation_history else 0)
# ... write messages[start_idx:] ...
self._last_flushed_db_idx = len(messages)

Uses getattr() fallback for backward compatibility with existing agent instances.

Testing

3 new tests in TestFlushMessagesDeduplication:

  • Second flush only writes new messages
  • Repeated flush with same messages writes nothing
  • Without prior state, falls back to len(conversation_history)

Full suite: 2863 passed.

_flush_messages_to_session_db() was called from 15+ exit paths in
run_conversation(), each time re-flushing all messages from
conversation_history start. Combined with _log_msg_to_db() and
gateway's append_to_transcript(), messages were written 3-4x,
inflating token usage by the same factor.

Track _last_flushed_db_idx on the agent instance so each flush
only writes truly new messages. Uses getattr() with fallback to
len(conversation_history) for backward compatibility with existing
sessions that have no prior state.

Closes NousResearch#860
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: SQLite session transcript accumulates duplicate messages (3-4x token inflation)

1 participant