Skip to content

Feature: Cognitive Encoding — LLM Auto-Classification + Contradiction Consolidation on Memory Write #676

@teknium1

Description

@teknium1

Overview

When Hermes Agent stores a memory today, it's a passive write — the agent provides flat text and the system appends it. There's no auto-classification, no contradiction detection, and no conflict resolution. If the agent stores "We use PostgreSQL" on Monday and "We switched to MySQL" on Friday, both coexist silently.

This issue adds cognitive encoding: when a memory is stored, an LLM pipeline auto-classifies it (scope, categories, importance) and checks for contradictions against existing memories. Conflicts are resolved at write time — not accumulated.

Inspired by CrewAI's EncodingFlow (MIT licensed).

Parent tracking issue: #509
Depends on: Memory storage migration (SQLite with scope/importance fields), Embedding infrastructure


What to Build

Auto-Classification on Write

When memory add is called, optionally run the auxiliary LLM to infer:

class MemoryAnalysis(BaseModel):
    suggested_scope: str    # e.g. "/infrastructure/database"
    categories: list[str]   # e.g. ["postgresql", "migration"]
    importance: float       # 0.0 to 1.0

The agent can override any field — if scope/importance are provided explicitly, skip that part of the LLM call. This gives us CrewAI's "Group A" fast path (0 LLM calls when all fields provided).

Consolidation on Write

Before inserting, embed the new content and search for similar existing memories:

  1. If top similarity >= 0.85, run LLM consolidation
  2. LLM produces a plan: for each similar memory, keep / update / delete
  3. Also decides whether to insert_new or not
  4. Execute the plan atomically
class ConsolidationAction(BaseModel):
    record_id: str
    action: Literal["keep", "update", "delete"]
    updated_content: str | None = None

class ConsolidationPlan(BaseModel):
    actions: list[ConsolidationAction]
    insert_new: bool

Configurable

# ~/.hermes/config.yaml
memory:
  cognitive: true              # Enable cognitive encoding (default: true)
  consolidation_threshold: 0.85  # Similarity threshold for triggering consolidation

Graceful Degradation

If the auxiliary LLM fails, fall back to simple insert with defaults (importance=0.5, scope="/"). Memory always gets stored — cognitive analysis is best-effort.


Implementation Details

  • Use existing agent/auxiliary_client.py for LLM calls (same pattern as session_search)
  • Use embedding infrastructure for similarity search
  • Consolidation runs inside the add action of memory_tool.py
  • System prompt for classification: short, focused on scope/categories/importance inference
  • System prompt for consolidation: compare new content vs existing, decide keep/update/delete

Files to Change

  • tools/memory_tool.py — Add encoding pipeline to add action
  • agent/prompts/ or inline — Classification and consolidation system prompts
  • tests/tools/test_memory_tool.py — Tests with mocked LLM responses

Acceptance Criteria

  • add with cognitive=true auto-infers scope, categories, importance via LLM
  • add with explicit scope/importance skips LLM classification (fast path)
  • Similar memories (>=0.85) trigger consolidation before insert
  • Consolidation resolves contradictions (update/delete old + insert new or merge)
  • LLM failure falls back to simple insert with defaults
  • cognitive: false in config disables all LLM analysis (pure storage)
  • Configurable consolidation threshold

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions