Feature request: idle TTL for MCP in-memory index cache

[AkatQuas]

Problem

The MCP server (uvx --from "semble[mcp]" semble) is a long-lived process that keeps indexes hot in RAM for the entire session. Memory grows as different (repo, content) combinations are searched and often does not shrink even when searches stop.

Observed behavior (macOS, semble 0.6.0):

• ~140–150 MB at startup (embedding model preload only)
• • hundreds of MB to 1 GB+ after indexing large monorepos (e.g. multi-thousand-file workspaces)
• RSS tends to climb monotonically across a cursor-cli / Cursor session

Current release mechanism (as documented in code)

From _IndexCache in mcp.py:

1. Count-based LRU only — _CACHE_MAX_SIZE = 10; evicts the least-recently-used entry when an 11th (path, content) variant is added.
2. No idle / TTL eviction — nothing runs when MCP is idle; indexes stay in memory until LRU fills or the process exits.
3. Stale-file revalidation — _evict_if_stale + _revalidate_after cooldown evict entries when on-disk files change, not when unused. This is a freshness check, not an idle timeout.
4. Embedding model — preloaded at startup via @cache in dense.py; never released for the process lifetime.
5. Disk cache exists — indexes persist under ~/Library/Caches/semble/ (or SEMBLE_CACHE_LOCATION). Reload from disk on next access is already supported via get_validated_cache / load_from_disk.

So disk cache already provides persistence across reloads, but memory cache has no “release when unused” path.

Why this matters

• MCP mode optimizes for sub-ms repeat queries by keeping indexes in RAM — that trade-off makes sense.
• For agent workflows (Cursor, cursor-cli) the MCP process can sit idle for long periods while the user edits code, yet retain very large in-memory indexes (BM25 + vectors + chunk text; BM25 dominates on big repos).
• LRU-by-count alone does not help when fewer than 10 variants are touched (e.g. always searching the same monorepo root) — memory stays high indefinitely.
• Even when LRU evicts, Python RSS often does not return to the OS, but dropping references after idle would still cap live heap usage and help on memory-constrained machines.

Suggested enhancement

Add an optional idle TTL for MCP in-memory entries, e.g. environment variable:

SEMBLE_MCP_CACHE_TTL=300   # seconds; 0 or unset = current behavior (no idle eviction)

Behavior sketch:

• Track last_access per cache key on each search / find_related.
• Background task (or check on each request): evict entries idle longer than TTL from memory only; disk cache unchanged.
• Next search reloads from disk cache if still valid (fast path today), or rebuilds if stale.
• Optionally apply TTL to _merged as well, or clear _merged when all its parts are evicted.

Nice-to-haves:

• Separate TTL for the embedding model (likely keep loaded) vs full indexes.
• Log/metric when idle evict occurs (debugging agent memory use).

Workaround today

Restart the MCP client (Cursor / cursor-cli) to reset the process. No CLI flag or env var for idle release exists yet.

Context

Use case: cursor-cli calling semble:search against large monorepos. .sembleignore and consistent project-root paths reduce index size, but a single large repo index can still be ~1 GB RSS — acceptable for active search, less so when idle for hours.

Happy to help test or refine the proposed API if useful.

[Pringled]

Hey @AkatQuas, thanks for the suggestion! I think we can add this, I'll have a look and will keep you posted.