fix(store): flush remote upserts for immediate search#543
Conversation
|
Fresh reproduction from a remote Milvus setup using upstream vanilla |
|
Ready for maintainer review and merge. I verified this PR is non-draft, mergeable, and has no configured checks reporting failures. |
…istency_level Supersedes the per-upsert flush approach (PR zilliztech#543). Per-upsert flush seals a tiny segment on every watcher write -> fragmentation + compaction pressure, and is the wrong lever: cross-process read-after-write (watcher writes, recall searches) only misses fresh data because Milvus default consistency is Bounded. - MilvusStore: add consistency_level (ctor + config/core/cli passthrough), applied to hybrid_search/dense_search/query as a per-call override. Sent only on remote (Lite is already strong-consistent), verified honoured in pymilvus 2.6.8 via kwargs, works on existing collections (no recreate). - Cold-start guard: get_collection_stats counts only sealed segments, so a fresh remote collection reports row_count=0 and the BM25 empty-guard would drop unsealed writes. When a consistency override is active, confirm emptiness with a consistency-honouring point query before short-circuiting. - search CLI gains --consistency; memory-recall skill opts into Strong so notes written earlier in a session are immediately recallable on remote Milvus. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Verification
Note: full suite currently has one unrelated failure in tests/test_embeddings_openai.py::test_embed_deterministic due exact float comparison drift.