schema overhaul, test consolidation, type cleanup

[Kristof Roomp (mcroomp)]

Schema migration (requires ts recreate on existing indexes):

• Every text field switches to the raw tokenizer. No more 40-char drop, no underscore splitting — add_text_field is one token, not three; InitializeNotificationHistoryAcrossDataCentersUSA (49 chars) stays whole.
• New path_tokens field stores per-directory + filename components, so a search for billing finds files under any billing/ directory at any depth. Replaces filename as the path-side query_by fallback.
• namespace is now multi-value: Acme.Billing.Service indexes as three separate searchable tokens.
• Minimal stored set — only {id, relative_path, mtime, filename, extension, language, path_segments} are stored=True. Every other field is indexed-for-search but not retrievable; AST stage carries line-level output.
• New member_sig_tokens field: every identifier inside a member's signature (attribute names, parameter names, default-value identifiers) extracted by walking the member AST minus the body.
• Listing modes (methods / classes / fields) now return line ranges. Output renders path:start-end: text so callers can Read(file, offset=start, limit=end-start+1) precisely.

Shared modules:

• indexserver/search_modes.py holds resolve_query_params and build_filter_by. Daemon /query-codebase and scripts/search.py share the same mode→query_by mapping — new modes reach both paths.
• mcp_server.py switched to indexserver.config.load_config() instead of its own JSON-reading copy. Roots are typed Root objects.

Test architecture:

• Killed _FakeBackend. ~30 unit tests migrated to real Tantivy on tempfile.mkdtemp() via a new make_test_backend() helper. Catches schema drift the fake silently hid.
• New TestSemanticFieldDiscrim + TestPySemanticFieldDiscrim: orthogonal fixtures where one identifier appears in exactly one structural role per file. Each test asserts search-by-field returns only the right files — verifies indexer→Tantivy round-trip instead of re-running the parser.
• New test_path_tokens.py (24 tests) and test_long_identifiers.py (10 tests) covering subpath search, namespace components, and the long-identifier round-trip.
• Test consolidation via LSP cross-reference: deleted 18 redundant per-mode test files (~267 tests) covered by canonical homes (test_indexer_query_consistency.py, query/tests/test_query_cs.py, discrimination tests). Pruned 34 of 49 dead fixtures in tests/fixtures.py. Removed LiveTestBase (no longer used).
• Sample E2E tests: _get_doc → doc.get(field) pattern replaced with _search(coll, value, query_by=field) → file-in-hits assertions.

Removed back-compat shims:

• X-TYPESENSE-API-KEY header → X-API-KEY across daemon, MCP server, CLI, VS Code extension, tests, and docs. Added rationale doc explaining why a localhost service still needs an API key.
• Root.to_external alias deleted (WSL-era leftover).
• _build_filter_by / _resolve_query_params shims in tsquery_server.py deleted; test imports from indexserver.search_modes directly.
• _OpenFailure unused dataclass + dataclasses import.
• _expand_type_refs, _q_all_call_sites_data, _collect_all_refs dead functions in query/cs.py (LSP confirmed no callers).
• Unused coll_name parameter on index_file_list (12 call sites updated).

Type cleanup (pyright):

• Backend._reopen_writer now None-guards wait_merging_threads.
• BackendResolver = Callable[[str], Backend | None] (production legitimately returns None for unknown collections; downstream code already handled it).
• _CsIndex factory function correctly typed as returning TreeIndex — 17 annotation sites in query/cs.py were treating it as a class.
• build_document typed -> dict | None.
• tsquery_server._backends: dict[str, Backend] (was object, which hid every Backend.method access).
• _Fence added to the OrderedDict value union in IndexQueue; payload: object widened to dict | str with runtime narrowing.
• _Handler.log_message matches BaseHTTPRequestHandler signature.
• Watcher narrows coll, native_root, root_exts together.

Tooling:

• pyrightconfig.json for full workspace background analysis.

Test suite: 1253 → 986 tests (267 deleted, 41 new) — strictly better signal, ~6s faster.