From 374dd51979c127503e42ab6266e3f66ff66a48c5 Mon Sep 17 00:00:00 2001 From: Kristof Roomp Date: Mon, 18 May 2026 14:21:28 +0200 Subject: [PATCH 1/2] qualified-call indexing, var-type resolver, visibility filter, API polish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This branch bundles two related streams of work for tscodesearch: QUALIFIED-CALL INDEXING & VAR-TYPE RESOLVER - New ``qualified_calls`` Tantivy field stores ``Type.Method`` tokens for call sites: static-style receivers (``Foo.Save()`` → ``Foo.Save``) and resolved-receiver forms (``store.Save()`` where ``store: IRepo`` → ``IRepo.Save``). The receiver type is pinned by a new method-scoped var-type map (``_build_var_type_map``) with conflict suppression — names that map to multiple types in one scope emit no qualified form. - Block-level scoping: ``block`` / ``catch_clause`` / ``for_statement`` / ``foreach_statement`` / ``using_statement`` are scope nodes, so sibling blocks (if/else, multiple catch arms, two for-loops) isolate their declarations. Fixes prior false negatives where shadowed names collapsed to ``None`` under method-wide flat scoping. - Best-guess inference heuristics in ``_infer_var_type``: await unwrap, generic-method type-argument (``c.Resolve()`` → ``IService``), static-factory pattern (``Widget.Create()`` → ``Widget``), ternary branch try-each, member-access (instance ``obj.Prop`` → ``Prop`` / static ``Type.Member`` → ``Type``), and foreach-over-collection element-type extraction across scope boundaries (handles ``foreach (var x in someField)`` where ``someField`` is in file scope). - ``calls`` mode now searches ``call_sites,qualified_calls,path_tokens`` so bare ``Save`` and qualified ``IRepo.Save`` both hit. The AST post-filter in ``q_calls`` is upgraded to match by resolved receiver too — ``q_calls("IRepo.Save")`` finds ``store.Save()`` when ``store`` resolves to ``IRepo``. - New ``var_type NAME`` listing mode: for each identifier occurrence, report the resolved type (or ``(unresolved)`` / ``(conflicting)``). Saves an ``at LINE:COL`` round-trip when you just want the type. - ``q_at`` fix: ``field_declaration`` and ``event_field_declaration`` carry their name on a nested ``variable_declarator`` (no direct ``name`` field), so the position scope chain was silently dropping the field. Now extracted via ``_field_declarator_name``. - ``q_calls`` anchors chained calls at the method-name token. The reported line and source text point at where the matched identifier actually appears, not the start of the outer multi-line invocation. - Windows file-handle flake fix: ``Backend.upsert_many`` retries the whole add+commit cycle on transient Windows IO errors (Defender / mmap-release races on a freshly-created collection's first commit). ``tests.helpers._delete_collection`` adds a parent-directory canary on Windows so tearDownClass doesn't return until the next setUpClass can write into ``.tantivy/``. API POLISH (Phases 0-5 from the improvement plan) - Phase 0 (docs/errors): ``uses_kind`` default-union now documented; ``accesses_of`` docstring cross-links to ``all_refs`` for implicit-this reads; ``query_codebase`` redirect message expanded to cover file-only modes (``body``/``at``/``params``/``var_type``) — not just listing modes; CLAUDE.md notes the required ``mcp__tscodesearch__*`` prefix. - Phase 1 (output trimming): replaced the box-drawing banner (``── [kind] name (lines X–Y) ──``) on ``body`` / ``declarations`` output with a compact ASCII header (``[kind] name S-E:``) across all five language modules. Saves ~30 chars per match and avoids ``cp1252`` codec errors on Windows consoles. - Phase 4a: tier-2/3 follow-up suggestions and the docstring examples drop the ``$SRC_ROOT/`` placeholder — the tool accepts the bare relative path that ``query_codebase`` itself returns. - Phase 5 (visibility filter): captures explicit C# access modifiers plus language defaults (top-level types → internal, nested types → private, interface members → public, enum body members → public, class/struct/record members → private). Compound modifiers collapse to their dominant role (``protected internal`` → ``protected``, ``private protected`` → ``private``). Exposed as a comma-separated ``visibility`` argument on ``query_codebase`` and ``query_single_file`` for the four declaration modes (declarations / classes / methods / fields). Two parallel Tantivy fields (``type_visibilities``, ``member_visibilities``) so a "files with at least one public member" query doesn't catch files whose only public thing is a top-level type. - Diagnostic: new ``scripts/probe_resolved.py`` categorises lowercase- receiver gap candidates so future tuning targets the biggest patterns. TESTS: +173 new cases (1099 total, all green; 10× full-suite stability sweep clean). Coverage spans var-type resolution, block scoping, inference heuristics, qualified-receiver matching, ``var_type`` mode, ``q_at`` field scope, output format, redirect routing, Windows IO retry, and visibility extraction + filtering. DOCS: CLAUDE.md updated with the new ``qualified_calls`` schema field, revised ``calls`` semantics, ``var_type`` mode entry, ``visibility`` filter section, and the MCP-prefix note. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 13 +- indexserver/backend.py | 102 ++- indexserver/indexer.py | 37 + indexserver/search_modes.py | 8 +- mcp_server.py | 102 ++- query/_util.py | 30 + query/cpp.py | 2 +- query/cs.py | 968 ++++++++++++++++++++---- query/dispatch.py | 10 +- query/js.py | 2 +- query/py.py | 2 +- query/rust.py | 2 +- query/tests/test_cs_at_and_body.py | 52 ++ query/tests/test_cs_foreach_accesses.py | 13 +- query/tests/test_cs_var_type_map.py | 743 ++++++++++++++++++ query/tests/test_cs_var_type_mode.py | 118 +++ query/tests/test_cs_visibility.py | 248 ++++++ query/tests/test_mode_declarations.py | 4 +- scripts/probe_resolved.py | 116 +++ tests/helpers.py | 56 +- tests/unit/test_indexer.py | 66 ++ tests/unit/test_mcp_server.py | 12 +- tests/unit/test_search_modes.py | 48 ++ tsquery_server.py | 9 +- 24 files changed, 2555 insertions(+), 208 deletions(-) create mode 100644 query/tests/test_cs_var_type_map.py create mode 100644 query/tests/test_cs_var_type_mode.py create mode 100644 query/tests/test_cs_visibility.py create mode 100644 scripts/probe_resolved.py create mode 100644 tests/unit/test_search_modes.py diff --git a/CLAUDE.md b/CLAUDE.md index 71acc27..e15d1a1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -146,6 +146,8 @@ There is **no longer a WSL or indexserver venv** — Tantivy runs in-process in ## Tool selection guide +MCP tools are invoked with the full namespaced name `mcp__tscodesearch__` (e.g. `mcp__tscodesearch__query_codebase`). Shortening to `mcp__tscodesearch` returns "No such tool available" — the suffix is required. + | Goal | Tool | |------|------| | Exact line-level results across the codebase | `query_codebase` | @@ -163,7 +165,7 @@ The daemon ignores any caller-supplied `query_by`/`weights` for `/query-codebase |------|---------------------|-------| | `declarations` (default) | `class_names`, `method_names`, `path_tokens` | [T1] — narrowed by `symbol_kind`: type kinds → `class_names,path_tokens`; member kinds → `method_names,path_tokens` | | `implements` | `base_types`, `class_names`, `path_tokens` | [T1] | -| `calls` | `call_sites`, `path_tokens` | [T1] | +| `calls` | `call_sites`, `qualified_calls`, `path_tokens` | [T1] — bare `Save` hits `call_sites`; qualified `IRepository.Save` / `Foo.Save` hits `qualified_calls` | | `uses` (default) | `type_refs`, `cast_types`, `path_tokens` | [T2] — narrowed by `uses_kind` (see below) | | `uses` `uses_kind=field` | `field_types`, `path_tokens` | [T1] | | `uses` `uses_kind=param` | `param_types`, `path_tokens` | [T1] | @@ -194,12 +196,14 @@ Only a small set of fields is `stored=True` (retrievable from the index at searc | `path_segments` | Cumulative ancestor folders for the `sub=` filter (`["services", "services/billing"]`). | | `mtime` | Read by the verifier to diff fs vs. index. | -Every other text field — `class_names`, `method_names`, `base_types`, `field_types`, `local_types`, `param_types`, `return_types`, `cast_types`, `type_refs`, `call_sites`, `member_accesses`, `attr_names`, `imports`, `namespace`, `tokens`, `path_tokens`, `member_sig_tokens` — is `stored=False`. The fields are indexed for search but never read back from the index. The daemon's pipeline pre-filters with Tantivy then runs tree-sitter on the candidate files, and the AST output is what carries line-level results all the way to the caller. +Every other text field — `class_names`, `method_names`, `base_types`, `field_types`, `local_types`, `param_types`, `return_types`, `cast_types`, `type_refs`, `call_sites`, `qualified_calls`, `member_accesses`, `attr_names`, `imports`, `namespace`, `tokens`, `path_tokens`, `member_sig_tokens` — is `stored=False`. The fields are indexed for search but never read back from the index. The daemon's pipeline pre-filters with Tantivy then runs tree-sitter on the candidate files, and the AST output is what carries line-level results all the way to the caller. `path_tokens` collects every directory name plus the filename, the filename stem, and the extension for one file — so a search for `billing` finds every file under any `billing/` directory at any depth, and a search for `Foo` matches `Foo.cs`. `namespace` is a multi-value field; the indexer splits on `.` for C#/Python/Java/JS (other languages can return a pre-split list from their extractor). `member_sig_tokens` is every identifier appearing in any member signature — attribute names, parameter names, generic args, default-value identifiers — collected by each language's AST extractor walking the member node and skipping the body. The legacy `member_sigs` (full signature strings) field is gone; the structured fields (`method_names`, `return_types`, `param_types`, `class_names`) plus `member_sig_tokens` cover the same searches. `tokens` is the per-file deduped bag of every identifier — identifiers inside string literals, char literals, and comments are excluded. +`qualified_calls` carries `Type.Method` tokens for call sites whose receiver type is *syntactically obvious*: PascalCase identifier receivers (`Foo.Save()` → `Foo.Save` — captures both static class calls and PascalCase locals literally) and receivers whose declared/inferred type was pinned by the per-file var-type map (`repo.Save()` where `repo: IRepository` → `IRepository.Save`). The map is method-scoped and conflict-suppressing: a name with two distinct types in one method's scope (e.g. shadowed across `if`/`else` branches) emits no qualified form for that call. Cases the map *doesn't* resolve — `var x = GetThing()`, generic inference, LINQ lambdas without typed params — leave the qualified form absent; the bare name in `call_sites` still finds the call. Agents that know the receiver's type query the qualified form for precision; agents that don't fall back to the bare method name. + ## tree-sitter query modes **One canonical mode name per concept across every language.** The dispatch raises a `ValueError` listing the supported modes when an unknown one is passed. Call `query_single_file("capabilities", file=…)` to enumerate the modes a given file's language actually supports. @@ -215,7 +219,7 @@ Every other text field — `class_names`, `method_names`, `base_types`, `field_t | `declarations` | NAME | The declaration(s) of NAME (narrow with `symbol_kind`) | all | | `body` | NAME | Full source of NAME's declaration | C# only | | `at` | LINE:COL | Deepest AST node at position + enclosing scope chain | C# only | -| `calls` | METHOD | Call sites of METHOD (`"Repo.Save"` to restrict by receiver). Pass a METHOD name only — a variable/receiver name silently returns empty; use `all_refs` on the variable for that. | all | +| `calls` | METHOD | Call sites of METHOD. Qualify with `Type.Method` to restrict by receiver — the qualifier matches both the literal receiver text (`Foo.Save()`) **and** any receiver whose declared/inferred type resolves to that name via the method-scoped var-type map (`store.Save()` where `store: IRepository` matches `IRepository.Save`). When the receiver's type is conflicted in its scope, the qualified match is skipped — the bare name still finds the call. Pass a METHOD name only — a variable/receiver name silently returns empty; use `all_refs` on the variable for that. | all | | `implements` | TYPE | Types that inherit/implement TYPE | all except SQL | | `uses` | TYPE | Type references; narrow with `uses_kind` (`field`/`param`/`return`/`cast`/`base`/`locals`) | C# only | | `casts` | TYPE | `(TYPE)expr` / `as TYPE` sites | C# only | @@ -223,6 +227,9 @@ Every other text field — `class_names`, `method_names`, `base_types`, `field_t | `accesses_of` | MEMBER | Access sites of property/field by name (`"Order.Status"` restricts) | C# only | | `accesses_on` | TYPE | `.Member` accesses on locals/params/fields typed as TYPE (plus `new T { … }` and `with` mutations). Returns nothing when the variable is only assigned, returned, or forwarded as an argument — no `.Member` exists. Fall back to `all_refs` on the variable name. | C# only | | `all_refs` | NAME | Every identifier occurrence (broadest — AST-only, skips strings/comments). For SQL this is a plain substring scan over lines. | all | +| `var_type` | NAME | For each occurrence of NAME, report the resolved type from the method-scoped var-type map, or `(unresolved)` / `(conflicting)` when the resolver can't pin it down. Saves an `at LINE:COL` round-trip when you just want the type. | C# only | + +**Visibility filter (declaration modes).** `declarations`, `classes`, `methods`, `fields` accept `visibility="public,internal,protected,private"` (comma-separated, any subset). The filter is applied AST-side per declaration using the same defaults the indexer uses: top-level types default to `internal`, nested types to `private`, interface members to `public`, enum members to `public`, class/struct/record members to `private`. Compound modifiers collapse to their dominant role (`protected internal` → `protected`, `private protected` → `private`). Languages other than C# currently don't capture visibility — passing the filter against them returns nothing rather than over-matching. --- diff --git a/indexserver/backend.py b/indexserver/backend.py index abe91b5..3fc9ffc 100644 --- a/indexserver/backend.py +++ b/indexserver/backend.py @@ -21,12 +21,45 @@ import os import re import shutil +import sys import threading +import time from pathlib import Path import tantivy +# ── Transient Windows IO retry ──────────────────────────────────────────────── +# +# On Windows, ``IndexWriter.commit()`` occasionally fails with +# ``Access is denied`` / ``PermissionDenied`` when opening a new segment +# file for write — even for a freshly-created collection directory. +# Suspected causes include Windows Defender scanning newly created files, +# stale mmap handles from earlier reader Backends releasing asynchronously +# in the same process, and the OS taking a moment to make a just-deleted +# directory's namespace available again. The errors are transient: a brief +# settle delay followed by re-running the same write succeeds. Retry only +# at the ``upsert_many`` level because the writer's in-memory buffer is +# discarded on a failed commit, so we have to re-add the docs. + +_COMMIT_RETRY_ATTEMPTS = 4 +_COMMIT_RETRY_BASE_DELAY = 0.2 # seconds; doubled each attempt + + +def _is_transient_windows_io_error(err: BaseException) -> bool: + """Return True for the Windows-only file-handle races we retry over.""" + if sys.platform != "win32": + return False + msg = str(err).lower() + return ( + "access is denied" in msg + or "permission denied" in msg + or "permissiondenied" in msg + or "os error 5" in msg + or "(os error 32)" in msg # The process cannot access the file. + ) + + # Schema definition — kept here because the same field set is consulted by # writer (build_document → backend.add) and reader (search.search). # @@ -46,8 +79,9 @@ "member_sig_tokens", "base_types", "field_types", "local_types", "param_types", "return_types", "cast_types", - "type_refs", "call_sites", "member_accesses", + "type_refs", "call_sites", "qualified_calls", "member_accesses", "attr_names", "imports", + "type_visibilities", "member_visibilities", "path_tokens", ) @@ -57,8 +91,9 @@ "member_sig_tokens", "base_types", "field_types", "local_types", "param_types", "return_types", "cast_types", - "type_refs", "call_sites", "member_accesses", + "type_refs", "call_sites", "qualified_calls", "member_accesses", "attr_names", "imports", + "type_visibilities", "member_visibilities", "namespace", "tokens", "path_tokens", @@ -104,8 +139,15 @@ def build_schema() -> tantivy.Schema: "member_sig_tokens", # every identifier inside a sig "base_types", "field_types", "local_types", "param_types", "return_types", "cast_types", - "type_refs", "call_sites", "member_accesses", + "type_refs", "call_sites", + "qualified_calls", # ``Type.Method`` forms (static + resolved-receiver) + "member_accesses", "attr_names", "imports", + # Canonical access modifiers captured per declaration. Two parallel + # fields so a query for "files that have at least one public + # member" doesn't catch files that only have public *types*. + "type_visibilities", + "member_visibilities", "tokens", # deduped bag of every identifier in the file "path_tokens", # per-directory + filename parts ): @@ -318,23 +360,51 @@ def upsert_many(self, docs: list[dict]) -> tuple[int, int]: Returns (n_ok, n_failed). On commit failure all docs are reported as failed (Tantivy's commit is atomic — partial application isn't a - thing here). + thing here). Transient Windows IO errors are retried with backoff: + the writer's in-memory buffer is discarded on a failed commit so + we re-run the entire add + commit sequence after a brief settle + delay. """ if not docs: return 0, 0 - n_added = 0 - for d in docs: + + last_err: BaseException | None = None + for attempt in range(_COMMIT_RETRY_ATTEMPTS): + n_added = 0 + for d in docs: + try: + self.add(d) + n_added += 1 + except Exception as e: + rel = d.get("relative_path", d.get("id", "?")) + print(f"[backend] add failed for {rel}: {type(e).__name__}: {e}", flush=True) try: - self.add(d) - n_added += 1 - except Exception as e: - rel = d.get("relative_path", d.get("id", "?")) - print(f"[backend] add failed for {rel}: {type(e).__name__}: {e}", flush=True) - try: - self.commit() - except Exception: - return 0, len(docs) - return n_added, len(docs) - n_added + self.commit() + return n_added, len(docs) - n_added + except Exception as commit_err: + last_err = commit_err + if (attempt + 1 >= _COMMIT_RETRY_ATTEMPTS + or not _is_transient_windows_io_error(commit_err)): + break + delay = _COMMIT_RETRY_BASE_DELAY * (2 ** attempt) + print( + f"[backend] commit attempt {attempt + 1}/{_COMMIT_RETRY_ATTEMPTS} " + f"hit a transient Windows IO error in {self.path}; " + f"retrying after {delay:.2f}s", + flush=True, + ) + # commit()'s error handler already rolled back and reopened + # the writer, so we just need to settle and retry. + gc.collect() + time.sleep(delay) + # Out of retries or non-transient error: caller treats as full failure. + if last_err is not None: + print( + f"[backend] upsert_many giving up on {len(docs)} docs in " + f"{self.path}: {type(last_err).__name__}: {last_err}", + flush=True, + ) + return 0, len(docs) def delete_many(self, ids: list[str]) -> int: if not ids: diff --git a/indexserver/indexer.py b/indexserver/indexer.py index 1451446..1790967 100644 --- a/indexserver/indexer.py +++ b/indexserver/indexer.py @@ -166,6 +166,24 @@ def flat_from_fd(fd) -> dict: local_types = [t for lv in fd.local_var_infos for t in _expand_type(lv.var_type)] member_accesses = [ma.member for ma in fd.member_access_infos] + # qualified_calls: ``Type.Method`` tokens that survive method-scope + # conflict suppression. Two contributors per call site: + # * Static-style receiver — ``Foo.Bar()`` where ``Foo`` is a PascalCase + # identifier; emit ``Foo.Bar`` (the receiver might be a class name + # or a misnamed PascalCase local, either way the literal form is + # what the agent typed). + # * Resolved-type receiver — the language extractor pinned the + # receiver's declared/inferred type via the var-type map; emit + # ``ResolvedType.Method`` for precise type-aware lookup. + # Both forms go in the same field so the agent can search whichever + # qualifier they know; AST post-filter narrows to actual hits. + qualified_calls: list = [] + for cs in fd.call_site_infos: + if cs.receiver and cs.receiver[:1].isupper() and cs.name: + qualified_calls.append(f"{cs.receiver}.{cs.name}") + if cs.resolved_type and cs.name: + qualified_calls.append(f"{cs.resolved_type}.{cs.name}") + type_refs = list(field_types) + list(param_types) + list(return_types) + list(base_types) + list(local_types) for cs in fd.call_site_infos: if cs.receiver and cs.receiver[0].isupper(): @@ -187,6 +205,19 @@ def flat_from_fd(fd) -> dict: for f in fd.fields: sig_tokens.extend(f.sig_tokens) + # Canonical access modifiers, indexed in two parallel multi-value + # fields so a query for "files containing at least one public member" + # doesn't accidentally fire on files whose only ``public`` thing is a + # top-level type. Empty strings (language doesn't track visibility) are + # dropped — searching ``public`` should never match those files. + type_visibilities = [ + c.visibility for c in fd.classes if getattr(c, "visibility", "") + ] + member_visibilities = ( + [m.visibility for m in fd.methods if getattr(m, "visibility", "")] + + [f.visibility for f in fd.fields if getattr(f, "visibility", "")] + ) + return { # ``namespace`` is multi-value raw — store each dot-separated # component as its own searchable token. @@ -195,6 +226,7 @@ def flat_from_fd(fd) -> dict: "method_names": _dedupe(method_names), "base_types": _dedupe(base_types), "call_sites": _dedupe(call_sites), + "qualified_calls": _dedupe(qualified_calls), "cast_types": _dedupe(cast_types), "member_sigs": _dedupe(member_sigs), # diagnostic only — not indexed "member_sig_tokens": _dedupe(sig_tokens), @@ -206,6 +238,8 @@ def flat_from_fd(fd) -> dict: "field_types": _dedupe(field_types), "local_types": _dedupe(local_types), "member_accesses": _dedupe(member_accesses), + "type_visibilities": _dedupe(type_visibilities), + "member_visibilities": _dedupe(member_visibilities), "tokens": _dedupe(fd.all_refs), } @@ -322,7 +356,10 @@ def build_document(full_path: str, relative_path: str) -> dict | None: "cast_types": meta["cast_types"], "type_refs": meta["type_refs"], "call_sites": meta["call_sites"], + "qualified_calls": meta["qualified_calls"], "member_accesses": meta["member_accesses"], + "type_visibilities": meta["type_visibilities"], + "member_visibilities": meta["member_visibilities"], "attr_names": meta["attr_names"], "imports": meta["imports"], } diff --git a/indexserver/search_modes.py b/indexserver/search_modes.py index 7c47560..56bca35 100644 --- a/indexserver/search_modes.py +++ b/indexserver/search_modes.py @@ -32,7 +32,13 @@ def resolve_query_params(ts_mode_flag: str, uses_kind: str = "", symbol_kind: st if ts_mode_flag == "implements": return "base_types,class_names,path_tokens", "4,3,2" if ts_mode_flag == "calls": - return "call_sites,path_tokens", "4,2" + # ``qualified_calls`` carries ``Type.Method`` tokens (static-style + # ``Foo.Bar`` plus method-scoped resolved-receiver forms like + # ``IRepository.Save`` when the indexer pinned the type). Querying + # both fields lets the agent pass either a bare ``Save`` or a + # qualified ``IRepository.Save`` without picking the right field + # themselves. + return "call_sites,qualified_calls,path_tokens", "4,4,2" if ts_mode_flag == "uses": k = (uses_kind or "all").lower().strip() if k == "field": return "field_types,path_tokens", "4,2" diff --git a/mcp_server.py b/mcp_server.py index 8b1fbf0..c713bb0 100644 --- a/mcp_server.py +++ b/mcp_server.py @@ -183,6 +183,7 @@ def query_codebase( include_body: bool = False, symbol_kind: str = "", uses_kind: str = "", + visibility: str = "", exclude_path: str = "", ) -> str: """Index pre-filter + tree-sitter AST. Returns one of three response shapes @@ -238,7 +239,12 @@ def query_codebase( `.Member` exists). When `accesses_on` is empty but you know the variable exists, fall back to `all_refs` on the variable name. - - `accesses_of` wants a MEMBER name (e.g. "Timeout"). + - `accesses_of` wants a MEMBER name (e.g. "Timeout"). It + only finds *qualified* reads (`expr.Timeout`). Bare + identifier reads in the declaring class itself — which + compile to implicit `this.Timeout` — are NOT matched. + For implicit-this reads use `all_refs` on the member + name. pattern: A single identifier. Examples that DO work: "BlobStore", "SaveChanges", "IDataStore". Examples that do NOT work: "using BlobStore", "(BlobStore)", "Save Changes", @@ -255,7 +261,18 @@ def query_codebase( root: Named source root (empty = default). include_body: For declarations — include full body. Default false. symbol_kind: For declarations — restrict to: method, class, interface, etc. - uses_kind: For uses — all, field, param, return, cast, base, locals. + uses_kind: For `uses` — narrow to one structural role. Values: + - omitted / "all" (default): union of `type_refs` + + `cast_types` — every type reference anywhere in the file. + - field, param, return, cast, base, locals: narrow to that + one role. + visibility: For declaration modes (declarations / classes / methods / + fields) — comma-separated access modifiers to keep. Values: + public, internal, protected, private. Empty = no filter. + Languages that don't capture visibility (e.g. SQL) match + nothing when this filter is set. (C# captures explicit + modifiers plus interface-public / enum-public / nested + type defaults.) exclude_path: Comma-separated list of folder paths to exclude from results. Each value is matched as an exact ancestor folder, not a glob — wildcards are not supported. Behavior: @@ -276,11 +293,25 @@ def query_codebase( query_codebase("calls", "SaveChanges", sub="services,vendor") query_codebase("calls", "SaveChanges", exclude_path="tests,generated") query_codebase("uses", "IRepo", sub="services", exclude_path="services/legacy")""" - _LISTING = {"methods", "fields", "classes", "imports", "capabilities"} + # File-targeted modes don't make sense for a codebase-wide search. + # `body`/`at`/`params`/`var_type` need an explicit file; listing modes + # (methods/fields/classes/imports/capabilities) only describe a single + # file's structure. Catch them here so the agent sees an actionable + # redirect instead of the daemon's generic "unknown mode" error. + _FILE_ONLY = { + "methods", "fields", "classes", "imports", "capabilities", + "body", "at", "params", "var_type", + } m = mode.lower().strip().replace("-", "_") - if m in _LISTING: - return (f"Mode '{m}' lists file contents without filtering — use query_single_file instead:\n" - f' query_single_file("{m}", file="$SRC_ROOT/path/to/File.cs")') + if m in _FILE_ONLY: + if m in ("body", "at", "params", "var_type"): + why = "needs a specific file" + example_arg = pattern or ('"SaveChanges"' if m != "at" else '"42:10"') + example = f' query_single_file("{m}", {example_arg}, file="path/to/File.cs")' + else: + why = "lists one file's contents without filtering" + example = f' query_single_file("{m}", file="path/to/File.cs")' + return (f"Mode '{m}' {why} — use query_single_file instead:\n{example}") try: status, data = _post("/query-codebase", { @@ -289,6 +320,7 @@ def query_codebase( "root": root or "", "limit": _QUERY_CODEBASE_LIMIT, "include_body": include_body, "symbol_kind": symbol_kind or "", "uses_kind": uses_kind or "", + "visibility": visibility or "", "exclude_path": exclude_path or "", }) except Exception as e: @@ -377,11 +409,16 @@ def query_codebase( files_with_matches.sort(key=lambda fm: -len(fm[1])) def _qsf_call(file_rel: str) -> str: - """A query_single_file call mirroring the current query_codebase params.""" + """A query_single_file call mirroring the current query_codebase params. + + Uses the bare relative path from the tier-2/3 listing — the tool + accepts that directly (it prepends the default root), so injecting + a ``$SRC_ROOT/`` placeholder just adds noise. + """ args = [f'"{m}"'] if pattern: args.append(f'"{pattern}"') - args.append(f'file="$SRC_ROOT/{file_rel}"') + args.append(f'file="{file_rel}"') if root: args.append(f'root="{root}"') if include_body: @@ -390,6 +427,8 @@ def _qsf_call(file_rel: str) -> str: args.append(f'symbol_kind="{symbol_kind}"') if uses_kind: args.append(f'uses_kind="{uses_kind}"') + if visibility: + args.append(f'visibility="{visibility}"') return "query_single_file(" + ", ".join(args) + ")" # Tier 2 — many files: filenames + counts only. @@ -445,6 +484,7 @@ def query_single_file( include_body: bool = False, symbol_kind: str = "", uses_kind: str = "", + visibility: str = "", head_limit: int = 250, offset: int = 0, ) -> str: @@ -476,12 +516,17 @@ def query_single_file( receiver — `obj.Foo()` is matched by calls("Foo") not calls("obj"); for variable usage use all_refs. implements TYPE Types that inherit/implement TYPE. - uses TYPE Type references; narrow with `uses_kind` (C# only) - among: field, param, return, cast, base, locals. + uses TYPE Type references. Omit `uses_kind` (or "all") for + the union of every role; narrow with `uses_kind` + ∈ {field, param, return, cast, base, locals}. + (C# only.) casts TYPE Explicit (TYPE)expr / as TYPE sites. attrs NAME? [Attribute] / @decorator usages (omit NAME to list all). params METHOD Parameters of METHOD. - accesses_of MEMBER Access sites of property/field MEMBER. (C# only.) + accesses_of MEMBER Qualified access sites of property/field MEMBER — + `expr.MEMBER`. Bare `MEMBER` (implicit `this.MEMBER` + inside the declaring class) is NOT matched; use + `all_refs MEMBER` for that. (C# only.) accesses_on TYPE .Member accesses on locals/params declared as TYPE. Returns NOTHING when the variable is only assigned, returned, or forwarded as an argument — no `.Member` @@ -489,6 +534,12 @@ def query_single_file( (C# only.) all_refs NAME Every identifier occurrence (broadest; AST-only, skips strings/comments). + var_type NAME For each occurrence of NAME, report its resolved + type from the method-scoped var-type map (or + `(unresolved)` / `(conflicting)`). Use this to + answer "what's the type of `foo` at line 42" + without having to find the exact column for `at`. + (C# only today.) Position mode — `pattern` is "LINE:COL" (1-indexed): at LINE:COL Identify the deepest AST node at the position and @@ -509,8 +560,16 @@ def query_single_file( symbol_kind: For `declarations` / `body` — restrict to method, ctor, class, interface, struct, enum, record, delegate, property, field, event, type, or member. - uses_kind: For `uses` — all (default), field, param, return, cast, base, - or locals. (C# only.) + uses_kind: For `uses` — narrow to one structural role. Omit (or pass + "all") for the union of every role; otherwise one of + field, param, return, cast, base, locals. (C# only.) + visibility: For declaration modes (declarations / classes / methods / + fields) — comma-separated access modifiers to keep + (public, internal, protected, private). Omit for no + filter. C# captures explicit modifiers and applies the + language's defaults (interface members ⇒ public, nested + types ⇒ private, top-level types ⇒ internal); other + languages currently return nothing when this filter is set. head_limit: Max results to return (default 250). Use with offset to page. offset: Skip first N results before applying head_limit (default 0). @@ -519,14 +578,16 @@ def query_single_file( lists the modes that ARE supported. Use `capabilities` to enumerate them programmatically before calling. -Examples: - query_single_file("capabilities", file="$SRC_ROOT/services/Widget.cs") - query_single_file("methods", file="$SRC_ROOT/services/Widget.cs") - query_single_file("body", "SaveChanges", file="$SRC_ROOT/data/Widget.cs") - query_single_file("at", "42:10", file="$SRC_ROOT/data/Widget.cs") - query_single_file("calls", "SaveChanges", file="$SRC_ROOT/data/Widget.cs") +Examples (relative paths resolve against the default root; ``$SRC_ROOT/`` +prefix is still accepted for back-compat but no longer required): + query_single_file("capabilities", file="services/Widget.cs") + query_single_file("methods", file="services/Widget.cs") + query_single_file("body", "SaveChanges", file="data/Widget.cs") + query_single_file("at", "42:10", file="data/Widget.cs") + query_single_file("calls", "SaveChanges", file="data/Widget.cs") + query_single_file("var_type", "store", file="data/Widget.cs") query_single_file("uses", "IRepository", uses_kind="param", - file="$SRC_ROOT/services/Widget.cs")""" + file="services/Widget.cs")""" if not file: return "file= is required." @@ -551,6 +612,7 @@ def query_single_file( include_body=include_body, symbol_kind=symbol_kind or None, uses_kind=uses_kind or None, + visibility=visibility or None, ) except ValueError as e: # Unknown extension or unsupported mode — propagate the helpful diff --git a/query/_util.py b/query/_util.py index 925c3c2..df66901 100644 --- a/query/_util.py +++ b/query/_util.py @@ -17,6 +17,12 @@ class CallSiteInfo: """A function or method call site.""" name: str receiver: str = "" # receiver identifier when it looks like a type (e.g. "Repo" in Repo.Save()) + # Resolved receiver type when the language's local analyser can pin it down + # unambiguously (static class name, declared field/param/local). Empty when + # the receiver isn't an identifier, the identifier isn't in scope, or the + # name resolves to conflicting types within its scope. Drives the indexer's + # ``qualified_calls`` field. + resolved_type: str = "" @dataclass @@ -61,6 +67,22 @@ class FileDescription: all_refs: set = dc_field(default_factory=set) +# Canonical visibility tokens. Each language extractor maps its native +# modifier set into one of these. ``""`` means "the language did not +# capture a visibility for this declaration" — typically because it's a +# language where the concept doesn't apply (Python module-level functions, +# Rust impl items, SQL columns). Callers filtering by visibility should +# treat empty as "unknown", not as "public". +VISIBILITY_PUBLIC = "public" +VISIBILITY_INTERNAL = "internal" +VISIBILITY_PROTECTED = "protected" +VISIBILITY_PRIVATE = "private" +KNOWN_VISIBILITIES = frozenset({ + VISIBILITY_PUBLIC, VISIBILITY_INTERNAL, + VISIBILITY_PROTECTED, VISIBILITY_PRIVATE, +}) + + @dataclass class ClassInfo: """A type declaration (class, struct, interface, enum, trait, union, …). @@ -74,6 +96,10 @@ class ClassInfo: kind: str bases: list = dc_field(default_factory=list) end_line: int = 0 + # Canonical access modifier: ``public`` / ``internal`` / ``protected`` / + # ``private`` / ``""`` (unknown). Captured by language extractors that + # have explicit modifier keywords; empty for languages that don't. + visibility: str = "" @property def text(self) -> str: @@ -103,6 +129,8 @@ class MethodInfo: # means the language doesn't yet emit them. sig_tokens: list = dc_field(default_factory=list) end_line: int = 0 + # Canonical access modifier — same values as ClassInfo.visibility. + visibility: str = "" @property def text(self) -> str: @@ -122,6 +150,8 @@ class FieldInfo: # field/property declaration excluding any initialiser body. sig_tokens: list = dc_field(default_factory=list) end_line: int = 0 + # Canonical access modifier — same values as ClassInfo.visibility. + visibility: str = "" @property def text(self) -> str: diff --git a/query/cpp.py b/query/cpp.py index 502569d..9d74374 100644 --- a/query/cpp.py +++ b/query/cpp.py @@ -411,7 +411,7 @@ def _append(node, decl_name, kind): content = "\n".join(lines[start_row:sig_end]).rstrip() else: content = "\n".join(lines[start_row:end_row + 1]) - header = f"── [{kind}] {name} (lines {start_row + 1}–{end_row + 1}) ──" + header = f"[{kind}] {name} {start_row + 1}-{end_row + 1}:" results.append((_line(node), f"{header}\n{content}")) # Classes / structs / enums diff --git a/query/cs.py b/query/cs.py index c69f83d..10b4865 100644 --- a/query/cs.py +++ b/query/cs.py @@ -303,6 +303,83 @@ def _build_sig(node, src) -> str: return f"{ret_txt} {name_txt}({params_txt})".strip() if ret_txt else f"{name_txt}({params_txt})" +_CS_VISIBILITY_TOKENS = ("public", "internal", "protected", "private") + + +def _cs_explicit_visibility(node) -> str: + """Return the most-specific access modifier present on ``node`` as a + canonical single-word string, or ``""`` if none is written. + + C# allows compound modifiers (``protected internal``, ``private + protected``). For filtering purposes the *outermost* visibility is the + one that matters: ``protected internal`` is reachable from outside the + assembly through inheritance, so it's classified as ``protected``; + ``private protected`` is the most-restricted form, classified as + ``private``. Anything else collapses to the matching keyword. + """ + mods = [c for c in node.children if c.type == "modifier"] + if not mods: + return "" + seen = [] + for m in mods: + for child in m.children: + t = child.type + if t in _CS_VISIBILITY_TOKENS and t not in seen: + seen.append(t) + if not seen: + return "" + if "private" in seen and "protected" in seen: + return "private" # private protected = most restricted + if "protected" in seen and "internal" in seen: + return "protected" # protected internal — reachable via inheritance + # Single-keyword forms. + for tok in _CS_VISIBILITY_TOKENS: + if tok in seen: + return tok + return "" + + +def _cs_type_visibility(node) -> str: + """Visibility for a top-level type declaration. Explicit modifier wins; + otherwise nested types in a class default to ``private``, and types at + namespace/file level default to ``internal`` (C# defaults).""" + explicit = _cs_explicit_visibility(node) + if explicit: + return explicit + # Walk up to find the immediate scope: another type → private; else + # namespace / compilation_unit → internal. + p = node.parent + while p is not None: + if p.type in _TYPE_DECL_NODES: + return "private" + if p.type in ("namespace_declaration", + "file_scoped_namespace_declaration", + "compilation_unit"): + return "internal" + p = p.parent + return "internal" + + +def _cs_member_visibility(node) -> str: + """Visibility for a member declaration (method, field, prop, event, + ctor). Explicit modifier wins; absence of any modifier inside an + interface or enum defaults to ``public`` (the language's rule). + Anywhere else, absence defaults to ``private``.""" + explicit = _cs_explicit_visibility(node) + if explicit: + return explicit + p = node.parent + while p is not None: + if p.type == "interface_declaration": + return "public" + if p.type == "enum_declaration": + return "public" + if p.type in _TYPE_DECL_NODES: + return "private" + p = p.parent + return "private" + + def _enclosing_type_name(node, src) -> str: p = node.parent while p: @@ -314,6 +391,518 @@ def _enclosing_type_name(node, src) -> str: return "" +# ── Block-scoped variable-type resolver ─────────────────────────────────────── + +# Node types that introduce a new variable scope. A call site inside one of +# these nodes resolves variable names against this node's local map first, +# then walks outward through every enclosing scope. The grammar's ``block`` +# nodes are included so sibling blocks (if/else branches, try and each catch, +# the two arms of a switch) isolate their declarations — without that, real +# code like ``if (b) { Foo x = ...; } else { Bar x = ...; }`` collides under +# a single method-wide map. +_SCOPE_NODES = frozenset({ + # Method-like containers (carry parameters but no body locals — those + # live in the body block, which is its own scope below). + "method_declaration", "constructor_declaration", "destructor_declaration", + "operator_declaration", "conversion_operator_declaration", + "local_function_statement", + "accessor_declaration", + "lambda_expression", "anonymous_method_expression", + # Block and the statements that declare a variable visible only inside + # their body. + "block", + "catch_clause", + "for_statement", + "foreach_statement", + "using_statement", +}) + +# Subset of scope nodes that carry a ``parameters`` field — used to decide +# which declaration channels the per-scope walker should consult. +_PARAMETERIZED_SCOPES = frozenset({ + "method_declaration", "constructor_declaration", "destructor_declaration", + "operator_declaration", "conversion_operator_declaration", + "local_function_statement", + "accessor_declaration", + "lambda_expression", "anonymous_method_expression", +}) + + +class _VarTypeMap: + """Method-scoped variable name → resolved type. + + ``resolve_at(name, node)`` walks up from ``node`` through every enclosing + scope node, returning the first matching declared/inferred type. A name + that maps to conflicting types within one scope is sentinelled to ``None`` + and never produces a qualified call form. + + Scope identity is keyed by ``tree_sitter.Node.id`` (the underlying C-side + node pointer) rather than Python ``id()``, because tree-sitter creates + fresh Python wrapper objects for each traversal — ``node.parent`` from + one call site produces a different Python object than the same node + found via a top-down walk, even though both wrap the same AST node. + """ + + __slots__ = ("_scope_maps", "_file_map") + + def __init__(self, scope_maps: dict[int, dict[str, str | None]], + file_map: dict[str, str | None]): + self._scope_maps = scope_maps + self._file_map = file_map + + def resolve_at(self, var_name: str, node) -> str | None: + """Return resolved type for ``var_name`` from ``node``'s position. + + Returns ``None`` when the name is unknown or conflicts in its scope. + """ + if not var_name: + return None + p = node + while p is not None: + sm = self._scope_maps.get(p.id) + if sm is not None and var_name in sm: + return sm[var_name] + p = p.parent + return self._file_map.get(var_name) + + +_MISSING = object() + + +def _scope_add(m: dict, name: str, type_txt: str) -> None: + """Insert name→type into m, suppressing on conflict (set to None).""" + if not name or not type_txt: + return + existing = m.get(name, _MISSING) + if existing is _MISSING: + m[name] = type_txt + elif existing is None: + return + elif existing != type_txt: + m[name] = None + + +def _generic_type_arg(generic_node, src) -> str: + """Return the first type argument from a ``generic_name``'s + ``type_argument_list`` child, or ``""`` if not present.""" + for c in generic_node.children: + if c.type == "type_argument_list": + for g in c.children: + if g.is_named: + return _text(g, src).strip() + return "" + + +def _infer_var_type(expr, src, scope_map: dict) -> str: + """Best-effort syntactic type inference for a ``var`` initialiser. + + Handles, in order: + * ``await E`` — unwrap and recurse + * ``new T(...)`` / ``new T[...]`` — exact type + * ``(T)expr`` / ``expr as T`` — cast/as target + * ``arr[i]`` where ``arr: T[]`` — element type from scope + * ``GenericMethod(...)`` — first type arg (DI/factory idiom) + * ``recv.GenericMethod(...)`` — same; ignores the receiver + * ``TypeName.Method(...)`` — assume TypeName is the result + (static factory idiom) + + The factory and generic heuristics deliberately favour over-emission: + AI agents post-filter results, and a missing qualified form is worse + than a few harmless extras (they don't match a real call line in the + AST stage anyway). + + Returns ``""`` when no plausible type is derivable. + """ + if expr is None: + return "" + + # Strip await wrappers — ``await E`` has the same observed type as E for + # our purposes (we don't model Task unwrapping, but the inner type + # is almost always more useful than nothing). + if expr.type == "await_expression": + inner = next((c for c in expr.children if c.is_named), None) + return _infer_var_type(inner, src, scope_map) + + t = expr.type + if t == "object_creation_expression": + tn = expr.child_by_field_name("type") + return _text(tn, src).strip() if tn else "" + if t == "array_creation_expression": + tn = expr.child_by_field_name("type") + return _text(tn, src).strip() if tn else "" + if t == "cast_expression": + tn = expr.child_by_field_name("type") + return _text(tn, src).strip() if tn else "" + if t == "as_expression": + tn = expr.child_by_field_name("right") or expr.child_by_field_name("type") + return _text(tn, src).strip() if tn else "" + if t == "element_access_expression": + obj = expr.child_by_field_name("expression") + if obj and obj.type == "identifier": + arr_type = scope_map.get(_text(obj, src).strip()) + if isinstance(arr_type, str) and arr_type.endswith("[]"): + return arr_type[:-2].strip() + return "" + if t == "invocation_expression": + fn = expr.child_by_field_name("function") + if fn is None: + return "" + # Bare ``GenericMethod(...)`` — the type argument is almost + # always the return type (DI ``Resolve``, ``Get`` patterns). + if fn.type == "generic_name": + ta = _generic_type_arg(fn, src) + if ta: + return ta + if fn.type == "member_access_expression": + name = fn.child_by_field_name("name") + receiver = fn.child_by_field_name("expression") + # ``recv.GenericMethod(...)`` — same heuristic. + if name is not None and name.type == "generic_name": + ta = _generic_type_arg(name, src) + if ta: + return ta + # ``TypeName.Method(...)`` — static factory pattern. Receiver is + # a bare PascalCase identifier and isn't a declared variable in + # this scope. False-positive friendly: a static method that + # returns something other than its enclosing type still yields + # a qualified form that AST post-filtering will reject if no + # matching call exists. + if (receiver is not None + and receiver.type == "identifier"): + rtxt = _text(receiver, src).strip() + if (rtxt and rtxt[:1].isupper() + and scope_map.get(rtxt, _MISSING) is _MISSING): + return rtxt + return "" + + if t == "member_access_expression": + name = expr.child_by_field_name("name") + receiver = expr.child_by_field_name("expression") + if name is None: + return "" + # Static property access: ``TypeName.Member`` — guess TypeName. + # Mirrors the invocation-side factory heuristic. + if (receiver is not None and receiver.type == "identifier"): + rtxt = _text(receiver, src).strip() + if (rtxt and rtxt[:1].isupper() + and scope_map.get(rtxt, _MISSING) is _MISSING): + return rtxt + # Instance property access: ``obj.PascalProperty`` — guess that the + # property's type matches its name (.NET convention; very common + # for typed wrapper/sub-object properties like + # ``request.RequestMetrics``, ``ctx.AuthContext``). False positives + # for primitive-named properties (``Count``, ``Length``, ``Name``) + # are tolerated — the qualified form they produce doesn't match + # any real call line at the AST stage. + if name.type == "identifier": + ptxt = _text(name, src).strip() + if ptxt and ptxt[:1].isupper(): + return ptxt + return "" + + if t == "conditional_expression": + # ``cond ? A : B`` — try each branch in turn; the first that yields + # a type wins. Captures patterns like + # ``var x = useNear ? group.Near : group.Far`` where both branches + # are property accesses (instance heuristic above resolves them + # individually). + for field in ("consequence", "alternative"): + branch = expr.child_by_field_name(field) + if branch is not None: + r = _infer_var_type(branch, src, scope_map) + if r: + return r + return "" + + return "" + + +def _get_init_expr(declarator): + children = declarator.children + if len(children) >= 3 and children[1].type == "=": + return children[2] + return None + + +def _add_variable_decl(node, src, scope_map: dict, + explicit: list, var_inferred: list) -> None: + """Split a ``variable_declaration`` into explicit / var-inferred buckets. + + ``explicit`` and ``var_inferred`` are appended to so the caller can apply + explicit declarations first (so element-access var-inference can resolve + against known array types). + """ + tn = node.child_by_field_name("type") + if tn is None: + return + ttxt = _text(tn, src).strip() + is_var = (ttxt == "var" or tn.type == "implicit_type") + for decl in _find_all(node, lambda x: x.type == "variable_declarator"): + vn = decl.child_by_field_name("name") + if vn is None: + continue + if is_var: + var_inferred.append((vn, _get_init_expr(decl))) + else: + explicit.append((vn, ttxt)) + + +def _collect_scope_locals(scope_node, src, scope_map: dict, + scope_maps: dict, file_map: dict) -> None: + """Populate scope_map with every variable declared **directly** in scope_node. + + Declarations inside nested scope nodes (other blocks, catch clauses, + nested methods, lambdas, …) belong to their own maps and are skipped. + The resolver walks the parent chain at lookup time, so an inner scope + transparently inherits names from outer scopes without needing the + inner map to copy them. + + ``scope_maps`` and ``file_map`` are the partial-state used for + cross-scope lookups at construction time — chiefly to resolve + ``foreach (var x in coll)`` where ``coll`` is a field or outer-method + parameter and the iterator type can be derived from its element type. + """ + nt = scope_node.type + + # ── Method-like nodes own their parameters; their body is a separate + # block scope that handles its own locals. + if nt in _PARAMETERIZED_SCOPES: + params = scope_node.child_by_field_name("parameters") + if params is not None: + for p in _find_all(params, lambda n: n.type == "parameter"): + pt = p.child_by_field_name("type") + pn = p.child_by_field_name("name") + if pt and pn: + _scope_add(scope_map, _text(pn, src).strip(), + _text(pt, src).strip()) + # Lambdas may have an expression body (no block) — pattern/decl + # bindings inside the expression head still bind into this scope. + body = scope_node.child_by_field_name("body") + if body is not None and body.type != "block": + _absorb_pattern_bindings(body, src, scope_map) + return + + # ── catch_clause: (TypeName ident) declares ``ident: TypeName``. + if nt == "catch_clause": + decl = next((c for c in scope_node.children if c.type == "catch_declaration"), None) + if decl is not None: + idents = [c for c in decl.children if c.type == "identifier"] + if len(idents) >= 2: + _scope_add(scope_map, _text(idents[1], src).strip(), + _text(idents[0], src).strip()) + return + + # ── for_statement: initializer may carry a variable_declaration. + if nt == "for_statement": + explicit: list = [] + var_inferred: list = [] + for c in scope_node.children: + if c.type == "variable_declaration": + _add_variable_decl(c, src, scope_map, explicit, var_inferred) + for vn, ttxt in explicit: + _scope_add(scope_map, _text(vn, src).strip(), ttxt) + for vn, expr in var_inferred: + inferred = _infer_var_type(expr, src, scope_map) + if inferred: + _scope_add(scope_map, _text(vn, src).strip(), inferred) + return + + # ── foreach_statement: declares ``left: type``. + if nt == "foreach_statement": + tn = scope_node.child_by_field_name("type") + nm = scope_node.child_by_field_name("left") + if tn is None or nm is None: + return + if tn.type != "implicit_type": + _scope_add(scope_map, _text(nm, src).strip(), _text(tn, src).strip()) + return + # ``foreach (var x in coll)`` — derive x's type from coll's + # collection element type. We need to look up ``coll`` across the + # enclosing scopes (it's commonly a field or method param, not a + # local in the foreach itself), which means consulting the partial + # scope state and the file map. + coll = scope_node.child_by_field_name("right") + if coll is None or coll.type != "identifier": + return + coll_name = _text(coll, src).strip() + coll_type = _walk_partial_scopes(coll_name, scope_node, scope_maps, file_map) + if not isinstance(coll_type, str): + return + elem = _collection_element_type(coll_type) + if elem: + _scope_add(scope_map, _text(nm, src).strip(), elem) + return + + # ── using_statement: ``using (var x = ...)`` or ``using (T x = ...)``. + if nt == "using_statement": + explicit = [] + var_inferred = [] + for c in scope_node.children: + if c.type == "variable_declaration": + _add_variable_decl(c, src, scope_map, explicit, var_inferred) + for vn, ttxt in explicit: + _scope_add(scope_map, _text(vn, src).strip(), ttxt) + for vn, expr in var_inferred: + inferred = _infer_var_type(expr, src, scope_map) + if inferred: + _scope_add(scope_map, _text(vn, src).strip(), inferred) + return + + # ── block: walk direct children, stopping at nested scope nodes. Picks + # up local_declaration_statement (variable_declaration), declaration + # patterns inside expressions, and out-var declarations. + explicit = [] + var_inferred = [] + pattern_nodes: list = [] + decl_expr_nodes: list = [] + + stack = list(scope_node.children) + while stack: + n = stack.pop() + sub_nt = n.type + if sub_nt in _SCOPE_NODES: + continue # nested scope owns its declarations + if sub_nt == "variable_declaration": + _add_variable_decl(n, src, scope_map, explicit, var_inferred) + elif sub_nt in ("declaration_pattern", "recursive_pattern"): + pattern_nodes.append(n) + elif sub_nt == "declaration_expression": + decl_expr_nodes.append(n) + stack.extend(n.children) + + for vn, ttxt in explicit: + _scope_add(scope_map, _text(vn, src).strip(), ttxt) + + for n in pattern_nodes: + tn = n.child_by_field_name("type") + nm = n.child_by_field_name("name") + if tn is not None and nm is not None: + _scope_add(scope_map, _text(nm, src).strip(), _text(tn, src).strip()) + + for n in decl_expr_nodes: + tn = n.child_by_field_name("type") + nm = n.child_by_field_name("name") + if tn is not None and nm is not None and tn.type != "implicit_type": + _scope_add(scope_map, _text(nm, src).strip(), _text(tn, src).strip()) + + for vn, expr in var_inferred: + inferred = _infer_var_type(expr, src, scope_map) + if inferred: + _scope_add(scope_map, _text(vn, src).strip(), inferred) + + +def _absorb_pattern_bindings(node, src, scope_map: dict) -> None: + """Collect declaration-pattern and out-var bindings inside ``node``. + + Used for lambdas with an expression body (`x => x is T t ? t.M() : null`). + Walks past nested scope nodes so nested lambdas don't leak. + """ + stack = [node] + while stack: + n = stack.pop() + nt = n.type + if n is not node and nt in _SCOPE_NODES: + continue + if nt in ("declaration_pattern", "recursive_pattern", "declaration_expression"): + tn = n.child_by_field_name("type") + nm = n.child_by_field_name("name") + if tn is not None and nm is not None and tn.type != "implicit_type": + _scope_add(scope_map, _text(nm, src).strip(), + _text(tn, src).strip()) + stack.extend(n.children) + + +def _walk_partial_scopes(name: str, start_node, scope_maps: dict, + file_map: dict): + """Walk up the parent chain looking for ``name`` in the partial state. + + Called from inside scope construction — ``scope_maps`` only contains + scopes built so far. The DFS-preorder iteration in + ``_build_var_type_map`` guarantees that every *enclosing* scope is + already built by the time we look at an inner one, so this works for + cross-scope name resolution at construction time (e.g. resolving a + field/property/outer-method-param from inside a foreach). + """ + if not name: + return None + p = start_node + while p is not None: + sm = scope_maps.get(p.id) + if sm is not None and name in sm: + return sm[name] + p = p.parent + return file_map.get(name) + + +def _collection_element_type(type_txt: str) -> str: + """Best-guess element type for a collection type string. + + Handles ``T[]`` (most precise), and PascalCase-generic forms with one + type arg (``List``, ``IEnumerable``, ``HashSet``, …). For + multi-arg generics (``Dictionary``, ``KeyValuePair``) the + element type isn't a single name, so we return "" — callers fall back + to leaving the iterator unresolved rather than guessing wrong. + """ + t = type_txt.strip() + if not t: + return "" + if t.endswith("[]"): + return t[:-2].strip() + lt = t.find("<") + if lt > 0 and t.endswith(">"): + inner = t[lt + 1:-1].strip() + if inner and "," not in inner: + return inner + return "" + + +def _build_var_type_map(tree, src) -> _VarTypeMap: + """Build a block-scoped variable-name → resolved-type map for the file. + + File scope holds fields/properties/events declared at the type level; + every block/catch/loop/method gets its own per-scope map layered on + top. Conflicting types within one scope are sentinelled to ``None``: + the resolver still reports the name as "known but ambiguous" so the + caller can choose not to emit a qualified form. + """ + file_map: dict[str, str | None] = {} + + # Fields and properties at the type level — visible inside every method + # of the enclosing type, so they belong to file scope for resolution. + for node in _find_all(tree.root_node, lambda n: n.type in ( + "field_declaration", "event_field_declaration")): + var_decl = next((c for c in node.children if c.type == "variable_declaration"), None) + if not var_decl: + continue + tn = var_decl.child_by_field_name("type") + if not tn: + continue + ttxt = _text(tn, src).strip() + for decl in _find_all(var_decl, lambda x: x.type == "variable_declarator"): + vn = decl.child_by_field_name("name") + if vn: + _scope_add(file_map, _text(vn, src).strip(), ttxt) + + for node in _find_all(tree.root_node, lambda n: n.type == "property_declaration"): + tn = node.child_by_field_name("type") + nm = node.child_by_field_name("name") + if tn and nm: + _scope_add(file_map, _text(nm, src).strip(), _text(tn, src).strip()) + + # Per-scope maps for every block/method-like/lambda/loop/catch. We rely + # on ``_find_all``'s DFS-preorder order: an outer scope is processed + # before any inner scope it contains, so when a foreach (or other + # cross-scope inference) walks up to resolve a name, the parent + # scope's map is already populated. + scope_maps: dict[int, dict[str, str | None]] = {} + for node in _find_all(tree.root_node, lambda n: n.type in _SCOPE_NODES): + m: dict[str, str | None] = {} + _collect_scope_locals(node, src, m, scope_maps, file_map) + scope_maps[node.id] = m + + return _VarTypeMap(scope_maps, file_map) + + # ── Shared traversal helpers ─────────────────────────────────────────────────── def _iter_single_field_locals(tree, src, type_name, node_types, name_field, *, @@ -475,7 +1064,8 @@ def _q_classes_data(src, idx: TreeIndex) -> list: name = _text(name_node, src).strip() bases = _base_type_names(node, src) results.append(ClassInfo(line=_line(node), end_line=_end_line(node), - name=name, kind=kind, bases=bases)) + name=name, kind=kind, bases=bases, + visibility=_cs_type_visibility(node))) return results @@ -486,6 +1076,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: ln = _line(node) end = _end_line(node) toks = _sig_tokens(node, src) + vis = _cs_member_visibility(node) if node.type == "field_declaration": type_txt = _field_type(node, src) for var in _find_all(node, lambda n: n.type == "variable_declarator"): @@ -494,7 +1085,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: name = _text(vn, src).strip() results.append(MethodInfo(line=ln, end_line=end, name=name, kind="field", sig=f"{type_txt} {name}".strip(), - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) elif node.type == "property_declaration": type_node = node.child_by_field_name("type") name_node = node.child_by_field_name("name") @@ -503,7 +1094,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: name = _text(name_node, src).strip() results.append(MethodInfo(line=ln, end_line=end, name=name, kind="prop", sig=f"{type_txt} {name}".strip(), - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) elif node.type == "event_declaration": type_node = node.child_by_field_name("type") name_node = node.child_by_field_name("name") @@ -512,7 +1103,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: name = _text(name_node, src).strip() results.append(MethodInfo(line=ln, end_line=end, name=name, kind="event", sig=f"{type_txt} {name}".strip(), - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) elif node.type == "event_field_declaration": type_txt = _field_type(node, src) for var in _find_all(node, lambda n: n.type == "variable_declarator"): @@ -521,7 +1112,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: name = _text(vn, src).strip() results.append(MethodInfo(line=ln, end_line=end, name=name, kind="event", sig=f"{type_txt} {name}".strip(), - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) elif node.type in ("method_declaration", "local_function_statement"): sig = _build_sig(node, src) if sig: @@ -539,7 +1130,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: results.append(MethodInfo(line=ln, end_line=end, name=name, kind="method", sig=sig, return_type=ret_txt, param_types=param_types, - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) elif node.type == "constructor_declaration": sig = _build_sig(node, src) if sig: @@ -554,7 +1145,7 @@ def _q_methods_data(src, idx: TreeIndex) -> list: param_types.append(_text(pt, src).strip()) results.append(MethodInfo(line=ln, end_line=end, name=name, kind="ctor", sig=sig, param_types=param_types, - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) return results @@ -565,6 +1156,7 @@ def _q_fields_data(src, idx: TreeIndex) -> list: ln = _line(node) end = _end_line(node) toks = _sig_tokens(node, src) + vis = _cs_member_visibility(node) if node.type == "field_declaration": type_txt = _field_type(node, src) for var in _find_all(node, lambda n: n.type == "variable_declarator"): @@ -573,7 +1165,7 @@ def _q_fields_data(src, idx: TreeIndex) -> list: name = _text(vn, src).strip() results.append(FieldInfo(line=ln, end_line=end, name=name, kind="field", field_type=type_txt, - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) else: type_node = node.child_by_field_name("type") type_txt = _text(type_node, src).strip() if type_node else "" @@ -582,7 +1174,7 @@ def _q_fields_data(src, idx: TreeIndex) -> list: name = _text(name_node, src).strip() results.append(FieldInfo(line=ln, end_line=end, name=name, kind="prop", field_type=type_txt, - sig_tokens=toks)) + sig_tokens=toks, visibility=vis)) return results @@ -621,8 +1213,17 @@ def _q_attrs_data(src, idx: TreeIndex, attr_name=None) -> list: return results -def _q_all_call_site_infos(src, idx: TreeIndex) -> list: - """Extract call sites as CallSiteInfo objects, capturing PascalCase receivers.""" +def _q_all_call_site_infos(src, idx: TreeIndex, var_map: _VarTypeMap | None = None) -> list: + """Extract call sites as ``CallSiteInfo`` objects. + + Captures the literal identifier receiver when one is present (``Foo`` in + ``Foo.Bar()`` or ``repo`` in ``repo.Save()``) and, when ``var_map`` is + provided, resolves that receiver to its declared/inferred type so the + indexer can emit a stable ``Type.Method`` token. Receivers whose name + isn't in scope or maps to conflicting types in the same scope leave + ``resolved_type`` empty — the bare-name + literal-receiver fallback in + the indexer keeps the call discoverable either way. + """ result = [] for node in idx.of("invocation_expression"): fn_node = node.child_by_field_name("function") @@ -632,7 +1233,16 @@ def _q_all_call_site_infos(src, idx: TreeIndex) -> list: expr = fn_node.child_by_field_name("expression") if nn: receiver = _text(expr, src).strip() if (expr and expr.type == "identifier") else "" - result.append(CallSiteInfo(name=_text(nn, src).strip(), receiver=receiver)) + resolved = "" + if receiver and var_map is not None: + rt = var_map.resolve_at(receiver, node) + if rt: + resolved = _strip_generic(rt.rsplit(".", 1)[-1]) + result.append(CallSiteInfo( + name=_text(nn, src).strip(), + receiver=receiver, + resolved_type=resolved, + )) elif fn_node.type == "identifier": result.append(CallSiteInfo(name=_text(fn_node, src).strip())) for name in _collect_ctor_names(idx, src): @@ -681,19 +1291,53 @@ def _q_all_local_types_data(src, idx: TreeIndex) -> list: # ── Query functions ──────────────────────────────────────────────────────────── -def q_classes(src, tree, lines): +def _parse_visibility_filter(visibility): + """Normalise the optional visibility filter into a set of canonical + tokens (or None when no filter is requested). Accepts a single value + (``"public"``), a comma-separated string (``"public,internal"``), or + any iterable. Unknown tokens are dropped silently — callers shouldn't + crash on a typo, just get back nothing matching their typo.""" + if not visibility: + return None + if isinstance(visibility, str): + toks = [t.strip().lower() for t in visibility.split(",")] + else: + toks = [str(t).strip().lower() for t in visibility] + keep = {t for t in toks if t} + return keep or None + + +def _visibility_keep(info_visibility: str, allowed) -> bool: + """True when ``info_visibility`` passes the optional filter. + + Empty string on the info means "language didn't capture a visibility" + and is treated as a hard miss — searching ``visibility="public"`` over + e.g. SQL files never matches, which is the right answer. + """ + if allowed is None: + return True + return bool(info_visibility) and info_visibility in allowed + + +def q_classes(src, tree, lines, visibility=None): + allowed = _parse_visibility_filter(visibility) return [(_r.line, _r.end_line, _r.text) - for _r in _q_classes_data(src, _CsIndex(src, tree, _TYPE_DECL_NODES))] + for _r in _q_classes_data(src, _CsIndex(src, tree, _TYPE_DECL_NODES)) + if _visibility_keep(getattr(_r, "visibility", ""), allowed)] -def q_methods(src, tree, lines): +def q_methods(src, tree, lines, visibility=None): + allowed = _parse_visibility_filter(visibility) return [(_r.line, _r.end_line, _r.text) - for _r in _q_methods_data(src, _CsIndex(src, tree, _MEMBER_DECL_NODES))] + for _r in _q_methods_data(src, _CsIndex(src, tree, _MEMBER_DECL_NODES)) + if _visibility_keep(getattr(_r, "visibility", ""), allowed)] -def q_fields(src, tree, lines): +def q_fields(src, tree, lines, visibility=None): + allowed = _parse_visibility_filter(visibility) return [(_r.line, _r.end_line, _r.text) - for _r in _q_fields_data(src, _CsIndex(src, tree, {"field_declaration", "property_declaration"}))] + for _r in _q_fields_data(src, _CsIndex(src, tree, {"field_declaration", "property_declaration"})) + if _visibility_keep(getattr(_r, "visibility", ""), allowed)] def q_calls(src, tree, lines, method_name): @@ -702,6 +1346,50 @@ def q_calls(src, tree, lines, method_name): else: qualifier, bare_name = None, method_name + # Build the var-type map lazily — only when a qualified pattern is + # supplied. Bare-method searches don't need receiver resolution. + var_map = _build_var_type_map(tree, src) if qualifier else None + + _qualifier_str: str = qualifier or "" + + def _qualifier_matches(expr_node) -> bool: + """True if ``expr_node`` (the receiver of a member access) matches + ``qualifier`` either by literal text or by resolved type.""" + if expr_node is None or not _qualifier_str: + return False + expr_txt = _text(expr_node, src).strip() + if expr_txt == _qualifier_str or expr_txt.endswith("." + _qualifier_str): + return True + # Resolved-type fallback: look up an identifier receiver in the + # method-scoped var-type map and compare its unqualified type name. + if expr_node.type == "identifier" and var_map is not None: + rt = var_map.resolve_at(expr_txt, expr_node) + if isinstance(rt, str) and rt: + resolved = _strip_generic(rt.rsplit(".", 1)[-1]) + if resolved == _qualifier_str: + return True + return False + + def _report(node, name_node): + """Build a result tuple pinpointing the method-name token. + + Reports the line of ``name_node`` (where the matched identifier + actually appears) rather than the start of the surrounding + invocation — for chained calls like ``a.B().Method(...)`` that + means the result lands on the line of ``Method``, not on + whichever line the chain begins. Source text is the single line + containing the name, not the multi-line invocation node. + """ + anchor = name_node if name_node is not None else node + row = anchor.start_point[0] + line_text = lines[row].strip() if 0 <= row < len(lines) else "" + if not line_text: + # Fall back to a truncated render of the node when the source + # row is empty/missing (defensive — shouldn't happen for real + # call sites). + line_text = _truncate_raw(node, src) + return (_line(anchor), line_text) + results = [] for node in _find_all(tree.root_node, lambda n: n.type == "invocation_expression"): if _in_literal(node): @@ -710,30 +1398,39 @@ def q_calls(src, tree, lines, method_name): if not fn: continue matched = None + match_name_node = None if fn.type == "member_access_expression": nn = fn.child_by_field_name("name") expr = fn.child_by_field_name("expression") if nn: matched = _strip_generic(_text(nn, src)) + match_name_node = nn if qualifier and matched == bare_name: - expr_txt = _text(expr, src).strip() if expr else "" - if not (expr_txt == qualifier or expr_txt.endswith("." + qualifier)): + if not _qualifier_matches(expr): matched = None elif fn.type == "conditional_access_expression": - # f?.Method(...) — method name is in the trailing member_binding_expression + # f?.Method(...) — method name is in the trailing + # member_binding_expression; receiver is the ``condition`` + # field on the conditional_access_expression. binding = next((c for c in fn.children if c.type == "member_binding_expression"), None) if binding: nn = binding.child_by_field_name("name") if nn: matched = _strip_generic(_text(nn, src)) + match_name_node = nn + if qualifier and matched == bare_name: + cond = fn.child_by_field_name("condition") + if not _qualifier_matches(cond): + matched = None elif fn.type in ("identifier", "generic_name"): if qualifier is None: nn = fn.child_by_field_name("name") if fn.type == "generic_name" else fn if nn: matched = _strip_generic(_text(nn, src)) + match_name_node = nn if matched == bare_name: - results.append((_line(node), _truncate_raw(node, src))) + results.append(_report(node, match_name_node)) if qualifier is None: for node in _find_all(tree.root_node, lambda n: n.type == "object_creation_expression"): @@ -746,7 +1443,8 @@ def q_calls(src, tree, lines, method_name): if not idents: continue if _strip_generic(_text(idents[-1], src)) == bare_name: - results.append((_line(node), _truncate_raw(node, src))) + # ``new T(...)`` — anchor at the type name token. + results.append(_report(node, idents[-1])) return results @@ -887,14 +1585,26 @@ def q_usings(src, tree, lines): return [(_r.line, _r.text) for _r in _q_usings_data(src, _CsIndex(src, tree, {"using_directive"}))] -def q_declarations(src, tree, lines, name, include_body=False, symbol_kind=None): +def q_declarations(src, tree, lines, name, include_body=False, symbol_kind=None, + visibility=None): kind_nodes = SYMBOL_KIND_TO_NODES.get((symbol_kind or "").lower().strip()) target_nodes = kind_nodes if kind_nodes is not None else (_TYPE_DECL_NODES | _MEMBER_DECL_NODES) + allowed = _parse_visibility_filter(visibility) results = [] for node in _find_all(tree.root_node, lambda n: n.type in target_nodes): name_node = node.child_by_field_name("name") if not name_node or _text(name_node, src).strip() != name: continue + if allowed is not None: + # Use the same defaults as the indexer so the AST stage stays + # consistent with the index pre-filter — type kinds get the + # type-level default, members get the member-level default. + if node.type in _TYPE_DECL_NODES: + vis = _cs_type_visibility(node) + else: + vis = _cs_member_visibility(node) + if not _visibility_keep(vis, allowed): + continue kind = _node_kind(node) start_row = node.start_point[0] end_row = node.end_point[0] @@ -904,7 +1614,7 @@ def q_declarations(src, tree, lines, name, include_body=False, symbol_kind=None) content = "\n".join(lines[start_row:sig_end_row]).rstrip() else: content = "\n".join(lines[start_row:end_row + 1]) - header = f"── [{kind}] {name} (lines {start_row + 1}–{end_row + 1}) ──" + header = f"[{kind}] {name} {start_row + 1}-{end_row + 1}:" results.append((_line(node), f"{header}\n{content}")) return results @@ -916,6 +1626,28 @@ def q_declarations(src, tree, lines, name, include_body=False, symbol_kind=None) ) +def _field_declarator_name(field_node, target_row: int, target_col: int): + """Find the ``variable_declarator``'s name inside a field_declaration or + event_field_declaration. When the field declares multiple variables on + one line (``int a, b, c``), pick the declarator whose range contains + the target; fall back to the first when none does (e.g. cursor sits on + a modifier keyword like ``readonly``).""" + var_decl = next((c for c in field_node.children + if c.type == "variable_declaration"), None) + if var_decl is None: + return None + declarators = [c for c in var_decl.children + if c.type == "variable_declarator"] + if not declarators: + return None + for d in declarators: + sr, sc = d.start_point + er, ec = d.end_point + if (sr, sc) <= (target_row, target_col) < (er, ec): + return d.child_by_field_name("name") + return declarators[0].child_by_field_name("name") + + def q_at(src, tree, lines, position: str): """Identify the symbol and enclosing scope chain at ``line:col``. @@ -979,6 +1711,13 @@ def _contains(n) -> bool: while walker is not None: if walker.type in _SCOPE_NODE_NAMES: name_node = walker.child_by_field_name("name") + # ``field_declaration`` / ``event_field_declaration`` carry their + # name inside a nested ``variable_declaration`` → ``variable_declarator`` + # rather than on a direct ``name`` field. Pick the declarator + # whose range contains the target, else the first declarator. + if name_node is None and walker.type in ( + "field_declaration", "event_field_declaration"): + name_node = _field_declarator_name(walker, target_row, target_col) if name_node: scopes.append({ "kind": _node_kind(walker), @@ -1191,112 +1930,12 @@ def q_casts(src, tree, lines, type_name): return results -def _add_typed_vars(tree, src, var_names, node_types, type_name, *, - name_field="name", skip_implicit=False): - """ - Walk nodes of the given types; for each, check that the 'type' field matches - `type_name`, then add the variable name to `var_names`. - If `skip_implicit` is True, nodes with an implicit_type (var) are skipped. - """ - for _, _, var_txt in _iter_single_field_locals( - tree, src, type_name, node_types, name_field, skip_implicit=skip_implicit): - var_names.add(var_txt) - - -def _get_init_expr(declarator): - children = declarator.children - if len(children) >= 3 and children[1].type == "=": - return children[2] - return None - - -def _collect_typed_var_names(tree, src, type_name): - """ - Return (var_names, array_names) for q_accesses_on. - - var_names: names of variables whose declared or inferred type is `type_name`. - array_names: names of variables whose type is an array of `type_name` elements. - - Covers explicitly-typed declarations, parameters, properties, foreach loops, - declaration patterns, out variables, and var-inferred declarations initialised - with object_creation_expression, array_creation_expression, cast_expression, or - as_expression. - """ - var_names = set() - array_names = set() - - # Explicitly typed variable declarations (locals and fields) - for node in _find_all(tree.root_node, lambda n: n.type == "variable_declaration"): - type_node = node.child_by_field_name("type") - if not type_node or type_name not in _type_names(_text(type_node, src).strip()): - continue - for decl in _find_all(node, lambda n: n.type == "variable_declarator"): - vn = decl.child_by_field_name("name") - if vn: - var_names.add(_text(vn, src).strip()) - - # Parameters, property declarations, foreach, declaration patterns, out vars - _add_typed_vars(tree, src, var_names, {"parameter", "property_declaration"}, type_name) - _add_typed_vars(tree, src, var_names, {"foreach_statement"}, type_name, - name_field="left", skip_implicit=True) - _add_typed_vars(tree, src, var_names, - {"declaration_pattern", "declaration_expression", "recursive_pattern"}, - type_name) - - # var-inferred declarations whose initialiser reveals the type - for node in _find_all(tree.root_node, lambda n: n.type == "variable_declaration"): - type_node = node.child_by_field_name("type") - if not type_node or _text(type_node, src).strip() != "var": - continue - for decl in _find_all(node, lambda n: n.type == "variable_declarator"): - vn = decl.child_by_field_name("name") - if not vn: - continue - expr = _get_init_expr(decl) - if not expr: - continue - name = _text(vn, src).strip() - if expr.type == "object_creation_expression": - t = expr.child_by_field_name("type") - if t and type_name in _type_names(_text(t, src)): - var_names.add(name) - elif expr.type == "array_creation_expression": - t = expr.child_by_field_name("type") - if t: - elem = t.child_by_field_name("type") if t.type == "array_type" else t - if type_name in _type_names(_text(elem, src)): - array_names.add(name) - elif expr.type == "cast_expression": - t = expr.child_by_field_name("type") - if t and type_name in _type_names(_text(t, src)): - var_names.add(name) - elif expr.type == "as_expression": - t = expr.child_by_field_name("right") or expr.child_by_field_name("type") - if t and type_name in _type_names(_text(t, src)): - var_names.add(name) - - # var-inferred element-access variables: var x = arr[i] where arr is T[] - if array_names: - for node in _find_all(tree.root_node, lambda n: n.type == "variable_declaration"): - type_node = node.child_by_field_name("type") - if not type_node or _text(type_node, src).strip() != "var": - continue - for decl in _find_all(node, lambda n: n.type == "variable_declarator"): - vn = decl.child_by_field_name("name") - if not vn: - continue - expr = _get_init_expr(decl) - if not expr or expr.type != "element_access_expression": - continue - obj = expr.child_by_field_name("expression") - if obj and obj.type == "identifier" and _text(obj, src).strip() in array_names: - var_names.add(_text(vn, src).strip()) - - return var_names, array_names - - def q_accesses_on(src, tree, lines, type_name): - var_names, _ = _collect_typed_var_names(tree, src, type_name) + var_map = _build_var_type_map(tree, src) + + def _matches(name: str, node) -> bool: + t = var_map.resolve_at(name, node) + return bool(t) and type_name in _type_names(t) results = [] seen_rows = set() @@ -1319,7 +1958,7 @@ def _emit(node, member_name): continue if obj.type != "identifier": continue - if _text(obj, src).strip() not in var_names: + if not _matches(_text(obj, src).strip(), node): continue _emit(node, _text(member, src).strip()) @@ -1330,7 +1969,7 @@ def _emit(node, member_name): cond = node.child_by_field_name("condition") if not cond or cond.type != "identifier": continue - if _text(cond, src).strip() not in var_names: + if not _matches(_text(cond, src).strip(), node): continue binding = next((c for c in node.children if c.type == "member_binding_expression"), None) @@ -1352,7 +1991,7 @@ def _emit(node, member_name): # With-expression member mutations (C# 9 records) — obj with { Prop = val } # Each member is emitted independently for the same reason as above. for wi, src_ident, prop in _iter_with_members(tree, src): - if _text(src_ident, src).strip() not in var_names: + if not _matches(_text(src_ident, src).strip(), wi): continue row = wi.start_point[0] line_text = lines[row].strip() if row < len(lines) else "" @@ -1379,10 +2018,54 @@ def q_all_refs(src, tree, lines, name): return results +def q_var_type(src, tree, lines, name): + """Report the resolved type of every occurrence of ``name`` in the file. + + For each identifier-position where ``name`` appears, run the method- + scoped var-type resolver and emit: + + L: name : when the resolver returns a type + L: name : (unresolved) when the resolver returns None + L: name : (conflicting) when the name is known but its + scope had conflicting declarations + + Identical (line, resolved) pairs are deduped so a name used multiple + times on the same line reports once. Identifiers inside string + literals or comments are skipped — matches mirror ``all_refs``. + """ + var_map = _build_var_type_map(tree, src) + results = [] + seen = set() + for node in _find_all(tree.root_node, lambda n: n.type == "identifier"): + if _text(node, src) != name: + continue + if _in_literal(node): + continue + resolved = var_map.resolve_at(name, node) + if resolved is None: + # Distinguish "name known but ambiguous" (sentinel set during + # construction) from "name never declared in scope". The map's + # resolve_at returns None for both, so re-check membership. + in_any = any( + name in m for m in var_map._scope_maps.values() # noqa: SLF001 + ) or name in var_map._file_map # noqa: SLF001 + label = "(conflicting)" if in_any else "(unresolved)" + else: + label = resolved + row = node.start_point[0] + key = (row, label) + if key in seen: + continue + seen.add(key) + line_text = lines[row].strip() if row < len(lines) else "" + results.append((_line(node), f"{name} : {label} ← {line_text}")) + return results + + # ── Process function ────────────────────────────────────────────────────────── def query_cs_bytes(src_bytes: bytes, mode: str, mode_arg: str, include_body=False, - symbol_kind=None, uses_kind=None, **kwargs): + symbol_kind=None, uses_kind=None, visibility=None, **kwargs): """Parse C# bytes and return list[{"line": N, "text": "..."}] for the given mode.""" src_bytes = _strip_else_branches(src_bytes) try: @@ -1394,9 +2077,9 @@ def query_cs_bytes(src_bytes: bytes, mode: str, mode_arg: str, include_body=Fals lines = src_bytes.decode("utf-8", errors="replace").splitlines() dispatch = { - "classes": lambda: q_classes(src_bytes, tree, lines), - "methods": lambda: q_methods(src_bytes, tree, lines), - "fields": lambda: q_fields(src_bytes, tree, lines), + "classes": lambda: q_classes(src_bytes, tree, lines, visibility=visibility), + "methods": lambda: q_methods(src_bytes, tree, lines, visibility=visibility), + "fields": lambda: q_fields(src_bytes, tree, lines, visibility=visibility), "calls": lambda: q_calls(src_bytes, tree, lines, mode_arg), "implements": lambda: q_implements(src_bytes, tree, lines, mode_arg), "uses": lambda: q_uses(src_bytes, tree, lines, mode_arg, uses_kind=uses_kind), @@ -1407,10 +2090,12 @@ def query_cs_bytes(src_bytes: bytes, mode: str, mode_arg: str, include_body=Fals "accesses_of": lambda: q_accesses_of(src_bytes, tree, lines, mode_arg), "imports": lambda: q_usings(src_bytes, tree, lines), "declarations": lambda: q_declarations(src_bytes, tree, lines, mode_arg, - include_body=include_body, symbol_kind=symbol_kind), + include_body=include_body, symbol_kind=symbol_kind, + visibility=visibility), "body": lambda: q_body(src_bytes, tree, lines, mode_arg, symbol_kind=symbol_kind), "at": lambda: q_at(src_bytes, tree, lines, mode_arg), "params": lambda: q_params(src_bytes, tree, lines, mode_arg), + "var_type": lambda: q_var_type(src_bytes, tree, lines, mode_arg), } return _run_dispatch(mode, "C#", dispatch) @@ -1428,6 +2113,11 @@ def describe_cs_file(src_bytes: bytes, ext: str = "") -> FileDescription: # extractor needs and collects literal-aware all_refs in the same pass. idx = _CsIndex(src_bytes, tree, _DESCRIBE_NODE_TYPES, collect_refs=True) + # Var-type map drives qualified-call resolution at index time. Built once + # per file; consulted by the call-site emitter to attach a stable + # ``Type.Method`` token to every receiver it can pin down syntactically. + var_map = _build_var_type_map(tree, src_bytes) + return FileDescription( language="cs", classes=_q_classes_data(src_bytes, idx), @@ -1436,7 +2126,7 @@ def describe_cs_file(src_bytes: bytes, ext: str = "") -> FileDescription: imports=_q_usings_data(src_bytes, idx), attrs=_q_attrs_data(src_bytes, idx), namespace=_q_namespace(src_bytes, idx), - call_site_infos=_q_all_call_site_infos(src_bytes, idx), + call_site_infos=_q_all_call_site_infos(src_bytes, idx, var_map), cast_infos=[CastInfo(target_type=t) for t in _q_all_cast_types_data(src_bytes, idx)], local_var_infos=[LocalVarInfo(var_type=t) for t in _q_all_local_types_data(src_bytes, idx)], member_access_infos=[MemberAccessInfo(member=m) for m in _q_all_member_accesses_data(src_bytes, idx)], diff --git a/query/dispatch.py b/query/dispatch.py index 64cdd40..bbfabd5 100644 --- a/query/dispatch.py +++ b/query/dispatch.py @@ -70,7 +70,8 @@ def _fn(src_bytes, mode, mode_arg, **kwargs): def query_file(src_bytes: bytes, ext: str, mode: str, mode_arg: str = "", - include_body=False, symbol_kind=None, uses_kind=None, **kwargs): + include_body=False, symbol_kind=None, uses_kind=None, + visibility=None, **kwargs): """Query src_bytes using the given mode. Returns ``list[{"line": N, "text": "..."}]`` on success. @@ -79,6 +80,10 @@ def query_file(src_bytes: bytes, ext: str, mode: str, mode_arg: str = "", the mode isn't supported for that language — explicit errors beat silent empties for tool-using agents. Use ``mode='capabilities'`` to ask which modes a given file supports. + + ``visibility`` is an optional comma-separated filter for declaration + modes (``classes``/``methods``/``fields``/``declarations``). Languages + that don't capture visibility silently ignore it. """ fn = _EXT_TO_QUERY_BYTES.get(ext) if fn is None: @@ -87,7 +92,8 @@ def query_file(src_bytes: bytes, ext: str, mode: str, mode_arg: str = "", f"Supported extensions: {', '.join(sorted(ALL_EXTS))}" ) return fn(src_bytes, mode, mode_arg, - include_body=include_body, symbol_kind=symbol_kind, uses_kind=uses_kind, **kwargs) + include_body=include_body, symbol_kind=symbol_kind, + uses_kind=uses_kind, visibility=visibility, **kwargs) def describe_file(src_bytes: bytes, ext: str) -> FileDescription: diff --git a/query/js.py b/query/js.py index 1bb73e1..c438a70 100644 --- a/query/js.py +++ b/query/js.py @@ -319,7 +319,7 @@ def js_q_declarations(src, tree, lines, name, include_body=False): content = "\n".join(lines[start_row:sig_end]).rstrip() else: content = "\n".join(lines[start_row:end_row + 1]) - header = f"── [{kind}] {name} (lines {start_row + 1}–{end_row + 1}) ──" + header = f"[{kind}] {name} {start_row + 1}-{end_row + 1}:" results.append((_line(node), f"{header}\n{content}")) return results diff --git a/query/py.py b/query/py.py index 852bdc9..9821b2b 100644 --- a/query/py.py +++ b/query/py.py @@ -263,7 +263,7 @@ def py_q_declarations(src, tree, lines, name, include_body=False, symbol_kind=No start_row = node.start_point[0] end_row = node.end_point[0] body_lines = "\n".join(lines[start_row:end_row + 1]) - header = f"── [{kind}] {name} (lines {start_row + 1}–{end_row + 1}) ──" + header = f"[{kind}] {name} {start_row + 1}-{end_row + 1}:" results.append((_line(node), f"{header}\n{body_lines}")) return results diff --git a/query/rust.py b/query/rust.py index 50aec49..b8cd80c 100644 --- a/query/rust.py +++ b/query/rust.py @@ -314,7 +314,7 @@ def rust_q_declarations(src, tree, lines, name, include_body=False): else: content = "\n".join(lines[start_row:end_row + 1]) - header = f"── [{kind}] {name} (lines {start_row + 1}–{end_row + 1}) ──" + header = f"[{kind}] {name} {start_row + 1}-{end_row + 1}:" results.append((_line(node), f"{header}\n{content}")) return results diff --git a/query/tests/test_cs_at_and_body.py b/query/tests/test_cs_at_and_body.py index 4800cd6..edb836b 100644 --- a/query/tests/test_cs_at_and_body.py +++ b/query/tests/test_cs_at_and_body.py @@ -124,6 +124,58 @@ def test_at_namespace_outside_any_class(self): self.assertNotIn("[method]", text) +# ── q_at: field/event scope-chain regression ────────────────────────────────── + + +_FIELDS_SRC = """\ +namespace Acme { + public class Container { + private int _count = 0; + private static readonly System.Guid SettingsKS = System.Guid.NewGuid(); + private string _first, _second; + public event System.Action OnChange; + } +} +""" + + +class TestQAtInsideFieldDeclarations(unittest.TestCase): + """Regression: ``field_declaration`` and ``event_field_declaration`` + don't expose a direct ``name`` field — the name lives inside a nested + ``variable_declarator``. q_at must still report the field in the + enclosing-scope chain rather than silently skipping it.""" + + def setUp(self): + b = _FIELDS_SRC.encode() + tree = _PARSER.parse(b) + self.fx = (b, tree, _FIELDS_SRC.splitlines()) + + def test_at_inside_simple_field(self): + # Line 3: `private int _count = 0;` — cursor on `_count`. + text = q_at(*self.fx, "3:24")[0][-1] + self.assertIn("[field] _count", text) + self.assertIn("[class] Container", text) + + def test_at_on_field_modifier_keyword(self): + # Cursor on the `readonly` keyword still resolves to the field's + # scope — fall-back picks the first declarator's name. + text = q_at(*self.fx, "4:24")[0][-1] + self.assertIn("[field] SettingsKS", text) + + def test_at_inside_multi_declarator_field_picks_correct_one(self): + # `private string _first, _second;` — point at `_second`. + text = q_at(*self.fx, "5:32")[0][-1] + self.assertIn("[field] _second", text) + # And pointing at `_first` picks _first. + text = q_at(*self.fx, "5:24")[0][-1] + self.assertIn("[field] _first", text) + + def test_at_inside_event_field(self): + # `public event System.Action OnChange;` — cursor on `OnChange`. + text = q_at(*self.fx, "6:36")[0][-1] + self.assertIn("[event field] OnChange", text) + + # ── q_body ──────────────────────────────────────────────────────────────────── class TestQBody(unittest.TestCase): diff --git a/query/tests/test_cs_foreach_accesses.py b/query/tests/test_cs_foreach_accesses.py index 0da9c62..7128c90 100644 --- a/query/tests/test_cs_foreach_accesses.py +++ b/query/tests/test_cs_foreach_accesses.py @@ -107,12 +107,11 @@ def test_nested_foreach_outer_variable_tracked(self): assert nested_line in _lines(r), \ f"Line {nested_line} (nested foreach access) missing: {r}" - def test_var_foreach_not_tracked(self): + def test_var_foreach_resolves_through_collection(self): """ - foreach (var entry in items) — 'entry' type cannot be resolved - without type inference; var-inferred iteration variables are not tracked. - The var loop (ProcessVar) must NOT produce spurious results. - (entry.Name must be absent since it is on a different line from explicit hits.) + ``foreach (var entry in items)`` where ``items: List`` resolves + the iteration variable to ``Item`` via the collection's element type. + ``entry.Name`` should appear in accesses_on Item results. """ r = self._on("Item") src_lines = _SRC.splitlines() @@ -121,8 +120,8 @@ def test_var_foreach_not_tracked(self): None ) assert var_line is not None, "ForeachAccess.cs must have entry.Name line" - assert var_line not in _lines(r), \ - f"var-inferred foreach line {var_line} must not appear in Item accesses: {r}" + assert var_line in _lines(r), \ + f"var-inferred foreach line {var_line} should resolve via List element type: {r}" def test_unrelated_type_returns_empty(self): assert self._on("NoSuchType") == [] diff --git a/query/tests/test_cs_var_type_map.py b/query/tests/test_cs_var_type_map.py new file mode 100644 index 0000000..3a8b975 --- /dev/null +++ b/query/tests/test_cs_var_type_map.py @@ -0,0 +1,743 @@ +""" +Unit tests for ``_build_var_type_map`` — the method-scoped variable-name → +resolved-type resolver, and the qualified-call form it produces on each +CallSiteInfo. +""" +from __future__ import annotations + +import unittest + +from tests.base import _parse +from query.cs import ( + _build_var_type_map, + _find_all, + _q_all_call_site_infos, + _CsIndex, + _DESCRIBE_NODE_TYPES, + describe_cs_file, +) + + +def _node_at(tree, src_bytes, text: str, node_type: str | None = None): + """Return the first node whose source matches ``text``.""" + for n in _find_all(tree.root_node, lambda x: True): + if src_bytes[n.start_byte:n.end_byte].decode() == text: + if node_type is None or n.type == node_type: + return n + raise AssertionError(f"node {text!r} not found") + + +def _call_at(tree, src_bytes, contains: str): + """Return the first invocation_expression whose text contains ``contains``.""" + for n in _find_all(tree.root_node, lambda x: x.type == "invocation_expression"): + if contains in src_bytes[n.start_byte:n.end_byte].decode(): + return n + raise AssertionError(f"call containing {contains!r} not found") + + +def _vm(src: str): + b, tree, _ = _parse(src) + return b, tree, _build_var_type_map(tree, b) + + +# ── _VarTypeMap ────────────────────────────────────────────────────────────── + + +class TestParametersResolve(unittest.TestCase): + def test_method_param_resolved(self): + src = "class C { void M(Repo r) { r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + def test_constructor_param_resolved(self): + src = "class C { public C(Repo r) { r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + +class TestExplicitLocalsResolve(unittest.TestCase): + def test_explicit_typed_local(self): + src = "class C { void M() { Repo r = null; r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + def test_var_with_new(self): + src = "class C { void M() { var r = new Repo(); r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + def test_var_with_cast(self): + src = "class C { void M(object o) { var r = (Repo)o; r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + def test_var_with_as_expression(self): + src = "class C { void M(object o) { var r = o as Repo; r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + def test_var_unresolvable_method_call(self): + # var x = GetRepo() — we don't know GetRepo's return type + src = "class C { void M() { var r = GetRepo(); r.Save(); } }" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) is None + + def test_array_element_inferred(self): + src = """class C { + void M() { + Repo[] arr = new Repo[10]; + var r = arr[0]; + r.Save(); + } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + assert vm.resolve_at("r", call) == "Repo" + + +class TestInferenceHeuristics(unittest.TestCase): + """Best-guess inference for AI-agent use: await unwrap, generic method + type args, static factory pattern. False positives are acceptable — + a wrong qualified form never matches a real call in the AST stage.""" + + def test_await_unwraps_to_inner_expression(self): + src = """class C { + async void M() { + var x = await new Repo(); + x.Save(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("x", _call_at(tree, b, "x.Save")) == "Repo" + + def test_generic_method_first_type_arg(self): + # ``Resolve``, ``Get``, ``As`` style — first type arg is + # idiomatically the return type. + src = """class C { + void M(IContainer c) { + var s = c.Resolve(); + s.Run(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("s", _call_at(tree, b, "s.Run")) == "IService" + + def test_bare_generic_method_first_type_arg(self): + src = """class C { + void M() { + var s = Resolve(); + s.Run(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("s", _call_at(tree, b, "s.Run")) == "IService" + + def test_static_factory_assumes_receiver_type(self): + # ``Foo.Create()`` is overwhelmingly likely to return Foo (or a + # subclass). Emitting Foo as the type is the false-positive + # friendly default. + src = """class C { + void M() { + var w = Widget.Create(); + w.Render(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("w", _call_at(tree, b, "w.Render")) == "Widget" + + def test_factory_skipped_when_receiver_is_known_local_same_block(self): + # If the PascalCase receiver name is a declared local in the *same* + # block where var is being inferred, prefer the declaration. + src = """class C { + void M() { + Widget Widget = null; + var w = Widget.Spawn(); + w.Render(); + } + }""" + b, tree, vm = _vm(src) + # ``Widget`` is a declared local in the same block — the factory + # heuristic skips and ``w`` stays unresolved (we don't know what + # an arbitrary instance's ``.Spawn()`` returns). + assert vm.resolve_at("w", _call_at(tree, b, "w.Render")) is None + + def test_factory_fires_even_when_outer_scope_shadows(self): + # Inference runs at scope-construction time and doesn't walk outer + # scopes — a param named the same as a type still lets the factory + # heuristic fire on inner blocks. False positive is acceptable per + # the project's AI-agent-friendly bias (worst case: a qualified + # form that AST post-filter rejects). + src = """class C { + void M(Widget Widget) { + var w = Widget.Spawn(); + w.Render(); + } + }""" + b, tree, vm = _vm(src) + # Best-guess inference fires: ``w`` is treated as ``Widget``. + assert vm.resolve_at("w", _call_at(tree, b, "w.Render")) == "Widget" + + def test_await_static_factory_combo(self): + # The patterns compose: ``await TypeName.CreateAsync()`` should + # still resolve to TypeName. + src = """class C { + async void M() { + var ws = await Workspace.CreateAsync(); + ws.Save(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("ws", _call_at(tree, b, "ws.Save")) == "Workspace" + + def test_await_generic_method_combo(self): + src = """class C { + async void M(IContainer c) { + var s = await c.GetAsync(); + s.Run(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("s", _call_at(tree, b, "s.Run")) == "IService" + + +class TestForeachVarIteratorInference(unittest.TestCase): + """foreach (var x in coll) should derive x's type from coll's element + type when coll is in scope — fixes a false negative where the iterator + was left unresolved despite the collection type being statically known.""" + + def test_foreach_over_array_field(self): + src = """class C { + private Item[] _items; + void M() { + foreach (var it in _items) { + it.Use(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("it", _call_at(tree, b, "it.Use")) == "Item" + + def test_foreach_over_method_param_list(self): + src = """class C { + void M(System.Collections.Generic.List items) { + foreach (var it in items) { + it.Use(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("it", _call_at(tree, b, "it.Use")) == "Item" + + def test_foreach_over_ienumerable_param(self): + src = """class C { + void M(IEnumerable items) { + foreach (var it in items) { + it.Use(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("it", _call_at(tree, b, "it.Use")) == "Item" + + def test_foreach_over_local_array(self): + # Local var array — collection lookup walks the parent block scope. + src = """class C { + void M() { + Item[] items = null; + foreach (var it in items) { + it.Use(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("it", _call_at(tree, b, "it.Use")) == "Item" + + def test_foreach_over_dictionary_skipped(self): + # Dictionary has two type args — the iterator is KeyValuePair, + # which we can't summarise as a single PascalCase name. We leave it + # unresolved rather than emit a wrong guess. + src = """class C { + void M(System.Collections.Generic.Dictionary map) { + foreach (var kv in map) { + kv.Key.ToString(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("kv", _call_at(tree, b, "kv.Key.ToString")) is None + + def test_foreach_over_unknown_collection_unresolved(self): + # Collection name not in scope at all — no inference. + src = """class C { + void M() { + foreach (var x in unknownCollection) { + x.Use(); + } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("x", _call_at(tree, b, "x.Use")) is None + + +class TestPropertyAccessHeuristic(unittest.TestCase): + """``var x = obj.PascalProperty`` infers x's type as the property name + (.NET convention that typed sub-objects are named after their type).""" + + def test_property_access_uses_property_name(self): + src = """class C { + void M(Context ctx) { + var meta = ctx.RequestMetadata; + meta.Touch(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("meta", _call_at(tree, b, "meta.Touch")) == "RequestMetadata" + + def test_property_access_on_this(self): + src = """class C { + private Widget _widget; + void M() { + var w = this._widget; + w.Render(); + } + }""" + b, tree, vm = _vm(src) + # ``_widget`` starts with underscore (not Pascal), so the property + # heuristic skips and the resolver finds nothing for w. + # Documented limitation: agents searching for Widget.Render on this + # line miss the qualified form but find the bare ``Render`` call. + assert vm.resolve_at("w", _call_at(tree, b, "w.Render")) is None + + def test_static_property_uses_receiver_type(self): + # ``Encoding.UTF8`` — receiver is PascalCase and not in scope, so + # the static path fires: result type = receiver = Encoding. + src = """class C { + void M() { + var enc = Encoding.UTF8; + enc.GetBytes(\"x\"); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("enc", _call_at(tree, b, "enc.GetBytes")) == "Encoding" + + +class TestTernaryInference(unittest.TestCase): + """``var x = cond ? a : b`` falls back to inferring each branch.""" + + def test_ternary_both_object_creation(self): + src = """class C { + void M(bool b) { + var x = b ? new Foo() : new Bar(); + x.Do(); + } + }""" + b, tree, vm = _vm(src) + # First branch wins — both branches independently produce a type; + # the consequence is tried first. + assert vm.resolve_at("x", _call_at(tree, b, "x.Do")) == "Foo" + + def test_ternary_property_access_branches(self): + src = """class C { + void M(Group group, bool near) { + var container = near ? group.NearContainer : group.FarContainer; + container.Use(); + } + }""" + b, tree, vm = _vm(src) + # Both branches are property accesses; the property-name heuristic + # fires on the first branch. + assert vm.resolve_at("container", _call_at(tree, b, "container.Use")) == "NearContainer" + + def test_ternary_falls_back_to_second_branch(self): + # First branch is unresolvable (bare identifier), second is a + # property access — we try both, the second wins. + src = """class C { + void M(bool b, Context ctx) { + var meta = b ? unknown : ctx.RequestMetadata; + meta.Touch(); + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("meta", _call_at(tree, b, "meta.Touch")) == "RequestMetadata" + + +class TestFileScopeResolve(unittest.TestCase): + def test_field_visible_inside_method(self): + src = """class C { + private Repo _repo; + void M() { _repo.Save(); } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "_repo.Save") + assert vm.resolve_at("_repo", call) == "Repo" + + def test_property_visible_inside_method(self): + src = """class C { + public Repo Backend { get; set; } + void M() { Backend.Save(); } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "Backend.Save") + assert vm.resolve_at("Backend", call) == "Repo" + + +class TestMethodScopingIsolation(unittest.TestCase): + """Same identifier in two methods with different types must not bleed.""" + + def test_two_methods_same_name_different_types(self): + src = """class C { + void A() { Repo r = null; r.Save(); } + void B() { Customer r = null; r.Touch(); } + }""" + b, tree, vm = _vm(src) + call_a = _call_at(tree, b, "r.Save") + call_b = _call_at(tree, b, "r.Touch") + assert vm.resolve_at("r", call_a) == "Repo" + assert vm.resolve_at("r", call_b) == "Customer" + + def test_method_local_shadows_field(self): + src = """class C { + private Repo r; + void M() { Customer r = null; r.Touch(); } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Touch") + assert vm.resolve_at("r", call) == "Customer" + + +class TestConflictSuppression(unittest.TestCase): + """Conflicting types in the **same** scope → None (not emitted).""" + + def test_redeclaration_in_same_block_conflicts(self): + # Two declarations directly in the same block — tree-sitter still + # parses it; we treat it as ambiguous and emit no qualified form. + src = """class C { + void M() { + Repo x = null; + x.Save(); + Customer x = null; + x.Touch(); + } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "x.Save") + assert vm.resolve_at("x", call) is None + + +class TestBlockScopingIsolatesBranches(unittest.TestCase): + """Sibling blocks (if/else, try/catch arms, switch sections) must each + see their own declarations — this is the false-negative that motivated + block-level scoping.""" + + def test_if_else_branches_isolate_same_name(self): + src = """class C { + void M(bool b) { + if (b) { Repo x = null; x.Save(); } + else { Customer x = null; x.Touch(); } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("x", _call_at(tree, b, "x.Save")) == "Repo" + assert vm.resolve_at("x", _call_at(tree, b, "x.Touch")) == "Customer" + + def test_multiple_catch_clauses_isolate_exception_var(self): + # Two catches per try-catch with the same variable name is the + # canonical real-world false-negative this resolves. + src = """class C { + void M() { + try { Do(); } + catch (Repo ex) { ex.Save(); } + catch (Customer ex) { ex.Touch(); } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("ex", _call_at(tree, b, "ex.Save")) == "Repo" + assert vm.resolve_at("ex", _call_at(tree, b, "ex.Touch")) == "Customer" + + def test_for_loop_variable_isolated_per_loop(self): + src = """class C { + void M() { + for (Repo i = null; i != null; i = i.Next) { i.Save(); } + for (Customer i = null; i != null; i = i.Next) { i.Touch(); } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("i", _call_at(tree, b, "i.Save")) == "Repo" + assert vm.resolve_at("i", _call_at(tree, b, "i.Touch")) == "Customer" + + def test_foreach_iteration_variable_scoped_to_loop(self): + src = """class C { + void M(Repo[] a, Customer[] b) { + foreach (Repo r in a) { r.Save(); } + foreach (Customer r in b) { r.Touch(); } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("r", _call_at(tree, b, "r.Save")) == "Repo" + assert vm.resolve_at("r", _call_at(tree, b, "r.Touch")) == "Customer" + + def test_using_statement_variable_scoped(self): + src = """class C { + void M() { + using (Repo s = null) { s.Save(); } + using (Customer s = null) { s.Touch(); } + } + }""" + b, tree, vm = _vm(src) + assert vm.resolve_at("s", _call_at(tree, b, "s.Save")) == "Repo" + assert vm.resolve_at("s", _call_at(tree, b, "s.Touch")) == "Customer" + + +class TestNestedScopes(unittest.TestCase): + def test_lambda_inherits_enclosing_method_var(self): + src = """class C { + void M() { + Repo r = null; + System.Action a = () => r.Save(); + } + }""" + b, tree, vm = _vm(src) + call = _call_at(tree, b, "r.Save") + # ``r`` is declared in the method body, not the lambda; the resolver + # walks up from the call to the lambda (no ``r``), then to the + # method (has ``r``), then returns ``Repo``. + assert vm.resolve_at("r", call) == "Repo" + + def test_typed_lambda_param_resolved(self): + # Typed lambda parameters land in a regular parameter_list — same + # handling as a method parameter. + src = """class C { + void M(Repo r) { + System.Func a = (Customer c) => c.Id; + } + }""" + b, tree, vm = _vm(src) + member = _node_at(tree, b, "c.Id", "member_access_expression") + assert vm.resolve_at("c", member) == "Customer" + # Outer-method param still visible from the lambda body. + assert vm.resolve_at("r", member) == "Repo" + + def test_implicit_lambda_param_unresolved(self): + # ``c => c.Id`` carries no syntactic type — resolution would need + # cross-expression Func<,> inference, which we don't do. The + # resolver returns None rather than guessing. + src = """class C { + void M() { + System.Func a = c => c.Id; + } + }""" + b, tree, vm = _vm(src) + member = _node_at(tree, b, "c.Id", "member_access_expression") + assert vm.resolve_at("c", member) is None + + +# ── _q_all_call_site_infos integration ─────────────────────────────────────── + + +class TestCallSiteResolvedType(unittest.TestCase): + """Verify ``resolved_type`` is set on CallSiteInfo when receiver resolves.""" + + def _infos(self, src: str): + b, tree, vm = _vm(src) + idx = _CsIndex(b, tree, _DESCRIBE_NODE_TYPES) + return _q_all_call_site_infos(b, idx, vm) + + def test_resolved_for_typed_local(self): + infos = self._infos("class C { void M() { Repo r = null; r.Save(); } }") + save = next(cs for cs in infos if cs.name == "Save") + assert save.receiver == "r" + assert save.resolved_type == "Repo" + + def test_not_resolved_for_unknown_local(self): + infos = self._infos("class C { void M() { var r = GetRepo(); r.Save(); } }") + save = next(cs for cs in infos if cs.name == "Save") + assert save.receiver == "r" + assert save.resolved_type == "" + + def test_pascal_receiver_static_call(self): + # ``Foo.Save()`` — Foo isn't a declared variable, so var-type map + # returns None; resolved_type is empty. The literal receiver + # captures Foo for the indexer's static-style qualifying. + infos = self._infos("class C { void M() { Foo.Save(); } }") + save = next(cs for cs in infos if cs.name == "Save") + assert save.receiver == "Foo" + assert save.resolved_type == "" + + def test_resolved_uses_unqualified_type(self): + # ``A.B.Repo`` field type → resolved_type stored as ``Repo`` + # so the qualified-call form is stable regardless of namespace. + infos = self._infos( + "class C { private A.B.Repo _r; void M() { _r.Save(); } }" + ) + save = next(cs for cs in infos if cs.name == "Save") + assert save.resolved_type == "Repo" + + def test_resolved_strips_generics(self): + infos = self._infos( + "class C { void M() { List xs = null; xs.Add(1); } }" + ) + add = next(cs for cs in infos if cs.name == "Add") + assert add.resolved_type == "List" + + +# ── q_calls qualified-receiver matching ────────────────────────────────────── + + +class TestQCallsQualifiedReceiver(unittest.TestCase): + """``q_calls("Type.Method")`` must match call sites whose receiver + resolves to ``Type`` via the var-type map, not just those whose literal + receiver text equals ``Type``.""" + + def _q(self, src: str, pattern: str): + from query.cs import q_calls + b, tree, _ = _parse(src) + return q_calls(b, tree, src.splitlines(), pattern) + + def test_literal_receiver_match_unchanged(self): + # ``Foo.Save()`` — receiver is literally "Foo"; legacy behaviour. + r = self._q("class C { void M() { Foo.Save(); } }", "Foo.Save") + assert len(r) == 1, r + + def test_typed_local_receiver_matches_via_resolved_type(self): + # ``r: Repo`` then ``r.Save()`` — should match q_calls("Repo.Save"). + r = self._q( + "class C { void M() { Repo r = null; r.Save(); } }", + "Repo.Save") + assert len(r) == 1, r + + def test_field_receiver_matches_via_resolved_type(self): + # Field declaration in file scope; method-local call uses it. + r = self._q( + "class C { private IRepo _repo; void M() { _repo.Save(); } }", + "IRepo.Save") + assert len(r) == 1, r + + def test_param_receiver_matches_via_resolved_type(self): + r = self._q( + "class C { void M(IRepo repo) { repo.Save(); } }", + "IRepo.Save") + assert len(r) == 1, r + + def test_unresolved_receiver_rejects_qualified_match(self): + # ``var r = GetRepo()`` is unresolvable; q_calls("Repo.Save") must + # not match (the qualifier check fails on literal and on resolution). + r = self._q( + "class C { void M() { var r = GetRepo(); r.Save(); } }", + "Repo.Save") + assert r == [], r + + def test_conflict_suppressed_receiver_rejects_qualified_match(self): + # Two same-name locals of different types in one block → conflict → + # no qualified match emitted. The bare-name search still finds them. + src = """class C { + void M() { + Repo x = null; + x.Save(); + Customer x = null; + x.Save(); + } + }""" + # Qualified form should NOT match (resolution is ambiguous). + r_qual = self._q(src, "Repo.Save") + assert r_qual == [], r_qual + # Bare-name should still find both calls. + r_bare = self._q(src, "Save") + assert len(r_bare) == 2, r_bare + + def test_null_conditional_uses_resolved_type(self): + # ``r?.Save()`` where r resolves to Repo. + r = self._q( + "class C { void M(Repo r) { r?.Save(); } }", + "Repo.Save") + assert len(r) == 1, r + + def test_literal_does_not_match_when_resolution_differs(self): + # ``r: Repo`` then ``r.Save()`` — must NOT match q_calls("OtherType.Save") + # since neither literal "OtherType" nor resolved type matches. + r = self._q( + "class C { void M() { Repo r = null; r.Save(); } }", + "OtherType.Save") + assert r == [], r + + +# ── q_calls anchors chained calls at the name token ────────────────────────── + + +class TestQCallsChainedReporting(unittest.TestCase): + """For multi-line chained calls ``a.B().Method(...)``, the reported line + should be where ``Method`` itself appears — not the start of the outer + invocation. The reported text is the single source line at that row.""" + + def _q(self, src: str, pattern: str): + from query.cs import q_calls + b, tree, _ = _parse(src) + return q_calls(b, tree, src.splitlines(), pattern) + + def test_single_line_call_unchanged(self): + # ``Foo.Save();`` — name and call start on the same row; result row + # is the obvious line (regression check that the new anchor logic + # doesn't shift simple cases). + src = "class C { void M() { Foo.Save(); } }" + r = self._q(src, "Save") + assert len(r) == 1, r + line, text = r[0] + assert line == 1 + assert "Foo.Save()" in text + + def test_chained_call_reports_name_line_not_chain_start(self): + # The outer invocation spans two lines (chain starts at L2, the + # ``ConfigureAwait`` name token sits on L3). The reported line + # should be L3 — the row containing the matched identifier. + src = ( + "class C { void M(System.Threading.Tasks.Task task) {\n" + " task.Result\n" + " .ToString();\n" + "} }\n" + ) + r = self._q(src, "ToString") + assert len(r) == 1, r + line, text = r[0] + assert line == 3, f"expected name-token line 3, got {line}: {r}" + # The reported text is the single line containing ToString, not + # the multi-line node render. + assert ".ToString()" in text + assert "task.Result" not in text + + def test_chained_call_at_method_call_receiver(self): + # Receiver of the outer ``.Save()`` is itself an invocation + # (``Get()``). The outer invocation begins at L2 (``Get`` line), + # but ``Save`` is on L3. + src = ( + "class C { void M() {\n" + " this.Get()\n" + " .Save();\n" + "} }\n" + ) + r = self._q(src, "Save") + assert len(r) == 1, r + line, _ = r[0] + assert line == 3, f"expected L3 (Save's line), got {line}: {r}" + + +# ── describe_cs_file end-to-end ────────────────────────────────────────────── + + +class TestDescribeFileEnd2End(unittest.TestCase): + """describe_cs_file must produce CallSiteInfo with resolved_type.""" + + def test_resolved_type_propagates(self): + src = b"class C { void M() { Repo r = null; r.Save(); } }" + fd = describe_cs_file(src) + save = next(cs for cs in fd.call_site_infos if cs.name == "Save") + assert save.resolved_type == "Repo" + + +if __name__ == "__main__": + unittest.main() diff --git a/query/tests/test_cs_var_type_mode.py b/query/tests/test_cs_var_type_mode.py new file mode 100644 index 0000000..d076f0f --- /dev/null +++ b/query/tests/test_cs_var_type_mode.py @@ -0,0 +1,118 @@ +""" +Tests for the ``var_type NAME`` query mode (C#). + +Exercises the user-facing wrapper around ``_VarTypeMap.resolve_at``: for +each occurrence of NAME in a file, report the resolved type at that scope +or a sentinel when the resolver can't pin it down. +""" +from __future__ import annotations + +import unittest + +import tree_sitter_c_sharp as tscsharp +from tree_sitter import Language, Parser + +from query.cs import q_var_type + +_CS = Language(tscsharp.language()) +_PARSER = Parser(_CS) + + +def _run(src: str, name: str): + b = src.encode() + tree = _PARSER.parse(b) + return q_var_type(b, tree, src.splitlines(), name) + + +def _text_of(results): + return [t for _, t in results] + + +class TestQVarTypeResolves(unittest.TestCase): + """Distinct scopes should report distinct types for the same name.""" + + def test_parameter_typed(self): + src = "class C { void M(Repo r) { r.Save(); } }" + out = _run(src, "r") + # Both occurrences (parameter list + body) resolve to Repo. + assert any("r : Repo" in t for t in _text_of(out)), out + assert all(": (unresolved)" not in t and ": (conflicting)" not in t + for t in _text_of(out)), out + + def test_typed_local_in_block(self): + src = "class C { void M() { Customer c = null; c.Touch(); } }" + out = _run(src, "c") + assert any("c : Customer" in t for t in _text_of(out)), out + + def test_different_methods_get_different_types(self): + # Method-scope isolation: same name, different types, no conflict. + src = """class C { + void A() { Repo r = null; r.Save(); } + void B() { Customer r = null; r.Touch(); } + }""" + out = _run(src, "r") + types_per_line = {ln: txt for ln, txt in out} + # Line 2 inside A — Repo. + assert "Repo" in types_per_line[2], types_per_line + # Line 3 inside B — Customer. + assert "Customer" in types_per_line[3], types_per_line + + +class TestQVarTypeUnresolved(unittest.TestCase): + """Unknown names report ``(unresolved)``; ambiguous-in-scope reports + ``(conflicting)`` so an agent can tell the two situations apart.""" + + def test_never_declared(self): + src = "class C { void M() { unknown.Method(); } }" + out = _run(src, "unknown") + assert any(": (unresolved)" in t for t in _text_of(out)), out + + def test_conflicting_redeclaration_in_same_block(self): + src = """class C { + void M() { + Repo x = null; + x.Save(); + Customer x = null; + x.Touch(); + } + }""" + out = _run(src, "x") + # Every emitted entry for x in this single (conflicted) scope is + # labelled "(conflicting)" — we never invent a winning type. + assert all(": (conflicting)" in t for t in _text_of(out)), out + + +class TestQVarTypeInferenceHeuristics(unittest.TestCase): + """The var-type map's inference heuristics should propagate to + ``var_type`` — e.g. ``var x = new Foo()`` reports x as Foo.""" + + def test_var_from_new(self): + src = "class C { void M() { var w = new Widget(); w.Render(); } }" + out = _run(src, "w") + assert any("w : Widget" in t for t in _text_of(out)), out + + def test_var_from_generic_method(self): + src = "class C { void M(IContainer c) { var s = c.Resolve(); s.Run(); } }" + out = _run(src, "s") + assert any("s : IService" in t for t in _text_of(out)), out + + +class TestQVarTypeSkipsLiterals(unittest.TestCase): + """Identifiers inside strings/comments are not matched (mirrors + ``all_refs``).""" + + def test_string_mention_excluded(self): + src = '''class C { void M() { + Repo r = null; + string s = "use r for storage"; + r.Save(); + } }''' + out = _run(src, "r") + # Only the real declaration + use line — not the inside of the string. + rows = [ln for ln, _ in out] + assert 2 in rows and 4 in rows, out + assert 3 not in rows, out + + +if __name__ == "__main__": + unittest.main() diff --git a/query/tests/test_cs_visibility.py b/query/tests/test_cs_visibility.py new file mode 100644 index 0000000..1414d8a --- /dev/null +++ b/query/tests/test_cs_visibility.py @@ -0,0 +1,248 @@ +""" +Tests for the C# visibility capture (modifier extraction + defaults) and +the visibility filter on declaration query modes. + +Covers: + * Explicit modifier resolution (public/internal/protected/private), + including compound forms like ``protected internal`` and + ``private protected``. + * Language defaults: top-level types default to internal; nested types + default to private; interface members default to public; enum members + default to public; class members default to private. + * Filter behavior: visibility="" matches everything; visibility="public" + keeps only public; comma-separated keeps the union; languages that + don't capture visibility match nothing under a filter. +""" +from __future__ import annotations + +import unittest + +import tree_sitter_c_sharp as tscsharp +from tree_sitter import Language, Parser + +from query.cs import ( + q_classes, q_methods, q_fields, q_declarations, + _cs_type_visibility, _cs_member_visibility, _cs_explicit_visibility, + _find_all, +) + +_CS = Language(tscsharp.language()) +_PARSER = Parser(_CS) + + +def _parse(src: str): + b = src.encode() + tree = _PARSER.parse(b) + return b, tree, src.splitlines() + + +def _find(tree, types): + return _find_all(tree.root_node, lambda n: n.type in types) + + +# ── Modifier extraction ────────────────────────────────────────────────────── + + +class TestExplicitVisibility(unittest.TestCase): + """Modifier-keyword parsing on a declaration node.""" + + def _vis(self, src: str, node_type: str) -> str: + _b, tree, _ = _parse(src) + nodes = _find(tree, {node_type}) + assert nodes, f"no {node_type} in source" + return _cs_explicit_visibility(nodes[0]) + + def test_public(self): + assert self._vis("public class C {}", "class_declaration") == "public" + + def test_internal(self): + assert self._vis("internal class C {}", "class_declaration") == "internal" + + def test_protected_internal_collapses_to_protected(self): + # ``protected internal`` = reachable via inheritance from outside + # the assembly — classifies as ``protected`` for filtering. + src = "class O { protected internal void M() {} }" + assert self._vis(src, "method_declaration") == "protected" + + def test_private_protected_collapses_to_private(self): + # ``private protected`` = most-restricted compound form. + src = "class O { private protected void M() {} }" + assert self._vis(src, "method_declaration") == "private" + + def test_no_modifier_returns_empty(self): + # Bare ``void M()`` with no modifier — explicit extractor returns "". + # Defaults are applied by the higher-level _cs_member_visibility. + src = "class O { void M() {} }" + assert self._vis(src, "method_declaration") == "" + + +# ── Language defaults ──────────────────────────────────────────────────────── + + +class TestTypeDefaults(unittest.TestCase): + """``_cs_type_visibility`` should apply C# defaults to types without + explicit modifiers.""" + + def test_top_level_class_defaults_to_internal(self): + _b, tree, _ = _parse("class TopLevel {}") + node = _find(tree, {"class_declaration"})[0] + assert _cs_type_visibility(node) == "internal" + + def test_nested_class_defaults_to_private(self): + _b, tree, _ = _parse("class Outer { class Nested {} }") + nested = _find(tree, {"class_declaration"})[1] + assert _cs_type_visibility(nested) == "private" + + +class TestMemberDefaults(unittest.TestCase): + """``_cs_member_visibility`` should apply C# defaults to members + without explicit modifiers (class ⇒ private; interface ⇒ public; + enum body ⇒ public).""" + + def test_class_member_defaults_to_private(self): + _b, tree, _ = _parse("class C { void M() {} }") + m = _find(tree, {"method_declaration"})[0] + assert _cs_member_visibility(m) == "private" + + def test_interface_member_defaults_to_public(self): + _b, tree, _ = _parse("interface I { void M(); }") + m = _find(tree, {"method_declaration"})[0] + assert _cs_member_visibility(m) == "public" + + def test_explicit_member_modifier_wins_over_default(self): + # Even inside an interface, an explicit ``private`` overrides the + # public default (C# 8 added this; we just defer to the modifier). + _b, tree, _ = _parse("interface I { private void M() {} }") + m = _find(tree, {"method_declaration"})[0] + assert _cs_member_visibility(m) == "private" + + +# ── Visibility filter on declaration queries ───────────────────────────────── + + +_SRC = """\ +namespace N { + public class Public { + public void PubMethod() {} + internal void IntMethod() {} + protected void ProtMethod() {} + private void PrivMethod() {} + public int PubProp { get; set; } + private int privField; + } + + internal class Internal { + public void OtherPub() {} + } + + interface I { + void IfaceMethod(); + } +} +""" + + +class TestQClassesVisibility(unittest.TestCase): + def setUp(self): + self.b, self.tree, self.lines = _parse(_SRC) + + def _names(self, results): + # Result text is "[class] Name : Bases" (no end-line in this tuple). + return [t.split("] ")[1].split(" ")[0].split(":")[0].strip() + for _, _, t in results] + + def test_no_filter_returns_all(self): + names = self._names(q_classes(self.b, self.tree, self.lines)) + assert set(names) == {"Public", "Internal", "I"}, names + + def test_public_only(self): + names = self._names(q_classes(self.b, self.tree, self.lines, + visibility="public")) + assert names == ["Public"], names + + def test_internal_includes_interface_default(self): + # ``interface I`` has no modifier → top-level default = internal. + names = self._names(q_classes(self.b, self.tree, self.lines, + visibility="internal")) + assert set(names) == {"Internal", "I"}, names + + def test_multi_value_visibility(self): + names = self._names(q_classes(self.b, self.tree, self.lines, + visibility="public,internal")) + assert set(names) == {"Public", "Internal", "I"}, names + + +class TestQMethodsVisibility(unittest.TestCase): + def setUp(self): + self.b, self.tree, self.lines = _parse(_SRC) + + def _method_names(self, results): + # Method text looks like "[method] " — sig has the name. + names = [] + for _, _, t in results: + # Skip non-method lines (props, fields). + if t.startswith("[method]"): + # ``[method] void PubMethod()`` → "PubMethod" + sig = t.split("] ", 1)[1] + # Last whitespace-separated token before ``(``. + names.append(sig.split("(")[0].strip().split(" ")[-1]) + return names + + def test_public_only_methods(self): + names = self._method_names(q_methods(self.b, self.tree, self.lines, + visibility="public")) + # PubMethod, OtherPub, IfaceMethod (interface default = public). + assert set(names) == {"PubMethod", "OtherPub", "IfaceMethod"}, names + + def test_private_includes_class_member_default(self): + # ``private void PrivMethod`` is explicit private. No implicit + # method in this fixture is "default-private", since the others + # have explicit modifiers. + names = self._method_names(q_methods(self.b, self.tree, self.lines, + visibility="private")) + assert names == ["PrivMethod"], names + + def test_unknown_visibility_returns_empty(self): + # Typo: "publik" — no match (treated as a hard filter, not a fallback). + out = q_methods(self.b, self.tree, self.lines, visibility="publik") + assert out == [], out + + +class TestQFieldsVisibility(unittest.TestCase): + def setUp(self): + self.b, self.tree, self.lines = _parse(_SRC) + + def test_public_props_only(self): + out = q_fields(self.b, self.tree, self.lines, visibility="public") + names = [t.split(" ")[-1] for _, _, t in out] + assert names == ["PubProp"], names + + def test_private_fields_only(self): + out = q_fields(self.b, self.tree, self.lines, visibility="private") + names = [t.split(" ")[-1] for _, _, t in out] + assert names == ["privField"], names + + +class TestQDeclarationsVisibility(unittest.TestCase): + def setUp(self): + self.b, self.tree, self.lines = _parse(_SRC) + + def test_filter_keeps_only_named_match_with_visibility(self): + # Three declarations named ``OtherPub`` etc. — but searching for + # ``Public`` restricted to public matches the public class only. + out = q_declarations(self.b, self.tree, self.lines, "Public", + visibility="public") + # Header text is ``[class] Public S-E:``. + headers = [t.splitlines()[0] for _, t in out] + assert any("[class] Public" in h for h in headers), headers + + def test_filter_drops_non_matching_visibility(self): + # Searching for ``Internal`` (the class name) restricted to public + # returns nothing — the class is internal. + out = q_declarations(self.b, self.tree, self.lines, "Internal", + visibility="public") + assert out == [], out + + +if __name__ == "__main__": + unittest.main() diff --git a/query/tests/test_mode_declarations.py b/query/tests/test_mode_declarations.py index e78d7ce..4f40f8f 100644 --- a/query/tests/test_mode_declarations.py +++ b/query/tests/test_mode_declarations.py @@ -58,9 +58,11 @@ def test_output_includes_kind(self): f"Output must include kind annotation: {texts}" def test_output_includes_line_numbers(self): + import re r = self._find(FIND_TARGET, "TargetMethod") texts = [t for _, t in r] - assert any("lines" in t for t in texts), \ + # Header format: ``[method] Name S-E:`` — verify a START-END pair. + assert any(re.search(r"\b\d+-\d+:", t) for t in texts), \ f"Output must include line range: {texts}" def test_output_includes_signature(self): diff --git a/scripts/probe_resolved.py b/scripts/probe_resolved.py new file mode 100644 index 0000000..bd158ff --- /dev/null +++ b/scripts/probe_resolved.py @@ -0,0 +1,116 @@ +"""Diagnostic: for each call site in a file, report whether the qualified-call +indexer pinned a resolved type. Surfaces the gap cases — receivers that look +identifier-like but produce no qualified form — so we can spot patterns the +var-type map should learn to handle. + +Usage: + python -m scripts.probe_resolved [ ...] + python -m scripts.probe_resolved --dir [--limit N] +""" +from __future__ import annotations + +import argparse +import os +import sys +from collections import Counter + +_REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if _REPO not in sys.path: + sys.path.insert(0, _REPO) + +from query.cs import ( + _build_var_type_map, _find_all, _text, _CsIndex, _DESCRIBE_NODE_TYPES, + _q_all_call_site_infos, +) +import tree_sitter_c_sharp as tscsharp +from tree_sitter import Language, Parser + +_CS = Language(tscsharp.language()) +_PARSER = Parser(_CS) + + +def _categorise_receiver(receiver: str, resolved: bool) -> str: + if resolved: + return "resolved" + if not receiver: + return "bare" # no receiver + if receiver[0].isupper(): + return "pascal_unresolved" # likely-static, still gets literal form + return "lower_unresolved" # gap candidate + + +def _scan(path: str, show_lower: int = 5): + with open(path, "rb") as f: + src = f.read() + tree = _PARSER.parse(src) + vm = _build_var_type_map(tree, src) + idx = _CsIndex(src, tree, _DESCRIBE_NODE_TYPES) + infos = _q_all_call_site_infos(src, idx, vm) + + counts: Counter[str] = Counter() + lower_examples: list[tuple[str, str, int]] = [] # (receiver, method, line) + + # Match each CallSiteInfo back to its tree location for line numbers. + # _q_all_call_site_infos walks idx.of("invocation_expression") then ctors, + # so we re-walk in the same order to align. + inv_nodes = idx.of("invocation_expression") + # Build aligned (info, node) pairs for the invocation-only prefix + # (constructors come after — without receivers — and aren't interesting + # for this diagnostic). + for i, info in enumerate(infos): + if i >= len(inv_nodes): + break + node = inv_nodes[i] + bucket = _categorise_receiver(info.receiver, bool(info.resolved_type)) + counts[bucket] += 1 + if bucket == "lower_unresolved" and len(lower_examples) < show_lower: + lower_examples.append( + (info.receiver, info.name, node.start_point[0] + 1)) + + total = sum(counts.values()) + print(f"\n=== {os.path.relpath(path, _REPO)} ({total} invocations) ===") + for b in ("resolved", "bare", "pascal_unresolved", "lower_unresolved"): + n = counts.get(b, 0) + pct = (100 * n / total) if total else 0 + print(f" {b:20s} {n:5d} ({pct:5.1f}%)") + if lower_examples: + print(" unresolved-lowercase examples (gap candidates):") + for r, m, ln in lower_examples: + print(f" L{ln}: {r}.{m}(...)") + return counts + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("files", nargs="*") + ap.add_argument("--dir", default=None) + ap.add_argument("--limit", type=int, default=10) + ap.add_argument("--show-lower", type=int, default=8) + args = ap.parse_args() + + paths: list[str] = list(args.files) + if args.dir: + for root, _dirs, names in os.walk(args.dir): + for n in names: + if n.endswith(".cs"): + paths.append(os.path.join(root, n)) + if len(paths) >= args.limit: + break + if len(paths) >= args.limit: + break + + grand: Counter[str] = Counter() + for p in paths: + c = _scan(p, show_lower=args.show_lower) + grand.update(c) + + print("\n=== AGGREGATE ===") + total = sum(grand.values()) + for b in ("resolved", "bare", "pascal_unresolved", "lower_unresolved"): + n = grand.get(b, 0) + pct = (100 * n / total) if total else 0 + print(f" {b:20s} {n:6d} ({pct:5.1f}%)") + + +if __name__ == "__main__": + main() diff --git a/tests/helpers.py b/tests/helpers.py index ae9d6a9..5dbcb19 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -59,11 +59,21 @@ def _delete_collection(collection: str, timeout: float = 10.0) -> None: even after ``Backend.close()`` clears the index and ``gc.collect()`` runs, the OS may briefly hold a mmap'd file. ``drop()`` already retries; this wrapper retries the whole drop a few times within ``timeout`` and - confirms ``os.path.exists`` is ``False`` before returning. Raises on - persistent failure so the next test fails early with a clear message - instead of indexing into a half-wiped directory. + confirms ``os.path.exists`` is ``False`` before returning. + + On Windows we additionally wait until the parent ``.tantivy/`` directory + is verifiably writable. The next test's setUpClass opens a fresh + ``IndexWriter`` and immediately commits — under Defender/AV scanning or + lingering mmap finalisation, even a freshly-created sibling directory + can transiently refuse a write. Blocking here until a write-then-delete + canary in the parent directory succeeds gives the OS time to settle and + prevents one slow handle release from cascading into the next test's + setUpClass commit. Raises on persistent failure so the next test fails + early with a clear message instead of indexing into a half-wiped + directory. """ import gc as _gc + import sys as _sys import time as _time from indexserver.config import load_config as _load_config, index_root from indexserver.backend import drop @@ -79,13 +89,43 @@ def _delete_collection(collection: str, timeout: float = 10.0) -> None: except Exception as e: last_err = e if not os.path.exists(index_dir): - return + break _gc.collect() _time.sleep(0.2) - raise RuntimeError( - f"_delete_collection({collection!r}) still sees {index_dir!r} after " - f"{timeout}s. Last drop error: {last_err}" - ) + else: + raise RuntimeError( + f"_delete_collection({collection!r}) still sees {index_dir!r} after " + f"{timeout}s. Last drop error: {last_err}" + ) + + if _sys.platform == "win32": + # Parent directory canary: create, write to, and delete a small + # sentinel inside ``.tantivy/``. If this fails (Access denied, + # PermissionError, etc.) the OS hasn't finished releasing whatever + # the just-closed Backend was holding. Retry briefly. + parent = os.path.dirname(index_dir) + canary_deadline = _time.time() + 2.0 + canary_err: Exception | None = None + while _time.time() < canary_deadline: + try: + os.makedirs(parent, exist_ok=True) + canary = os.path.join( + parent, f".release_canary_{collection}.tmp") + with open(canary, "wb") as f: + f.write(b"ok") + os.remove(canary) + return + except OSError as e: + canary_err = e + _gc.collect() + _time.sleep(0.1) + # If the canary keeps failing the next test will hit the same + # problem; surface it now rather than letting it manifest as a + # confusing commit failure inside setUpClass. + raise RuntimeError( + f"_delete_collection({collection!r}): parent {parent!r} not " + f"writable after release-settle wait. Last canary error: {canary_err}" + ) def _make_git_repo(files: dict) -> str: diff --git a/tests/unit/test_indexer.py b/tests/unit/test_indexer.py index 09082cd..47ac0ac 100644 --- a/tests/unit/test_indexer.py +++ b/tests/unit/test_indexer.py @@ -47,6 +47,72 @@ def test_call_sites(self): f"call_sites: {meta['call_sites']}" ) + def test_qualified_calls_static_pascal_receiver(self): + """``Foo.Bar()`` produces ``Foo.Bar`` in qualified_calls.""" + src = b"class C { void M() { Foo.Bar(); } }" + meta = extract_metadata(src, ".cs") + self.assertIn("Foo.Bar", meta["qualified_calls"]) + + def test_qualified_calls_resolved_typed_local(self): + """``Repo r; r.Save()`` resolves to ``Repo.Save``.""" + src = b"class C { void M() { Repo r = null; r.Save(); } }" + meta = extract_metadata(src, ".cs") + self.assertIn("Repo.Save", meta["qualified_calls"]) + + def test_qualified_calls_resolved_field(self): + """Field-typed receiver picks up the field's type.""" + src = b"class C { private IRepository _repo; void M() { _repo.Save(); } }" + meta = extract_metadata(src, ".cs") + self.assertIn("IRepository.Save", meta["qualified_calls"]) + + def test_qualified_calls_no_resolution_for_unknown_var(self): + """``var x = Get(); x.Save()`` — no qualified form (just bare ``Save``).""" + src = b"class C { void M() { var x = Get(); x.Save(); } }" + meta = extract_metadata(src, ".cs") + self.assertNotIn("x.Save", meta["qualified_calls"]) + self.assertIn("Save", meta["call_sites"]) + + def test_qualified_calls_conflict_suppressed_same_block(self): + """Two same-name declarations *in one block* → no qualified emission.""" + src = b"""class C { + void M() { + Repo x = null; + x.Save(); + Customer x = null; + x.Save(); + } + }""" + meta = extract_metadata(src, ".cs") + self.assertNotIn("Repo.Save", meta["qualified_calls"]) + self.assertNotIn("Customer.Save", meta["qualified_calls"]) + # The bare name still survives so the call is still discoverable. + self.assertIn("Save", meta["call_sites"]) + + def test_qualified_calls_independent_branches_both_emitted(self): + """if/else branches each resolve independently — both forms appear.""" + src = b"""class C { + void M(bool b) { + if (b) { Repo x = null; x.Save(); } + else { Customer x = null; x.Save(); } + } + }""" + meta = extract_metadata(src, ".cs") + self.assertIn("Repo.Save", meta["qualified_calls"]) + self.assertIn("Customer.Save", meta["qualified_calls"]) + + def test_qualified_calls_method_scope_isolation(self): + """Same variable name in two methods doesn't cross-contaminate.""" + src = b"""class C { + void A() { Repo r = null; r.Save(); } + void B() { Customer r = null; r.Touch(); } + }""" + meta = extract_metadata(src, ".cs") + self.assertIn("Repo.Save", meta["qualified_calls"]) + self.assertIn("Customer.Touch", meta["qualified_calls"]) + # Cross-pollination would yield these — they must not appear. + self.assertNotIn("Repo.Touch", meta["qualified_calls"]) + self.assertNotIn("Customer.Save", meta["qualified_calls"]) + def test_imports(self): src = b"using System; using System.Collections.Generic;" meta = extract_metadata(src, ".cs") diff --git a/tests/unit/test_mcp_server.py b/tests/unit/test_mcp_server.py index be3dbdb..560bd1e 100644 --- a/tests/unit/test_mcp_server.py +++ b/tests/unit/test_mcp_server.py @@ -484,9 +484,12 @@ def test_threshold_reached_at_20(self): assert "(3 hits)" in result # No grep-style line:content body assert "src/F0.cs:1:" not in result - # Suggestion to drill into a single file + # Suggestion to drill into a single file: the file= argument uses the + # bare relative path (no $SRC_ROOT/ placeholder — the tool accepts + # relative paths directly). assert "query_single_file" in result - assert 'file="$SRC_ROOT/' in result + assert 'file="src/F0.cs"' in result + assert '$SRC_ROOT' not in result def test_below_threshold_is_tier3_not_tier2(self): """19 files → tier 3 (per-line content shown).""" @@ -573,10 +576,11 @@ def test_caps_at_10_lines_per_file(self): # Lines 11-25 absent for i in range(11, 26): assert f"src/Big.cs:{i}:" not in result - # Per-file suggestion appended + # Per-file suggestion appended — uses bare relative path, not the + # legacy $SRC_ROOT/ placeholder. assert "25 total hits" in result assert "query_single_file" in result - assert 'file="$SRC_ROOT/src/Big.cs"' in result + assert 'file="src/Big.cs"' in result def test_no_suggestion_when_under_cap(self): """File with <=10 hits shouldn't trigger a per-file suggestion.""" diff --git a/tests/unit/test_search_modes.py b/tests/unit/test_search_modes.py new file mode 100644 index 0000000..519b718 --- /dev/null +++ b/tests/unit/test_search_modes.py @@ -0,0 +1,48 @@ +""" +Unit tests for ``indexserver.search_modes.resolve_query_params``. + +Verifies the mode → ``(query_by, weights)`` mapping the daemon and the +standalone search CLI share. +""" +from __future__ import annotations + +import unittest + +from indexserver.search_modes import resolve_query_params + + +class TestResolveQueryParams(unittest.TestCase): + + def test_calls_includes_qualified_calls(self): + """``calls`` mode must query both bare and ``Type.Method`` fields. + + Allows the agent to pass either ``Save`` (bare) or + ``IRepository.Save`` (resolved) without picking the right field. + """ + query_by, weights = resolve_query_params("calls", "", "") + fields = query_by.split(",") + assert "call_sites" in fields + assert "qualified_calls" in fields + # Weights must be parallel to fields (same length, all numeric). + weight_parts = weights.split(",") + assert len(weight_parts) == len(fields), (query_by, weights) + for w in weight_parts: + float(w) # raises if non-numeric + + def test_implements_unchanged(self): + query_by, _ = resolve_query_params("implements", "", "") + assert "base_types" in query_by + assert "class_names" in query_by + + def test_uses_field_kind_unchanged(self): + query_by, _ = resolve_query_params("uses", "field", "") + assert "field_types" in query_by + + def test_unknown_mode_falls_back(self): + query_by, _ = resolve_query_params("totally-unknown", "", "") + # Falls back to the broad all_refs mapping. + assert "tokens" in query_by + + +if __name__ == "__main__": + unittest.main() diff --git a/tsquery_server.py b/tsquery_server.py index eb6ae88..3585931 100644 --- a/tsquery_server.py +++ b/tsquery_server.py @@ -106,7 +106,7 @@ def _get_query_module(): def _run_query(mode: str, pattern: str, files: list[Path], include_body: bool = False, symbol_kind: str = "", - uses_kind: str = "") -> list: + uses_kind: str = "", visibility: str = "") -> list: _q = _get_query_module() results = [] for path in files: @@ -120,7 +120,8 @@ def _run_query(mode: str, pattern: str, files: list[Path], matches = _q.query_file(src_bytes, ext, mode, pattern, include_body=include_body, symbol_kind=symbol_kind, - uses_kind=uses_kind) + uses_kind=uses_kind, + visibility=visibility) if matches: results.append({"file": str(native), "matches": matches}) return results @@ -484,6 +485,7 @@ def _handle(self) -> None: include_body = bool(body.get("include_body", False)) symbol_kind = str(body.get("symbol_kind", "") or "") uses_kind = str(body.get("uses_kind", "") or "") + visibility = str(body.get("visibility", "") or "") exclude_path = str(body.get("exclude_path", "") or "") if mode not in _EXT_TO_TS_AND_AST: @@ -556,7 +558,8 @@ def _handle(self) -> None: ast_results = _run_query(ast_mode, pattern, file_list, include_body=include_body, - symbol_kind=symbol_kind, uses_kind=uses_kind) + symbol_kind=symbol_kind, uses_kind=uses_kind, + visibility=visibility) response_hits = [] for ast_item in ast_results: From 9f52440d496bb1d957ff60f524cc889883a41f5b Mon Sep 17 00:00:00 2001 From: Kristof Roomp Date: Mon, 18 May 2026 14:38:47 +0200 Subject: [PATCH 2/2] normalise the codebase to ASCII Strip non-ASCII codepoints (em dashes, arrows, box drawing, ellipsis, bullets, math symbols, check marks) from every .py / .md / .mjs / .js file under the repo and replace each with its ASCII analogue: EM DASH (U+2014) / EN DASH (U+2013) -> '--' or '-' RIGHTWARDS / LEFTWARDS ARROW -> '->' / '<-' RIGHTWARDS DOUBLE ARROW (U+21D2) -> '=>' HORIZONTAL ELLIPSIS (U+2026) -> '...' BOX DRAWINGS LIGHT / DOUBLE HORIZONTAL -> '-' / '=' BOX DRAWINGS LIGHT VERTICAL -> '|' BULLET (U+2022) -> '*' GREATER-/LESS-THAN OR EQUAL TO -> '>=' / '<=' ELEMENT OF (U+2208) -> ' in ' MULTIPLICATION SIGN (U+00D7) -> 'x' CHECK MARK / BALLOT X -> 'OK' / 'NO' Touches 80 files and ~17k character positions. Mostly cosmetic section-separator comments and prose dashes, but also includes the ``q_accesses_on`` output marker (``<-``) and a handful of f-strings that the daemon and tools emit. Adds CLAUDE.md "ASCII only" rule with the Unicode-name-keyed reference table (so the rule's own text doesn't violate the rule). Adds two helper scripts: scripts/find_nonascii.py audits the tree and reports every non-ASCII codepoint by file region (docstring / string literal / comment / code), separating hot-path modules (query / indexserver / mcp_server / tsquery_server) from cold. scripts/replace_nonascii.py the mass-rewriter used for this commit; dry-run by default, --apply to write. All 1099 tests still pass; no test assertion depended on the old Unicode forms. Motivation: Windows console encoding (cp1252) and many downstream tools misrender or fail on the previously-used codepoints, and the ``[method] name S-E:`` tool output had been triggering ``cp1252 codec can't encode`` errors when piped to Python sub-processes. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 119 ++++++++------ README.md | 78 ++++----- guidedloop.md | 56 +++---- indexserver/__init__.py | 2 +- indexserver/backend.py | 48 +++--- indexserver/config.py | 20 +-- indexserver/index_queue.py | 22 +-- indexserver/indexer.py | 26 +-- indexserver/query_util.py | 10 +- indexserver/search.py | 24 +-- indexserver/search_modes.py | 4 +- indexserver/verifier.py | 24 +-- indexserver/watcher.py | 2 +- mcp_server.py | 152 +++++++++--------- query/__init__.py | 2 +- query/_util.py | 26 +-- query/cpp.py | 30 ++-- query/cs.py | 144 ++++++++--------- query/dispatch.py | 6 +- query/js.py | 14 +- query/py.py | 8 +- query/rust.py | 22 +-- query/sql.py | 26 +-- .../tests/test_cs_accesses_of_initializers.py | 12 +- query/tests/test_cs_at_and_body.py | 32 ++-- query/tests/test_cs_attrs_and_implements.py | 20 +-- query/tests/test_cs_cast_as_expression.py | 6 +- query/tests/test_cs_foreach_accesses.py | 18 +-- query/tests/test_cs_null_conditional.py | 12 +- .../test_cs_object_initializer_accesses.py | 14 +- query/tests/test_cs_out_var_accesses.py | 14 +- query/tests/test_cs_pattern_locals.py | 10 +- query/tests/test_cs_pattern_match_accesses.py | 16 +- query/tests/test_cs_recursive_pattern.py | 10 +- query/tests/test_cs_throttle.py | 40 ++--- query/tests/test_cs_uses_kind.py | 40 ++--- query/tests/test_cs_using_statement_locals.py | 20 +-- query/tests/test_cs_var_type_map.py | 64 ++++---- query/tests/test_cs_var_type_mode.py | 10 +- query/tests/test_cs_visibility.py | 26 +-- .../tests/test_cs_with_expression_accesses.py | 10 +- query/tests/test_main.py | 2 +- query/tests/test_mode_accesses_on.py | 6 +- query/tests/test_mode_all_refs.py | 10 +- query/tests/test_mode_declarations.py | 6 +- query/tests/test_mode_params.py | 4 +- query/tests/test_query_cs.py | 44 ++--- run_tests.mjs | 10 +- scripts/find_nonascii.py | 110 +++++++++++++ scripts/parse_perf.py | 2 +- scripts/probe_resolved.py | 6 +- scripts/replace_nonascii.py | 121 ++++++++++++++ scripts/search.py | 6 +- setup.mjs | 22 +-- tests/fixtures.py | 56 +++---- tests/helpers.py | 4 +- tests/integration/conftest.py | 2 +- tests/integration/test_indexer.py | 42 ++--- tests/integration/test_long_identifiers.py | 16 +- tests/integration/test_path_tokens.py | 26 +-- tests/integration/test_path_translation.py | 4 +- tests/integration/test_python.py | 20 +-- tests/integration/test_sample_e2e.py | 50 +++--- tests/integration/test_verifier.py | 12 +- tests/integration/test_watcher.py | 4 +- tests/unit/test_api_dispatch_tables.py | 6 +- tests/unit/test_cpp.py | 66 ++++---- tests/unit/test_indexer.py | 34 ++-- tests/unit/test_indexer_query_consistency.py | 78 ++++----- tests/unit/test_js_ts.py | 14 +- tests/unit/test_mcp_server.py | 82 +++++----- tests/unit/test_python.py | 34 ++-- tests/unit/test_rust.py | 24 +-- tests/unit/test_search_ext_filter.py | 10 +- tests/unit/test_search_modes.py | 2 +- tests/unit/test_sql.py | 8 +- tests/unit/test_verifier.py | 12 +- tests/unit/test_watcher.py | 16 +- ts.mjs | 40 ++--- tsquery_server.py | 56 +++---- 80 files changed, 1280 insertions(+), 1026 deletions(-) create mode 100644 scripts/find_nonascii.py create mode 100644 scripts/replace_nonascii.py diff --git a/CLAUDE.md b/CLAUDE.md index e15d1a1..6df4a9c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,12 +1,35 @@ -# codesearch — developer notes for Claude - -## CRITICAL: no worktrees, no subagents — work serially - -**Never use git worktrees or spawn subagents.** Make all edits directly in the main working directory (`Q:\spocore\tscodesearch`). Work step by step. +# codesearch -- developer notes for Claude + +## CRITICAL: ASCII only -- no Unicode in source files + +**Never introduce non-ASCII characters in code, comments, docstrings, output strings, or docs.** The Windows console encoding (`cp1252`) and many downstream tools (LLM consumers, terminal renderers, log aggregators) misrender or fail outright on Unicode codepoints that look harmless in an editor. The whole codebase has been normalised to ASCII; keep it that way. + +When you're tempted to reach for a "pretty" character, use these ASCII equivalents (the table refers to codepoints by name to avoid embedding the very characters this file forbids): + +| Tempting Unicode | Use ASCII instead | +|------------------|-------------------| +| EM DASH (U+2014), EN DASH (U+2013) | `--` or `-` | +| RIGHTWARDS ARROW (U+2192) | `->` | +| LEFTWARDS ARROW (U+2190) | `<-` | +| LEFT RIGHT ARROW (U+2194) | `<->` | +| RIGHTWARDS DOUBLE ARROW (U+21D2) | `=>` | +| HORIZONTAL ELLIPSIS (U+2026) | `...` | +| BOX DRAWINGS LIGHT HORIZONTAL (U+2500) | `-` | +| BOX DRAWINGS DOUBLE HORIZONTAL (U+2550) | `=` | +| BOX DRAWINGS LIGHT VERTICAL (U+2502) | `\|` | +| BULLET (U+2022) | `*` or `-` | +| GREATER-THAN OR EQUAL TO (U+2265) | `>=` | +| LESS-THAN OR EQUAL TO (U+2264) | `<=` | +| ELEMENT OF (U+2208) | ` in ` | +| MULTIPLICATION SIGN (U+00D7) | `x` | +| CHECK MARK (U+2713) | `OK` | +| BALLOT X (U+2717) | `NO` | + +Run `python -m scripts.find_nonascii` from the repo root to verify no non-ASCII codepoint has slipped in (it categorises by file region: docstring, string literal, comment, code). `python -m scripts.replace_nonascii --apply` will mass-rewrite the known codepoints if anything sneaks back in. ## CRITICAL: running Python scripts from the Bash tool -Everything runs in the **client venv** on Windows — there is no separate WSL venv anymore. +Everything runs in the **client venv** on Windows -- there is no separate WSL venv anymore. ```bash .client-venv/Scripts/python.exe