Skip to content

fix: one unloadable grammar no longer takes down the whole CLI (#323) - #337

Open
Ercaner1988 wants to merge 2 commits into
trailhq:mainfrom
Ercaner1988:grammar-load-isolation
Open

fix: one unloadable grammar no longer takes down the whole CLI (#323)#337
Ercaner1988 wants to merge 2 commits into
trailhq:mainfrom
Ercaner1988:grammar-load-isolation

Conversation

@Ercaner1988

@Ercaner1988 Ercaner1988 commented Sep 10, 2026

Copy link
Copy Markdown

Implements suggestion (3) from #323, with (2) for Kotlin as a second commit you can drop on its own.

What happens today

src/graph/extract.ts imports all nine depth-tier grammars at the top of the module. They are native (node-gyp) modules, so any one of them failing to load takes the CLI down at import time, before argv is read — with a node-gyp-build stack trace that never says "graft".

On current main, with tree-sitter-kotlin made unloadable:

$ graft --version
node:internal/modules/run_main:107
Error: No native build was found for platform=win32 arch=x64 runtime=node abi=137 uv=1 node=24.16.0
    loaded from: tree-sitter-kotlin

$ graft --help                          # same
$ graft init --dry-run --agents claude   # same — the repro in the issue
$ graft build <repo with no .kt in it>   # same

I have hit this twice from unrelated causes: Kotlin, as in this issue, and tree-sitter-r under Bun, where the binding asks for a prebuild filename that the scoped package does not ship (r-lib/tree-sitter-r#205, one-line fix in r-lib/tree-sitter-r#206). Both were one line to fix once found, and neither was findable from the error, which is the part fix (3) changes.

Commit 1 — the isolation (fix 3)

createRequire plus a try/catch per grammar, keeping whatever loads. The extension table is filtered by that, which is all the rest of the pipeline needs: entryFor stops claiming the language's extensions, and its files then take the paths a language graft has no grammar for takes today — the breadth tier where a generic row claims the extension, unindexed where none does. listSourceFiles never enumerates them, so build.ts's generic! stays safe.

The warning is once per language, from entryFor — the first time a file that language would have claimed comes past. A repo with no Kotlin in it stays quiet; a Kotlin repo never indexes short in silence:

graft: .kt/.kts files are not parsed with their own grammar — tree-sitter-kotlin failed to load (No native build was found for platform=win32 arch=x64 runtime=node abi=137 uv=1 node=24.16.0). Reinstalling graft rebuilds it; every other language indexes as usual.

This is the contract your two WASM tiers already state for themselves — "an unavailable grammar is skipped rather than fatal" (container.ts), "degrades to a file node only (never throws)" (generic.ts). The native tier was the one that could not say it.

tree-sitter itself stays a static import: it is the shared dependency of all nine, and there is no depth tier without it.

Commit 2 — Kotlin's fallback row (fix 2), droppable on its own

With commit 1 alone that machine starts but indexes no Kotlin at all, which on a Kotlin repo is still a bad day. tree-sitter-wasm ships a kotlin grammar and src/graph/queries/kotlin.scm has been in the tree unused since Kotlin was promoted to the depth tier, so the fallback costs one registry row.

A row whose extension the depth tier also claims is unreachable while the native grammar loads — .java has been exactly that all along. Both warmGenericGrammars calls now apply the tier precedence their own parse loops apply, so a fallback row cannot warm a WASM grammar the run will never call (already true of .java; a second row would have doubled it).

Measured on a three-file repo (.ts, .kt, .rs):

tree-sitter-kotlin result
loads (today) 11 nodes, 8 edges [kotlin, rust, typescript]
broken, main the CLI does not start
broken, commit 1 6 nodes, 4 edges [rust, typescript], plus the warning
broken, commits 1+2 11 nodes, 5 edges [kotlin, rust, typescript]

On a healthy install the graph is identical before and after commit 2 — same nodes, same edges, compared field by field. The row really is unreachable.

graft check agrees with the build it follows on the fallback path (graph check: OK, with Repo, describe, reload and openRepo all present), so this does not reopen the tier-mismatch shape of #236.

Tests

test/grammar-unavailable.test.ts drives the real CLI in a child process, with test/break-grammar-preload.cjs standing in for the missing native build — so it runs on a runner that has a compiler, which is why CI never caught the original. The preload breaks both ways a module can be reached, import and require, so the test still fails if the static imports ever come back. Each of the three fails without the commit it belongs to.

npm test on this branch: 1223 tests, 1210 pass, 8 fail, 5 skipped — the same 8 that fail on upstream/main before it, and no others. They are toLocaleString() assertions that assume a , thousands separator, where this tr-TR machine renders ~100.000. Nothing to do with this PR: they are #338, fixed separately in #345, and this branch is deliberately independent of that one.

The three tests this PR adds pass; each fails without the commit it belongs to.

One thing you may see and should not blame on this branch: test/mcp-server.test.ts timed out on two of my five full-suite runs (its rpc() helper waits a fixed 15s, then reads undefined.result). It is a load-sensitive deadline — those tests need ~2s to get a first response when run alone, and I measured MCP server startup identical on this branch and on main (1.7–2.0s across five spawns each), so nothing here made the server slower. But this PR does add a test file that spawns three short-lived CLI processes, so it adds parallel load, and that deadline has little headroom on a busy machine.

Environment: Node v24.16.0, Windows 11 x64, rebased onto de8456e (0.18.0).

No CHANGELOG entry

I dropped the one I had. 0.18.0 is already released, so there is no unreleased section to add to, and de8456e shows the release commit is what writes them. Happy to add a bullet wherever you want it.

Not in this PR

Loading the nine grammars costs ~220 ms, paid by every command including --version. GRAMMAR_MODULES makes lazy per-language loading a small follow-up, but that is a performance change and does not belong in a fix.

🤖 Generated with Claude Code

@trailhq-graft

trailhq-graft Bot commented Sep 10, 2026

Copy link
Copy Markdown

🌱 graft blast radius

1 area changed → 8 areas can be affected. 41 dependent symbols, depth 2.
Tests: Graph Analysis has tests the diff did not touch.
Tag: @anirudhkumar-nanonets — 6 of 9 areas · @shhdwi — 7 of 9 areas · @Frankie-Xu — Context Build

flowchart TB
  A0(("Graph Construction<br/>16 symbols"))
  A1(("Review Processing<br/>10 symbols"))
  A2(("CLI Engine<br/>6 symbols"))
  A3(("MCP Tools<br/>4 symbols"))
  A4(("Language Enrichment<br/>2 symbols"))
  AX(("3 smaller areas<br/>3 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3,A4 reached;
  classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
  class AX tail;
Loading
Can be affected Symbols Nearest hop Reached from
Graph Construction 16 src/graph/container.ts:L1-L209 container.ts — imports, depth 1 Graph Analysis
Review Processing 10 src/app/brain-build.ts:L251-L358 readRepository — calls, depth 1 Graph Analysis
CLI Engine 6 src/engine.ts:L91-L101 graph — calls, depth 1 Graph Analysis
MCP Tools 4 src/mcp/tools.ts:L216-L244 callTool — calls, depth 2 Graph Analysis
Language Enrichment 2 src/graph/lsp/enrich.ts:L1-L141 enrich.ts — imports, depth 1 Graph Analysis
Viewer Build 1 scripts/build-viewer.mjs:L1-L45 build-viewer.mjs — calls, depth 2 Graph Analysis
2 smaller areas 2 Context Build, Sync Execution see below
Who knows this code — 4 people across 9 areas
Area Who knows it
Graph Analysis · changed @anirudhkumar-nanonets — 20 commits, last 14d ago · @shhdwi — 15 commits, last 28d ago
Graph Construction · affected @anirudhkumar-nanonets — 20 commits, last 16d ago · @shhdwi — 19 commits, last 28d ago
Review Processing · affected @anirudhkumar-nanonets — 12 commits, last today
CLI Engine · affected @anirudhkumar-nanonets — 41 commits, last today · @shhdwi — 24 commits, last 28d ago
MCP Tools · affected @shhdwi — 14 commits, last 29d ago · @anirudhkumar-nanonets — 7 commits, last 8d ago
Language Enrichment · affected @shhdwi — 2 commits, last 29d ago
Viewer Build · affected @shhdwi — 2 commits, last 29d ago
Context Build · affected @anirudhkumar-nanonets — 4 commits, last 21d ago · @Frankie-Xu — 2 commits, last 10d ago
…1 further area

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 41 dependent symbols, grouped by area

Graph Construction — 16 symbols in 8 files

  • src/graph/container.ts:L1-L209 — container.ts (imports, depth 1)
    19: * worse than not indexing the file at all: graft's promise is that its
  • src/graph/refresh.ts:L150-L227 — ensureFreshGraph (calls, depth 1)
    158: // worktree, whose parent checkout's `graft/` git could not check out. Copy it
  • src/graph/resolve.ts:L1-L658 — resolve.ts (imports, depth 1)
    77: * and refusing edges for every extension graft cannot name would lose real ones.
  • src/graph/source-files.ts:L20-L22 — supportedExtensions (calls, depth 1)
    21: return [...new Set([...depthExtensions(), ...genericExtensions(), ...containerExtensions()])].sort();
  • src/graph/source-files.ts:L1-L121 — source-files.ts (imports, depth 1)
    14: import { languageOf, depthExtensions } from "./extract.js";
  • src/graph/workspace.ts:L630-L655 — federateCheck (calls, depth 1)
    638: const g = await checkGraph(join(root, child));
  • src/graph/fingerprint.ts:L1-L197 — fingerprint.ts (imports, depth 2)
    4: * Every graft retrieval call runs this, so it has to be ~free on the common
  • src/graph/map.ts:L132-L139 — sortedLanguages (calls, depth 2)
  • src/graph/refresh.ts:L235-L261 — ensureFreshChildren (calls, depth 2)
    249: // clobbering the rest.) A child's graph always lives in its own `<child>/graft`,
  • src/graph/refresh.ts:L1-L274 — refresh.ts (imports, depth 2)
    5: * background `graft build` *after* the turn ended, so every query an agent made
  • src/graph/resolve.ts:L67-L71 — familyOf (calls, depth 2)
  • src/graph/source-files.ts:L64-L85 — listSourceFiles (calls, depth 2)
    75: // All three must agree here or `build` and `check` would enumerate different sets.
  • src/graph/source-files.ts:L35-L38 — unsupportedExtensions (calls, depth 2)
  • src/graph/workspace-cli.ts:L49-L70 — buildChild (calls, depth 2)
  • src/graph/workspace-cli.ts:L126-L130 — runWorkspaceCheck (calls, depth 2)
  • src/graph/workspace.ts:L1-L725 — workspace.ts (imports, depth 2)
    5: * `graft/` (byte-identical to building that child standalone), and the parent

Review Processing — 10 symbols in 6 files

  • src/app/brain-build.ts:L251-L358 — readRepository (calls, depth 1)
    282: await buildGraph(checkout.dir, { graphOnly: true });
  • src/app/review.ts:L45-L99 — reviewPullRequest (calls, depth 1)
    55: await buildGraph(checkout.dir);
  • src/app/brain-build-worker.ts:L1-L83 — brain-build-worker.ts (calls, depth 2)
    36: message: string;
  • src/app/brain-build-worker.ts:L29-L32 — DoneMessage (references, depth 2)
  • src/app/brain-build.ts:L237-L239 — buildRepoIntoBrain (calls, depth 2)
  • src/app/brain-build.ts:L1-L428 — brain-build.ts (imports, depth 2)
    10: * GitHub App credentials, only it can clone, and only graft can build a symbol
  • src/app/review-process.ts:L179-L183 — childReviewer (references, depth 2)
  • src/app/review-worker.ts:L67-L87 — run (calls, depth 2)
    84: // message is safe to put in the parent's log — which is where it is going.
  • src/app/review.ts:L1-L174 — review.ts (imports, depth 2)
    6: * functions the CLI calls, which is what keeps the App's comment and `graft
  • src/app/server.ts:L34-L46 — AppSeams (references, depth 2)

CLI Engine — 6 symbols in 2 files

  • src/engine.ts:L91-L101 — graph (calls, depth 1)
    92: return buildGraph(dir, {
  • src/engine.ts:L82-L84 — checkGraph (calls, depth 1)
    82: checkGraph(dir: string): Promise<GraphCheckResult> {
  • src/cli.ts:L168-L178 — refreshBefore (calls, depth 2)
  • src/cli.ts:L135-L144 — warnUnsupportedExtensions (calls, depth 2)
  • src/cli.ts:L1-L1407 — cli.ts (imports, depth 2)
    3: * `graft` CLI. Commands: build, ask, check, viz, mcp, callers, skeleton, grep,
  • src/engine.ts:L1-L161 — engine.ts (imports, depth 2)
    21: import { buildGraph, type GraphBuildOptions, type GraphBuildResult } from "./graph/build.js";

MCP Tools — 4 symbols in 1 file

  • src/mcp/tools.ts:L216-L244 — callTool (calls, depth 2)
    242: return { text: err instanceof Error ? err.message : String(err), isError: true };
  • src/mcp/tools.ts:L247-L329 — callSingleTool (calls, depth 2)
    272: const g = await engine.checkGraph(root);
  • src/mcp/tools.ts:L153-L199 — callWorkspaceTool (calls, depth 2)
  • src/mcp/tools.ts:L1-L330 — tools.ts (imports, depth 2)
    36: const NO_GRAPH = 'no graph found — run `graft build` first';

Language Enrichment — 2 symbols in 1 file

  • src/graph/lsp/enrich.ts:L1-L141 — enrich.ts (imports, depth 1)
    2: * Opt-in LSP enrichment (`graft build --lsp`): add compiler-grade call edges the
  • src/graph/lsp/enrich.ts:L25-L25 — langOf (calls, depth 2)

Viewer Build — 1 symbol in 1 file

  • scripts/build-viewer.mjs:L1-L45 — build-viewer.mjs (calls, depth 2)
    4: * so `graft viz` needs no install or build step at runtime.

Context Build — 1 symbol in 1 file

  • src/context/build.ts:L1-L468 — build.ts (imports, depth 2)
    67: * whitelist recorded in the graph fingerprint (mirrors `checkGraph`). */

Sync Execution — 1 symbol in 1 file

  • src/claude/sync-run.ts:L19-L33 — runSync (calls, depth 2)
Test signal per changed area — 1 ⚠

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • Graph Analysis — 2 of 5 reached · 34 test files reach it, none changed here
    • not reached: warnUnavailable, entryFor, depthExtensions
39 test suites also reference this code

48 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/ask-index.test.ts
  • test/ask.test.ts
  • test/container-extract.test.ts
  • test/context-only-dir.test.ts
  • test/context.test.ts
  • test/covers.test.ts
  • test/generic-extract.test.ts
  • test/generic-node24-probe.ts
  • test/graph-cross-language.test.ts
  • test/graph-enrich-pending.test.ts
  • test/graph-go.test.ts
  • test/graph-incremental.test.ts
  • test/graph-invariants.test.ts
  • test/graph-java.test.ts
  • test/graph-languages.test.ts
  • test/graph-php.test.ts
  • test/graph-posix-paths.test.ts
  • test/graph-python.test.ts
  • test/graph-r-classes.test.ts
  • test/graph-r-phase3.test.ts
  • …19 more

graft blast · origin/main...HEAD · depth 2 · 6 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

Ercaner1988 and others added 2 commits September 10, 2026 21:21
extract.ts imported all nine depth-tier grammars at the top of the module.
They are native (node-gyp) modules, so any one of them failing to load took
the whole CLI down at import time, before argv was read — `--version` and
`--help` died exactly like a build, with a node-gyp-build stack trace that
never says "graft". On the machine in trailhq#323 that grammar is tree-sitter-kotlin,
which ships no prebuilds at all and so cannot load without a C toolchain.

Load them through createRequire instead and keep what loads. The extension
table is filtered by that, which is all the rest of the pipeline needs: an
unavailable language stops claiming its files, and they take the paths a
language graft has no grammar for takes today — the breadth tier where a
generic row claims the extension, unindexed where none does.

Warned once per language, the first time a file it would have claimed comes
past, so a repo with no Kotlin in it stays quiet and a Kotlin repo never
indexes short in silence.

The test breaks a grammar both ways a module can be reached, `import` and
`require`, so it still fails if the static imports ever come back.

Refs trailhq#323

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…r is gone

The previous commit keeps the CLI alive when a native grammar will not load,
but a language with no breadth-tier row then indexes nothing at all — and the
machine in trailhq#323 is one where tree-sitter-kotlin can never load, because the
package ships no prebuilds and there is no compiler to build one. Starting and
indexing no Kotlin is not much of an answer on a Kotlin repo.

tree-sitter-wasm ships a kotlin grammar and queries/kotlin.scm has been in the
tree unused since Kotlin was promoted to the depth tier, so the fallback is one
registry row. A row whose extension the depth tier also claims is unreachable
while the native grammar loads — `.java` has been exactly that all along, which
is why the "must not collide" note needed rewriting rather than an exception.

Both warmGenericGrammars calls now apply the tier precedence their own parse
loops apply, so a fallback row cannot warm a WASM grammar the run will never
call. That was already true of `.java`; a second row would have doubled it.

Measured on a three-file repo (.ts/.kt/.rs) with tree-sitter-kotlin broken:
6 nodes [rust, typescript] before this commit, 11 nodes [kotlin, rust,
typescript] after. With the grammar healthy the graph is identical either way.

Refs trailhq#323

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant