Fix/wasm parser lifetime - #340
Conversation
…ed runtime extractGeneric and parseWasm created a Parser per file and never deleted it or the Tree. Both live in the WASM heap (2 GB cap, no finalizer), so a 65k-file C repo aborted after ~12k files and every later parse failed with Aborted(). Reuse one parser per grammar and delete every tree. After a real abort, fail every later parse with one message instead of one per file. parseWasm now returns the tree for the container tier to free, and rethrows instead of caching a symbol-less file as a success.
A cached error was replayed until the file's bytes changed, so one abort pinned 52k files as failed. Skip the cache hit for an errored entry. The fingerprint keeps its hash, so a refresh does not rebuild just to retry it.
🌱 graft blast radius2 areas changed → 7 areas can be affected. 20 dependent symbols, depth 2. flowchart TB
A0(("Graph Workspace Orchestration<br/>7 symbols"))
A1(("Pull Request Review<br/>6 symbols"))
A2(("Engine Graph<br/>2 symbols"))
A3(("Context Building<br/>2 symbols"))
A4(("Viewer Build Script<br/>1 symbol"))
AX(("2 smaller areas<br/>2 symbols"))
classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
class A0,A1,A2,A3,A4 reached;
classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
class AX tail;
Who knows this code — 4 people across 9 areas
Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no All 20 dependent symbols, grouped by areaGraph Workspace Orchestration — 7 symbols in 5 files
Pull Request Review — 6 symbols in 4 files
Engine Graph — 2 symbols in 1 file
Context Building — 2 symbols in 1 file
Viewer Build Script — 1 symbol in 1 file
MCP Tools — 1 symbol in 1 file
Claude Sync Execution — 1 symbol in 1 file
Test signal per changed area — 1 ✓ · 1 –Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.
35 test suites also reference this code47 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.
Open the interactive graph → — click an area to see its dependent symbols at file:line. |
Was working with
grafton a large codebase (~65k files) to stress-test, noticed some failure doing so -> attempted a fix pass to move it towards functional. Feel free to leave inputs or modify/reject!I have a subsequent PR that improves the
graft buildmemory tolerance + performance using a streaming worker pool (~2.5x speedup on that codebase) that I can open afterwards.Problem
extractGenericandparseWasmcreated aParserper file and never deleted it or theTree. Both live in the WASM heap (2 GB cap, no finalizer), so a 65k-file C repo aborted after ~12k files and every subsequent parse failed withAborted()after a few minutes.Solution
Reuse one parser per grammar and delete every tree. After a real abort, fail every later parse with one message instead of one per file.
parseWasmnow returns the tree for the container tier to free, and rethrows instead of caching a symbol-less file as a success.