fix(graph): re-parse a cached entry that holds a parse error - #317
fix(graph): re-parse a cached entry that holds a parse error#317datrixlab wants to merge 1 commit into
Conversation
A grammar abort is a process-global, run-dependent failure, but the extraction cache stores it as a per-file entry keyed on content hash and replays it on every later build. An untouched file then stays broken forever, and a whole language can drop out of a graph that still exits 0 with a success line. Let an entry carrying an error fall through to the parse path instead of being replayed. The entry is still persisted, so the freshness probe does not report the file as new on every query; it just never stands in for a result again. Re-parsing costs only files that produced no nodes, and a run that succeeds repairs the entry. Fixes trailhq#312
🌱 graft blast radius1 area changed → 6 areas can be affected. 12 dependent symbols, depth 2. flowchart TB
A0(("Pull Request Review<br/>4 symbols"))
A1(("Graph Freshness<br/>3 symbols"))
A2(("Graph Initialization<br/>2 symbols"))
A3(("Viewer Build<br/>1 symbol"))
A4(("Tool Invocation<br/>1 symbol"))
AX(("1 smaller area<br/>1 symbol"))
classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
class A0,A1,A2,A3,A4 reached;
classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
class AX tail;
Who knows this code — 3 people across 7 areas
Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no All 12 dependent symbols, grouped by areaPull Request Review — 4 symbols in 4 files
Graph Freshness — 3 symbols in 2 files
Graph Initialization — 2 symbols in 2 files
Viewer Build — 1 symbol in 1 file
Tool Invocation — 1 symbol in 1 file
Synchronous Execution — 1 symbol in 1 file
Test signal per changed area — 1 ✓Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.
32 test suites also reference this code40 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.
Open the interactive graph → — click an area to see its dependent symbols at file:line. |
- generic-extract.test.ts: the union merge of trailhq#196, trailhq#200 and trailhq#201 spliced their appended tests into one another and left the file unterminated. Rebuilt as trailhq#196's file plus trailhq#200's and trailhq#201's appended blocks verbatim. - The trailhq#139 test asserted that a cached parse failure is replayed without a re-parse; trailhq#317 (fixes trailhq#312) deliberately re-parses it instead and did not update the test. It now asserts the re-parse and that the error still comes back. - The trailhq#319 symlink test (trailhq#328) needed symlink privilege on Windows. Directory links are junctions there, as test/graph-root and test/ingest-fs already do, and the file-link case is skipped with a diagnostic when it cannot be created. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What
An extraction-cache entry that carries an
erroris no longer replayed.buildGraphre-parses it instead, so a failure that belonged to the run cannot outlive the run.Fixes #312
Why this option
@Ercaner1988 listed three. This is option 2 — persist the entry, never let it stand in for a result — and the other two both have a hole:
graft checkreports drift that a rebuild can't clear. Persisting without theerrorfield is worse still: the file would then replay as successfully parsed with zero nodes, and the error disappears instead of merely being wrong.Aborted()andRuntimeError: memory access out of boundsagainst messages that come out of wasm, which is a list to keep up to date and a silent wrong answer whenever it's out of date.Re-parsing is cheap by construction: the only entries it re-parses are the ones that produced no nodes. On a healthy repo that's zero files.
The self-healing property matters as much as the fix. A cache poisoned by a run that has since been fixed — a smaller workspace, a
--only-dir, more memory — repairs itself on the next ordinarygraft build. Today it needs someone to know that--no-reuseexists, which requires first noticing a graph that reports✓and exit 0.The docstring already promised this
ExtractEntry.errorinsrc/graph/extract-cache.tssaid:That is the invariant the whole file is built on — its header states it twice ("an incremental build must produce a byte-identical
wiring.jsonto a cold one"). A replayed abort breaks it: the cold build parseskanit.rsand gets 11 nodes, the incremental one replays a failure and gets none. The comment has been updated to say what the code now does.The same repo already gets this right one line up
A file that can't be read recovers by itself.
an unreadable file is still recorded, so it can't look new on every probechmods a file to000, builds twice, chmods it back, and asserts the next ordinary build indexes it — no--no-reuseneeded. That works because a read error is recorded withhash: "", which can never match, so the entry is retried every build.A file that can't be parsed gets a real hash, so it is never retried. Same cache, same kind of entry, opposite lifetime — and the parse case is the one that can fail for a reason that has nothing to do with the file.
Test
One test in
test/graph-incremental.test.ts, which is where that invariant lives.It builds cold, then poisons the memo the way a grammar abort does — the entry keeps the file's real content hash, so nothing about the file looks stale — and rebuilds normally. It asserts the three things the report distinguishes:
wiring.jsonis byte-identical to the cold build (the nodes come back)languagesstill contains the language (the symptom:[python]vs[python, rust])errorsis empty (the stale message is not re-reported), and the entry on disk no longer carries one, so the next build is cheap againTwo-way: reverting only
src/graph/build.tsfails it on the first assertion.Not in scope
The report also suggests warning when a language that has files in the index produces zero nodes, and surfacing it in
graft check. That's a real gap and it stays open with #312 — but it's a new signal with its own false-positive question (a language whose files are all.d.ts-shaped, say), not part of making the cache honest. Happy to follow up separately if you want it.