The finding this comes from
On a 116-file repo, graphify's hyperedge output surfaced something no single file states: six documentation nodes across five files (README, install guide, update guide, a skill reference, a setup guide) all asserting one principle — nothing gets installed, logged in, or removed without explicit user consent.
I then checked each claim against the code. Five were enforced by real code with a test beside them. The sixth was not: a preservation guard whose flag defaulted the wrong way, making the guard dead code and the "preserved" branch unreachable, so the documented-as-safe operation silently deleted a user's customized files. The docs promised it; nothing enforced it; and critically, the five enforced claims each had a test asserting the behavior, and the broken one had tests only for a neighboring code path.
That last sentence is the whole signal, and the graph already contains everything needed to compute it — it just cannot express it.
Request
Make "this claim is backed by a test" a first-class relation, e.g. locked_by, so the interesting query becomes one traversal instead of five subagents:
Which documentation claims reach code but never reach a test?
Concretely:
- Emit
locked_by edges from a code node to the test node(s) that exercise it. The AST layer can already see test files (they were 702 of my 1191 AST nodes); what is missing is marking test→subject edges as a distinct relation rather than generic uses/calls.
- Let the semantic layer emit
rationale_for / references from a doc node to code — it already can; my extraction produced exactly these.
- Add a report section or subcommand that walks doc → code → test and lists the doc claims whose transitive closure contains no test node. Something like
graphify coverage --from docs.
Why this fits graphify specifically
The project's stated value is an honest audit trail — EXTRACTED / INFERRED / AMBIGUOUS on every edge, no vector store, everything explainable. "This documented promise has no test behind it" is the same kind of claim: checkable, deterministic, and derived from structure rather than from a model's opinion. It is also the one question a knowledge graph can answer that grep genuinely cannot, because it requires crossing the doc→code→test boundary that no single file spans.
The god-node and betweenness questions in suggest_questions today point at structure. This points at risk, and in my case it pointed straight at a real bug.
Related: #2293 (query scores labels only, never the rationale attribute, so "why" questions miss) — the rationale attribute is part of the same underexploited doc-side signal.
The finding this comes from
On a 116-file repo, graphify's hyperedge output surfaced something no single file states: six documentation nodes across five files (README, install guide, update guide, a skill reference, a setup guide) all asserting one principle — nothing gets installed, logged in, or removed without explicit user consent.
I then checked each claim against the code. Five were enforced by real code with a test beside them. The sixth was not: a preservation guard whose flag defaulted the wrong way, making the guard dead code and the "preserved" branch unreachable, so the documented-as-safe operation silently deleted a user's customized files. The docs promised it; nothing enforced it; and critically, the five enforced claims each had a test asserting the behavior, and the broken one had tests only for a neighboring code path.
That last sentence is the whole signal, and the graph already contains everything needed to compute it — it just cannot express it.
Request
Make "this claim is backed by a test" a first-class relation, e.g.
locked_by, so the interesting query becomes one traversal instead of five subagents:Concretely:
locked_byedges from a code node to the test node(s) that exercise it. The AST layer can already see test files (they were 702 of my 1191 AST nodes); what is missing is marking test→subject edges as a distinct relation rather than genericuses/calls.rationale_for/referencesfrom a doc node to code — it already can; my extraction produced exactly these.graphify coverage --from docs.Why this fits graphify specifically
The project's stated value is an honest audit trail — EXTRACTED / INFERRED / AMBIGUOUS on every edge, no vector store, everything explainable. "This documented promise has no test behind it" is the same kind of claim: checkable, deterministic, and derived from structure rather than from a model's opinion. It is also the one question a knowledge graph can answer that grep genuinely cannot, because it requires crossing the doc→code→test boundary that no single file spans.
The god-node and betweenness questions in
suggest_questionstoday point at structure. This points at risk, and in my case it pointed straight at a real bug.Related: #2293 (query scores labels only, never the
rationaleattribute, so "why" questions miss) — the rationale attribute is part of the same underexploited doc-side signal.