Summary
prune_sources matches nodes on their source_file. Extractors create per-file nodes for imported external symbols — Path from pathlib, Counter from collections, Graph from networkx — and those carry no source_file, because they are defined outside the corpus. Every edge they have points at symbols in the one file they were created for.
So when that file is deleted or excluded, the prune removes the file's own nodes and leaves these behind: degree 0, named after a file the corpus no longer contains, and unreachable by any future prune, since there is no source_file to match on.
Reproduce
graphify's own package, 2287 nodes / 5396 edges, pruning one real file:
after pruning 'graphify/callflow_html.py': 2147 nodes, 5120 edges
[graphify] Pruned 137 node(s) from 1 deleted or excluded source file(s).
orphaned (degree-0) nodes now: 3 (new: 2)
'graphify_callflow_html_py_path' label='Path' source_file=''
'graphify_callflow_html_py_counter' label='Counter' source_file=''
nodes still named after the pruned file: 2
'graphify_callflow_html_py_path' label='Path' source_file='' deg=0
'graphify_callflow_html_py_counter' label='Counter' source_file='' deg=0
137 of the file's 139 nodes go; 2 stay forever.
This corpus has 97 source-less nodes across 81 files, so roughly one to two per file are candidates. Excluding a directory of any size leaves a proportional residue, and each subsequent prune adds more — they cannot be cleaned up by re-running anything.
Why it matters
Nothing warns. The prune line reports 137 node(s) and looks complete.
Suggested fix
After the prune removes nodes and edges, sweep the source-less nodes it just isolated:
before = {n for n, d in G.nodes(data=True)
if not d.get("source_file") and G.degree(n) == 0}
# ... existing prune ...
newly_isolated = [n for n, d in G.nodes(data=True)
if not d.get("source_file") and G.degree(n) == 0
and n not in before]
G.remove_nodes_from(newly_isolated)
Scoping it to nodes this prune isolated matters: a source-less node that was already isolated before the prune is someone else's business, and this graph had one such node that should not be swept up by a fix for a different problem.
A node with no source_file and no edges carries no information — it cannot be located in the corpus and it connects nothing — so removing it loses nothing, whereas keeping it inflates every count that reads the graph.
I have this working with tests and will open a PR shortly.
Environment
|
|
| graphify |
v8 @ 4fca621 (0.9.44) |
| Python |
3.12 |
Summary
prune_sourcesmatches nodes on theirsource_file. Extractors create per-file nodes for imported external symbols —Pathfrom pathlib,Counterfrom collections,Graphfrom networkx — and those carry nosource_file, because they are defined outside the corpus. Every edge they have points at symbols in the one file they were created for.So when that file is deleted or excluded, the prune removes the file's own nodes and leaves these behind: degree 0, named after a file the corpus no longer contains, and unreachable by any future prune, since there is no
source_fileto match on.Reproduce
graphify's own package, 2287 nodes / 5396 edges, pruning one real file:
137 of the file's 139 nodes go; 2 stay forever.
This corpus has 97 source-less nodes across 81 files, so roughly one to two per file are candidates. Excluding a directory of any size leaves a proportional residue, and each subsequent prune adds more — they cannot be cleaned up by re-running anything.
Why it matters
GRAPH_REPORT.mdopens withN nodes · M edges · K communities, computed off the graph, so the totals drift upward from reality with every prune.to_obsidianwrites one note per node, and the wiki andgraph.jsoncarry them too, so a vault keeps aPath.mdfor a file that is no longer in the corpus.deleted_files, an exclusion byexcluded_files(save_manifest retains scan-excluded files, causing permanent false deleted_files #1908) or_stale_graph_sources(0.9.16: semantic cache emits .graphifyignore'd files as graph nodes #1909). A node with nosource_fileis outside all three.Nothing warns. The prune line reports
137 node(s)and looks complete.Suggested fix
After the prune removes nodes and edges, sweep the source-less nodes it just isolated:
Scoping it to nodes this prune isolated matters: a source-less node that was already isolated before the prune is someone else's business, and this graph had one such node that should not be swept up by a fix for a different problem.
A node with no
source_fileand no edges carries no information — it cannot be located in the corpus and it connects nothing — so removing it loses nothing, whereas keeping it inflates every count that reads the graph.I have this working with tests and will open a PR shortly.
Environment
v8@4fca621(0.9.44)