update to icebug-disk v1

[aheev]

No description provided.

[aheev]

@adsharma

• do we need generators(generate/*.py) at all?
• should we follow a more general approach to input tables? something like a config file with (ofc we need to tidy it up)

"node_table_files" : [
"nt1" : "node_table1.parquet",
"nt2" :  "node_table2.parquet",
 ... ]
"rel_table_files" : [
                             "rel_table1" : ["from_table_name", "to_table_name", "from_col", "to_col", "rel_table.parquet"],
                             "rel_table2" : [...],
 ... 
]

so that users can just bring their parquet files without modifying source tables

[adsharma]

ok to get rid of generate/*.py in a separate PR

Leaning towards convention over configuration ruby style. If someone brings a directory full of parquet files we could add some scripts that rename/generate the schema.cypher instead of requiring them to read docs and write such a config file.

[adsharma]

What was the motivation for removing the node table entry from schema.cypher? Because they're not explicitly used?
Without the node table, the rel table entry becomes semantically invalid cypher.

The main reason why they're there is to help the icebug-format cli to understand acceptable node types for rel tables. Also important for multi rel tables IIRC.

[aheev]

What was the motivation for removing the node table entry from schema.cypher? Because they're not explicitly used? Without the node table, the rel table entry becomes semantically invalid cypher.

The main reason why they're there is to help the icebug-format cli to understand acceptable node types for rel tables. Also important for multi rel tables IIRC.

CLI is not treating it as cypher

content = schema_path.read_text()

# Parse REL TABLE definitions: CREATE REL TABLE Follows(FROM User TO User, ...);
# Also handles backtick-quoted identifiers: CREATE REL TABLE `edges` (FROM `nodes` TO `nodes`, ...)
rel_pattern = (
  r"CREATE\s+REL\s+TABLE\s+`?(\w+)`?\s*\(\s*FROM\s+`?(\w+)`?\s+TO\s+`?(\w+)`?"
)
for match in re.finditer(rel_pattern, content, re.IGNORECASE):
  edge_name = match.group(1).lower()
  from_node = match.group(2).lower()
  to_node = match.group(3).lower()
  edge_relationships[edge_name] = (from_node, to_node)

[adsharma]

That was a practical compromise to avoid linking a ladybug cypher parser into icebug-format cli.

Generating output schema.cypher without the node table definition will require parsing parquet and converting types.

[adsharma]

Looks good. Two minor nits:

• ladybug >= 0.17.0 won't work until its released. 0.17.0.dev20260522 < 0.17.0 in some versioning schemes
• examples/karate/duckdb/icebug-disk could move to examples/karate/icebug-disk/

Ideal invocation:

uv run icebug-format.py --source-db kg_history.duckdb --schema schema.cypher

and it could infer:

 --output-db kg_history_csr.duckdb --csr-table kg_history

from --source-db. Remove the hard coded defaults

  --source-db SOURCE_DB
                       Source DuckDB database path (default: karate_random.duckdb)
 --output-db OUTPUT_DB
                       Output DuckDB database path (default: csr_graph.db)
 --csr-table CSR_TABLE
                       Table name prefix for CSR data (default: csr_graph)

[aheev]

Looks good. Two minor nits:

• ladybug >= 0.17.0 won't work until its released. 0.17.0.dev20260522 < 0.17.0 in some versioning schemes

are we going to release icebug-format before ladybug?

[aheev]

do we need ladybug in dependencies at all?

[adsharma]

Good point. Ok to remove.

[aheev]

Good point. Ok to remove.

done