Skip to content

rfc: resolve type references to StableTypeId below the compiler #1079

Open

Description

@ragnorc

Problem

Below the compiler, OmniGraph refers to types by name.

  • The query IR carries names: IROp::NodeScan { type_name } and IROp::Expand { src_type, dst_type } (crates/omnigraph-compiler/src/ir/mod.rs).
  • The planner and engine build table keys from those names, format!("node:{type_name}") and format!("edge:{edge_type}") (crates/omnigraph/src/engine/scan.rs, engine/operators/scan/input.rs, engine/graph.rs, crates/omnigraph-planner/src/source.rs).
  • A snapshot's tables are a HashMap<String, DatasetEntry> keyed by table key (crates/omnigraph-catalog/src/lib.rs).

Within one snapshot this is correct, because each (kind, name) pair maps to exactly one StableTypeId. Three upcoming changes make it costly:

  • Two vocabularies. Integer node identity makes the engine carry NodeRef { type_id: StableTypeId, id } through traversal frontiers, dedup, hydration, cascades and the graph index. A plan that still names types by string must translate names to ids at every boundary. The polymorphic-types prototype is direct evidence of the risk: two identity rules chosen per site produced four defects (findings P1, P2, P4 and P6 in that RFC).
  • Routine renames. Schema namespaces turn moving a type into a namespace into a rename, so a schema reorganization becomes many renames in one apply. Derived state keyed by name pays for each one:
    • the graph-index topology cache stamps type and edge names (graph_index/persist.rs) and rebuilds on a mismatch;
    • name-keyed caches miss after a rename;
    • change-feed cursors hash their type-name filter.
  • Kind ambiguity. Node, edge and interface names are unique only per kind, so invariant 9 needs a rule that execution never re-picks between a node and an edge of the same name. StableTypeIds come from one allocator across kinds, so the case cannot arise with ids.

Proposed change

Resolve every type reference once, in the compiler, and key everything below it by id.

  • IR. Type references in the query IR become TypeRefIR { type_id, type_name }, the pair SchemaIR already uses (catalog/schema_ir.rs). The id is the authority; the name is a label for explain output and errors.
  • Planner and engine.
    • Table lookups, plan assumptions and pins, caches and NodeRef construction use the id.
    • A snapshot indexes its registrations by stable_table_id, which __manifest rows already carry.
    • The catalog gains id-keyed maps; its id lookups are linear scans by name today.
  • Serialized plans. The bound plan's serialized form records ids. Its envelope version bumps, and explain keeps rendering names so GQT plan claims stay readable.
  • The frozen v1 engine. Adding type_id beside type_name keeps it compiling: it matches IR nodes with .. patterns. Replacing the name outright would break its pinned sources.
  • Not changed:
    • query text and every wire field, which keep canonical names;
    • the persisted table_key alias in __manifest;
    • stored-query text, which stored-query pinning by id already covers in the polymorphic-types RFC.
  • Sequencing. Ship it with, or directly after, integer node identity and before polymorphic endpoints, so the engine never runs with both vocabularies. Schema namespaces do not depend on it.

This likely needs an RFC: it changes the serialized plan format and the IR contract.

Affected areas

Alternatives considered

  • Keep names below the compiler. Correct within one snapshot, but it needs a name-to-id translation at every boundary once integer node identity lands, and it rebuilds or invalidates derived state on every rename.
  • Replace names with ids entirely. It loses readable explain output and diagnostics, and it breaks the frozen v1 engine's sources.
  • Translate only at the storage boundary. This is today's shape plus a translation layer. It is the dual-vocabulary design the prototype found error-prone.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature proposalneeds-triageAwaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions