Repository navigation
rfc: resolve type references to StableTypeId below the compiler #1079
Copy link
Copy link
Open
Labels
featureFeature proposalFeature proposalneeds-triageAwaiting maintainer triageAwaiting maintainer triage
Description
Problem
Below the compiler, OmniGraph refers to types by name.
- The query IR carries names:
IROp::NodeScan { type_name }andIROp::Expand { src_type, dst_type }(crates/omnigraph-compiler/src/ir/mod.rs). - The planner and engine build table keys from those names,
format!("node:{type_name}")andformat!("edge:{edge_type}")(crates/omnigraph/src/engine/scan.rs,engine/operators/scan/input.rs,engine/graph.rs,crates/omnigraph-planner/src/source.rs). - A snapshot's tables are a
HashMap<String, DatasetEntry>keyed by table key (crates/omnigraph-catalog/src/lib.rs).
Within one snapshot this is correct, because each (kind, name) pair maps to exactly one StableTypeId. Three upcoming changes make it costly:
- Two vocabularies. Integer node identity makes the engine carry
NodeRef { type_id: StableTypeId, id }through traversal frontiers, dedup, hydration, cascades and the graph index. A plan that still names types by string must translate names to ids at every boundary. The polymorphic-types prototype is direct evidence of the risk: two identity rules chosen per site produced four defects (findings P1, P2, P4 and P6 in that RFC). - Routine renames. Schema namespaces turn moving a type into a namespace into a rename, so a schema reorganization becomes many renames in one apply. Derived state keyed by name pays for each one:
- the graph-index topology cache stamps type and edge names (
graph_index/persist.rs) and rebuilds on a mismatch; - name-keyed caches miss after a rename;
- change-feed cursors hash their type-name filter.
- the graph-index topology cache stamps type and edge names (
- Kind ambiguity. Node, edge and interface names are unique only per kind, so invariant 9 needs a rule that execution never re-picks between a node and an edge of the same name.
StableTypeIds come from one allocator across kinds, so the case cannot arise with ids.
Proposed change
Resolve every type reference once, in the compiler, and key everything below it by id.
- IR. Type references in the query IR become
TypeRefIR { type_id, type_name }, the pair SchemaIR already uses (catalog/schema_ir.rs). The id is the authority; the name is a label for explain output and errors. - Planner and engine.
- Table lookups, plan assumptions and pins, caches and
NodeRefconstruction use the id. - A snapshot indexes its registrations by
stable_table_id, which__manifestrows already carry. - The catalog gains id-keyed maps; its id lookups are linear scans by name today.
- Table lookups, plan assumptions and pins, caches and
- Serialized plans. The bound plan's serialized form records ids. Its envelope version bumps, and explain keeps rendering names so GQT plan claims stay readable.
- The frozen v1 engine. Adding
type_idbesidetype_namekeeps it compiling: it matches IR nodes with..patterns. Replacing the name outright would break its pinned sources. - Not changed:
- query text and every wire field, which keep canonical names;
- the persisted
table_keyalias in__manifest; - stored-query text, which stored-query pinning by id already covers in the polymorphic-types RFC.
- Sequencing. Ship it with, or directly after, integer node identity and before polymorphic endpoints, so the engine never runs with both vocabularies. Schema namespaces do not depend on it.
This likely needs an RFC: it changes the serialized plan format and the IR contract.
Affected areas
omnigraph-compiler: the IR types and lowering.omnigraph-planner: plan sources, assumptions, the bound-plan mirrors and explain.omnigraph-engine: scans, expansion, hydration and graph-index stamps.omnigraph-catalog: snapshot indexing.- The GQT plan-claim parser.
- Related:
- rfc: polymorphic types with interfaces, unions and polymorphic edges #884, polymorphic types and integer node identity;
- rfc: schema namespaces and multi-file schemas #1015, schema namespaces and multi-file schemas;
- rfc: granular permissioning #951, actor views (future per-type rules bind to ids).
Alternatives considered
- Keep names below the compiler. Correct within one snapshot, but it needs a name-to-id translation at every boundary once integer node identity lands, and it rebuilds or invalidates derived state on every rename.
- Replace names with ids entirely. It loses readable explain output and diagnostics, and it breaks the frozen v1 engine's sources.
- Translate only at the storage boundary. This is today's shape plus a translation layer. It is the dual-vocabulary design the prototype found error-prone.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
featureFeature proposalFeature proposalneeds-triageAwaiting maintainer triageAwaiting maintainer triage
Activity