scope uniqueness constraints to affected objects by ajtmccarty · Pull Request #10019 · opsmill/infrahub

ajtmccarty · 2026-07-23T01:46:15Z

Why

Uniqueness validation scanned the entire population of a kind on every data change. Merging, rebasing, or proposing a change that touched a handful of nodes of a large kind still ran a population-wide scan — slow on big kinds (hundreds of thousands of nodes) and it shipped large duplicate result sets back to Python.

Goal: scope a data-triggered uniqueness check to only the nodes the diff actually affects, so the work is proportional to the size of the change rather than the size of the kind.

Non-goals: this does not change node-save-time uniqueness (grouped_uniqueness), does not touch other constraint validators, and does not change the uniqueness error surfaced to users.

Note: uniqueness constraints do not support attribute of relationships (e.g. "device__name"). some parts of the code support this kind of uniqueness constraint, but not all of it and it is still blocked when creating/updating a schema.

Planned follow-ups

refactor MergeConstraintValidator construction. schema_branch should probably be an argument for this class and its dependents, not a dependency
check on integration/functional-level testing for uniqueness constraint validation and add more if necessary
issue for supporting attributes on relationships in uniqueness constraints

Part of IFC-2796 (epic IFC-2706).

What changed

Behavioral

Data-triggered uniqueness validation (merge, rebase, proposed-change integrity) now validates only the changed/affected nodes via a batched, index-anchored targeted query instead of a full-population scan.
A uniqueness check is scoped to exactly the nodes whose changed field participates in the constraint — changing one node's unique field no longer drags in every other changed node of the kind.
A newly added or broadened constraint (schema-diff origin, with no affected-node set) still runs the full-population scan, as before.
Cross-kind constraints that read a peer's attribute (e.g. owner__name) fall back to full-population validation, which supports them; the targeted query does not. Again, this type of uniqueness constraint cannot be added to schemas currently.
Uniqueness error messages are unchanged — attribute values render identically to the full-population path regardless of their type.

Implementation notes

A node_uuids carrier is threaded end to end: None means full scan, a list means validate exactly those nodes. It rides from the diff summary through the determiner, the constraint info, and the validator request into the checker.
NodeDiffFieldSummary/NodeDiffIndex now keep the correlation between each changed field and the nodes that changed it (per-field UUID maps), so scoping targets a specific field's nodes rather than every changed node of the kind.
New here (constructor-injected, wired by build_constraint_validator_determiner): UniquenessConstraintScoper decides, per kind, whether uniqueness must run and which nodes it affects (None → full scan); the ConstraintValidatorDeterminer update threads the affected set through; constraint dedup/merge helpers collapse a node's uniqueness check into its generic's where the generic already covers it.
UniquenessChecker now runs the targeted query for a scoped set, and falls back to the full-population scan when a constraint reads a peer attribute.

What stayed the same

The full-population path and the node-save-time uniqueness checker (grouped_uniqueness) are untouched.
No GraphQL/API contract changes, no database schema or migration changes.

Suggested review order

core/diff/model/path.py, core/diff/query/field_summary.py, core/validators/node_diff_index.py — how per-field changed-node UUIDs are captured.
core/validators/uniqueness/scope.py + core/validators/determiner.py — how the affected-node set (or None) is decided.
core/validators/uniqueness/checker.py — running the targeted query for a scoped set vs the full-population fallback.
proposed_change/tasks.py, merge/constraints.py, branch/tasks.py — the three call sites that thread the affected set through.
Tests.

How to review

Focus on the scope decision (UniquenessConstraintScoper._compute_scope) and the checker's targeted-vs-full-population branch (UniquenessChecker.check / _supports_targeted) — those govern correctness. Mechanical: the attribute_node_uuids conversions across the test files.

Impact & rollout

Backward compatibility: no breaking changes; uniqueness error messages are identical to before.
Performance: the headline win. On a large local dataset (≈500k-node kind) the full-population scan ran 8+ minutes before OOM-killing Neo4j; the targeted query validates ~9,400 changed nodes of that kind in ~12s and ~23,500 in ~29s, and all implicated kinds complete in tens of seconds. Work is now bounded by the change, not the population.
Config/env changes: none.
Deployment notes: safe to deploy; no coordinated release or migration required.

Checklist

Tests added/updated
Changelog entry added
External docs updated (no user-facing surface change)
Internal .md docs updated (internal knowledge and AI code tools knowledge)
I have reviewed AI generated content

Summary by cubic

Scope node-level uniqueness validation to only the nodes that changed each unique field (and their dependents), using diff-derived IDs and batched, index-backed queries. Full scans now run only when a constraint is added/broadened or when peer-attribute elements are present, cutting worst-case merge and schema-update time (IFC-2796).

Bug Fixes
- Fall back to full-population validation for peer-attribute paths (e.g., owner__name) unsupported by the targeted query.
- Reset scoper cache on determiner re-init to prevent stale scopes between diffs.
- Stringify targeted values in error paths to match full-scan messages.
Refactors
- Batch checks per constraint group via TargetedUniquenessValidationQuery, paging over changed nodes with one read-only session per change.
- UniquenessChecker reads branch/schema from the validator request; dependency builder and m018 migration updated.
- Carry node_uuids through SchemaUpdateConstraintInfo and validator requests across proposed-change, merge, rebase, and migrations; merge/dedup constraints and standardize on ConstraintIdentifier.NODE_UNIQUENESS_CONSTRAINTS_UPDATE.

^{Written for commit 7a09ce8. Summary will update on new commits.}

…niquenessChecker Replace the per-node targeted uniqueness path with a single batched TargetedUniquenessValidationQuery per constraint group, paged over the changed nodes. Take branch and schema_branch from the validator request rather than the constructor, and open one read-only session per change instead of per query. Update the dependency builder and the m018 migration for the new signatures, and switch the node-scoped tests to exact violation-set assertions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would auto-approve. Optimizes uniqueness validation by scoping checks to only the nodes affected by data diffs, reducing unnecessary full scans. All changes are implementation-level; cubic reviewed and found no issues.

Re-trigger cubic}

codspeed-hq · 2026-07-23T01:54:54Z

Merging this PR will improve performance by 30.86%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 10 untouched benchmarks

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	`test_base_schema_duplicate_CoreProposedChange`	2.2 ms	1.5 ms	+49.29%
⚡	`test_schemabranch_duplicate`	7.7 ms	6.7 ms	+14.7%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing ifc-2796-node-scope-uniqueness-2 (7a09ce8) with stable (c993b19)¹}

No successful run was found on ifc-2796-node-scope-uniqueness-full (517c77a) during the generation of this report, so stable (c993b19) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

cubic-dev-ai

All reported issues were addressed

_{Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic}

ajtmccarty · 2026-07-23T03:03:39Z

        self, candidate_schema: SchemaBranch, schema_diff_constraints: list[SchemaUpdateConstraintInfo]
    ) -> MergeConstraintValidationResult:
-        determiner = ConstraintValidatorDeterminer(schema_branch=candidate_schema)
+        determiner = build_constraint_validator_determiner(


this should be built as part of construction, but the dependencies require schema_branch which is not ready until runtime. I think the right solution is to move schema_branch from dependency to runtime argument for all of these, but I'm going to do that separately

The constraint result model gained a node_uuids field, so model_dump() now emits 'node_uuids': None. Update the two schema-validate tests whose hardcoded expected dicts predate the field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The targeted validation path emitted attribute values with their native graph type, so an integer attribute rendered as value=8 while the full-population path rendered '8'. Stringify the value when building the data path so error messages are identical across both paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The determiner re-initializes its node-diff index on each get_constraints call, but the uniqueness scoper's per-kind scope cache was never cleared, so a reused determiner could return scopes computed from a previous diff. Add a reset() on the scoper and call it alongside index initialization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The batched targeted query cannot read an attribute of a related peer (a constraint element such as owner__name) and raises on one. Detect such constraints and validate the whole population for that schema instead of scoping to the changed nodes, since the full-population path supports them. Removes the xfail markers on the two peer-attribute tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The diff field summary collapsed all changed node uuids per kind, so changing one node's unique field validated every changed node of that kind. Track uuids per changed field through NodeDiffFieldSummary, the diff query, NodeDiffIndex, and the uniqueness scoper, so a uniqueness check is scoped to only the nodes whose changed field participates in it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 14 files (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would auto-approve. Scopes uniqueness validation to affected nodes per diff, with fallback to full scan for unsupported paths; a bounded, clearly beneficial performance optimization. Implementation verified by Cubic.

Re-trigger cubic}

ajtmccarty and others added 11 commits July 22, 2026 16:30

retrieve UUIds in NodeDiffFieldSummary

81908cb

add UniquenessConstraintScoper

f096473

update ConstraintValidatorDeterminer with new components

0b49394

add uniqueness constraints to ConstraintIdentifier enum

87a8d52

add uniqueness constraints deduplicator

7563e6f

add ConstraintInfoMerger

5cfaad6

pass node_uuids for constraints through PC pipeline

7bd6ceb

build and pass constraint classes through rebase flow

79eb72c

update PC pipeline tests

5bf5e60

update MergeConstraintValidator

57f47ca

github-actions Bot added the group/backend Issue related to the backend (API Server, Git Agent) label Jul 23, 2026

formatting

77b106c

cubic-dev-ai Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread backend/infrahub/proposed_change/tasks.py Outdated

Comment thread backend/infrahub/core/validators/determiner.py

Comment thread backend/infrahub/core/diff/query/field_summary.py Outdated

ajtmccarty commented Jul 23, 2026

View reviewed changes

ajtmccarty and others added 5 commits July 22, 2026 20:57

cubic-dev-ai Bot reviewed Jul 23, 2026

View reviewed changes

ajtmccarty marked this pull request as ready for review July 23, 2026 05:17

ajtmccarty requested a review from a team as a code owner July 23, 2026 05:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

scope uniqueness constraints to affected objects#10019

scope uniqueness constraints to affected objects#10019
ajtmccarty wants to merge 17 commits into
ifc-2796-node-scope-uniqueness-fullfrom
ifc-2796-node-scope-uniqueness-2

ajtmccarty commented Jul 23, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ajtmccarty Jul 23, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

ajtmccarty commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Suggested review order

How to review

Impact & rollout

Checklist

Summary by cubic

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 30.86%

Performance Changes

Footnotes

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ajtmccarty Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ajtmccarty commented Jul 23, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading