perf(computed-attribute): batch Python transform recompute

[saltas888]

Why

When a change touches data that Python-transform computed attributes read, every affected node was recomputed through its own full path: a git repository init per node, one GraphQL update mutation per node, and every mutation's events re-triggered target queries and further recompute dispatches. One device-type rename could keep an instance churning for minutes, and at scale it takes the instance down. This is the "merge leaves the instance unusable" pain reported by large deployments.

Goal: absorb a wide fan-out as one bounded pass per chunk, with the same final values and the same events for real changes.

Non-goals: scoping the fan-out itself (every node of the kind vs only the affected ones) and a recovery surface for failed nodes, both are follow-ups.

The improvements

The scenario for all of them is a Python computed attribute whose source data changes with many reader nodes, most visibly a branch merge or rebase, but the same path fires on a direct data change or a transform update that fans out to a whole kind.

1. One git repository init per batch instead of one per node. The recompute flow receives a chunk of node ids (up to ~250) and initializes the transform's repository once, before this every single node paid its own repository init.
2. One bulk write instead of one GraphQL mutation per node. Values persist through the shared bulk recompute writer in bounded transactions (100 nodes per transaction), no per-node InfrahubUpdateComputedAttribute mutation, no per-node client round-trip.
3. Unchanged values emit no events, which removes the echo storm. Before, every per-node mutation emitted NodeUpdated events even when the recomputed value was identical to the stored one, and those events re-triggered the query-computed-attribute-transform-targets and graphql-query-group-update automations, which dispatched further recomputes, a self-sustaining storm. Now a node whose save produces no effective change emits nothing and dispatches nothing, so a wave of recomputes that mostly lands on already-correct values simply stops.
4. One bad node no longer hurts its siblings. A transform that raises or returns a non-string for one node is skipped with its previous value intact and a logged reason, the rest of the batch persists, and the flow ends with a submitted/written/skipped summary line.

Plus one visibility fix: the branch tag is set on each submission at creation so the recompute runs stay visible in branch-filtered task views, tags added mid-run can be dropped by later in-flow tag updates.

How we know it is an improvement

Measured, not inferred. The TestMergeRecomputePython suite in infrahub-private-tests restores the same dataset backup, renames a device type on a branch, merges, waits until every affected device is recomputed, and measures how long the instance keeps churning afterwards and how many flow runs the merge caused. We ran it twice on develop_x-small, once with develop (baseline run) and once with this branch (run with this PR), same dataset, same scenario, the only difference between the two runs is the commits in this PR.

Metric (merge)	develop	this branch
Trailing recompute churn after merge	275.2 s	~1 s
query-computed-attribute-transform-targets runs	1,321	1
Process flow runs	169	1
Client-visible update mutations	1 per node	0

At scale develop does not degrade gracefully, it collapses: on develop_large, the same scenario on develop spawned 73,149 flow runs in 110 minutes (99% dispatch echo, 53k group updates plus 19k target queries, near-zero actual transform work) and died with Neo4j bolt-thread and file-descriptor exhaustion. The same scenario with this branch, on a deliberately smaller database because the shared CI hosts kept OOM-killing full-size setups, survived without taking the host down. The churn that remains there is the reverse-index bookkeeping from dispatching to every node of the kind, which this PR deliberately does not touch, so the at-scale data confirms fan-out scoping as the necessary follow-up rather than a nice-to-have.

How we know nothing broke

• The perf suite asserts correctness, not just speed: after the merge, every affected device's computed value matches the expected new value, on both runs above.
• The existing functional computed-attribute suite (real git repository, real transforms, end to end) runs unchanged and passes.
• The write path is not new code, it is the same bulk writer the Jinja2 computed attributes already persist through.
• The behavior contract is preserved: same final values, same per-node NodeUpdated events for real changes, the public InfrahubUpdateComputedAttribute mutation stays (just no longer used internally), and the flow parameters are unchanged so the Prefect deployment contract is stable.
• New tests pin the new behavior: unit tests for the partition/skip logic, a functional test asserting an oversized fan-out splits into chunks with every id exactly once and the branch tag on every submission, and a component test characterizing today's whole-kind fan-out so the future scoping work has a baseline to assert against.

Built on the Jinja2 work, and what Python transforms still cannot do

This deliberately reuses the bulk recompute machinery Guillaume built for the Jinja2 computed attributes (BulkRecomputeDispatcher/BulkRecomputeWriter), no second write path, and it is verified with his TestMergeRecomputePython suite from infrahub-private-tests #29.

The part we cannot inherit: Jinja2 values are template renders the backend computes directly, so on merge they are recomputed in one coalesced in-process pass. Python transforms execute user code from a git repository on the task workers, they cannot join that pass, so they remain event-dispatched flows and a merge still dispatches a recompute for every node of the kind rather than only the changed ones. This PR makes that dispatch cheap and non-amplifying, scoping it to the affected nodes is the next big lever (see next steps).

Suggested review order

1. backend/infrahub/computed_attribute/tasks.py, the process_transform rewrite plus the tagged submission sites.
2. Tests: unit (partition/skip, submitter tags), functional (chunked submission and tags), component (fan-out characterization).
3. The add_tags commit pair (de6664ded reverted by 58680ba2e) documents a rejected fix, kept for history, the final approach tags at submission instead.

The design artifacts (spec, plan, critique, tasks) live in dev/specs/004-batch-python-recompute/.

How to test

uv run pytest backend/tests/unit/computed_attribute/ -q
uv run pytest backend/tests/functional/computed_attributes/ -q

At-scale validation runs through infrahub-private-tests test-dataset.yml with test_filter=TestMergeRecomputePython, links above.

Impact & rollout

• Backward compatibility: no schema changes, no API changes, event contract unchanged for real value changes.
• Performance: see above, the trailing recompute after a wide-impact change drops from minutes to about a second on the reference dataset.
• Deployment notes: safe to deploy, rollback is a clean revert, no migration.

Next steps

• Handle failed/skipped nodes more gracefully, likely raise after the dispatch so the flow run shows failed while healthy siblings stay persisted:

  await dispatcher.dispatch(writes=writes, ...)   # healthy siblings already persisted
  if skipped:
      raise RecomputeError(f"{len(skipped)} node(s) failed: {skipped}")

• A queryable recompute-failure surface with a one-click re-run (the RecomputeComputedAttribute mutation is the existing lever).

• Scope the merge fan-out to affected nodes instead of every node of the kind, that is the next big lever for giant-branch merges and the one thing the Jinja2 path also still needs.

• Phase-level spans/counters (repo-init, read, execute, write) to restore the per-node observability the batch path removed.

Checklist

• Tests added/updated
• Changelog entry added
• External docs updated (not user-facing, behavior-preserving)
• Internal .md docs updated (dev/knowledge/backend/computed-attributes.md)
• I have reviewed AI generated content

🤖 Generated with Claude Code

[cubic-dev-ai]

0 issues found across 2 files (changes from recent commits).

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would auto-approve. Performance optimization batching Python transform recompute with shared repo init and bulk write, tested at scale. The change is bounded and uses existing bulk writer machinery.

Re-trigger cubic</sub>

[codspeed-hq]

Merging this PR will not alter performance

✅ 12 untouched benchmarks

---

_{Comparing python-computed-attr-bulk-recompute (e67ebee) with develop (d781920)}

[cubic-dev-ai]

All reported issues were addressed

<sub>Shadow auto-approve: would not auto-approve because issues were found.
Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.

Re-trigger cubic</sub>

[cubic-dev-ai]

1 issue found across 17 files (changes from recent commits).

Confidence score: 5/5

• In dev/specs/004-batch-python-recompute/data-model.md, the AttributeValueWrite.value type omits the list[str] case defined in backend/infrahub/core/recompute/bulk_write.py, which could lead contributors to build against the wrong contract and introduce avoidable validation/serialization mismatches in HFI-related paths — align the spec type union with the implementation (str | list[str] | None).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="dev/specs/004-batch-python-recompute/data-model.md">

<violation number="1" location="dev/specs/004-batch-python-recompute/data-model.md:13">
P3: AttributeValueWrite.value type in the table shows `str | None` but the referenced source (`backend/infrahub/core/recompute/bulk_write.py`) defines it as `str | list[str] | None`. The `list[str]` variant is used by HFID writes and this path only enqueues `str` values, so the constraint note is right — but the type column should match the source type for readers who cross-reference.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: AttributeValueWrite.value type in the table shows str | None but the referenced source (backend/infrahub/core/recompute/bulk_write.py) defines it as str | list[str] | None. The list[str] variant is used by HFID writes and this path only enqueues str values, so the constraint note is right — but the type column should match the source type for readers who cross-reference.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At dev/specs/004-batch-python-recompute/data-model.md, line 13:

<comment>AttributeValueWrite.value type in the table shows `str | None` but the referenced source (`backend/infrahub/core/recompute/bulk_write.py`) defines it as `str | list[str] | None`. The `list[str]` variant is used by HFID writes and this path only enqueues `str` values, so the constraint note is right — but the type column should match the source type for readers who cross-reference.</comment>

<file context>
@@ -0,0 +1,32 @@
+|---|---|---|
+| node_id | str | target node |
+| field | str | computed attribute name |
+| value | str \| None | recomputed value; only `str` reaches the writer on this path |
+
+Source: `backend/infrahub/core/recompute/bulk_write.py`. Validation: this feature only enqueues writes whose `value` is `str` (R4).
</file context>

Suggested change

	| value | str \| None | recomputed value; only `str` reaches the writer on this path |
	| value | str \| list[str] \| None | recomputed value (only `str` reaches the writer on this path) |

[ajtmccarty]

looks good to me. @gmazoyer might want to look at it too

[ajtmccarty]

looks like _partition could handle logging the failures and only return the writes instead of splitting them and returning both.

[gmazoyer]

I would keep it returning both. The function is probably easier to unit-test as it stands. The flow also still needs len(skipped) for the summary line. So moving the logging inside would probably not help much.

[ajtmccarty]

do we need a task to cover the rest of the places where we add tag(s) mid-run?

[cubic-dev-ai]

0 issues found across 4 files (changes from recent commits).

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic</sub>

[gmazoyer]

written=len(writes) counts what we send to the writer, not what really got saved. The writer skips unchanged nodes, but dispatch() throws away its return value. So the number of skipped no-ops never shows up. The plan asked for written/unchanged/skipped, but that is not what we log. Maybe return the written list from dispatch() and log written and unchanged. Or just rename this to dispatched=.

[gmazoyer]

schema_branch does not change in the loop. So you can build the dispatcher once, above the loop, like process_jinja2 does.

[gmazoyer]

I would keep it returning both. The function is probably easier to unit-test as it stands. The flow also still needs len(skipped) for the summary line. So moving the logging inside would probably not help much.

[gmazoyer]

The if not all_ids: return is after add_tags. So an empty batch still tags. Moving the guard up matches what T008 wanted.

[gmazoyer]

@fatih-acar's input about this could be nice to have. I wonder if this needs to be documented somewhere for production environments too.