fix: evict stale GraphQL schema caches and bound test memory usage

[fatih-acar]

Summary

API worker processes retain a full GraphQL schema (~40MB of graphene types, ~1s to build) for every schema version a branch has ever served: the registry caches managers by schema hash, but nothing evicts the old hash when a branch's schema changes — purge_inactive only fires when a branch is deleted. Long-lived servers under schema churn grow until restart. Found while profiling the backend integration suites, where the same mechanism (plus driver garbage accumulation and host-sized neo4j auto-configuration) made memory the bottleneck that caps pytest-xdist parallelism.

Key Changes

• API servers release the previous GraphQL schema when a branch's schema changes. get_manager_for_branch now records each branch's active hash (using the previously dead _branch_hash_activation_by_branch_name field) and evicts a hash once no branch is on it, including registered types no remaining schema references — the cleanup _reference_hash_schema_map's docstring always described. purge_inactive reuses the same eviction path.
• Integration test classes reuse the pristine core-schema GraphQL manager instead of clearing the registry and rebuilding it per class (~1s CPU + ~40MB allocation churn × ~74 classes).
• Integration test workers stay small enough to scale out: cyclic garbage from the async neo4j driver is collected and freed pages are returned to the OS once per test class, and neo4j testcontainers get a bounded heap/page cache instead of sizing themselves off host memory (3–4GB each).

Measurements (62GB/32-core host, full backend/tests/integration)

Configuration	Result
-n 4 before	green, 664s, worker RSS ends at 1.9–2.7GB
-n 8 before	fails (host OOM → cascading fixture timeouts)
-n 8 with this PR + MALLOC_ARENA_MAX=2	green, 461s, worker RSS peaks at 1.1–1.7GB

In test_generic_migrations (a schema-churn-heavy module) the registry previously accumulated one full GraphQL schema per schema update within the class (~1GB growth per class); with eviction it holds at most the active ones.

Test Plan

• New unit tests for the registry: backend/tests/unit/graphql/test_registry.py (hash reuse across calls and branches, activation-based eviction, protection while another branch still uses a hash, orphaned-type pruning, purge_inactive behaviour).
• Full backend/tests/integration suite green locally at -n 4 and -n 8 with all changes.
• CI: backend-tests-integration and backend-docker-integration exercise the registry end to end.

🤖 Generated with Claude Code

---

Summary by cubic

Fixes unbounded memory growth in API workers by evicting stale GraphQL schema caches when branches update, and reduces test worker RSS so we can run more xdist workers. CI now runs the integration suite with 8 workers faster and with lower memory.

• Bug Fixes

  • Evict a branch’s previous schema hash when it activates a new one; keep it only if another branch still uses it.
  • Centralize eviction to also prune unreferenced GraphQL types; purge_inactive reuses this and clears per-branch hash activation.

• Performance

  • Reuse the pristine core-schema GraphQL manager across test classes instead of rebuilding it.
  • Bound neo4j testcontainer memory (heap 1g, page cache 512m).
  • After each test class, collect cycles and call malloc_trim to return freed pages to the OS.
  • CI: run integration tests with 8 xdist workers and set MALLOC_ARENA_MAX=2.
  • Results: -n 8 is green at 461s; worker RSS peaks ~1.1–1.7GB (previously 1.9–2.7GB at -n 4; -n 8 failed).

^{Written for commit 18a6cc7. Summary will update on new commits.}

[codspeed-hq]

Merging this PR will degrade performance by 23.39%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 10 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_base_schema_duplicate_CoreProposedChange	1.5 ms	2.2 ms	-32.4%
❌	test_schemabranch_duplicate	6.7 ms	7.7 ms	-13.17%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing fac/optimize-integration-tests-4oa27 (18a6cc7) with stable (2a8e5c8)}

[cubic-dev-ai]

No issues found across 6 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would require human review. Modifies core GraphQL schema eviction logic; requires human review to verify correctness and avoid unintended side effects.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 1 file (changes from recent commits).

<sub>Shadow auto-approve: would require human review. Core registry logic and eviction strategy changes require human review to ensure no schema serving errors or memory leaks. High-impact refactor affecting API workers.

Re-trigger cubic</sub>

[ogenstad]

One thing to consider here is if a user searches using at then we might end up with another schema hash. I think there are problems with that now if the schema differs, at the very least if the old schema has node kinds that have been removed in the newer schema. It could be that we shouldn't consider that aspect just now but I think at some point we should have a timestamp for when the schema was created and perhaps only delete schemas if they are unused and older than 15 minutes or so. Not requesting a change only mentioning this.