Skip to content

feat(analytics): World ID unique-verifications data layer (1/3) - #2191

Open
soamdesai-tfh wants to merge 12 commits into
mainfrom
feat/world-id-analytics-data
Open

feat(analytics): World ID unique-verifications data layer (1/3)#2191
soamdesai-tfh wants to merge 12 commits into
mainfrom
feat/world-id-analytics-data

Conversation

@soamdesai-tfh

@soamdesai-tfh soamdesai-tfh commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

First of three stacked PRs implementing single-metric World ID analytics (Unique Verifications across legacy v3 and v4 actions).

Scope

  • action_v3_stats_daily / action_v4_stats_daily mirror rollup tables (cascading FKs to their action parents), singleton world_id_analytics_state watermark, tracked empty return-shape table
  • rollup_world_id_analytics(): atomic dual-source absolute rebuild under advisory lock (533214,43), 5-minute safety delay, ~25h overlap horizon, monotonic watermark; deletion always wins races (FK failure aborts the run; the next 5-minute cron heals)
  • world_id_analytics_app_daily(): STABLE combined v3+v4 app-day aggregation in PostgreSQL
  • Service-role-only Hasura metadata; */5 cron trigger; protected /_rollup-world-id-analytics route behind a fail-closed WORLD_ID_ANALYTICS_ROLLUP_ENABLED gate (cron no-ops unless it is exactly "true")
  • Operator scripts: out-of-band, validity-gated CREATE INDEX CONCURRENTLY on nullifier(created_at) and nullifier_v4(created_at), plus backfill validation SQL. Neither index lives in the transactional migration — a plain build there would block verification inserts for the build duration while old pods still serve during a rolling deploy. The backfill gate refuses to run until both indexes are valid.
  • Fresh-stack contract harness (isolated Postgres+Hasura from this checkout's migrations/metadata) with the data-layer tests; regenerated GraphQL schema snapshot/base types (generated files, review-skippable)

Verification

  • Data-slice fresh-stack suite (smoke + backfill/validation gate) green on this tree: 10/10 (4 smoke + 6 gate cases, including missing/invalid-index rejections) via pnpm test:world-id-analytics:fresh
  • Rollup route unit tests 6/6; tsc --noEmit; prettier clean
  • Full-stack evidence (integration 19/19 + opt-in million-row gate: 1M-row backfill 264ms, catch-up 43ms, endpoint p95 ~29ms, plans prove rollup-only reads) lives with PR 2/3; re-run it there after merging this branch forward

Production rollout gates (deliberately outstanding, see the ops scripts): concurrent v3 + v4 index builds + validation, real backfill + canonical comparisons, uses = 0 evidence, production p95 vs latency budget, HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true verification.

Dual v3/v4 action-day rollup tables with cascading FKs, the singleton
watermark state table, the tracked return-shape table, the atomic
five-minute dual-source rollup function under advisory lock (533214,43),
the combined app-daily read function, service-only Hasura metadata, the
protected cron route with a rollout-disable gate, operator scripts for
the out-of-band v3 index and backfill validation, and the fresh-stack
contract harness with its data-layer tests.
The default and --million paths invoked integration.test.ts and
million.test.ts, which land with the read slice; jest exits 1 on a
pattern with no matches, so both entrypoints failed on this branch.
The read slice restores them alongside the files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@soamdesai-tfh
soamdesai-tfh marked this pull request as ready for review August 3, 2026 00:49

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f175224c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

A plain CREATE INDEX in the transactional migration blocks nullifier_v4
verification inserts for the build duration while old pods serve during
a rolling deploy. Build it out of band with CONCURRENTLY like the v3
twin: the operator script now creates and validity-gates both indexes,
and the backfill gate refuses to run until both are valid.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ats_daily

The API contract (legacy_actions) and UI scope type (source: "legacy")
already named this concept 'legacy'; only the SQL layer said 'v3'. v3
was never a real protocol generation here - nullifier/action predate
v4 entirely and internally split by engine (on-chain/cloud), not by
any versioning scheme. Align the table/index name and source tags with
the terminology the other two layers already settled on.

Regenerated graphql.schema.json/graphql.ts against the renamed table;
tsc and prettier clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@soamdesai-tfh
soamdesai-tfh force-pushed the feat/world-id-analytics-data branch from 2cd5cd5 to 3c2ff71 Compare August 3, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant