Skip to content

Add index for nullifier hash integer lookups - #2129

Open
soamdesai-tfh wants to merge 4 commits into
mainfrom
perf/nullifier-hash-int-index
Open

Add index for nullifier hash integer lookups#2129
soamdesai-tfh wants to merge 4 commits into
mainfrom
perf/nullifier-hash-int-index

Conversation

@soamdesai-tfh

@soamdesai-tfh soamdesai-tfh commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

⚠️ Do not merge until

  • Verified prod state of nullifier_hash_int_idx with the catalog query in the Deployment note: no row, or the exact partial-index definition below with indisvalid = true and indisready = true
  • Pre-created the index on prod with CREATE INDEX CONCURRENTLY, outside a transaction (SQL in the Deployment note)

Merging without the pre-create means the migration auto-applies on deploy (the cli-migrations-v3 image runs migrations at container startup) as a plain CREATE INDEX, which write-locks public.nullifier for the whole build — blocking /api/v2/verify and OIDC authorize writes and potentially stalling Hasura startup. With the pre-create done, the migration is an instant no-op via IF NOT EXISTS.

What changed

  • Added a non-unique partial B-tree index on public.nullifier(nullifier_hash_int) for rows where nullifier_hash_int IS NOT NULL.
  • Kept IF NOT EXISTS so a matching index can be created concurrently before deployment on a large target.
  • Added a reversible down migration that removes only the index.
  • Kept the existing GraphQL operations and application behavior unchanged.

Why

AtomicUpsertNullifier updates nullifier rows with nullifier_hash_int equality as the only predicate; with no index on that column, PostgreSQL performed a full sequential scan of nullifier on every successful verification. FetchAppAction performs the same equality lookup nested under actions, so it was likely already served by the existing action_id_uses index rather than a sequential scan; the new index still gives it a direct lookup path. The existing unique index on nullifier_hash cannot serve this different predicate.

The column is nullable and was added after the table without a backfill migration. Excluding NULL values keeps legacy rows that cannot match these equality lookups out of the index, reducing its storage and write-maintenance cost without changing the supported query plans.

The pinned Hasura v2.47 migration runner executes migration SQL inside a transaction, so the checked-in migration cannot use CREATE INDEX CONCURRENTLY.

Impact

There is no API or functional behavior change. PostgreSQL gains a direct lookup path for both equality consumers, preventing their execution time from growing linearly with table size. The tradeoff is the normal storage and write-maintenance cost of one targeted B-tree index.

Validation

  • Applied the full migration set successfully to a fresh isolated PostgreSQL and Hasura environment.
  • At 100,000 nullifier rows, the update changed from a sequential scan at approximately 13.1 ms to an index scan at approximately 0.8 ms.
  • Simulated a legacy population with 100,000 rows and only one non-NULL value; the partial index was 16 KB.
  • Forced PostgreSQL to use a generic prepared plan and confirmed nullifier_hash_int = $1 still used the partial index, executing in approximately 0.8 ms.
  • Called AtomicUpsertNullifier twice and confirmed one row was retained while uses incremented from 1 to 2.
  • Applied the down migration, confirmed verification still worked and data remained, then reapplied the up migration successfully.
  • Confirmed Hasura metadata remained consistent.
  • pnpm format:check
  • npx tsc --noEmit

Deployment note

First inspect any same-named index. IF NOT EXISTS checks the name only, and an interrupted CREATE INDEX CONCURRENTLY can leave behind an invalid index whose definition still looks correct:

SELECT
  c.relkind,
  pg_get_indexdef(i.indexrelid) AS index_definition,
  i.indisvalid,
  i.indisready
FROM pg_class AS c
JOIN pg_namespace AS n ON n.oid = c.relnamespace
LEFT JOIN pg_index AS i ON i.indexrelid = c.oid
WHERE n.nspname = 'public'
  AND c.relname = 'nullifier_hash_int_idx';

Proceed only if this returns no row, or returns the exact partial-index definition below with indisvalid = true and indisready = true. If it returns an invalid index (a failed earlier concurrent build), drop it first with DROP INDEX CONCURRENTLY "public"."nullifier_hash_int_idx"; and re-create.

If the target table is large, pre-create the semantically identical partial index outside the transactional migration. Run it outside any transaction and without IF NOT EXISTS, so a name conflict fails loudly instead of silently skipping:

CREATE INDEX CONCURRENTLY "nullifier_hash_int_idx"
  ON "public"."nullifier" USING btree ("nullifier_hash_int")
  WHERE "nullifier_hash_int" IS NOT NULL;

The checked-in migration then safely no-ops via IF NOT EXISTS because a valid matching index already exists under the same name.

@soamdesai-tfh
soamdesai-tfh marked this pull request as ready for review July 21, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants