Skip to content

Add indexes to tables improve performance#1891

Open
AkramBitar wants to merge 1 commit into
mainfrom
add_indexes
Open

Add indexes to tables improve performance#1891
AkramBitar wants to merge 1 commit into
mainfrom
add_indexes

Conversation

@AkramBitar

Copy link
Copy Markdown
Contributor

Fixes: 1873

Add the indexes, to improve query performance to the following tables:

  1. token_locks: index on consumer_tx_id.
  2. validations: index on status/stored_at (matching what transactions/requests already have).
  3. tokens: extend or add an index covering (is_deleted, spent_at) for the SKI cleanup query).

@AkramBitar AkramBitar added this to the Q3/26 milestone Jul 10, 2026
@AkramBitar AkramBitar self-assigned this Jul 10, 2026
@AkramBitar AkramBitar requested review from HayimShaul and adecaro July 10, 2026 16:58
@adecaro

adecaro commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Hi @AkramBitar , thanks for submitting this. The issue calls also for a way to verify the indices are in place. Is this possible?

Signed-off-by: AkramBitar <akram@il.ibm.com>
@adecaro

adecaro commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

some additional analysis worth including in this PR:

Missing indexes (real scan risk)

  1. requests.recovery_claim_expires_at — CleanupExpiredClaims (postgres/transactions.go:248) does WHERE recovery_claim_expires_at < NOW() with no status filter, so it can't use the
    existing partial index idx_recovery_claim (status, recovery_claim_expires_at, stored_at) WHERE status = 1. This is a full scan of the requests table every sweep. Fix: CREATE INDEX ...
    ON requests (recovery_claim_expires_at) WHERE recovery_claimed_by IS NOT NULL — small partial index since most rows aren't claimed.
  2. tokens.owner_identity, owner_type — HasTokenForIdentity (tokens.go:915), used by the SKI cleanup signer check (cmd/skicleanup), filters on these two columns with zero supporting
    index → full scan of the tokens table per identity checked.
  3. tokens.issuer, redeemed — IssuedBalance, RedeemedBalance, and ListHistoryIssuedTokens (tokens.go:642, 681) all filter issuer = true AND redeemed = . is_deleted is deliberately
    not filtered here (per the comment at tokens.go:637), so issued tokens accumulate forever — this scan only gets worse over time with no index backing it.
  4. token_locks.created_at — Cleanup (postgres/tokenlock.go:82) does WHERE OlderThan(created_at, ...) OR EXISTS(...). The created_at branch has no index, so this OR generally forces a
    sequential scan of the whole table on every periodic sweep.

Redundant indexes (pure write-path cost, no read value)

  • identity.go: idx_ic_id_type_%s (id, type, url) — exact duplicate of PRIMARY KEY(id, type, url).
  • identity.go: idx_audits_%s (identity_hash) on IdentityInfo — duplicate of its single-column PRIMARY KEY(identity_hash).
  • identity.go: idx_signers_%s (identity_hash) on Signers — same, duplicate of PK.
  • wallet.go: idx_identity_hash_and_wallet_and_role%s (identity_hash, wallet_id, role_id) — exact duplicate of PRIMARY KEY(identity_hash, wallet_id, role_id).
  • wallet.go: idx_identity_hash_%s (identity_hash) — redundant, already a prefix of idx_identity_hash_and_role%s and the PK.
  • tokens.go: idx_tx_id_%s (tx_id) — redundant prefix of PRIMARY KEY(tx_id, idx).

These get rebuilt on every insert/update for no query benefit; dropping them is a pure win.

Lower-confidence, worth an EXPLAIN before touching

idx_recovery_claim_%s (status, recovery_claim_expires_at, stored_at) WHERE status = 1 is used by ClaimPendingTransactions, which needs status = 1 AND stored_at < X ORDER BY stored_at,
but recovery_claim_expires_at sits between status and stored_at in the index and isn't equality-constrained (it's only checked via a non-sargable OR with recovery_claimed_by). That
likely blocks the index from satisfying the ORDER BY without an extra sort. Reordering to (status, stored_at) might be tighter, but I'd want to see a real EXPLAIN ANALYZE before
recommending a change here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: three storage tables are missing indexes for their hot query paths

2 participants