Skip to content

fix(indexer): use indexed exact-match participant columns instead of ILIKE on JSON - #921

Open
sasasamaes wants to merge 2 commits into
FinChippay:mainfrom
sasasamaes:fix/indexer-column-queries
Open

fix(indexer): use indexed exact-match participant columns instead of ILIKE on JSON#921
sasasamaes wants to merge 2 commits into
FinChippay:mainfrom
sasasamaes:fix/indexer-column-queries

Conversation

@sasasamaes

@sasasamaes sasasamaes commented Aug 26, 2026

Copy link
Copy Markdown

Closes #889

Summary

Participant queries in backend/src/services/eventIndexer.js (queryEventsByPublicKey, queryEventsByType, getEventStats) previously used payload::text ILIKE '%<publicKey>%' — a full-text substring scan over the entire contract_events table. This caused:

  1. Performance: full-table scan on every dashboard call, unusable by any index.
  2. Correctness: a public key (55 chars) appearing in unrelated payload fields (e.g. a memo or nested array) produced false matches — users could see events they weren't part of.

The table already stores parsed from_addr / to_addr columns (populated on insert via eventParser.js), but the query helpers never used them.

Changes

  • Rewrite all three query helpers to match exactly with from_addr = $1 OR to_addr = $2
  • Add migration 028_events_participants_index.js: cross-dialect composite index idx_events_participants (from_addr, to_addr) (uses CREATE INDEX IF NOT EXISTS, has down() rollback per CONTRIBUTING.md)
  • Keep the in-memory fallback store in sync — now uses exact column match too
  • Add _seedForTest helper to seed events into the in-memory store for tests

Tests

New suite backend/__tests__/eventIndexer-column-queries.test.js (13 tests):

  • Exact-participant filtering returns only the user's events (queryEventsByPublicKey, queryEventsByType)
  • A public key string appearing in a memo does NOT match (false-positive regression test)
  • getEventStats counts only true participants and does not inflate from memo matches
  • Descending ledger order, unknown-key empty results

All 13 new tests pass; existing 15 integration-eventIndexer tests still pass; eslint src/**/*.js reports no issues in changed files.

Acceptance Criteria

Criteria Status
Participant queries use indexed exact-match columns
Memo containing a public key no longer produces false matches
≥4 tests; lint clean ✅ (13 tests)
In-memory fallback kept in sync, documented as best-effort

@github-actions github-actions Bot added the needs-review PR ready for Greptile AI code review label Aug 26, 2026
@github-actions

Copy link
Copy Markdown

🤖 Greptile AI Code Review

Greptile will automatically review this PR (3 file(s) changed).

Review gates:

  • ✅ CodeQL Security Scan
  • ✅ Custom rules (.greptile/config.json)
  • ✅ Architecture guidelines (.greptile/rules.md)

To manually trigger a re-review, comment @greptileai on this PR.
To skip review, add the skip-review label.

…ILIKE on JSON (FinChippay#889)

Participant queries (queryEventsByPublicKey, queryEventsByType,
getEventStats) previously scanned the serialized payload with
payload::text ILIKE '%<publicKey>%' — a full-table scan that also
produced false matches when the key appeared in unrelated fields
(e.g. a memo).

- Rewrite all three query helpers to match exactly on from_addr/to_addr
- Add migration 028: composite index idx_events_participants
  (from_addr, to_addr) for fast lookups (cross-dialect)
- Keep the in-memory fallback store in sync (exact column match)
- Add 13 tests covering exact-participant filtering and rejection of
  memo substring false positives

Closes FinChippay#889
@sasasamaes
sasasamaes force-pushed the fix/indexer-column-queries branch from e84d746 to 5cdf7ec Compare August 26, 2026 03:29
GitHub Actions does not allow the `secrets` context inside step-level
`if:` conditionals, which made contract-type-check.yml fail workflow
validation. Expose TESTNET_CONTRACT_ID as a job-level env var and gate
the steps on `env.*` instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review PR ready for Greptile AI code review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#108 — Uses on Serialized JSON: Wrong Matches and Slow Queries

1 participant