perf: replace LATERAL jsonb_each_text label queries with indexed lookup

[rh-jfuller]

The fetch_labels query scanned every row in advisory/sbom, expanded each row's JSONB labels column via LATERAL jsonb_each_text(), then applied ILIKE filtering on the expanded text. No index could accelerate this pattern.

• Creates advisory_label and sbom_label tables — each stores one row per key/value label pair, with a pre-computed key_value text column (e.g., "type=spdx")
• Creates GIN (key_value gin_trgm_ops) indexes on both tables for fast ILIKE substring matching
• Creates trigger functions that automatically sync the lookup tables on INSERT, UPDATE, and DELETE of the source tables
• Backfills from existing data
• Rewrote the fetch_labels query to read directly from advisory_label/sbom_label instead of doing LATERAL jsonb_each_text() expansion

The new query hits the GIN trigram index for ILIKE filtering and reads from a pre-expanded table — no more full-table JSONB expansion. The trigram index supports the existing substring matching behavior (e.g., searching "pd" still finds "spdx").

Addresses TC-5204

Summary by Sourcery

Introduce denormalized advisory and SBOM label lookup tables and update label fetching to use them for more efficient indexed searches.

New Features:

• Add advisory_label and sbom_label tables to store per-label key/value pairs with a precomputed search column for label lookups.

Enhancements:

• Change the fetch_labels query to read from the new label lookup tables using indexed key/value search instead of expanding JSONB labels per row.
• Add database triggers and helper function to keep label lookup tables synchronized with advisory and sbom label changes and backfill them from existing data.

[sourcery-ai]

Reviewer's Guide

Replaces expensive LATERAL jsonb_each_text-based label searching with denormalized label lookup tables backed by trigram indexes, and wires them into the application and migration system for performant ILIKE substring searches.

Sequence diagram for fetch_labels using denormalized label lookup tables

sequenceDiagram
    participant Service
    participant Database
    participant advisory_label_kv_trgm_idx
    participant sbom_label_kv_trgm_idx

    Service->>Database: fetch_labels(type, pattern)
    alt [DocumentType.Advisory]
        Database->>Database: SELECT DISTINCT ON (key, value) FROM advisory_label WHERE key_value ILIKE pattern
        Database->>advisory_label_kv_trgm_idx: use GIN gin_trgm_ops index on key_value
        advisory_label_kv_trgm_idx-->>Database: matching rows
    else [DocumentType.Sbom]
        Database->>Database: SELECT DISTINCT ON (key, value) FROM sbom_label WHERE key_value ILIKE pattern
        Database->>sbom_label_kv_trgm_idx: use GIN gin_trgm_ops index on key_value
        sbom_label_kv_trgm_idx-->>Database: matching rows
    end
    Database-->>Service: Vec<serde_json::Value>

Loading

Entity relationship diagram for new label lookup tables

erDiagram
    advisory {
        UUID id
        JSONB labels
    }

    advisory_label {
        UUID advisory_id
        TEXT key
        TEXT value
        TEXT key_value
    }

    sbom {
        UUID sbom_id
        JSONB labels
    }

    sbom_label {
        UUID sbom_id
        TEXT key
        TEXT value
        TEXT key_value
    }

    advisory ||--o{ advisory_label : has_labels
    sbom ||--o{ sbom_label : has_labels

Loading

File-Level Changes

Change	Details	Files
Refactor fetch_labels query to use pre-expanded label lookup tables instead of per-row JSONB expansion.	Change SELECT to read directly from key and value columns instead of jsonb_each_text alias fields. Replace FROM base advisory/sbom tables with FROM advisory_label/sbom_label based on DocumentType. Filter using key_value ILIKE pattern on the lookup table rather than CASE expression on key/value. Retain DISTINCT ON and ordering semantics over key and value, adjusting column references accordingly.	modules/fundamental/src/common/service.rs
Introduce label lookup tables with trigram indexes and triggers to mirror JSONB labels for advisory and sbom records, including backfill and rollback logic.	Register new migration m0002240_label_lookup_tables in the migrator pipeline. Create advisory_label and sbom_label tables with foreign keys, composite primary keys, and key_value text column. Add GIN trigram indexes on key_value columns to accelerate substring ILIKE searches. Define immutable helper function label_key_value to normalize key/value into search text consistent with previous behavior. Implement PL/pgSQL trigger functions to sync lookup tables on INSERT, UPDATE of labels, and DELETE from source tables. Backfill advisory_label and sbom_label from existing advisory and sbom labels data during migration up. Provide down migration to drop triggers, functions, lookup tables, and helper function.	migration/src/lib.rs migration/src/m0002240_label_lookup_tables.rs

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• If labels can ever be NULL on advisory/sbom, consider wrapping jsonb_each_text(NEW.labels) in COALESCE(NEW.labels, '{}'::jsonb) in the trigger functions to avoid unexpected behavior when labels are absent.
• Instead of manually maintaining key_value via triggers and the label_key_value helper, consider defining key_value as a generated column (e.g., GENERATED ALWAYS AS (...) STORED) to remove the risk of drift between key/value and key_value.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- If `labels` can ever be NULL on advisory/sbom, consider wrapping `jsonb_each_text(NEW.labels)` in `COALESCE(NEW.labels, '{}'::jsonb)` in the trigger functions to avoid unexpected behavior when labels are absent.
- Instead of manually maintaining `key_value` via triggers and the `label_key_value` helper, consider defining `key_value` as a generated column (e.g., `GENERATED ALWAYS AS (...) STORED`) to remove the risk of drift between `key`/`value` and `key_value`.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}