Skip to content

fix(api): normalize destination topics on write#994

Merged
alexluong merged 2 commits into
hookdeck:mainfrom
mvanhorn:fix/normalize-destination-topics
Jul 14, 2026
Merged

fix(api): normalize destination topics on write#994
alexluong merged 2 commits into
hookdeck:mainfrom
mvanhorn:fix/normalize-destination-topics

Conversation

@mvanhorn

Copy link
Copy Markdown
Contributor

Summary

Topics.Validate checks each entry independently, so a destination's stored
topic set can contain redundant entries that are accepted verbatim:

  • exact duplicates: ["user.created", "user.created"]
  • a wildcard plus an entry it covers: ["user.*", "user.created"]
  • overlapping patterns plus a covered entry: ["*.created", "user.created", "user.*"]

Delivery is unaffected (topic matching is any-entry-wins), but the persisted
array is ambiguous as data: duplicates echo back on every read (API/SDK/portal
and tenant.subscriptions.updated payloads), and there is no defined edit
semantics for a redundant set. Since #912 (TOPICS_ALLOW_WILDCARDS) the portal
itself can create these mixed states, so they occur organically.

This adds Topics.Normalize() and calls it after Validate in the destination
create and update handlers:

  • dedupes exact duplicates, keeping the first occurrence;
  • folds an entry that a sibling wildcard pattern strictly covers
    (["user.*", "user.created"] -> ["user.*"]), where "strictly covers" means
    the pattern covers the entry but the entry does not cover the pattern;
  • leaves mutually-non-covering pattern pairs untouched
    (["*.created", "user.*"] is unchanged);
  • leaves ["*"] and already-minimal sets untouched;
  • preserves first-seen order so API responses stay stable.

Because it only removes entries already covered by a retained entry, MatchTopic
(delivery) results are identical before and after. A create/update with
["user.*", "user.created"] now stores and returns ["user.*"].

Why this matters

This implements the "normalize on write" direction from the issue author
(@alexluong): "dedupe exact duplicates and fold entries covered by a sibling
pattern (["user.*", "user.created"] -> ["user.*"])", reaffirmed as "if
user.* then should not have user.created". Normalizing at the write boundary
makes the persisted topic set unambiguous, so every downstream reader (API, SDK,
portal, subscription-update events) sees a single canonical representation. It
also gives the portal topic editor a well-defined data model to build on.

Out of scope (deliberately)

  • Portal TopicPicker.tsx hasWildcard onChange fixes. That is a separate
    frontend change requiring the portal running to verify; this backend
    normalization is the prerequisite that makes the persisted data unambiguous.
  • Rejecting non-trailing wildcards such as *.created ("wildcard as last param",
    raised by @alexbouchardd). That is a validation-tightening change that could
    reject currently-valid stored data, a distinct decision from normalization -
    left for a maintainer as a known follow-up.

Per the issue's own note, pattern-vs-pattern overlap where neither covers the
other (["*.created", "user.*"]) is intentionally left intact.

Tests

  • New TestDestinationTopics_Normalize table test covering dedupe, fold,
    mutually-non-covering patterns, ["*"], already-minimal sets, and order
    preservation.
  • Existing TestDestinationTopics_Validate unchanged.
  • go build ./..., go test ./internal/models/... ./internal/apirouter/...,
    and go vet on the changed packages all pass.

Closes #990

Add Topics.Normalize() and call it after Validate in the destination
create and update handlers. It dedupes exact duplicate entries and folds
an entry that a sibling wildcard pattern strictly covers
(["user.*","user.created"] -> ["user.*"]), while leaving mutually-non-
covering pattern pairs (["*.created","user.*"]) and ["*"] untouched.
Preserves first-seen order so API responses stay stable.

Normalization only removes entries already covered by a retained entry,
so MatchTopic (delivery) results are unchanged.

Closes hookdeck#990

@alexluong alexluong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, LGTM!

One small comment is can we add some tests at the handler level where we send POST/PUT/PATCH request with different topics input and confirm that the topic list is normalized?

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alexluong alexluong merged commit c54782f into hookdeck:main Jul 14, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Destination topics: duplicates and redundant wildcard states are persisted

2 participants