Skip to content

Fix search and filter bugs against the Ballerina Central API - #50

Merged
daneshk merged 3 commits into
wso2:mainfrom
daneshk:fix/search-filter-issue-1853
Aug 5, 2026
Merged

Fix search and filter bugs against the Ballerina Central API#50
daneshk merged 3 commits into
wso2:mainfrom
daneshk:fix/search-filter-issue-1853

Conversation

@daneshk

@daneshk daneshk commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #1853 and two related filtering bugs found while auditing search/filter behavior against the live Ballerina Central search API.

  • Multi-word search returned nothing (the reported issue) — escapeSolrQuery escaped the space character, turning a query like "SAP Business" into the wildcard term *sap\ business*. The API's *...* wildcard is a literal, non-analyzed pattern, so the escaped backslash-space never matches. Space is no longer escaped.
  • Area filters could silently return 0 results when combined with another filter — Area values containing & (e.g. "Finance & Accounting") make the API's query parser return 0 results when followed by another AND keyword:... clause. Verified live that the identical clauses in the opposite order parse correctly, so Area filters are now always added last in the built query.
  • The API's keyword: field isn't an exact match — it can silently mix in wrong connectors. Verified live that keyword:Vendor/OpenAI also matches azure.openai.text (real vendor: Microsoft), purely because it carries an unrelated bare keyword "Azure OpenAI". The same effect made a "Finance & Accounting" + "Microsoft" filter return 30 connectors instead of the correct 11, including several from unrelated areas (Accounts Payable, Tax, HR, etc). Added filterByExactKeywords(), a client-side post-filter that re-checks the exact Area//Vendor//Type/ tag, mirroring the existing excludeHidden/filterByRelevance post-processing pattern.
  • The "HRMS" sidebar area matched zero live connectors — the actual tag Ballerina Central uses is Area/Human Resources (0 packages have Area/HRMS, 2 have Area/Human Resources). Renamed the hardcoded sidebar entry.

All of the above were verified directly against the live api.central.ballerina.io search endpoint before and after the fix, not just against mocked tests.

Test plan

  • npm run typecheck passes
  • npm run lint passes
  • npm test — 87 tests pass, including 3 new regression tests:
    • multi-word query no longer escapes the space in the wildcard term
    • Area filter is placed after Vendor/Type filters in the built query
    • a loosely-matched false positive (azure.openai.text under a Vendor/OpenAI filter) is dropped by filterByExactKeywords
  • Manually reproduced and re-verified all four issues against the live API (see commit message / PR description for the exact before/after query results)

🤖 Generated with Claude Code

Fixes wso2/product-integrator#1853.

- Multi-word text search returned no results: escapeSolrQuery escaped
  the space character, turning "SAP Business" into the wildcard term
  *sap\ business*, which never matches since the API's wildcard is a
  literal (non-analyzed) pattern. Space is no longer escaped.

- Area filters combining with another AND clause could silently return
  0 results when the Area value contains "&" (e.g. "Finance &
  Accounting" + a Vendor filter) — verified live that the same clauses
  in the opposite order parse correctly. Area filters are now built
  last in the query.

- The search API's keyword: field isn't an exact match on one tag —
  it matches loosely against a package's whole keyword list (e.g.
  keyword:Vendor/OpenAI also matches azure.openai.text, whose real
  vendor is Microsoft, because of an unrelated bare keyword "Azure
  OpenAI"). This silently mixed wrong connectors into filtered
  results. Added filterByExactKeywords() to re-check the exact
  Area/Vendor/Type tag client-side, mirroring the existing
  excludeHidden/filterByRelevance post-processing.

- The hardcoded "HRMS" sidebar area matched zero live connectors; the
  actual tag is "Human Resources".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary

  • Fixed multi-word wildcard search handling for the Ballerina Central API.
  • Ordered area filters after other filters to prevent parser failures.
  • Added exact client-side matching for Area/, Vendor/, and Type/ tags.
  • Added complete-result filtering and client-side pagination for filtered searches.
  • Renamed the HRMS sidebar area to Human Resources.
  • Added regression tests for search, filter ordering, exact matching, and pagination.
  • Type checking, linting, formatting, and all 89 tests pass.

Walkthrough

The search client preserves spaces in Solr terms and splits non-wildcard multi-word searches into separate AND clauses. It places area filters after vendor and type filters. Filtered searches fetch complete result sets before exact tag filtering, deduplication, sorting, relevance filtering, and pagination. Unfiltered searches retain server-side pagination. Regression tests cover these behaviors and the predefined HRMS area label is renamed to Human Resources.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The check is inconclusive because multiple pull request templates are available, but pre-merge checks support only one description template. Configure a single supported pull request description template, then rerun this check.
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary search and filtering fixes in the pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/rest-client/rest-client.test.ts`:
- Around line 173-184: Add a Connector entry to the types filter in the
searchPackages call, then capture the URL index for keyword:Type/Connector and
assert it appears before the existing area clause, while preserving the
vendor-before-area ordering assertions.

In `@src/lib/rest-client/rest-client.ts`:
- Around line 526-541: Apply pagination only after exact-keyword filtering:
update the single-combination path around executeSingleSearch to retrieve and
filter the complete logical result set, then calculate the exact count and slice
the requested page. Apply the same ordering to each multi-combination result
path at src/lib/rest-client/rest-client.ts lines 574-575 before merged
pagination; both sites require changes, and remove the
fixed-buffer/proportional-count approach.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b5e89c4e-0465-4039-849c-9b0b49e95eeb

📥 Commits

Reviewing files that changed from the base of the PR and between 83c4c9f and 13e46ec.

📒 Files selected for processing (3)
  • src/lib/rest-client/rest-client.test.ts
  • src/lib/rest-client/rest-client.ts
  • src/pages/HomePage.tsx

Comment thread src/lib/rest-client/rest-client.test.ts
Comment thread src/lib/rest-client/rest-client.ts Outdated
… search bug

- npm run lint runs prettier --check, which failed on the previous commit
  since it was never run through `npm run format`. Fixed formatting.

- CodeRabbit review: the vendor/type-before-area regression test only
  asserted vendor ordering, not type. Added a Type filter and assert it
  too.

- CodeRabbit review (major): the server-side-pagination fast path used a
  fixed HIDDEN_PACKAGES-sized overfetch buffer to compensate for
  filterByExactKeywords removing false positives, but that buffer isn't
  sized for an open-ended problem (unlike the small, static hidden-package
  list) — a page could come up short, or the reported count could be a
  rough proportional estimate rather than exact. Restructured
  searchPackages so any Area/Vendor/Type filter now always fetches the
  complete result set, filters exactly, and paginates client-side —
  the same pattern already used and trusted for text search and name
  sorts. The buffer/estimate approach is now used only for the
  filter-free browse case, where HIDDEN_PACKAGES is the only possible
  exclusion and its bounded size actually justifies the buffer.

- Found while re-verifying the PR live: searching "dynamics 365" returned
  nothing. The API's `*dynamics 365*` literal wildcard term returns 0
  results (verified live) even though `*dynamics*` and `*365*` each
  match, and the connector's own keywords contain the phrase. Multi-word
  search queries are now split into separate ANDed wildcard terms
  (`*dynamics* AND *365*`) instead of one literal phrase term — verified
  this doesn't regress the original "SAP Business" fix.

All typecheck/lint/format/test checks pass locally (89 tests), and all
of the above were re-verified against the running app and the live
Central API, not just mocks.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread src/pages/HomePage.tsx Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/lib/rest-client/rest-client.test.ts (1)

285-319: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Make complete-fetch regression mocks request-sensitive.

The mocks return all packages regardless of request parameters. These tests can pass if batch requests use a page-sized limit.

  • src/lib/rest-client/rest-client.test.ts#L285-L319: make the mock apply the requested limit and offset, so partial fetches cannot produce the expected pages.
  • src/lib/rest-client/rest-client.test.ts#L321-L340: use a count above the normal page size and assert each batch request has a limit sufficient for the complete result set.

As per path instructions, provide concise, actionable feedback focused on correctness and best practices.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/rest-client/rest-client.test.ts` around lines 285 - 319, Update the
regression tests around searchPackages in
src/lib/rest-client/rest-client.test.ts:285-319 and 321-340. Make the mockFetch
response honor each request’s limit and offset so page-sized partial fetches
cannot satisfy the expected results; in the second test, use a dataset larger
than the normal page size and assert every batch request requests enough results
to cover the complete set.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/rest-client/rest-client.test.ts`:
- Around line 285-319: Update the regression tests around searchPackages in
src/lib/rest-client/rest-client.test.ts:285-319 and 321-340. Make the mockFetch
response honor each request’s limit and offset so page-sized partial fetches
cannot satisfy the expected results; in the second test, use a dataset larger
than the normal page size and assert every batch request requests enough results
to cover the complete set.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f8de1fcd-a856-4224-a1f3-f61d5f2a9888

📥 Commits

Reviewing files that changed from the base of the PR and between 13e46ec and 6735244.

📒 Files selected for processing (2)
  • src/lib/rest-client/rest-client.test.ts
  • src/lib/rest-client/rest-client.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/rest-client/rest-client.ts

Comment thread src/pages/HomePage.tsx Outdated

@RDPerera RDPerera left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR body's fourth bullet says the sidebar entry was renamed HRMSHuman Resources. Seems that is missing or re-aligned for ealier state for some reason ?

@daneshk

daneshk commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

PR body's fourth bullet says the sidebar entry was renamed HRMSHuman Resources. Seems that is missing or re-aligned for ealier state for some reason ?

I reverted the change, as we can't change the category now

@daneshk
daneshk merged commit 4d9cf61 into wso2:main Aug 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Connector Search won't work for Two or More words

2 participants