feat(discovery): Worker Discovery API with geo-radius search, skill facets & reputation-weighted ranking (#43) - #54
Conversation
203a5bf to
fc059cb
Compare
Architecture and the decisions that shape the implementation, written up front so the code that follows can point at it: the two-stage index-backed geo filter, reputation materialised off the request path with a documented staleness bound and fallback, the configurable ranking blend, keyset cursor pagination semantics, and the facet-count contract. Refs workman-labs#43
GeoBox.around turns (lat, lon, radiusKm) into the lat/lon box the search leads with so a radius query is served by a B-tree index rather than a per-row distance computation; it is a superset of the true circle and degrades safely to the full longitude range at the poles and across the antimeridian. GeoDistance is the exact Haversine, mirrored verbatim in the search SQL. Refs workman-labs#43
worker_reputation_snapshots holds a per-worker materialised copy of the reputation contract's Rating aggregate so the search query can LEFT JOIN a reputation score instead of making a chain call. The id is the worker id (assigned), so the entity is Persistable to get an INSERT rather than a SELECT-then-UPDATE on upsert. ReputationProperties binds the staleness bound, fallback score and refresh batch size. Refs workman-labs#43
ReputationSnapshotService refreshes the most-stale snapshots on a schedule, reading each worker's Rating aggregate through ReputationContractClient. HttpReputationContractClient calls a read-model / indexer that projects the contract aggregate rather than hand-encoding Soroban ledger-entry XDR - the same call EscrowReconciliationService already made. On a read failure an existing snapshot is kept and retried; a worker with none gets a neutral FALLBACK snapshot so search always has a value. The scheduler delay is defaulted to an hour in surefire, like every other poller. Refs workman-labs#43
WorkerRankingCalculator is the reference implementation of the ranking blend - proximity, materialised reputation, availability - with the weights bound from guildworkman.discovery.ranking.* rather than baked into a comparator. The score is normalised by the weight total so it stays in [0,1] whatever weights an operator picks. The search SQL computes the same expression; the test pins the two together. Refs workman-labs#43
SearchCursor is the (rankScore, workerId) keyset position; CursorCodec encodes it as an opaque versioned base64url token and validates it on the way back in. Anything unparseable, tampered, or from another version is an InvalidSearchCursorException - never silently dropped, which would restart pagination from the top and repeat rows. Refs workman-labs#43
The whole search in SQL: bounding-box prefilter, exact Haversine, composed skill/category/availability filters, the ranking blend, and the keyset page - plus the two grouped facet-count queries, each ignoring its own dimension's filter. The Haversine is shared between all three queries. SkilledWorker gains the composite geo index and the category index the queries rely on, and a nullable `available` column (null = available). Refs workman-labs#43
WorkerDiscoveryService turns a WorkerSearchCriteria into the bounding box, issues the single ranked query (one row over the page size, to know if a next page exists) and the two facet queries, and assembles the response including the next-page cursor. DiscoveryProperties binds the radius and page-size bounds and the skill-facet cap. Refs workman-labs#43
GET /api/v1/discovery/workers - validated parameters, OpenAPI annotations, and the app-wide RFC 7807 error contract (a bad cursor is 400 invalid-search-cursor via GlobalExceptionHandler). Public, same access level as /api/v1/skilledWorker/** and /api/v1/booking/**, since finding a worker has to work before anyone signs in. Config defaults land in application.properties. Refs workman-labs#43
WorkerDiscoveryIntegrationTest seeds a dataset against a real Postgres and asserts the ranked order matches the documented formula, that the radius excludes far workers, that filters and facets compose correctly, and that a keyset scroll stays stable when a top-ranked row is inserted mid-scroll (where OFFSET pagination would repeat a row). WorkerDiscoveryIndexUsageTest proves the query plan uses idx_skilled_workers_geo, not a Seq Scan. The Test workflow gains an explicit, pom-keyed ~/.m2 cache step. Refs workman-labs#43
fc059cb to
d1a81d6
Compare
|
Verified locally against Postgres with the CI command ( Force-pushed once since opening to fix a whitespace bug in the assembled native SQL and to correct |
meshackyaro
left a comment
There was a problem hiding this comment.
Nice piece of work — this is a real discovery API, not just a filter bolted onto the existing endpoint, and the design choices are backed up by tests rather than just asserted. Well done!
What this does
Closes #43.
Adds the search endpoint the marketplace was missing. Before this, a client
could only reach a worker whose id it already had
(
GET /api/v1/skilledWorker/findById); the oneGET /api/v1/skilledWorker/nearbyroute loaded every worker row and filtered it in memory with a per-row
Haversine. This replaces that with a real discovery API:
Returns a ranked page,
facets.category/facets.skillcounts for buildingfilter UI without extra round-trips, and a keyset
pageInfo.nextCursor.Public, same access level as
/api/v1/skilledWorker/**and/api/v1/booking/**— finding a worker has to work before anyone signs in.
Full design write-up:
backend-api/docs/WORKER_DISCOVERY.md.How each hard part is handled
Geo filtering stays index-backed
Two stages.
GeoBox.around(lat, lon, radiusKm)produces a lat/lon bounding box;the query leads with
latitude BETWEEN ? AND ? AND longitude BETWEEN ? AND ?, served by a newcomposite index
idx_skilled_workers_geo (latitude, longitude). The exactHaversine (
GeoDistance, mirrored verbatim in the SQL) then runs only over therows the box already narrowed to — to drop the box corners and to produce
distanceKm. The box widens longitude by1/cos(lat)and degrades safely tothe full range at the poles / across the antimeridian. No PostGIS or
earthdistanceextension — plain B-tree + trigonometry.WorkerDiscoveryIndexUsageTestrunsEXPLAINand asserts the plan usesidx_skilled_workers_geo, not aSeq Scan.Reputation is read off the request path
worker_reputation_snapshotsholds a per-worker materialised copy of thereputationcontract'sRatingaggregate(rating_count, average_rating, reputation_score, source, refreshed_at). The search queryLEFT JOINs it andCOALESCEs a missing score to a configured fallback — the request pathtouches Postgres only, never chain infrastructure.
ReputationSnapshotServicerefreshes the most-stale snapshots on a@Scheduledloop throughReputationContractClient. The implementation(
HttpReputationContractClient) calls a read-model / indexer that projects thecontract aggregate rather than hand-encoding a contract-data
LedgerKeyanddecoding
ScValXDR — the same decisionEscrowReconciliationServicealreadymade and documented, for the same reasons.
guildworkman.discovery.reputation.staleness-bound(default
PT15M) — the upper bound on how stale a worker's reputationcontribution to ranking can be.
source = FALLBACKrow withthe neutral
fallback-score(default0.5). Read fails + prior snapshotexists → left in place, retried next tick. Search always has a value.
Ranking formula is explicit and configurable
WorkerRankingCalculator+RankingWeights(@ConfigurationProperties, notconstants in a comparator):
Defaults
0.5 / 0.3 / 0.2. The divisor normalisesrankScoreto[0,1]forany non-negative weights, so retuning doesn't change the score's scale. The SQL
computes the identical expression (so ordering + pagination stay in the DB);
WorkerRankingCalculatorTestpins the two together. Order isrankScore DESC, workerId ASC— total and deterministic.Pagination is stable
Keyset, not offset. The cursor is an opaque versioned base64url token encoding
the last row's
(rankScore, workerId); the next page addsWHERE rankScore < ? OR (rankScore = ? AND workerId > ?). Rows inserted ordeleted on an earlier page don't shift the window.
WorkerDiscoveryIntegrationTestscrolls a page, inserts a new top-rankedworker, and asserts page 2 neither repeats nor skips a row (where
OFFSETpagination would repeat one). A malformed/tampered/wrong-version cursor is
400 invalid-search-cursorviaGlobalExceptionHandler— never silentlydropped.
Facet counts
facets.categoryandfacets.skillare counted over the same filtered setminus their own dimension (selecting a category doesn't collapse the
category facet, but does narrow the skill facet). Both facet queries reuse the
bounding-box prefilter. Skill facets capped at
guildworkman.discovery.max-skill-facets(default 25), most-common first.Schema changes
ddl-auto=update, no Flyway/Liquibase in this repo (seeESCROW_ORCHESTRATION.md). All additive, nothing dropped or retyped:+ available BOOLEAN NULL(null = "not stated" = available)skilled_workers+ INDEX idx_skilled_workers_geo (latitude, longitude)skilled_workers+ INDEX idx_skilled_workers_category (category)skilled_workers+ TABLE worker_reputation_snapshotsError contract & OpenAPI
Every failure path is RFC 7807
application/problem+jsonthrough the existingGlobalExceptionHandler(400 invalid-search-cursor,400 constraint-violationfor out-of-range params,
400 type-mismatchfor a badcategory). The endpointand DTOs carry
@Operation/@Parameter/@Schemaannotations, so/swagger-ui.htmland/v3/api-docsdescribe it fully.Tests
GeoBoxTestWorkerRankingCalculatorTestCursorCodecTestHttpReputationContractClientTestReputationSnapshotServiceTestWorkerDiscoveryIntegrationTestWorkerDiscoveryIndexUsageTestEXPLAINshowsidx_skilled_workers_geo, notSeq Scan on skilled_workersCI
.github/workflows/test.ymlgets an explicitactions/cache@v4step for~/.m2/repositorykeyed onbackend-api/pom.xml, withrestore-keysfallback(replacing the implicit
setup-javacache so the key and paths are visible andtunable). The reputation refresher's
@Scheduleddelay is defaulted to an hourin the surefire config, matching every other poller in the repo.
Commits
10, one logical unit each:
docs→ geo utils → reputation entity/repo/config →reputation client + refresher → ranking → cursor → repository + indexes →
service → REST endpoint + wiring → integration tests + CI.
New dependencies
None. Uses the existing
okhttp(client), Jackson, Spring Data JPA, springdoc,and
mockwebserver(test) already inpom.xml.